crbro-memory 1.14.0 → 1.15.0

package/README.md

@@ -16,6 +16,7 @@ Free and open source (MIT). All 23 tools included — no license, no account, no
 
 - **🧬 Biological Architecture** — Knowledge organized as neurons (cortex), connections (synapses), and session memory (hippocampus)
 - **🔍 Fact-Level Search** — Powered by [Orama](https://orama.com/). Every fact is indexed on its own, so a topic with hundreds of facts stays as findable as one with three. Each result comes back with the exact line that matched, when it was recorded, a `confidence` label (`weak` = little of the question was covered) and, for the top results, the topic's next best lines. A short bilingual synonym table widens the question without inventing terms *(v1.13+)*
+- **🗣️ The model in the loop** — Two levers no embedding model replaces, measured blind: keywords written at save time (the caller knows the synonyms: a line about Hetzner gets *hosting, alojamiento, servidor*) and several phrasings searched at once, fused by rank. Zero disk, zero RAM; numbers in the table below *(v1.15+)*
 - **🧭 Semantic layer, opt-in** — Two commands and one env var add a local embedding model (`multilingual-e5-small`, int8) fused with the keyword engine: paraphrases the words do not cover start to land. Measured +8 points of recall@1 on the blind benchmark; ~500 MB on disk, never installed or loaded unless you ask *(v1.14+)*
 - **🔥 Heat Scores** — Automatic relevance tracking based on frequency, recency, and connectivity. Topics written in the same session are linked at consolidation, so the graph fills itself in *(v1.13+)*
 - **✏️ Correctable** — Knowledge can be superseded or retracted, not just piled up. A memory that only appends keeps serving yesterday's answer with today's confidence
@@ -39,7 +40,8 @@ Every number below comes from a deterministic benchmark in [`benchmarks/`](bench
 | What | Result | The honest part |
 |------|--------|-----------------|
 | **Retrieval** (48 blind paraphrased queries, written by someone who never saw the stored text) | recall@1 **71%** · recall@3 **77%** · MRR 0.74 — and **79% / 85%** counting the neuron's `also_matched` lines | Was 56% / 69% in 1.12. Of the 13 misses, 8 were the *right neuron answering with the wrong line* (its name chunk, or a sibling fact) — fixed in the engine; the rest are vocabulary gaps, which a short bilingual synonym table now closes in part. A naive substring search scores 38% / 58%. Still no semantic model: the remaining misses are listed in the benchmark output |
-| **Retrieval with the opt-in semantic layer** (same 48 queries, `CRBRO_SEMANTIC=1`) | recall@1 **79%** · recall@3 **83%** · MRR 0.81 — **88% / 92%** counting `also_matched` | Vectors from `multilingual-e5-small` (int8) fused with BM25 by reciprocal rank. Alone, the model scores 60% / 83%; fused, it adds 8 points at recall@1 and no distractor reaches a real hit's score (0 of 14; 12 return something, 11 of them labelled `weak`). The cosine floor under which a vector-only candidate is dropped (0.84) was picked on this same set — a tuned number, not a blind one. Costs ~500 MB on disk, a one-time embedding pass (~18 ms per line) and ~13 s of model load per process. Off by default *(v1.14+)* |
+| **Retrieval with the opt-in semantic layer** (same 48 queries, `CRBRO_SEMANTIC=1`) | recall@1 **79%** · recall@3 **83%** · MRR 0.81 — **88% / 92%** counting `also_matched` | Vectors from `multilingual-e5-small` (int8) fused with BM25 by reciprocal rank. Alone, the model scores 60% / 83%; fused, it adds 8 points at recall@1 and no distractor reaches a real hit's score (0 of 14; 12 return something, 11 of them labelled `weak`). The cosine floor under which a vector-only candidate is dropped (0.84) was picked on this same set — a tuned number, not a blind one. Costs ~500 MB on disk, ~0.5 GB of RAM while the server runs, a one-time embedding pass (~3 min for a 4k-line brain) and ~13 s of model load per process. Off by default *(v1.14+)* |
+| **Retrieval with the model in the loop** (same 48 queries; keywords and rewrites written blind by a model that saw only one half of the test) | keywords alone: recall@1 **83%** · recall@3 **90%** — everything on (keywords + rewrites + semantic layer): **90% / 92%**, and **96% / 98%** counting `also_matched` | The biggest lever costs nothing: 2-5 keywords written when a fact is saved close exactly the gaps no embedding model closed. Rewrites alone barely move the keyword engine (71% → 71% / 79%); they add up on top of keywords. Every configuration and the three questions still missed are in [`benchmarks/README.md`](benchmarks/README.md) *(v1.15+)* |
 | **Retrieval — false confidence** (14 questions about things that are NOT stored) | 11 return *something*; **2** at a real hit's score; **10 of 11** labelled `weak` | A keyword memory answers almost anything. Every result now carries `confidence`, and the label catches nearly every distractor — at the price of also calling 18 of 48 real hits weak. Weak means "little of the question was covered", not "wrong" |
 | **Secret redaction** (20 credentials in adversarial disguises, 19 near-miss innocents) | **100%** caught · **0%** false positives | 100% on *this frozen set* — a floor, not a security proof. The set grows as new evasion shapes appear; four of its entries were misses in the first run and were fixed, not hidden |
 | **Cost** (what CRBRO adds to a session) | ~**750 tokens** at boot · **~5.4k tokens** of tool definitions · **<1 ms** local recall over 300 facts | The boot block is paid once. The 23 tool definitions are paid on every request by clients that load all tools (Claude Desktop, Cursor); Claude Code defers them and pays only for the ones it uses. 1.12 measured 6.3k and did not say so |
@@ -116,10 +118,10 @@ Session context never reaches Task-spawned subagents, so this hook can inject th
 |------|-------------|
 | `crbro_boot` | Boot the brain at session start — loads hot topics and context |
 | `crbro_status` | Brain status — neurons, synapses, sessions count |
-| `crbro_learn` | Store a fact, decision, pattern, preference, error or debt |
+| `crbro_learn` | Store a fact, decision, pattern, preference, error or debt — with the keywords a future question may use |
 | `crbro_neuron` | Read a specific neuron (topic) with all its knowledge |
 | `crbro_neurons` | List neurons with optional filters (domain, type, heat) |
-| `crbro_recall` | Search every stored line, not just topic names — returns what matched, how confidently, and the topic's next best lines |
+| `crbro_recall` | Search every stored line, not just topic names — returns what matched, how confidently, and the topic's next best lines. Several phrasings at once are fused by rank |
 | `crbro_connect` | Create or strengthen a connection between neurons |
 | `crbro_connections` | Get all connections for a neuron |
 | `crbro_session_log` | Log a session summary |
@@ -280,11 +282,13 @@ The keyword engine has no synonyms, and the blind benchmark shows exactly where
 ```bash
 npx crbro-memory semantic install     # once per machine → ~/.crbro/.semantic
 # then add CRBRO_SEMANTIC=1 to the crbro server's env in your MCP client config
-npx crbro-memory semantic build       # once: embeds every line of the brain (~18 ms each)
+npx crbro-memory semantic build       # once: embeds every line of the brain (a 4k-line brain: ~3 min)
 ```
 
 From then on every new line is embedded when it is saved (ids are content hashes, so nothing is embedded twice), the model warms in the background after boot, and `crbro_recall` fuses both rankings by reciprocal rank. Results the vectors ranked carry `semantic_score`; a vector-only match is `strong` from cosine 0.86. Set `CRBRO_SEMANTIC=0` (or just unset it) and the layer disappears: no vectors are read, no model is loaded, recall is the 1.13 engine byte for byte.
 
+The model is `multilingual-e5-small` and stays so on purpose. `CRBRO_SEMANTIC_MODEL` accepts any e5-family model, and `e5-base` and `e5-large` were measured on the same benchmark: the large one is the better model alone (71% vs 63% recall@1) but fused with the keyword engine it scores the same or worse (75% / 85% vs 79% / 83%) for 4× the disk, 1.2 GB of RAM and 6× the time per line. The table is in [`benchmarks/README.md`](benchmarks/README.md).
+
 What it buys on the frozen benchmark, and what it does not, is in the table above and in [`benchmarks/README.md`](benchmarks/README.md) — including the fact that the 0.84 cosine floor was chosen on that same set. One limit worth knowing before you install 500 MB: the model does not understand the question. Queries that share no concrete word with the stored line ("which machine serves the pages" for a fact about a Hetzner VPS) land in a flat 0.80–0.84 cosine band with near-random ordering — measured, and the reason the floor exists. What it adds is tolerance to vocabulary variation and to entities, which is where the benchmark gain comes from.
 
 ### Measuring retrieval