hippo-memory 3.7.0 → 3.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
- # 🦛 Hippo
1
+ # <img src="https://hippo-memory.com/logo.svg" alt="" width="48" height="42"> Hippo
2
2
 
3
- **Your corrections, remembered by Claude Code, Codex and Cursor.**
3
+ **One memory for every agent you'll ever use.**
4
4
 
5
- Hippo is open-source memory that runs on your machine and gives the rules it captured back to the agent when a new session starts. In our Claude Code test on 29 repositories, adding it to built-in memory cut repeat mistakes from 21 in 100 to 11, at the same cost.
5
+ Inspired by the human hippocampus, hippo saves what you teach one coding agent in your repo, and your other agents read it, so switching agents stops meaning starting over.
6
6
 
7
7
  [![npm](https://img.shields.io/npm/v/hippo-memory)](https://npmjs.com/package/hippo-memory)
8
8
  [![npm downloads](https://img.shields.io/npm/dm/hippo-memory)](https://npmjs.com/package/hippo-memory)
@@ -14,81 +14,153 @@ Hippo is open-source memory that runs on your machine and gives the rules it cap
14
14
  <img src="https://raw.githubusercontent.com/kitfunso/hippo-memory/master/assets/hippo-init.svg" alt="hippo init adding memory to one project" width="720">
15
15
  </p>
16
16
 
17
- Hippo keeps your coding agents' memories in a SQLite store on your machine, with markdown mirrors you can read and commit. Search is BM25 out of the box, with no model and no network call; embeddings are an optional install. `hippo init` installs hooks for Claude Code and OpenCode, adds 2 hooks to Codex's `hooks.json` when Codex is installed (Codex runs them once you trust them in `/hooks`), and adds instructions to an existing `AGENTS.md` for Codex, Cursor, OpenClaw and Pi. Any MCP client can connect. Mark a memory wrong and it ranks lower; run `hippo supersede` and the old fact leaves recall. Zero runtime deps.
17
+ ```
18
+ Works with: Claude Code, Codex, Cursor, Pi, OpenClaw, OpenCode, any MCP client
19
+ Imports from: ChatGPT, Claude (CLAUDE.md), Cursor (.cursorrules), Slack, markdown
20
+ Storage: SQLite with markdown mirrors you can read and commit
21
+ Dependencies: Zero runtime deps. Node.js 22.16+. Embeddings: `hippo init` installs the local Transformers.js embedder (`@huggingface/transformers@4.3.2`, about 480 MB) into a global npm install, and `--no-embeddings` skips it. An API embedder (OpenAI/Voyage/Cohere) is opt-in.
22
+ ```
18
23
 
19
- Install it, then run `hippo init` inside one project. Init creates the project's `.hippo/` store and adds a block to the `CLAUDE.md` or `AGENTS.md` already there. On your machine it adds hooks for the agents it finds, such as Claude Code's in `~/.claude/settings.json`, and a daily 6:15am run. [What hippo init changes](#what-hippo-init-changes) lists all of it and the flags that skip each part.
24
+ **Contents:** [Install](#install) · [Follow one memory](#follow-one-memory) · [What hippo does](#what-hippo-does) · [Results](#repeat-mistakes-per-100-chances) · [Setup in detail](#setup-in-detail) · [Agent setup](#framework-integrations) · [MCP server](#mcp-server) · [How it works](#how-it-works) · [CLI](#cli-reference) · [Comparison](#comparison) · [Benchmarks](#benchmarks) · [FAQ](#faq) · [Contributing](#contributing)
20
25
 
21
- ```bash
22
- npm install -g hippo-memory && hippo init
26
+ ---
27
+
28
+ ## Install
29
+
30
+ Paste one prompt into your agent:
31
+
32
+ ```text
33
+ Install hippo-memory for me by following https://github.com/kitfunso/hippo-memory/blob/master/llms-install.md step by step. Run every command yourself and finish with hippo doctor.
23
34
  ```
24
35
 
25
- Setting up every git repo under a folder in one go is a second step. The [Quick start](#quick-start) says what it changes, then gives the command.
36
+ Or by hand:
37
+
38
+ ```bash
39
+ npm install -g hippo-memory
40
+
41
+ # then, in your project
42
+ hippo init
43
+ ```
26
44
 
27
45
  Installing the package alone captures nothing, and what runs after init depends on the agent.
28
46
 
29
- Having an AI agent install it? Point it at [llms-install.md](llms-install.md): it installs, wires hippo into the agents it finds, and verifies with `hippo doctor`.
47
+ `hippo init` creates a `.hippo/` store in the project, adds a short block to an existing `CLAUDE.md` or `AGENTS.md`, installs Claude Code's hooks, schedules a daily run at 6:15am that stores lessons from the day's commits and installs the local embedder for recall by meaning. `--no-hooks`, `--no-schedule` and `--no-embeddings` skip those parts. Codex runs hippo's hooks only once you trust them in `/hooks`. `hippo doctor` checks the install. [What hippo init changes](#what-hippo-init-changes) lists every file it writes, and [llms-install.md](llms-install.md) is the guide the prompt above follows.
30
48
 
31
- ```
32
- Works with: Claude Code, Codex, Cursor, OpenClaw, OpenCode, Pi, any MCP client
33
- Imports from: ChatGPT, Claude (CLAUDE.md), Cursor (.cursorrules), Slack, markdown
34
- Storage: SQLite backbone with markdown mirrors. Git-trackable, human-readable.
35
- Dependencies: Zero runtime deps. Node.js 22.16+. Optional embeddings: bring-your-own local Transformers.js (`npm i @huggingface/transformers`, or legacy `@xenova/transformers`) or an opt-in API embedder (OpenAI/Voyage/Cohere). Nothing is auto-installed.
36
- ```
49
+ ---
37
50
 
38
- **Contents:** [Why](#why-this-exists) · [Receipts](#receipts) · [Quick start](#quick-start) · [Agent setup](#framework-integrations) · [MCP server](#mcp-server) · [How it works](#how-it-works) · [Features](#key-features) · [CLI](#cli-reference) · [Comparison](#comparison) · [Benchmarks](#benchmarks) · [FAQ](#faq) · [Contributing](#contributing)
51
+ ## Follow one memory
52
+
53
+ 1. **You correct your agent once.** Say always, never or must, and hippo's session-end hook pins it.
54
+ 2. **Next session, it's already there.** Your rules load before your first prompt.
55
+ 3. **Every agent reads it.** All your agents read one store, so switching loses nothing.
56
+ 4. **Each prompt gets what's relevant.** Your pinned rules, plus up to 5 memories that share its words.
57
+ 5. **A wrong memory gets weaker.** After a bad answer, `hippo outcome --bad` weakens the memories behind it.
58
+ 6. **A new fact replaces the old.** With `hippo supersede`, the old one leaves recall but stays on record.
59
+ 7. **Unused fades, pinned stays.** Unused memories weaken over a year or two, and pinned ones never fade.
39
60
 
40
61
  ---
41
62
 
42
- ## Why this exists
63
+ ## What hippo does
43
64
 
44
- Most "AI memory" systems save everything and search later. That's storage with search on top. A note that turned out wrong ranks the same as one that held up, and an old fact sits beside the one that replaced it.
65
+ | | What it does | Where |
66
+ |---|---|---|
67
+ | **Capture** | Rules and errors from each session, found by word patterns. | `SessionEnd` hook |
68
+ | **Recall** | Keyword search with no model or network call, and `--why` shows the match. | `hippo recall "why did the deploy break" --why` |
69
+ | **Outcomes** | A bad mark weakens a memory, and each recall keeps it longer. | `hippo outcome --bad` |
70
+ | **Mirror** | A markdown copy of every memory to grep, commit and review. | `.hippo/` |
71
+ | **Dashboard** | Memory health by project, on localhost. | `hippo dashboard` |
72
+ | **Git history** | Learns from the last 30 days of commits, then each day's. | `hippo learn --git` |
73
+ | **Sleep** | Merges related memories and stores faded ones, ready to restore. | `hippo sleep` |
74
+ | **Import** | Brings in the notes your agents already keep, unchanged. | `hippo import --agents` |
45
75
 
46
- Hippo learns from outcomes. When a recalled memory turns out wrong, mark it bad and it drops out of the top results. Memories you keep using get stronger. Those two are the parts we measured helping retrieval, on a synthetic test ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)). When a fact changes, run `hippo supersede` and the old version leaves recall; we have not measured whether that helps. The design borrows from the hippocampus (decay, three layers, sleep consolidation), but that is inspiration. In our tests, decay tied with decay switched off and sleep lowered recall ([Benchmarks](#benchmarks)).
76
+ ---
47
77
 
48
- It also fixes the portability problem. Your ChatGPT memories don't travel to Claude. Your `.cursorrules` don't travel to Codex. Hippo is one store behind every agent. CLAUDE.md, Cursor rules, ChatGPT exports, Slack history, all in one SQLite store, all queryable from any tool that speaks MCP or HTTP.
78
+ ## Repeat mistakes per 100 chances
79
+
80
+ Real sessions on 29 repositories, with the plan registered before the run.
81
+
82
+ | Claude Code with | Repeats per 100 chances |
83
+ |---|--:|
84
+ | No memory | 83 |
85
+ | Built-in memory | 21 |
86
+ | Built-in plus hippo | **11** |
87
+
88
+ Cost with hippo: 1.006 times, the same.
89
+
90
+ The gain was the same with lesson capture switched off, so this run cannot say hippo's stored lessons caused it. The test agent was Claude Code only, so there's no measured result for Codex or Cursor. The [result](docs/evals/2026-11-08-z0-round3-result.md) has every figure, and [Benchmarks](#benchmarks) has the rest of the test.
91
+
92
+ ### Public memory tests
93
+
94
+ One answer model and scorer for every system we ran in a test.
95
+
96
+ | Test | Questions | hippo | Other system |
97
+ |---|--:|--:|---|
98
+ | DMR | 495 | 77.6 | full chat 83.6 |
99
+ | PersonaMem v1 | 584 | 72.4 | MemOS 61.2, published |
100
+ | LongMemEval-S | 495 | 88.5 | |
101
+ | LoCoMo | 395 | 89.4 | Mem0 94.9 |
102
+
103
+ hippo used its local embedder, plus its local reranker on DMR. MemOS's score is from its own setup. [How we test](#benchmarks).
49
104
 
50
105
  ---
51
106
 
52
- ## Receipts
107
+ ## Zero model calls to store a memory
108
+
109
+ Other memory tools run what you save through a language model, an LLM on top of your LLM.
110
+
111
+ | Tool | LLM calls per memory | Unit |
112
+ |---|--:|---|
113
+ | hippo | 0 | per memory, default path |
114
+ | Mem0 | 1 | per add |
115
+ | Zep (Graphiti) | 2+ | per episode, more for duplicates and dates |
116
+ | Letta | 2+ | per turn that writes memory, V1 server |
117
+ | Supermemory | count not public | uses a model, per its docs (low confidence) |
118
+ | Cognee | 1 | per text chunk, with a key |
119
+ | LangMem | 1+ | per run |
120
+ | MemOS | 1 | per message window |
53
121
 
54
- Numbers, not adjectives. Every claim links to the benchmark or the test that proves it.
55
- Every measurement we have ever published is indexed in [`docs/evals/`](docs/evals/README.md),
56
- pre-registrations kept next to their results, including the runs that failed and the one
57
- claim we retracted.
122
+ Storing the LoCoMo test chats took Mem0 5,882 language-model calls, one per turn, and hippo none ([the count](benchmarks/public/results/2026-11-12-answer-accuracy/mem0-locomo-ingest-gate.json)). Counts come from each product's public code and docs, 13 Nov 2026.
58
123
 
59
- - **Correct your agent once, and with hippo it repeats that mistake about half as often. Z0 round 3, real Claude Code sessions on 29 repositories, hippo against built-in memory alone: 11 repeats in 100 against 21, 2 to 4 tasks after the correction, at no extra cost (ratio 1.006 [0.964, 1.049]).** With no memory the agent repeats it 83 times in 100. Over every first chance to repeat, the pre-registered read is 14.9 points fewer [5.5, 24.3], adjusted p 0.006, against a full-win line of 15. The gain is the same with lesson capture switched off, so round 4 tests how much of it the stored lessons carry. ([result](docs/evals/2026-11-08-z0-round3-result.md); [round 2](docs/evals/2026-11-01-z0-round2-result.md), on 16 repositories, was inconclusive at -8.9 points)
60
- - **Sequential Learning Benchmark.** [benchmarks/sequential-learning/](benchmarks/sequential-learning/). 50 tasks, 10 buried traps. Measures whether agents learn from past mistakes, not just retrieve text. v0.11.0 informal magnitude RETRACTED v1.7.9; mechanism remains shipped. See [CHANGELOG.md](./CHANGELOG.md) v1.7.9 entry.
61
- - **LongMemEval oracle split, all 500 questions in one pooled store, BM25 only, no embeddings, v0.11: R@5 = 74.0%** ([benchmarks/README.md](benchmarks/README.md), scripts in [benchmarks/longmemeval/](benchmarks/longmemeval/)). A different setup from the per-haystack results under [Benchmarks](#benchmarks), so the two are not a before and after.
62
- - **On a private 300-query developer store, R@1 0.41 to 0.62 with `hippo recall "<query>" --reranker jev`**, the free local cross-encoder against Jev ([full eval](docs/evals/2026-09-19-jev-reranker.md)). The opt-in [TypeSafe Jev](https://typesafe.ai) reranker, off by default, about 0.0004 USD a recall. 2000-draw paired bootstrap; the margin held in 20 of 20 seeds and a permutation null reached it in 0 of 200 runs. Ranking only: three graded tests on one 150-question LongMemEval set did **not** show a better answer rate than the free local cross-encoder, and that negative result is in the same doc. What it buys today is a shorter context: on that set, 2 memories ranked by Jev answered as well as 5 ranked by the cross-encoder.
63
- - **Staged Slack corpus, 10 incident scenarios: recall beat transcript replay in 10 of 10** ([benchmarks/e1.3/](benchmarks/e1.3/)). The answers sit mid-channel by design. In every scenario hippo's top 10 results held all the answer messages, and the channel's last 10 messages held none.
64
- - **Slack connector, 1000-event ingestion smoke: 0 outbound HTTP** ([benchmarks/e1.3/](benchmarks/e1.3/)). Proven by a `globalThis.fetch` spy that throws on call, not a hardcoded zero. Recall makes no network call by default. One default does: `hippo sleep` sends memory text to Anthropic for fact extraction when `ANTHROPIC_API_KEY` is set. Sleep runs at the end of every Claude Code and OpenCode session and in the daily job, so with the key set the call happens without you asking. `{"extraction":{"enabled":false}}` in `.hippo/config.json` turns that off. Opt-in features such as the Jev reranker above, the LLM reranker and the API embedders also call out.
65
- - **3,500+ tests on a real database.** No module mocks and no mocked store; only paid network calls are stubbed. Project rule. The one mocks-vs-prod divergence that bit us early is now the constraint that kept the next ten releases honest.
66
- - **3-cluster fixture where BM25 alone cannot discriminate: dlPFC goal-conditioned cluster discrimination passes 3 of 3 queries.** Full goal stack with policy weighting and lifespan-windowed outcome propagation, one query per goal; deterministic test in [`benchmarks/micro/results/b3-depth.json`](benchmarks/micro/results/b3-depth.json).
124
+ The exception is `hippo sleep`, which sends text to Anthropic when `ANTHROPIC_API_KEY` is set, and `{"extraction":{"enabled":false}}` turns it off.
67
125
 
68
126
  ---
69
127
 
70
- ## What it does for your agent
128
+ ## One memory, three readers
71
129
 
72
- - **Keeps errors longer.** Tag a failure with `--tag error` and it gets twice the half-life of an ordinary memory, so the lesson is still in the store the next time a recall matches it. In Claude Code, a hook stores failed tool calls as error memories for you.
73
- - **Survives tool switches.** Use Claude Code on Monday, Cursor on Tuesday, Codex on Wednesday. They all read the same `.hippo/` store, so the memories come with you.
74
- - **Ingests systems of record.** Slack and GitHub today (`POST /v1/connectors/slack/events`, `POST /v1/connectors/github/events`). Jira and Notion next. Webhooks land as `kind='raw'` memories with full provenance and GDPR-correct deletion.
75
- - **Knows where every memory came from.** Every row carries `kind`, `scope`, `owner`, and `artifact_ref`. Right-to-be-forgotten is a single API call, not an audit nightmare.
76
- - **Plays nice with multi-tenant.** API keys, scrypt-hashed. Audit log on every mutation. Tenant A literally cannot see tenant B's memories. Proven by negative test.
130
+ You, your team and your platform lead read the same store.
131
+
132
+ - **The developer.** Say a rule with never, always or must, and every session starts with it.
133
+ - **The team.** Commit the markdown copy and review a rule in a pull request.
134
+ - **The platform lead.** One team server gives everyone the same store and their own key.
135
+
136
+ | Agent | What hippo adds |
137
+ |---|---|
138
+ | Claude Code | `CLAUDE.md` + 8 hooks |
139
+ | Codex | `AGENTS.md` + 2 hooks |
140
+ | Cursor | reads `AGENTS.md` |
141
+ | Pi | reads `AGENTS.md` |
142
+ | OpenClaw | plugin |
143
+ | OpenCode | plugin |
144
+ | Any MCP client | `hippo mcp`, 13 tools |
145
+
146
+ `hippo dashboard` opens a local page with each project's memory health.
77
147
 
78
148
  ---
79
149
 
80
- ## Quick start
150
+ ## Memory is infrastructure you own
81
151
 
82
- Start in one project. [What hippo init changes](#what-hippo-init-changes) lists everything the second command writes. A step-by-step version is on the website: [hippo-memory.com/quickstart](https://hippo-memory.com/quickstart/).
152
+ Your agents' memory belongs in a file you can read, diff and delete, whichever agent you use next year.
83
153
 
84
- ```bash
85
- npm install -g hippo-memory
154
+ - **Today:** a team server that shares one store over MCP, with API keys, roles, an audit log, and Slack and GitHub connectors.
155
+ - **Planned:** a commercial edition with SSO, SCIM, SIEM export, hosting and an SLA.
86
156
 
87
- # In a project: create its store and wire in the agents it uses
88
- hippo init
89
- ```
157
+ There's no team result yet, so [open an issue](https://github.com/kitfunso/hippo-memory/issues/new?title=Pilot%20request&body=Company%2C%20team%20size%2C%20agents%20in%20use%3A) for a holdout pilot. [Open source and commercial](#open-source-and-commercial) says what each edition holds.
158
+
159
+ ---
90
160
 
91
- **Optional: many repos at once.** Read what it changes first. `hippo init --scan <folder>` looks for git repos in the folder and up to three levels below it, skipping dot-folders and `node_modules`. Each repo gets a `.hippo/` store, seeded with lessons from the last 365 days of its commits and with its [agent memories](#agent-memories), and is added to the daily run's list. For the agents it finds, it installs the same user-level hooks as `hippo init`: 8 Claude Code hook entries in `~/.claude/settings.json` and the OpenCode plugin. It also sets up the daily 6:15am run, a crontab line on Linux and macOS or a scheduled task on Windows. It adds no block to any repo's `CLAUDE.md` or `AGENTS.md`. `--no-hooks`, `--no-schedule` and `--no-learn` leave out the hooks, the daily run and the history import.
161
+ ## Setup in detail
162
+
163
+ **Many repos at once.** Read what it changes first. `hippo init --scan <folder>` looks for git repos in the folder and up to three levels below it, skipping dot-folders and `node_modules`. Each repo gets a `.hippo/` store, seeded with lessons from the last 365 days of its commits and with its [agent memories](#agent-memories), and is added to the daily run's list. For the agents it finds, it installs the same user-level hooks as `hippo init`: 8 Claude Code hook entries in `~/.claude/settings.json` and the OpenCode plugin. It also sets up the daily 6:15am run, a crontab line on Linux and macOS or a scheduled task on Windows. It adds no block to any repo's `CLAUDE.md` or `AGENTS.md`. `--no-hooks`, `--no-schedule` and `--no-learn` leave out the hooks, the daily run and the history import.
92
164
 
93
165
  ```bash
94
166
  hippo init --scan ~
@@ -123,6 +195,7 @@ Run `hippo init` inside one project. This is everything it writes, in the projec
123
195
  - **OpenCode,** when the project has `.opencode/` or `opencode.json`: a plugin at `~/.config/opencode/plugins/hippo.ts`.
124
196
  - **Codex,** when the project has `AGENTS.md` or `.codex` and Codex is installed (`$CODEX_HOME`, else `~/.codex`, exists): 2 hook entries in Codex's `hooks.json`, one on UserPromptSubmit that sends your pinned memories plus up to 5 that match the prompt with every prompt and one on SessionStart after a compaction. **Codex runs them only after you trust them once in `/hooks`.** Init also prints `hippo hook install codex`, the opt-in that wraps the Codex launcher to capture sessions; `hippo hook uninstall codex` removes hippo's hooks and the wrapper.
125
197
  - **A daily run at 6:15am,** one per machine: a crontab line on Linux and macOS, a scheduled task named `hippo-daily-runner` on Windows. It runs `hippo learn --git --days 1` and then `hippo sleep` in every project listed in `~/.hippo/workspaces.json`, and init adds this project to that list.
198
+ - **The local embedder,** when hippo is a global npm install and no embedder is installed yet: `@huggingface/transformers@4.3.2` in the same global folder, about 480 MB. It goes in before anything else, can take a few minutes, and runs with `--ignore-scripts`, so no install script in that package tree runs. Its model (`Xenova/all-MiniLM-L6-v2`, 87 MB) downloads from Hugging Face the first time hippo embeds. Under npx, pnpm, yarn, bun or as a project dependency, init prints the command for that setup instead. `npm uninstall -g hippo-memory` leaves the embedder in place; `npm uninstall -g @huggingface/transformers` removes it.
126
199
  - **Agent memories.** On every run, the notes your coding agents keep about this project go into its store, and the ones about you go into the global store. [Agent memories](#agent-memories) lists what is read.
127
200
 
128
201
  ```bash
@@ -138,7 +211,7 @@ hippo init
138
211
  # Scheduled machine-level daily runner (6:15am) via crontab
139
212
  ```
140
213
 
141
- To leave parts out: `--no-hooks` skips the instruction files and hooks, `--no-schedule` the daily run, and `--no-learn` the git history and agent memory import. `HIPPO_SKIP_AUTO_INTEGRATIONS=1` skips the same files and hooks that `--no-hooks` does, and `HIPPO_SKIP_SCHEDULE=1` skips the daily run as `--no-schedule` does.
214
+ To leave parts out: `--no-hooks` skips the instruction files and hooks, `--no-schedule` the daily run, `--no-embeddings` the embedder install, and `--no-learn` the git history and agent memory import. `HIPPO_SKIP_AUTO_INTEGRATIONS=1` skips the same files and hooks that `--no-hooks` does, `HIPPO_SKIP_SCHEDULE=1` skips the daily run as `--no-schedule` does, and `HIPPO_SKIP_EMBEDDER_INSTALL=1` skips the embedder as `--no-embeddings` does.
142
215
 
143
216
  ### Agent memories
144
217
 
@@ -165,7 +238,7 @@ Not read: Windsurf (the file format is not documented, and Cascade reached end o
165
238
 
166
239
  ## Cross-Tool Import
167
240
 
168
- Your memories shouldn't be locked inside one tool. Hippo pulls them in from anywhere.
241
+ Hippo imports the notes and rules you already keep in other tools. `hippo import --agents` reads what your coding agents remember ([Agent memories](#agent-memories)); these commands read files:
169
242
 
170
243
  ```bash
171
244
  # ChatGPT memory export
@@ -188,7 +261,7 @@ All import commands support `--dry-run` (preview without writing), `--global` (w
188
261
 
189
262
  ### Conversation Capture
190
263
 
191
- Extract memories from raw conversation text. No LLM needed: pattern-based heuristics find decisions, rules, errors, and preferences.
264
+ Store memories from conversation text. Word patterns, not a model, find decisions, rules, errors and preferences.
192
265
 
193
266
  ```bash
194
267
  # Pipe a conversation in
@@ -227,9 +300,11 @@ Backfill an existing channel: `SLACK_BOT_TOKEN=xoxb-... hippo slack backfill --c
227
300
 
228
301
  Multi-workspace deployments populate `slack_workspaces (team_id, tenant_id)` to route events per tenant; single-workspace falls back to `HIPPO_TENANT`.
229
302
 
303
+ GitHub webhooks land the same way at `POST /v1/connectors/github/events`. Backfill a repository's issues and comments with `hippo github backfill --repo <owner/name>`. Every memory carries `kind`, `scope`, `owner` and `artifact_ref`, and one API call deletes a memory.
304
+
230
305
  ### Active task snapshots
231
306
 
232
- Long-running work needs short-term continuity, not just long-term memory. Hippo can persist the current in-flight task so a later `continue` has something concrete to recover.
307
+ Hippo can save the task in progress, so a later session can pick it up where this one stopped.
233
308
 
234
309
  ```bash
235
310
  hippo snapshot save \
@@ -246,7 +321,7 @@ hippo snapshot clear
246
321
 
247
322
  ### Session event trails
248
323
 
249
- Manual snapshots are useful, but real work also needs a breadcrumb trail. Hippo can now store short session events and link them to the active snapshot so context output shows the latest steps, not just the last summary. The trail prints for the calling session's own snapshot; for another session's, the command in the pointer line prints it.
324
+ Hippo can store short session events and link them to the active snapshot, so context output shows the latest steps as well as the last summary. The trail prints for the calling session's own snapshot; for another session's, the command in the pointer line prints it.
250
325
 
251
326
  ```bash
252
327
  hippo session log \
@@ -296,28 +371,30 @@ hippo wm push --scope repo \
296
371
 
297
372
  hippo wm read --scope repo # show current working notes
298
373
  hippo wm clear --scope repo # wipe the scratchpad
299
- hippo wm flush --scope repo # flush on session end
374
+ hippo wm flush --scope repo # same as clear; nothing runs it at session end
300
375
  ```
301
376
 
302
377
  The buffer holds a maximum of 20 entries per scope. When full, the lowest-importance entry is evicted.
303
378
 
304
379
  ### Explainable recall
305
380
 
306
- See why a memory was returned:
381
+ `--why` shows, for each memory recall returns, the words of your query it matched, the embedding similarity when an embedder is on, and where the memory came from:
307
382
 
308
383
  ```bash
309
384
  hippo recall "data pipeline" --why --limit 5
310
-
311
- # --- mem_a1b2c3 [episodic] [observed] [local] score=0.847
312
- # BM25: matched [data, pipeline]; cosine: 0.82
313
- # ...memory content...
314
385
  ```
315
386
 
387
+ `hippo explain "<query>"` prints the full score breakdown for each result.
388
+
316
389
  ---
317
390
 
318
391
  ## How It Works
319
392
 
320
- Input enters the buffer. Important things get encoded into episodic memory. During "sleep," related episodes are merged into one semantic memory, by word overlap. Weak memories decay and disappear.
393
+ Every memory has a strength. It fades on a half-life, grows each time the memory is recalled, and drops when you mark it wrong. The hippocampus inspired the design, and we tested each part on its own. On synthetic tests, outcome marks and recall strengthening helped and decay made no measurable difference. Sleep lowered recall on one scorer, and the rest is unmeasured. The design notes are in [PLAN.md](PLAN.md#core-principles) and [RESEARCH.md](RESEARCH.md).
394
+
395
+ - **Buffer.** New information lands here for the session.
396
+ - **Episodic.** Timestamped memories that fade unless they are used.
397
+ - **Semantic.** Patterns merged from related episodes during sleep, by word overlap.
321
398
 
322
399
  The store is SQLite (`.hippo/hippo.db`). The markdown files are mirrors written after each change. `index.json` is no longer refreshed by writes, deletes or recalls, so a copy an older version left on disk goes stale. Read the store through the CLI, the MCP server or the HTTP API.
323
400
 
@@ -362,7 +439,7 @@ sequenceDiagram
362
439
 
363
440
  ### Decay by default
364
441
 
365
- Every memory has a half-life: 365 days by default. Until 1.46.0 the default was 7 days. A pre-registered evaluation found 7 days lost the current version of a fact far more often: it was in the top five 29% of the time at 7 days and 75% at 365 ([result](docs/evals/2026-09-24-decay-default-result.md)). 730 days and decay off both tied with 365. So 365 was not tuned: it is the tested value that tied with the others, and on that test decay made no measurable difference to recall. `hippo sleep` moves memories still on the old 7-day base to the new one, once, and records each move in the audit log. Set `defaultHalfLifeDays` in `.hippo/config.json` to choose your own.
442
+ A memory's strength halves each time a half-life passes without use, and the default half-life is 365 days. The default used to be 7 days, which turned out to be a bug. On a [test we registered first](docs/evals/2026-09-24-decay-default-result.md), the current version of a fact was in the top five 29.2% of the time at 7 days and 74.7% at 365. We did not tune 365. It tied with 730 days and with decay switched off, so on that test decay made no measurable difference. `hippo sleep` moves memories still on the old 7-day base to the new one, once, and records each move in the audit log. Set `defaultHalfLifeDays` in `.hippo/config.json` to choose your own, or pin a memory so it never decays.
366
443
 
367
444
  ```bash
368
445
  hippo remember "always check cache contents after refresh"
@@ -375,9 +452,9 @@ hippo inspect mem_a1b2c3
375
452
 
376
453
  ---
377
454
 
378
- ### Retrieval strengthens
455
+ ### Recall makes a memory last longer
379
456
 
380
- Use it or lose it. Each recall boosts the half-life by 2 days.
457
+ Each recall adds 2 days to a memory's half-life, so 365 days becomes 367 and then 369. Recall ranks memories by relevance, strength and recency, and the strongest matches fill the token budget first. In the first round of our audit this helped on the synthetic test, in its best case.
381
458
 
382
459
  ```bash
383
460
  hippo recall "cache issues"
@@ -388,14 +465,13 @@ hippo recall "cache issues"
388
465
  hippo recall "cache issues" # again next week
389
466
  # retrieval_count: 2 -> 3
390
467
  # half_life: 367d -> 369d
391
- # this memory is learning to survive
392
468
  ```
393
469
 
394
470
  ---
395
471
 
396
472
  ### Active invalidation
397
473
 
398
- When you migrate from one tool to another, old memories about the replaced tool should die immediately. Hippo detects migration and breaking-change commits during `hippo learn --git` and actively weakens matching memories.
474
+ When you move from one tool to another, old memories about the replaced tool should stop showing up at once. Hippo detects migration and breaking-change commits during `hippo learn --git` and actively weakens matching memories.
399
475
 
400
476
  ```bash
401
477
  hippo learn --git
@@ -433,17 +509,15 @@ hippo decide "Use CockroachDB for global services" \
433
509
 
434
510
  ---
435
511
 
436
- ### Error memories stick
512
+ ### Errors last twice as long
437
513
 
438
- Tag a memory as an error and it gets 2x the half-life automatically.
514
+ A memory stored with `--error`, or captured from a failed tool call, gets a 730-day half-life against the usual 365. We haven't measured whether that helps.
439
515
 
440
516
  ```bash
441
517
  hippo remember "deployment failed: forgot to run migrations" --error
442
518
  # half_life: 730d instead of 365d
443
519
  # emotional_valence: negative
444
520
  # strength formula applies 2.0x multiplier (HIPPO_LOSS_AVERSION_RATIO=0.75 to keep v1.13.4 1.5x)
445
-
446
- # production incidents don't fade quietly
447
521
  ```
448
522
 
449
523
  ---
@@ -466,13 +540,11 @@ When context is generated, confidence is shown inline:
466
540
  [inferred] The flaky test might be a race condition
467
541
  ```
468
542
 
469
- Agents can see at a glance what's established fact vs. a pattern worth questioning.
470
-
471
543
  A memory not recalled for 30 days is shown as aged when it is read, and recalling it clears that. Pinned and `verified` memories are exempt. Nothing in the store changes.
472
544
 
473
545
  ### Conflict tracking
474
546
 
475
- Hippo detects obvious contradictions between overlapping memories and keeps them visible instead of silently letting both masquerade as truth. Shared tags alone do not count; the statements themselves need to overlap in content.
547
+ When two memories overlap in content and contradict each other, hippo records a conflict that `hippo conflicts` lists. Shared tags alone do not count; the statements themselves need to overlap in content. `hippo supersede <id> "<new fact>"` stores the new version of a fact and links the old one to it, `hippo recall --as-of <date>` shows what was current on a given day, and `hippo reject <id> --reason "<why>"` stops a value coming back at all. None of this is measured yet.
476
548
 
477
549
  ```bash
478
550
  hippo sleep # refreshes open conflicts
@@ -518,9 +590,7 @@ hippo sleep
518
590
  # New semantic: 2
519
591
  ```
520
592
 
521
- Two or more related episodes get merged into a single semantic memory. The originals decay. The pattern survives.
522
-
523
- Sleep keeps the store tidy. It has not been shown to improve recall. In round 2 of the mechanism audit, a slept LongMemEval store scored 3.6 points lower at hit@5 than the same store never slept, and no scorer showed sleep helping ([PR #232](https://github.com/kitfunso/hippo-memory/pull/232)).
593
+ Sleep merges two or more related episodes into one pattern, and the originals fade. It keeps the store tidy. It has not been shown to improve recall, and on one scorer a slept store scored 3.6 points lower than the same store never slept ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)).
524
594
 
525
595
  **Experimental: learned memory-value rescue (opt-in, default off).** With
526
596
  `{"memoryValue":{"enabled":true}}` in `.hippo/config.json`, sleep consults a learned
@@ -529,9 +599,8 @@ top 30% of its tenant by learned value is kept ("rescued") even though its stren
529
599
  below the decay threshold. The scorer can only rescue, never delete: with the flag on,
530
600
  sleep deletes a strict subset of what it would delete with the flag off. Every rescue is
531
601
  recorded in the audit log (`hippo audit list --op mv_rescue`). The weights were learned
532
- on the LongMemEval retention benchmark (held-out retention 0.4897 vs 0.4203 for the best
533
- hand-set baseline); caveat: their usage-feature signs reflect that benchmark's simulated
534
- usage, NOT real usage value, so treat the flag as an experiment, not a recommendation.
602
+ on a LongMemEval retention test with simulated usage, so they do not measure real usage
603
+ value: treat the flag as an experiment.
535
604
  Tenants with fewer than 10 non-pinned memories never rescue (rank statistics are noise at
536
605
  tiny scale).
537
606
 
@@ -619,7 +688,7 @@ are not recorded. Set the rate to 0 only after the pilot window closes, because
619
688
 
620
689
  ### Outcome feedback
621
690
 
622
- Did the recalled memories actually help? Tell Hippo. It tightens the feedback loop.
691
+ `hippo outcome --good` or `--bad` marks the memories from the last recall.
623
692
 
624
693
  ```bash
625
694
  hippo recall "why is the gold model broken"
@@ -636,16 +705,16 @@ hippo outcome --bad
636
705
 
637
706
  Outcomes are cumulative. A memory with 5 positive outcomes and 0 negative has a reward factor of ~1.42, making its effective half-life 42% longer. A memory with 0 positive and 3 negative has a factor of 0.625, so it decays 1.6 times as fast; each bad mark past the good ones also halves its strength, up to three times, and recall stops strengthening it. Mixed outcomes converge toward neutral (1.0).
638
707
 
639
- This is the mechanism with the clearest measured win. On the synthetic E1 test, plain BM25 plus the outcome nudge cut how often a marked-bad memory stayed in the top five from 71.9% to 0.0% ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)). Every mark in E1 is correct; real marks are noisier, since `--bad` marks the whole recall batch.
708
+ On a synthetic test where every mark was correct, a bad mark cut how often a wrong memory stayed in the top five from 71.9% to 0.0% ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)). Real marks are noisier, because `--bad` marks the whole recall batch.
640
709
 
641
710
  ---
642
711
 
643
712
  ### Token budgets
644
713
 
645
- Recall only what fits. No context stuffing.
714
+ Recall prints only what fits the token budget.
646
715
 
647
716
  ```bash
648
- # fits within Claude's 2K token window for task context
717
+ # a small block for task context
649
718
  hippo recall "deployment checklist" --budget 2000
650
719
 
651
720
  # need more for a big task
@@ -712,6 +781,7 @@ The command runs with no shell, so each argument reaches the program as written.
712
781
  | `hippo init` | Create `.hippo/`, install agent hooks and the daily run ([what it changes](#what-hippo-init-changes)) |
713
782
  | `hippo init --global` | Create global store at `~/.hippo/` |
714
783
  | `hippo init --no-hooks` | Create `.hippo/` without auto-installing hooks |
784
+ | `hippo setup` | Detect installed AI tools, install their hooks and import each agent's user-level memories |
715
785
  | `hippo remember "<text>"` | Store a memory |
716
786
  | `hippo remember "<text>" --tag <t>` | Store with tag (repeatable) |
717
787
  | `hippo remember "<text>" --error` | Store as error (2x half-life) |
@@ -720,12 +790,14 @@ The command runs with no shell, so each argument reaches the program as written.
720
790
  | `hippo remember "<text>" --observed` | Set confidence: observed |
721
791
  | `hippo remember "<text>" --inferred` | Set confidence: inferred |
722
792
  | `hippo remember "<text>" --global` | Store in global `~/.hippo/` store |
793
+ | `hippo supersede <id> "<text>"` | Replace a memory with a new version; the old one leaves recall and points at it |
723
794
  | `hippo recall "<query>"` | Retrieve relevant memories (local + global) |
724
795
  | `hippo recall "<query>" --budget <n>` | Recall within token limit (default: 4000) |
725
796
  | `hippo recall "<query>" --limit <n>` | Cap result count |
726
797
  | `hippo recall "<query>" --why` | Show match reasons and source buckets |
727
798
  | `hippo recall "<query>" --hops <n>` | Also surface memories N hops away in the entity/relation graph (0..3, default off) |
728
799
  | `hippo recall "<query>" --json` | Output as JSON |
800
+ | `hippo recall "<query>" --as-of <date>` | Return the memories that were current on that date |
729
801
  | `hippo context --auto` | Smart context injection (auto-detects task from git) |
730
802
  | `hippo context "<query>" --budget <n>` | Context injection with explicit query (default: 1500) |
731
803
  | `hippo context --limit <n>` | Cap memory count in context |
@@ -739,6 +811,8 @@ The command runs with no shell, so each argument reaches the program as written.
739
811
  | `hippo import --file <path>` | Import from any text file |
740
812
  | `hippo import --dry-run` | Preview import without writing |
741
813
  | `hippo import --global` | Write imported memories to `~/.hippo/` |
814
+ | `hippo import --agents [--dry-run]` | Bring in every coding agent's own memories now ([Agent memories](#agent-memories)) |
815
+ | `hippo export [file] [--format markdown]` | Export all memories as JSON (default) or markdown |
742
816
  | `hippo capture --stdin` | Extract memories from piped conversation text |
743
817
  | `hippo capture --file <path>` | Extract memories from a file |
744
818
  | `hippo capture --dry-run` | Preview extraction without writing |
@@ -750,6 +824,7 @@ The command runs with no shell, so each argument reaches the program as written.
750
824
  | `hippo outcome --id <id> --good` | Target a specific memory |
751
825
  | `hippo inspect <id>` | Full detail on one memory |
752
826
  | `hippo forget <id>` | Force remove a memory |
827
+ | `hippo reject <id> --reason "<why>"` | Stop a value coming back: it refuses re-ingestion (`hippo rejections` lists them) |
753
828
  | `hippo dormant [<query>]` | List faded memories sleep kept instead of deleting |
754
829
  | `hippo dormant restore <id>` | Bring a dormant memory back to active memory |
755
830
  | `hippo dormant forget <id>` | Delete a dormant memory permanently |
@@ -859,7 +934,7 @@ For Claude Code, it also adds 8 hook entries to `~/.claude/settings.json`:
859
934
  - a `SessionEnd` hook that runs `hippo sleep` and then `hippo capture` when the session exits. Capture matches the last 20 user and 10 assistant messages of the transcript against word patterns for decisions, rules, errors and preferences. It uses no model unless you turn on `capture.followUpRules` (see Conversation Capture). The patterns do not read earlier turns, so a correction older than that window becomes a pinned rule only with `hippo remember --pin` by hand. Capture also saves each message you typed whole, so recall still finds the older correction (see What you type).
860
935
  - a `SessionStart` hook that prints the previous session's consolidation log on stderr, where `claude --debug` shows it. When the last sleep failed or a compaction summary was set aside, it shows you one line that points to `hippo doctor`.
861
936
  - a second `SessionStart` hook that runs `hippo session-rules`, which prints the rules you taught under `## Rules from the user` and nothing else. It has no matcher, so a resumed or cleared session and a compaction get the rules again.
862
- - a `UserPromptSubmit` hook that runs `hippo context --pinned-only --include-recent 5 --format additional-context` every turn. It re-injects pinned memories (`hippo remember <text> --pin`) plus up to 5 memories that share words with your prompt. When nothing matches, it adds only the pinned ones. Since 1.55.0 this replaces the 5 newest memories, which cut the median block from 847 to 533 tokens in our eval. `{"pinnedInject":{"promptRecall":false}}` brings back the 5 newest, so fresh same-session lessons appear on the next prompt whatever you ask. The block is rendered without live strength percentages, so it stays byte-identical while its memories do not change, and it is sent only when it changed since the session's last prompt: an unchanged block is skipped, resent every 10 skips (`pinnedInject.refreshTurns`, `0` never resends) and resent after compaction. The prompt-matched memories go in a separate block that is never skipped, so they are sent on every prompt they match. `{"pinnedInject":{"skipUnchanged":false}}` sends the pinned block every turn as before. `{"pinnedInject":{"excludeTags":["session-digest"]}}` leaves out every memory that carries a listed tag, whether it is pinned, recent or matched to the prompt; the list is empty by default. Opt out entirely with `{"pinnedInject":{"enabled":false}}` in `.hippo/config.json`.
937
+ - a `UserPromptSubmit` hook that runs `hippo context --pinned-only --include-recent 5 --format additional-context` every turn. It re-injects pinned memories (`hippo remember <text> --pin`) plus up to 5 memories that share words with your prompt. When nothing matches, it adds only the pinned ones. Since 1.55.0 this replaces the 5 newest memories. `{"pinnedInject":{"promptRecall":false}}` brings back the 5 newest, so fresh same-session lessons appear on the next prompt whatever you ask. The block is rendered without live strength percentages, so it stays byte-identical while its memories do not change, and it is sent only when it changed since the session's last prompt: an unchanged block is skipped, resent every 10 skips (`pinnedInject.refreshTurns`, `0` never resends) and resent after compaction. The prompt-matched memories go in a separate block that is never skipped, so they are sent on every prompt they match. `{"pinnedInject":{"skipUnchanged":false}}` sends the pinned block every turn as before. `{"pinnedInject":{"excludeTags":["session-digest"]}}` leaves out every memory that carries a listed tag, whether it is pinned, recent or matched to the prompt; the list is empty by default. Opt out entirely with `{"pinnedInject":{"enabled":false}}` in `.hippo/config.json`.
863
938
  - a `PreCompact` hook that runs `hippo pre-compact` before the transcript gets summarized. It records the compaction in the store, saves a working-state snapshot (task/summary/next step) so mid-session compaction can't drop it, and asks the summariser to end its summary with a "Memories for hippo" list: the lessons, decisions and corrections from the session that should outlive it. The `SessionEnd` hook still owns extracting durable memories from the transcript.
864
939
  - a second `SessionStart` hook (matcher `compact`) that runs `hippo compact-resume`, printing that snapshot back into context right after compaction, if it is under 15 minutes old.
865
940
  - a `PostCompact` hook that runs `hippo post-compact`. It keeps the summary in the store with secrets scrubbed, and saves each item of that list as a memory that sleep never deletes: at most 10 per compaction, skipping an item an earlier compaction already saved and any item that looks like a secret. An item over 500 characters stays in the compaction's record only. It then prints one line, such as "Hippo saved 3 memories from this compaction and restored your task snapshot." If the store is busy, the summary waits in the store's `compactions-spool/` folder and `hippo sleep` finishes the save; `hippo doctor` names any compaction left unfinished for over 10 minutes. It also counts spool files that are waiting, stale (left by a replay that stopped) or set aside as `.bad`. The session that compacted does not have those memories injected back into its own prompts, since it just read them in the summary; `hippo recall` still finds them. The hook prints nothing when there is no store to save to.
@@ -930,7 +1005,7 @@ The headers are a filter the caller picks, not access control: any member key ca
930
1005
 
931
1006
  Native plugin with auto-context injection, workspace-aware memory lookup, and
932
1007
  tool hooks for auto-learn / auto-sleep. When `autoSleep` is enabled, the
933
- OpenClaw plugin now launches `hippo sleep` in a detached background worker at
1008
+ OpenClaw plugin launches `hippo sleep` in a detached background worker at
934
1009
  session end so the live session can exit immediately.
935
1010
 
936
1011
  Query-time retrieval still uses the active workspace store plus the shared
@@ -952,64 +1027,33 @@ Full integration details: [integrations/](integrations/)
952
1027
 
953
1028
  ---
954
1029
 
955
- ## The Neuroscience
956
-
957
- Hippo's design borrows seven properties of the human hippocampus. This section is design inspiration, not measured benefit. For what we measured, see the Receipts and [PR #232](https://github.com/kitfunso/hippo-memory/pull/232).
958
-
959
- **Why two stores?** The brain uses a fast hippocampal buffer + a slow neocortical store (Complementary Learning Systems theory, McClelland et al. 1995). If the neocortex learned fast, new information would overwrite old knowledge. The buffer absorbs new episodes; the neocortex extracts patterns over time.
960
-
961
- **Why decay at all?** In the brain, new neurons born in the dentate gyrus disrupt old memory traces (Frankland et al. 2013), which may reduce interference from outdated information. That is why hippo has decay. In hippo's own tests, age-based decay made no measurable difference to recall: at the 365-day default it tied with decay switched off. A bad outcome mark is the forgetting that measured helpful. Supersession, a newer fact replacing an old one, has not been measured.
962
-
963
- **Why do errors stick?** The amygdala modulates hippocampal consolidation based on emotional significance. Fear and error signals boost encoding. Your first production incident is burned into memory. Your 200th uneventful deploy isn't.
964
-
965
- **Why does retrieval strengthen?** Recalled memories undergo "reconsolidation" (Nader et al. 2000). The act of retrieval destabilizes the trace, then re-encodes it stronger. This is the testing effect. Hippo borrows the idea: each recall adds 2 days to a memory's half-life.
966
-
967
- **Why does sleep consolidate?** During sleep, the hippocampus replays compressed versions of recent episodes and "teaches" the neocortex by repeatedly activating the same patterns. Hippo's `sleep` command borrows the idea for a consolidation pass. In hippo's own audit it lowered recall ([Sleep consolidation](#sleep-consolidation) has the numbers).
968
-
969
- The 7 mechanisms in full: [PLAN.md#core-principles](PLAN.md#core-principles)
970
-
971
- For how these mechanisms connect to LLM training, continual learning, and open research problems: **[RESEARCH.md](RESEARCH.md)**
972
-
973
- **Why does reward modulate decay?** In spiking neural networks, reward-modulated STDP strengthens synapses that contribute to positive outcomes and weakens those that don't. Hippo's reward-proportional decay (v0.11.0) borrows this idea: memories with consistent positive outcomes decay slower, negatives decay faster, with no fixed deltas. Inspired by [MH-FLOCKE](https://github.com/MarcHesse/mhflocke)'s R-STDP architecture for quadruped locomotion, where the same mechanism produces stable learning with 11.6x lower variance than PPO.
974
-
975
- **Prior art in agent memory simulation.** The idea that human-like memory produces human-like behavior as an emergent property was explored in IEEE research from 2010-2011 ([5952114](https://ieeexplore.ieee.org/document/5952114), [5548405](https://ieeexplore.ieee.org/document/5548405), [5953964](https://ieeexplore.ieee.org/document/5953964)). Walking between rooms and forgetting why you went there doesn't need direct simulation; it emerges naturally from a memory system with capacity limits and decay. Hippo takes the mechanisms as design ideas. Whether better agent behavior follows from them has not been shown.
976
-
977
- **Related work:** [HippoRAG](https://arxiv.org/abs/2405.14831) (Gutierrez et al., 2024) applies hippocampal indexing to RAG via knowledge graphs. [MemPalace](https://github.com/milla-jovovich/mempalace) (Sigman & Jovovich, 2026) organizes memory spatially (wings/halls/rooms) with AAAK compression, achieving 100% on [LongMemEval](https://arxiv.org/abs/2410.10813). [MH-FLOCKE](https://github.com/MarcHesse/mhflocke) (Hesse, 2026) uses spiking neurons with R-STDP for embodied cognition. Each system tackles a different facet: HippoRAG optimizes retrieval quality, MemPalace optimizes retrieval organization, MH-FLOCKE optimizes embodied learning, and Hippo works on the memory lifecycle.
978
-
979
- ---
980
-
981
1030
  ## Comparison
982
1031
 
983
- The AI-memory category matured fast in 2026. Hippo's specific take (bio-decay, strengthen-on-use, outcome-weighted half-lives) is one stance among several. The table below is a feature snapshot, not a verdict: graph-first systems ([gbrain](https://hermesatlas.com/projects/garrytan/gbrain), [Zep](https://www.getzep.com/), [Cognee](https://www.cognee.ai/)), agent-managed systems ([Letta](https://github.com/letta-ai/letta-code)), and version-control / skill-distillation takes ([Memoria](https://github.com/matrixorigin/Memoria), [EverMind](https://evermind.ai/)) all solve adjacent problems with different mechanics.
984
-
985
- | Feature | Hippo | [MemPalace](https://github.com/milla-jovovich/mempalace) | [Mem0](https://github.com/mem0ai/mem0) | [Basic Memory](https://github.com/basicmachines-co/basic-memory) | [gbrain](https://hermesatlas.com/projects/garrytan/gbrain) | [Zep](https://www.getzep.com/) | [Letta](https://github.com/letta-ai/letta-code) | [Cognee](https://www.cognee.ai/) | [Memoria](https://github.com/matrixorigin/Memoria) | [EverMind](https://evermind.ai/) |
986
- |---------|-------|-----------|------|-------------|--------|-----|-------|--------|---------|----------|
987
- | Decay by default | Yes | No | No | No | No | No | No | No | No | No |
988
- | Retrieval strengthening | Yes | No | No | No | No | No | No | Partial (recall tuning) | No | Partial (Skill Memory distills patterns) |
989
- | Reward-proportional decay | Yes | No | No | No | No | No | No | No | No | No |
990
- | Hybrid search (BM25 + embeddings) | Yes | Embeddings + spatial | Yes (semantic + BM25 + entity) | No | Yes (vec + rerank + graph) | Yes (graph + vec) | ? | Yes (GraphRAG) | Yes (vector + full-text) | Yes (mRAG, multi-modal) |
991
- | Schema acceleration / knowledge graph | Yes (schema) | No | Partial (entity linking; graph memory on Pro) | No | Yes (typed KG, self-wiring) | Yes (temporal KG) | No | Yes (auto-ontologies) | No (typed claims) | Yes (hierarchical: user/group/agent) |
992
- | Conflict detection + resolution | Yes | No | Partial (hosted platform marks superseded facts) | No | Yes (eval-surfaced) | Yes (auto-invalidate stale facts) | No | No | Yes (auto-detect + quarantine) | Partial (temporal tracking) |
993
- | Multi-agent shared memory | Yes | No | No | No | Yes (brain repo, team mounts) | Yes | Yes (shared memory blocks) | Yes | Yes (branch/merge across sessions) | Yes (multi-agent coordination) |
994
- | Transfer scoring | Yes | No | No | No | No | No | No | No | No | No |
995
- | Outcome tracking | Yes | No | No | No | No | No | No | No | No | Partial (Cases: agent trajectories) |
996
- | Confidence tiers | Yes | No | No | No | No (typed facts) | No | No | No | No | No |
997
- | Spatial organization | No | Yes (wings/halls/rooms) | No | No | No | No | No | No | No | No |
998
- | Lossless compression | No | Yes (AAAK, 30x) | No | No | No | No | No | No | No | No |
999
- | Cross-tool import (ChatGPT/Claude/Cursor) | Yes | No | No | No | Partial (data sources) | ? | No | Partial (28 data sources) | No (Git ops) | Partial (mRAG: PDFs/images/URLs) |
1000
- | Auto-hook install | Yes | No | No | No | No | No | No | No | No | No |
1001
- | MCP server | Yes | Yes | Yes (hosted, needs an account) | Yes | Yes (stdio + HTTP/OAuth) | Yes (hosted, needs an account) | Yes (hosted, needs an API key) | Yes (first-party Claude/LangGraph) | Yes | ? |
1002
- | Zero runtime deps | Yes | No (ChromaDB) | No | No | No (PGLite or PG+pgvector) | No (managed service) | No (npm deps) | No (Python deps) | Yes (single Rust binary) | No (managed + OSS) |
1003
- | LongMemEval (best published) | 98.0% any / 88.5% all R@5\* (local MiniLM; 99.8% any-evidence with voyage-3-large; s_cleaned, per-haystack) | 96.6% raw / 100% reranked R@5 | 94.4 (hosted platform)\*\* | N/A | 95.53% all R@5\* (paid reranker; 93.19% without; s_cleaned) | 90.2% accuracy\*\* (LoCoMo 94.7%) | N/A | N/A | 88.78% accuracy\*\* (overall, with a reader) | 83.00% accuracy\*\* (overall; LoCoMo 93.05%, HaluMem 93.04%) |
1004
- | Git-friendly | Yes | No | No | Yes | Yes | No | Yes (memory tracked in git) | No | Yes (Git is the model) | ? |
1005
- | Framework agnostic | Yes | Yes | Partial | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
1006
- | License | MIT | (open) | Apache-2.0 | (open) | MIT | Proprietary cloud (Graphiti: Apache-2.0) | Apache-2.0 | MIT (core) | Apache-2.0 | Apache-2.0 (OSS) + cloud |
1007
-
1008
- \* Hippo's figures are on `longmemeval_s_cleaned` with a per-question haystack, each the best of five retrieval settings in the benchmark scripts, not `hippo recall`. Any-evidence R@5 counts a hit when any answer session is in the top 5, over all 500 questions: 98.0% with the free local MiniLM embedder (an optional install) and 99.8% with voyage-3-large (measured 2026-06-09, not re-run). All-evidence R@5 counts a hit only when every answer session is in the top 5, over the 470 questions that have an answer: 86.8 to 88.5% with MiniLM. gbrain first published 97.6%, an any-evidence score over all 500; its [report](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-05-07-longmemeval-s.md) now leads with all-evidence, 95.53% (449 of 470) with the paid Voyage rerank-2.5 reranker and 93.19% without it. On all-evidence recall gbrain is ahead. The June 2026 build scored 98.6 any-evidence; [`docs/evals/2026-09-23-longmemeval-reproduction.md`](docs/evals/2026-09-23-longmemeval-reproduction.md) has both runs. An older hippo number, 86.8% R@5 on `longmemeval_oracle` under pooled (non-per-haystack) retrieval, is not comparable to per-haystack figures.
1009
-
1010
- \*\* Different metric: these are end-to-end answer scores, not retrieval R@5. Mem0's 94.4 comes from its hosted platform, which its README says includes optimizations the open-source SDK lacks. Zep's 90.2% and 94.7% are accuracy figures from its homepage. Memoria's 88.78% and EverMind's 83% are overall accuracy with a reader LLM. Higher denominator + LLM helps. Not directly comparable to retrieval-only R@5 numbers above. The Mem0, Zep and Letta columns were last checked against each vendor's own pages on 2026-09-28.
1011
-
1012
- Different tools answer different questions. Mem0 and Basic Memory implement "save everything, search later." MemPalace implements "store everything, organize spatially for retrieval." gbrain, Zep, and Cognee implement "extract typed entities and relationships into a knowledge graph." Letta implements "the agent edits its own memory blocks." Memoria implements "Git-style version control over the memory state itself." EverMind implements "self-evolving Skill Memory + multi-modal retrieval over hierarchical scopes." Hippo implements "learn what is wrong and stop repeating it." These are complementary takes, not a single-axis ranking: bio-lifecycle (Hippo) + GraphRAG (gbrain/Cognee/Zep) + agent-self-edit (Letta) + memory-VCS (Memoria) + skill-distillation (EverMind) cover different parts of the same problem.
1032
+ Hippo is for coding agents on machines you control. [Mem0](https://hippo-memory.com/vs/mem0/) fits better when the memory belongs to your app's end users, [Zep](https://hippo-memory.com/vs/zep/) when you want a managed knowledge graph built from business data, and [Letta Code](https://hippo-memory.com/vs/letta/) when you want an agent that edits its own memory. The table is a feature snapshot, not a verdict, and it holds no scores: [Benchmarks](#benchmarks) has those, split by setup.
1033
+
1034
+ | Feature\* | Hippo | [MemPalace](https://github.com/milla-jovovich/mempalace) | [Mem0](https://github.com/mem0ai/mem0) | [Basic Memory](https://github.com/basicmachines-co/basic-memory) | [Zep](https://www.getzep.com/) | [Letta](https://github.com/letta-ai/letta-code) | [Cognee](https://www.cognee.ai/) | [Memoria](https://github.com/matrixorigin/Memoria) | [EverMind](https://evermind.ai/) |
1035
+ |---------|-------|-----------|------|-------------|-----|-------|--------|---------|----------|
1036
+ | Decay by default | Yes | No | No | No | No | No | No | No | No |
1037
+ | Retrieval strengthening | Yes | No | No | No | No | No | Partial (recall tuning) | No | Partial (Skill Memory distills patterns) |
1038
+ | Reward-proportional decay | Yes | No | No | No | No | No | No | No | No |
1039
+ | Hybrid search (BM25 + embeddings) | Yes | Embeddings + spatial | Yes (semantic + BM25 + entity) | No | Yes (graph + vec) | ? | Yes (GraphRAG) | Yes (vector + full-text) | Yes (mRAG, multi-modal) |
1040
+ | Schema acceleration / knowledge graph | Yes (schema) | No | Partial (entity linking; graph memory on Pro) | No | Yes (temporal KG) | No | Yes (auto-ontologies) | No (typed claims) | Yes (hierarchical: user/group/agent) |
1041
+ | Conflict detection + resolution | Yes | No | Partial (hosted platform marks superseded facts) | No | Yes (auto-invalidate stale facts) | No | No | Yes (auto-detect + quarantine) | Partial (temporal tracking) |
1042
+ | Multi-agent shared memory | Yes | No | No | No | Yes | Yes (shared memory blocks) | Yes | Yes (branch/merge across sessions) | Yes (multi-agent coordination) |
1043
+ | Transfer scoring | Yes | No | No | No | No | No | No | No | No |
1044
+ | Outcome tracking | Yes | No | No | No | No | No | No | No | Partial (Cases: agent trajectories) |
1045
+ | Confidence tiers | Yes | No | No | No | No | No | No | No | No |
1046
+ | Spatial organization | No | Yes (wings/halls/rooms) | No | No | No | No | No | No | No |
1047
+ | Lossless compression | No | Yes (AAAK, 30x) | No | No | No | No | No | No | No |
1048
+ | Cross-tool import (ChatGPT/Claude/Cursor) | Yes | No | No | No | ? | No | Partial (28 data sources) | No (Git ops) | Partial (mRAG: PDFs/images/URLs) |
1049
+ | Auto-hook install | Yes | No | No | No | No | No | No | No | No |
1050
+ | MCP server | Yes | Yes | Yes (hosted, needs an account) | Yes | Yes (hosted, needs an account) | Yes (hosted, needs an API key) | Yes (first-party Claude/LangGraph) | Yes | ? |
1051
+ | Zero runtime deps | Yes | No (ChromaDB) | No | No | No (managed service) | No (npm deps) | No (Python deps) | Yes (single Rust binary) | No (managed + OSS) |
1052
+ | Git-friendly | Yes | No | No | Yes | No | Yes (memory tracked in git) | No | Yes (Git is the model) | ? |
1053
+ | Framework agnostic | Yes | Yes | Partial | Yes | Yes | Yes | Yes | Yes | Yes |
1054
+ | License | MIT | (open) | Apache-2.0 | (open) | Proprietary cloud (Graphiti: Apache-2.0) | Apache-2.0 | MIT (core) | Apache-2.0 | Apache-2.0 (OSS) + cloud |
1055
+
1056
+ \* The Mem0, Zep and Letta columns were last checked against each vendor's own pages on 2026-09-28.
1013
1057
 
1014
1058
  Side-by-side pages on the website: [hippo vs mem0](https://hippo-memory.com/vs/mem0/) · [hippo vs Zep](https://hippo-memory.com/vs/zep/) · [hippo vs Letta](https://hippo-memory.com/vs/letta/) · [CLAUDE.md vs MEMORY.md](https://hippo-memory.com/vs/claude-memory/).
1015
1059
 
@@ -1017,101 +1061,119 @@ Side-by-side pages on the website: [hippo vs mem0](https://hippo-memory.com/vs/m
1017
1061
 
1018
1062
  ## Benchmarks
1019
1063
 
1020
- Three benchmarks testing three different things. Full details in [`benchmarks/`](benchmarks/).
1021
-
1022
- ### LongMemEval (retrieval accuracy)
1023
-
1024
- [LongMemEval](https://arxiv.org/abs/2410.10813) (ICLR 2025) is the industry-standard benchmark: 500 questions across 5 memory abilities, embedded in 115k+ token chat histories.
1025
-
1026
- **`hippo recall` (measured 2026-09-28 on hippo-memory 1.52.5, re-run 2026-10-06 on 1.64.0).** On a default install, `hippo recall` puts an answer session in its top 5 for 85.6% of the 500 `_s` questions (95% CI 82.4 to 88.6), and 87.8% with the optional MiniLM embedder (84.8 to 90.6; 87.6% on 1.52.5). Most of the gap to the scripts below is the default 4,000-token budget. Each memory in this test is a whole session, about 2,600 tokens at the median, so recall returns a median of 2 sessions. With the budget lifted, the same rankings score 96.8% (95.2 to 98.2) and 97.4% (96.0 to 98.6). A lifted call returns every candidate, a median of 47 sessions and 123,491 tokens per question, so those two figures measure the ranking, not an amount of text an agent could take in. Result: [`docs/evals/2026-09-28-recall-cli-longmemeval-result.md`](docs/evals/2026-09-28-recall-cli-longmemeval-result.md); re-run: [`docs/evals/2026-10-06-recall-cli-longmemeval-rerun-1.64.0.md`](docs/evals/2026-10-06-recall-cli-longmemeval-rerun-1.64.0.md).
1064
+ We built hippo and ran these tests, so read them as a vendor's numbers. We ran rivals on the same answer model where we could. Scores marked published come from each team's own setup, so we give no ranks. The same results are on [hippo-memory.com/benchmarks](https://hippo-memory.com/benchmarks/).
1027
1065
 
1028
- **The benchmark scripts (`_s` split; MiniLM re-measured 2026-09-23, voyage measured 2026-06-09).** Each question is scored against its own ~48-session haystack, the same way gbrain and other published systems report. The v1.23.0 pluggable embedding provider lets you choose the embedder:
1066
+ ### Repeat mistakes in real Claude Code sessions
1029
1067
 
1030
- | Embedder | Dense-only R@5 | Best of five settings, R@5 | R@1 |
1031
- |----------|----------------|-----------------|-----|
1032
- | MiniLM-L6 (local, optional install) | 96.8 | 98.0 | 88.4 |
1033
- | voyage-3-large (opt-in, paid) | 99.8 | 99.8 | 94.6 |
1068
+ Each run corrects Claude Code once and then, 2 to 4 tasks later, gives it a task where the same mistake is possible. We ran it on 29 repositories, [registered first](docs/evals/2026-11-08-z0-round3-result.md).
1034
1069
 
1035
- These are any-evidence scores: a hit when any answer session is in the top 5, over all 500 questions. Counting a hit only when every answer session is in the top 5, over the 470 questions that have an answer, the MiniLM runs score 86.8 to 88.5. gbrain first published 97.6 any-evidence and now reports 95.53 all-evidence with a paid reranker, 93.19 without, so on all-evidence recall gbrain is ahead. These numbers come from the scripts in `benchmarks/longmemeval/`, which index every turn and fuse BM25 with dense ranks; they are not `hippo recall`, and a default install has no embedder. Re-measure: [`docs/evals/2026-09-23-longmemeval-reproduction.md`](docs/evals/2026-09-23-longmemeval-reproduction.md). Any-evidence recall is near its ceiling on this task; all-evidence recall is not. Method and the global-pool comparison: [`docs/evals/2026-06-09-longmemeval-per-haystack-dual.md`](docs/evals/2026-06-09-longmemeval-per-haystack-dual.md).
1070
+ | Claude Code with | Repeats in 100 |
1071
+ |---|--:|
1072
+ | No memory | 83 |
1073
+ | Claude Code's built-in memory | 21 |
1074
+ | Built-in memory plus hippo | **11** |
1036
1075
 
1037
- The differentiator is what happens as one store grows. Point retrieval at a single unified memory of tens of thousands of sessions, with no pre-scoped haystack, and recall stops being free (MiniLM 47, voyage 56 on the 19,195-session `_s` store, June 2026). That is where we expect the memory lifecycle to matter, and it is what hippo measures next (see ROADMAP Part III). It is not shown yet. Decay and sleep are design choices, and the tests so far do not favour them:
1076
+ The chart counts first chances 2 to 4 tasks after the correction. The registered measure, the first chance in each mistake family over 148 units, gives 14.9 points fewer repeats with hippo [5.5, 24.3], at the same cost, a ratio of 1.006 [0.964, 1.049]. On tasks with no lesson to apply, neither the cost nor the share solved changed.
1038
1077
 
1039
- - **Decay tied with decay switched off.** On hippo's synthetic lifecycle test, full@365 minus decay-off is -0.7 points [-1.4, 0.1] on currentR5, no measurable effect. That test runs 20 sessions, so a 365-day half-life barely decays inside it ([mechanism audit, round 2](docs/evals/2026-09-23-mechanism-audit-round2-result.md); [decay default](docs/evals/2026-09-24-decay-default-result.md)).
1040
- - **Sleep lowered LongMemEval recall.** The slept store loses 3.6 points of hit@5 [-5.8, -1.4] to the never-slept one under the audit's declared scorer, and no scorer there shows sleep helping recall ([mechanism audit, round 2](docs/evals/2026-09-23-mechanism-audit-round2-result.md)).
1041
- - **Outcome marks helped on the synthetic test.** Plain BM25 plus the fast outcome nudge drops trap persistence from 71.9% to 0.0% (round 2). Round 1 found outcome feedback and retrieval strengthening each help there, in the test's best case: every outcome mark is right, and every scheduled recall repeats the probe's query ([round 1](docs/evals/2026-09-23-mechanism-audit-result.md)). Supersession is not measured yet.
1078
+ What this run can't tell us is why. With lesson capture removed the result was the same, so we can't say hippo's lessons caused it. Round 4 tests that. The corrections were scripted, the agent was Claude Code only, and the test code and run records are not in the public repo. The round before was inconclusive, at 8.9 points fewer on 16 repositories.
1042
1079
 
1043
- None of this shows hippo making agents better at their work; nothing published has shown that.
1080
+ ### Official-setup rows side by side
1044
1081
 
1045
- **Hippo v0.28.0 oracle-split results (hybrid BM25 + cosine, full 500 questions, pooled retrieval):**
1082
+ Hippo 3.5.1 run with the same answer model, judge and settings as the published rows.
1046
1083
 
1047
- | Metric | v0.28 | v0.11 (BM25 only) |
1048
- |--------|-------|-------------------|
1049
- | Recall@1 | 46.6% | 50.4% |
1050
- | Recall@3 | **67.0%** | 66.6% |
1051
- | Recall@5 | 73.8% | 74.0% |
1052
- | Recall@10 | 81.0% | 82.6% |
1053
- | Answer in content@5 | **49.6%** | 46.6% |
1084
+ | Benchmark | hippo | Mem0 | Zep | MemOS | Supermemory | Honcho | Memobase | Other |
1085
+ |---|---|---|---|---|---|---|---|---|
1086
+ | **LoCoMo, Mem0 paper setup**<br><sub>Long chat recall, gpt-4o-mini answers; official setup</sub> | **69.22**<br><sub>embedder, 30 per speaker, all 1,540</sub> | [66.88](https://arxiv.org/html/2504.19413)<br><sub>gpt-4o-mini</sub> | [65.99](https://arxiv.org/html/2504.19413)<br><sub>gpt-4o-mini</sub> | - | - | - | - | [68.44](https://arxiv.org/html/2504.19413)<br><sub>Mem0g · gpt-4o-mini</sub> |
1087
+ | **LongMemEval-S, gpt-4o reader**<br><sub>Long chat recall, gpt-4o answers; official setup</sub> | **81.8**<br><sub>reranker, top 10 (embedder 78.2), official gpt-4o judge</sub> | - | [71.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o</sub> | - | [81.6](https://arxiv.org/html/2512.12818v1)<br><sub>gpt-4o</sub> | - | - | [86.0](https://www.emergence.ai/blog/sota-on-longmemeval-with-rag)<br><sub>EmergenceMem Internal · gpt-4o-2024-08-06</sub> |
1088
+ | **PersonaMem v1, MemOS suite setup**<br><sub>Personalisation, gpt-4o-mini answers; official setup</sub> | **61.97**<br><sub>embedder, top 20, mean of 3 runs</sub> | [43.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [57.8](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [61.2](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [53.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | [58.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.8](https://arxiv.org/html/2507.03724v4)<br><sub>MemU · gpt-4o-mini</sub> |
1089
+ | **EverMemBench, GPT-4.1-mini answers**<br><sub>Changing facts, GPT-4.1-mini answers; official setup, with a caveat</sub> | **44.20**<br><sub>embedder, top 10, mean of 9 categories</sub> | [37.09](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [39.97](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [42.55](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | - | - | [34.27](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [37.44](https://arxiv.org/html/2602.01313v3)<br><sub>Full Context · GPT-4.1-mini</sub> |
1090
+ | **HaluMem-Medium**<br><sub>Hallucination control, GPT-4o answers; official setup, with a caveat</sub> | **70.23**<br><sub>embedder, top 20, GPT-4o answers and judge</sub> | [53.02](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [55.47](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [67.23](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [54.07](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | - | [35.33](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [54.66](https://arxiv.org/html/2511.03506v3)<br><sub>Mem0-Graph · GPT-4o</sub> |
1091
+ | **HaluMem-Long**<br><sub>Hallucination control, GPT-4o answers; official setup, with a caveat</sub> | *paused* | [28.11](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [50.19](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [64.44](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [53.77](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | - | [33.60](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [32.44](https://arxiv.org/html/2511.03506v3)<br><sub>Mem0-Graph · GPT-4o</sub> |
1092
+ | **DolphinBench, official Claude Code setup**<br><sub>Agent memory, Claude Code with Sonnet 5; official setup</sub> | *in build* | [32.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/mem0/README.md)<br><sub>Claude Code, Sonnet 5 · 194/600</sub> | - | - | - | [35.8](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/honcho/README.md)<br><sub>Claude Code, Sonnet 5 · 215/600</sub> | - | [26.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/builtin/README.md)<br><sub>Built-in · Claude Code, Sonnet 5 · 158/600</sub> |
1054
1093
 
1055
- | Question Type | Count | R@5 | R@10 |
1056
- |---------------|-------|-----|------|
1057
- | single-session-assistant | 56 | 100.0% | 100.0% |
1058
- | knowledge-update | 78 | 89.7% | 96.2% |
1059
- | multi-session | 133 | 72.2% | 82.0% |
1060
- | temporal-reasoning | 133 | 72.9% | 78.9% |
1061
- | single-session-user | 70 | 62.9% | 71.4% |
1062
- | single-session-preference | 30 | 20.0% | 33.3% |
1063
-
1064
- For context: MemPalace scores 96.6% (raw) using ChromaDB embeddings + spatial indexing. Hippo v0.28 achieves 73.8% R@5 with hybrid BM25 + cosine. Hybrid scoring trades a little R@1 accuracy for better top-5 content relevance (answer_in_content@5 +3pp vs v0.11).
1065
-
1066
- Hippo's strongest categories (single-session-assistant 100% R@5, knowledge-update 89.7%) are where keyword overlap between question and stored content is highest. The weakest (preference 20%) involves indirect references that need deeper semantic understanding.
1094
+ <details>
1095
+ <summary>How each row ran</summary>
1067
1096
 
1068
- > Note: v0.28 R@10 is 1.6pp below v0.11's BM25-only result. The earlier v0.27 benchmark showed an apparent 35pp regression; that was a methodology bug (budget-limited retrieval vs unlimited), fixed in v0.28 with the `minResults` option. See [`evals/README.md`](evals/README.md) for the full investigation and per-type breakdown.
1097
+ - **LoCoMo, Mem0 paper setup.** Run as the Mem0 paper ran it, with Mem0's released evaluation code (mem0ai/mem0 aae5989e78): all 1,540 questions in categories 1 to 4, 30 memories per speaker, gpt-4o-mini answers under the paper's prompt, and the code's gpt-4o-mini judge. The paper itself does not name its judge. One run. The paper's full-context baseline scores 72.9, above every memory row. Compare the overall score only: our category labels follow our own mapping.
1098
+ - **LongMemEval-S, gpt-4o reader.** The answer model and the judge match the published gpt-4o rows: gpt-4o-2024-08-06 reads hippo's top 10 memories, and the official script's prompts on gpt-4o-2024-08-06 judge all 500 questions. Each system keeps its own retrieval, as in the published rows. The cell shows the reranker setup, 81.8; the embedder setup scores 78.2. We chose the reranker after seeing its Sonnet scores and recorded that before its run. EmergenceMem does not name its judge.
1099
+ - **PersonaMem v1, MemOS suite setup.** Run as MemOS's evaluation suite runs every vendor row (MemTensor/MemOS 3a16e5fe): the same data files and top 20, PM_ANSWER_PROMPT as one system message to gpt-4o-mini-2024-07-18 at temperature 0, 3 runs per question, scored as the mean of the run accuracies over all 589 rows. Each system keeps its own retrieval and memory text.
1100
+ - **EverMemBench, GPT-4.1-mini answers.** The answer model, prompts, top 10, multiple-choice scorer and the 9-category mean match the paper's Table 4 (EverMind-AI/EverMemBench e10b3d5): gpt-4.1-mini-2025-04-14 at temperature 0 answers all 2,400 items. One setting differs: open answers are judged by gpt-5.6-sol through codex with the authors' prompts, and the repository's default judge is gemini-3-flash-preview; the paper names no judge. A second judge, gpt-4o-2024-08-06 with the same prompts, agrees with codex on 96.9% of open verdicts and scores hippo 44.75. The default setup scores 41.66 with codex and 42.50 with gpt-4o.
1101
+ - **HaluMem-Medium.** Question answering only, with the paper's GPT-4o answers and judge (gpt-4o-2024-08-06, temperature 0) and the same data, question set, top 20 and prompts. We do not produce the extraction and update scores, so we cannot enter the HaluMem board yet.
1102
+ - **HaluMem-Long.** Question answering only, with the paper's GPT-4o answers and judge (gpt-4o-2024-08-06, temperature 0), the same setup as the HaluMem-Medium row. The run paused when the OpenAI credit ran out on 16 Nov, so the row has no score yet. We do not produce the extraction and update scores, so we cannot enter the HaluMem board yet.
1103
+ - **DolphinBench, official Claude Code setup.** The same setup as the published Claude Code rows: one Claude Code turn with the hippo plugin for each of the 13,539 messages, each test in a fresh sandbox with memory writes off, Sonnet 5 and the gpt-5.6-sol judge. The run is in build, so there is no score yet.
1069
1104
 
1070
- ```bash
1071
- cd benchmarks/longmemeval
1072
- python ingest_direct.py --data data/longmemeval_oracle.json --store-dir ./store
1073
- python retrieve_fast.py --data data/longmemeval_oracle.json --store-dir ./store --output results/retrieval.jsonl
1074
- python evaluate_retrieval.py --retrieval results/retrieval.jsonl --data data/longmemeval_oracle.json
1075
- ```
1105
+ </details>
1076
1106
 
1077
- ### LoCoMo (conversational evidence recall)
1107
+ ### Our own setup
1108
+
1109
+ Every other benchmark we ran. Our setup differs from the published rows, so these scores do not sit beside theirs.
1110
+
1111
+ | Benchmark | hippo | Mem0 | Zep | MemOS | Supermemory | Honcho | Hindsight | EverMemOS | Memobase | Other |
1112
+ |---|---|---|---|---|---|---|---|---|---|---|
1113
+ | **LoCoMo**<br><sub>Long chat recall, Sonnet 5 answers</sub> | **91.5**<br><sub>embedder, top 200, gpt-4o-mini judge, 400 sample</sub> | [92.5](https://mem0.ai/research-3)<br><sub>not stated · unit not stated</sub> | [75.14](https://www.getzep.com/blog/lies-damn-lies-statistics-is-mem0-really-sota-in-agent-memory/)<br><sub>not stated</sub> | - | - | [89.9](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>Claude Haiku 4.5</sub> | [89.61](https://arxiv.org/html/2512.12818v1)<br><sub>Gemini-3 Pro</sub> | [93.05](https://evermind.ai/blogs/everos-sota-results-across-four-memory-benchmarks-and-what-it-means-for-llm-agents)<br><sub>not stated</sub> | [75.78](https://arxiv.org/html/2512.12818v1)<br><sub>not stated</sub> | [90.0](https://backboard.io/blog/best-ai-memory-score-in-the-world)<br><sub>Backboard · not stated</sub> |
1114
+ | **LongMemEval-S**<br><sub>Long chat recall, Sonnet 5 answers</sub> | **88.4**<br><sub>embedder, top 200, official gpt-4o judge</sub> | [94.4](https://mem0.ai/blog/locomo-benchmark)<br><sub>not stated · unit not stated</sub> | [71.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o</sub> | - | [85.2](https://arxiv.org/html/2512.12818v1)<br><sub>gemini-3-pro</sub> | [92.6](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>Gemini 3 Pro</sub> | [91.4](https://arxiv.org/html/2512.12818v1)<br><sub>Gemini-3 Pro</sub> | [83.00](https://evermind.ai/blogs/everos-sota-results-across-four-memory-benchmarks-and-what-it-means-for-llm-agents)<br><sub>not stated</sub> | - | [94.87](https://mastra.ai/research/observational-memory)<br><sub>Mastra Observational Memory · gpt-5-mini</sub> |
1115
+ | **LongMemEval-M**<br><sub>Very long chat recall, Sonnet 5 answers</sub> | **78.4**<br><sub>default, top 200, official gpt-4o judge</sub> | - | - | - | - | - | - | - | - | - |
1116
+ | **DMR**<br><sub>Multi-session recall, Sonnet 5 answers</sub> | **77.6**<br><sub>reranker, top 20</sub> | - | [98.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o-mini</sub> | - | - | - | - | - | - | [98.0](https://arxiv.org/html/2501.13956v1)<br><sub>Full-conversation baseline · gpt-4o-mini</sub> |
1117
+ | **DMR, gpt-4o-mini answers**<br><sub>Multi-session recall, gpt-4o-mini answers</sub> | **85.60**<br><sub>reranker, top 20, Speaker 1 role</sub> | - | [98.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o-mini</sub> | - | - | - | - | - | - | [98.0](https://arxiv.org/html/2501.13956v1)<br><sub>Full-conversation baseline · gpt-4o-mini</sub> |
1118
+ | **PersonaMem v1**<br><sub>Personalisation, Sonnet 5 answers</sub> | **72.4**<br><sub>embedder, top 20</sub> | [43.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [57.8](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [61.2](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [53.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | - | - | [58.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.8](https://arxiv.org/html/2507.03724v4)<br><sub>MemU · gpt-4o-mini</sub> |
1119
+ | **PersonaMem v2**<br><sub>Personalisation, Sonnet 5 answers</sub> | **50.1**<br><sub>embedder, top 20</sub> | - | - | [40.58](https://raw.githubusercontent.com/MemTensor/MemOS/main/README.md)<br><sub>not stated</sub> | - | - | - | [53.25](https://evermind.ai/blogs/top-ai-memory-systems-benchmarked-in-2026)<br><sub>not stated</sub> | - | [55.2](https://arxiv.org/html/2512.06688)<br><sub>Qwen3-4B agentic memory · Qwen3-4B-Instruct-2507</sub> |
1120
+ | **PrefEval**<br><sub>Preference following, Sonnet 5 answers</sub> | **83.0**<br><sub>default, top 5</sub> | [63.7](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [39.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [71.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.7](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | - | - | [34.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [92.15](https://arxiv.org/html/2502.09597)<br><sub>Claude 3 Sonnet + RAG · Claude 3 Sonnet</sub> |
1121
+ | **EverMemBench**<br><sub>Changing facts, Sonnet 5 answers</sub> | **56.4**<br><sub>default, top 10, mean of 9 categories</sub> | [37.09](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [39.97](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [42.55](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | - | - | - | - | [34.27](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [72.61](https://arxiv.org/html/2602.01313v3)<br><sub>Full Context · Gemini-3-Flash</sub> |
1122
+ | **BEAM-100K**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **72.4**<br><sub>embedder</sub> | - | - | - | - | [63.0](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [73.4](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [79.0](https://www.cognee.ai/benchmarking-cognee-on-beam)<br><sub>cognee · not stated</sub> |
1123
+ | **BEAM-500K**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **70.6**<br><sub>embedder</sub> | - | - | - | - | [64.9](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [71.1](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [35.9](https://arxiv.org/html/2510.27246)<br><sub>LIGHT · Llama-4-Maverick</sub> |
1124
+ | **BEAM-1M**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **69.9**<br><sub>default</sub> | [64.1](https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm)<br><sub>gpt-5</sub> | - | - | - | [63.1](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [73.9](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [33.6](https://arxiv.org/html/2510.27246)<br><sub>LIGHT · Llama-4-Maverick</sub> |
1125
+ | **BEAM-10M**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **54.3**<br><sub>embedder</sub> | [48.6](https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm)<br><sub>gpt-5</sub> | - | - | - | [40.6](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [64.1](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [up to 67.0](https://www.cognee.ai/benchmarking-cognee-on-beam)<br><sub>cognee · not stated</sub> |
1126
+ | **ConvoMem**<br><sub>Conversation memory, Sonnet 5 answers</sub> | **59.6**<br><sub>default, top 20</sub> | - | - | - | - | - | - | - | - | [92.9](https://vectorize.io/articles/mempalace-benchmarks)<br><sub>MemPalace · not stated</sub> |
1127
+ | **HotPotQA-24**<br><sub>Multi-hop questions, F1, Sonnet 5 answers</sub> | **91.0**<br><sub>reranker, top 15, all 24 questions, one run</sub> | [12.0](https://www.cognee.ai/knowledge-graph-memory-benchmarks)<br><sub>not stated</sub> | - | - | - | - | - | - | - | [84.1](https://www.cognee.ai/knowledge-graph-memory-benchmarks)<br><sub>Cognee · not stated</sub> |
1128
+ | **DolphinBench, shortcut run**<br><sub>Agent memory, Claude Code with Sonnet 5</sub> | **45.0**<br><sub>270/600 · Claude Code, Sonnet 5</sub> | [32.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/mem0/README.md)<br><sub>Claude Code, Sonnet 5 · 194/600</sub> | - | - | - | [35.8](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/honcho/README.md)<br><sub>Claude Code, Sonnet 5 · 215/600</sub> | - | - | - | [26.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/builtin/README.md)<br><sub>Built-in · Claude Code, Sonnet 5 · 158/600</sub> |
1129
+ | **Terminal-Bench 2.0**<br><sub>Agent coding tasks, Claude Code with Sonnet 5</sub> | **8/10**<br><sub>first 10 of 89 tasks, 1 trial; Claude Code with no memory 9/10</sub> | - | - | - | - | - | - | - | - | - |
1078
1130
 
1079
- [LoCoMo](https://arxiv.org/abs/2402.17753) is 10 long multi-session conversations (5,882 turns, 1,986 questions). Hippo scores it with a deterministic metric: did the gold evidence turn land in the top 5 recalled memories? No LLM judge is in the scoring path, so the numbers are not comparable to the LLM-as-judge accuracy Mem0 and Letta publish for LoCoMo.
1131
+ <details>
1132
+ <summary>How each row ran</summary>
1133
+
1134
+ - **LoCoMo.** Judged again with the Mem0 paper's prompt on gpt-4o-mini, the judge in Mem0's released code. This row still scores a 400-question sample with Sonnet 5 answers, and the published rows use all 1,540 questions. The "LoCoMo, Mem0 paper setup" row runs that setup in full.
1135
+ - **LongMemEval-S.** Judged with the official script's prompt for each question type and its judge, gpt-4o-2024-08-06, over all 500 questions. One setting differs: the answer model is Sonnet 5, and each published row uses its own. The Mem0 cell used its runner's lenient judge prompt, so it stays grey. The "LongMemEval-S, gpt-4o reader" row uses a gpt-4o reader.
1136
+ - **LongMemEval-M.** Judged with the official script's prompts and its gpt-4o-2024-08-06 judge, over all 500 questions. The answer model is Sonnet 5. No product publishes an answer score for M.
1137
+ - **DMR.** Our harness caps the score: the whole chat scores 83.6% here, against 94% to 98% in the Zep paper.
1138
+ - **DMR, gpt-4o-mini answers.** gpt-4o-mini answers, as in Zep's rows, and the prompt now says the answerer is Speaker 1. The harness check failed: the whole chat scores 92.4 here, against 98.0 for the paper's full-conversation row with the same model, so our setup still caps the score. Compare each system with the whole chat in its own setup only: Zep +0.2 points, hippo's reranker setup -6.8 points. Neither paper names its judge.
1139
+ - **PersonaMem v1.** The vendor rows all answer with gpt-4o-mini under MemOS's prompt, one model for every system. Against the authors' own board of full-context models (best 52%), our score compares with a caveat. A rule that picks the longest option scores 48.9%.
1140
+ - **PersonaMem v2.** The paper's row uses the same text split and scorer with another answer model. The MemOS and EverMemOS rows do not say how they ran. A rule that picks the longest option scores 45.0%.
1141
+ - **PrefEval.** The paper's RAG rows keep the whole chat in the prompt; ours shows 5 retrieved turns. The vendor rows use another data split, top 10 and gpt-4o-mini.
1142
+ - **EverMemBench.** Every published row answers with GPT-4.1-mini, one model for every system, and ours answers with Sonnet 5.
1143
+ - **BEAM-100K.** Hindsight uses another answer prompt and Gemini models with a budget it does not state. Honcho scores event ordering another way. cognee ran one conversation. Mem0 publishes no 100K score.
1144
+ - **BEAM-500K.** Hindsight uses another answer prompt and Gemini models with a budget it does not state. Honcho scores event ordering another way. The LIGHT row comes from the paper's scorer.
1145
+ - **BEAM-1M.** Mem0's row uses the same runner code, prompts and scorer. Mem0 answered and judged with gpt-5 and showed 200 memories; we use Sonnet 5, gpt-5.6-sol and 100. The other rows use other scorers or setups.
1146
+ - **BEAM-10M.** Mem0's row uses the same runner code, prompts and scorer. Mem0 answered and judged with gpt-5 and showed 200 memories; we use Sonnet 5, gpt-5.6-sol and 100. The other rows use other scorers or setups.
1147
+ - **ConvoMem.** The sample design is ours, and no published number exists at 50 conversations. The MemPalace row is a 300-item sample with no stated model or judge.
1148
+ - **HotPotQA-24.** F1 over all 24 questions with Cognee's own scorer, from one run with Sonnet 5 answers. Cognee averages 45 runs and names no answer model, and one question moves the score by about 4 points.
1149
+ - **DolphinBench, shortcut run.** Our run wrote each message straight to the store with no agent turn, used a recall hook we wrote, and judged through codex, which the official validator rejects.
1150
+ - **Terminal-Bench 2.0.** The first 10 of 89 tasks, 1 trial each at medium effort; the rule is 5 trials, and the matching Claude Code row used max effort. The 2.0 board is closed, and the open board is 4.1.
1080
1151
 
1081
- **v1.25.0 baseline (measured 2026-07-05, single run).** Zero-dependency default embedder (`Xenova/all-MiniLM-L6-v2`), fresh store per conversation, `hippo recall --budget 4000`, top-k 5, 1,982 scored questions:
1152
+ </details>
1082
1153
 
1083
- | Category | n | Evidence recall@5 |
1084
- |----------|--:|------------------:|
1085
- | single-hop | 282 | 0.239 |
1086
- | multi-hop | 321 | 0.491 |
1087
- | temporal-reasoning | 92 | 0.169 |
1088
- | open-domain | 841 | 0.450 |
1089
- | adversarial | 446 | 0.226 |
1090
- | **overall** | 1,982 | **0.363** |
1154
+ ### Finding the right session
1091
1155
 
1092
- Read these as a point estimate, not an exact value: n=1, and the run predates the v1.26.0 determinism fix and the harness fix in [#126](https://github.com/kitfunso/hippo-memory/pull/126) (0.9% of stored rows lost their tags at run time). The table has not been re-run on a newer build. Overall recall is 2.10x the April v0.32.0 baseline under the identical protocol. Informational only, gates no feature. Full protocol, caveats and regeneration commands: [`benchmarks/LOCOMO_INVESTIGATION.md`](benchmarks/LOCOMO_INVESTIGATION.md); harness in [`benchmarks/locomo/`](benchmarks/locomo/).
1156
+ On LongMemEval-S, hippo's benchmark scripts put an answer session in the top five for 98.0% of 500 questions with the free local MiniLM embedder. That is the best of five settings, chosen after the run, on a measure close to its ceiling. Requiring every answer session, the same runs score 86.8 to 88.5%. The `hippo recall` command on a default install scores 85.6% [82.4, 88.6]. On one store of all 19,195 sessions, recall fell to 47.2% in June 2026.
1093
1157
 
1094
- ### Sequential Learning Benchmark (agent improvement over time)
1158
+ ### What the two rerankers bought
1095
1159
 
1096
- No other public benchmark tests whether memory systems produce learning curves. LongMemEval tests retrieval on a fixed corpus. This benchmark tests whether an agent with memory *performs better on task 40 than task 5*.
1160
+ `--reranker cross-encoder` reorders the top 50 memories with a free model on your machine, and is off by default. [On DMR](benchmarks/public/results/2026-11-12-answer-accuracy/dmr-best.txt) it lifted hippo's answers from 73.1 with the embedder alone to 77.6, by 4.4 points [0.6, 8.3]. On PersonaMem v1 and HotPotQA-24 the change was within the noise.
1097
1161
 
1098
- 50 tasks, 10 trap categories, each appearing 2-3 times across the sequence.
1162
+ `--reranker jev` sends the top candidates to the hosted TypeSafe Jev API, and is off by default. On 300 developer queries from a private store, the right memory ranked first for 62% of them, against 41% with the free local cross-encoder. [Three graded-answer tests](docs/evals/2026-09-19-jev-reranker.md) showed no better answer rate. It buys a shorter context for about 0.0004 USD a recall, and it sends your query and candidate text to a third party.
1099
1163
 
1100
- > **v0.11.0 informal results: RETRACTED v1.7.9.** The 78% → 14% magnitude does NOT reproduce on the formal sequential-learning benchmark. Three pre-registered workload variants (v1.7.5 full-late, v1.7.6 budget sweep, v1.7.7 `--restrict-late-to 4`) all returned C2 hippo-base late mean = 0.0% across every seed (the workload's late phase saturates structurally). The mechanism (dlPFC goal-stack: `pushGoal`/`completeGoal` hooks, `--use-goal-stack`) is shipped and exercisable. **The magnitude is RETRACTED. The mechanism is shipped; no magnitude is currently claimed.** v1.8.0 (queued) explores adversarial trap categories as mechanism characterisation under the magnitude-smuggling guard in `docs/RETRACTION.md`. Pre-registration trail: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md`, `docs/evals/2026-05-09-v1.7.6-calibration-result.md`, `docs/evals/2026-05-09-v1.7.7-goal-stack-eval-result.md`. CHANGELOG: see v1.7.9 entry.
1164
+ ### What didn't work
1101
1165
 
1102
- <details>
1103
- <summary>Original v0.11.0 informal numbers (RETRACTED, preserved as audit trail in git, not reproduced here)</summary>
1166
+ - Sleep lowered recall by 3.6 points [1.4, 5.8] on one scorer and raised it a little on another, so we claim no benefit for it.
1167
+ - Decay made no measurable difference, on a synthetic test that gives a 365-day half-life little time to act.
1168
+ - The old 7-day default was a bug that cost recall, so the default moved to 365 days.
1169
+ - Seven smaller experiments failed, stalled or were thrown out, among them a correction detector at 0.82 precision against a 0.90 bar.
1170
+ - One early headline number did not reproduce, and [we retracted it](docs/RETRACTION.md).
1104
1171
 
1105
- v0.11.0 reported a single-run informal headline citing late-phase trap-rate decline on the sequential-learning benchmark. The specific numbers are archived at git tag `v0.11.0` and the corresponding `CHANGELOG.md` historical entry. Retained in version control, not reproduced here, since reproduction risks accidental re-citation. See `git show v0.11.0 -- README.md` for the original wording.
1172
+ ### Check our work
1106
1173
 
1107
- </details>
1174
+ The score files for five of these public tests and their write-ups are in the repo, under `benchmarks/public`.
1108
1175
 
1109
- The benchmark, harness, and adapter contract remain shipped. Any memory system can run this benchmark by implementing the [adapter interface](benchmarks/sequential-learning/adapters/interface.mjs).
1110
-
1111
- ```bash
1112
- cd benchmarks/sequential-learning
1113
- node run.mjs --adapter all
1114
- ```
1176
+ [Benchmark results](benchmarks/public) · [Eval documents](docs/evals/README.md) · [Report a mistake](https://github.com/kitfunso/hippo-memory/issues/new)
1115
1177
 
1116
1178
  ---
1117
1179
 
@@ -1180,7 +1242,7 @@ On repeat mistakes, yes, in [one test of our own](https://github.com/kitfunso/hi
1180
1242
  Issues and PRs welcome. Before contributing, run `hippo status` in the repo root to see the project's own memory.
1181
1243
 
1182
1244
  The interesting problems:
1183
- - **LongMemEval retrieval.** `hippo recall` on a default install scores 85.6% R@5 inside its 4,000-token budget and 96.8% with the budget lifted ([result](docs/evals/2026-09-28-recall-cli-longmemeval-result.md)); a budget that fits five median sessions is the next run. The benchmark scripts' best of five settings reach 98.0% with the free local embedder (re-measured 2026-09-23; the June build gave 98.6) and 99.8% with voyage-3-large (measured 2026-06-09), counting a hit when any answer session is in the top 5. That measure is near its ceiling. The all-evidence one, every answer session in the top 5 over the 470 questions with an answer, is not; there the MiniLM runs score 86.8 to 88.5% against gbrain's 95.53%. The lifecycle stress eval (ROADMAP Part III) is the next measurement.
1245
+ - **Recall as one store grows.** On LongMemEval-S, `hippo recall` on a default install scores 85.6% [82.4, 88.6] ([result](docs/evals/2026-09-28-recall-cli-longmemeval-result.md)), and on one store of all 19,195 sessions recall fell to 47.2% in June 2026. Requiring every answer session in the top five, the benchmark scripts score 86.8 to 88.5%. The lifecycle stress eval (ROADMAP Part III) is the next measurement.
1184
1246
  - Better consolidation heuristics (LLM-powered merge vs current text overlap)
1185
1247
  - Web UI / dashboard for visualizing decay curves and memory health
1186
1248
  - Optimal decay parameter tuning from real usage data
@@ -1190,9 +1252,11 @@ The interesting problems:
1190
1252
 
1191
1253
  ## Open source and commercial
1192
1254
 
1193
- Hippo is open core. This repository provides the MIT core for individual developers and self-hosted teams: the CLI, the MCP server, supported hooks and adapters, connectors, the dashboard, tenants, API keys, admin/member roles, per-key scope grants and the audit log. Automatic capture and delivery depend on the configured integration; the all-agent low-touch acceptance work remains planned. Code published here stays MIT and stays here.
1255
+ Hippo is open core. This repository is the MIT core for individual developers and self-hosted teams: the CLI, the MCP server, the supported hooks and adapters, and the dashboard. It includes an HTTP server with tenants, API keys, admin and member roles, per-key scope grants and an audit log on every change. Keys are stored scrypt-hashed, and one API call deletes a memory. Slack and GitHub connectors bring in what your team has already written down, with Jira and Notion connectors next. Automatic capture and delivery depend on the configured integration. Code published here stays MIT and stays here.
1256
+
1257
+ The fair test for a team is a holdout, where a random share of sessions runs without memory and you compare cost per merged PR and repeat errors between the two groups. We have no team result to show. If your team wants to try, [open an issue](https://github.com/kitfunso/hippo-memory/issues/new) and tell us what you'd need. More on [hippo-memory.com/teams](https://hippo-memory.com/teams/).
1194
1258
 
1195
- The commercial edition is planned; its private repository is a scaffold, not a released enterprise product. It is intended for larger companies as a separate package under a commercial licence from KITFUNSO LTD. Planned capabilities include SSO (OIDC and SAML sign-in), SCIM, organisation/team/project policy, an org admin view, the pilot report and telemetry join, SIEM export of the audit log, offline licence keys, serving agent hooks to laptops that keep no store of their own, hosted SaaS, and support with an SLA. Pull requests implementing those commercial features belong there; see [CONTRIBUTING.md](CONTRIBUTING.md). Release availability and independently verified benefit are separate milestones in the [roadmap](https://github.com/kitfunso/hippo-memory/blob/master/ROADMAP.md#current-execution-index).
1259
+ The commercial edition is planned; its private repository is a scaffold, not a released enterprise product. None of it is released. It is intended for larger companies as a separate package under a commercial licence from KITFUNSO LTD. Planned capabilities include SSO (OIDC and SAML sign-in), SCIM, organisation/team/project policy, an org admin view, the pilot report and telemetry join, SIEM export of the audit log, offline licence keys, serving agent hooks to laptops that keep no store of their own, hosted SaaS, and support with an SLA. Pull requests implementing those commercial features belong there; see [CONTRIBUTING.md](CONTRIBUTING.md). Release availability and independently verified benefit are separate milestones in the [roadmap](https://github.com/kitfunso/hippo-memory/blob/master/ROADMAP.md#current-execution-index).
1196
1260
 
1197
1261
  ## License
1198
1262