hippo-memory 3.7.0 → 3.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +277 -213
- package/dist/base/util/env.d.ts +1 -0
- package/dist/base/util/env.js +2 -0
- package/dist/base/util/version.d.ts +1 -1
- package/dist/base/util/version.js +1 -1
- package/dist/surface/cli/embedder-install.d.ts +18 -0
- package/dist/surface/cli/embedder-install.js +121 -0
- package/dist/surface/cli/init.js +32 -1
- package/dist/surface/cli/maintenance.js +2 -2
- package/dist/surface/cli/usage.d.ts +1 -1
- package/dist/surface/cli/usage.js +1 -1
- package/dist/surface/cli/verbs/memory.js +3 -1
- package/extensions/openclaw-plugin/README.md +5 -13
- package/extensions/openclaw-plugin/openclaw.plugin.json +2 -2
- package/extensions/openclaw-plugin/package.json +2 -2
- package/openclaw.plugin.json +2 -2
- package/package.json +4 -5
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
#
|
|
1
|
+
# <img src="https://hippo-memory.com/logo.svg" alt="" width="48" height="42"> Hippo
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**One memory for every agent you'll ever use.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Inspired by the human hippocampus, hippo saves what you teach one coding agent in your repo, and your other agents read it, so switching agents stops meaning starting over.
|
|
6
6
|
|
|
7
7
|
[](https://npmjs.com/package/hippo-memory)
|
|
8
8
|
[](https://npmjs.com/package/hippo-memory)
|
|
@@ -14,81 +14,153 @@ Hippo is open-source memory that runs on your machine and gives the rules it cap
|
|
|
14
14
|
<img src="https://raw.githubusercontent.com/kitfunso/hippo-memory/master/assets/hippo-init.svg" alt="hippo init adding memory to one project" width="720">
|
|
15
15
|
</p>
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
```
|
|
18
|
+
Works with: Claude Code, Codex, Cursor, Pi, OpenClaw, OpenCode, any MCP client
|
|
19
|
+
Imports from: ChatGPT, Claude (CLAUDE.md), Cursor (.cursorrules), Slack, markdown
|
|
20
|
+
Storage: SQLite with markdown mirrors you can read and commit
|
|
21
|
+
Dependencies: Zero runtime deps. Node.js 22.16+. Embeddings: `hippo init` installs the local Transformers.js embedder (`@huggingface/transformers@4.3.2`, about 480 MB) into a global npm install, and `--no-embeddings` skips it. An API embedder (OpenAI/Voyage/Cohere) is opt-in.
|
|
22
|
+
```
|
|
18
23
|
|
|
19
|
-
Install
|
|
24
|
+
**Contents:** [Install](#install) · [Follow one memory](#follow-one-memory) · [What hippo does](#what-hippo-does) · [Results](#repeat-mistakes-per-100-chances) · [Setup in detail](#setup-in-detail) · [Agent setup](#framework-integrations) · [MCP server](#mcp-server) · [How it works](#how-it-works) · [CLI](#cli-reference) · [Comparison](#comparison) · [Benchmarks](#benchmarks) · [FAQ](#faq) · [Contributing](#contributing)
|
|
20
25
|
|
|
21
|
-
|
|
22
|
-
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## Install
|
|
29
|
+
|
|
30
|
+
Paste one prompt into your agent:
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
Install hippo-memory for me by following https://github.com/kitfunso/hippo-memory/blob/master/llms-install.md step by step. Run every command yourself and finish with hippo doctor.
|
|
23
34
|
```
|
|
24
35
|
|
|
25
|
-
|
|
36
|
+
Or by hand:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
npm install -g hippo-memory
|
|
40
|
+
|
|
41
|
+
# then, in your project
|
|
42
|
+
hippo init
|
|
43
|
+
```
|
|
26
44
|
|
|
27
45
|
Installing the package alone captures nothing, and what runs after init depends on the agent.
|
|
28
46
|
|
|
29
|
-
|
|
47
|
+
`hippo init` creates a `.hippo/` store in the project, adds a short block to an existing `CLAUDE.md` or `AGENTS.md`, installs Claude Code's hooks, schedules a daily run at 6:15am that stores lessons from the day's commits and installs the local embedder for recall by meaning. `--no-hooks`, `--no-schedule` and `--no-embeddings` skip those parts. Codex runs hippo's hooks only once you trust them in `/hooks`. `hippo doctor` checks the install. [What hippo init changes](#what-hippo-init-changes) lists every file it writes, and [llms-install.md](llms-install.md) is the guide the prompt above follows.
|
|
30
48
|
|
|
31
|
-
|
|
32
|
-
Works with: Claude Code, Codex, Cursor, OpenClaw, OpenCode, Pi, any MCP client
|
|
33
|
-
Imports from: ChatGPT, Claude (CLAUDE.md), Cursor (.cursorrules), Slack, markdown
|
|
34
|
-
Storage: SQLite backbone with markdown mirrors. Git-trackable, human-readable.
|
|
35
|
-
Dependencies: Zero runtime deps. Node.js 22.16+. Optional embeddings: bring-your-own local Transformers.js (`npm i @huggingface/transformers`, or legacy `@xenova/transformers`) or an opt-in API embedder (OpenAI/Voyage/Cohere). Nothing is auto-installed.
|
|
36
|
-
```
|
|
49
|
+
---
|
|
37
50
|
|
|
38
|
-
|
|
51
|
+
## Follow one memory
|
|
52
|
+
|
|
53
|
+
1. **You correct your agent once.** Say always, never or must, and hippo's session-end hook pins it.
|
|
54
|
+
2. **Next session, it's already there.** Your rules load before your first prompt.
|
|
55
|
+
3. **Every agent reads it.** All your agents read one store, so switching loses nothing.
|
|
56
|
+
4. **Each prompt gets what's relevant.** Your pinned rules, plus up to 5 memories that share its words.
|
|
57
|
+
5. **A wrong memory gets weaker.** After a bad answer, `hippo outcome --bad` weakens the memories behind it.
|
|
58
|
+
6. **A new fact replaces the old.** With `hippo supersede`, the old one leaves recall but stays on record.
|
|
59
|
+
7. **Unused fades, pinned stays.** Unused memories weaken over a year or two, and pinned ones never fade.
|
|
39
60
|
|
|
40
61
|
---
|
|
41
62
|
|
|
42
|
-
##
|
|
63
|
+
## What hippo does
|
|
43
64
|
|
|
44
|
-
|
|
65
|
+
| | What it does | Where |
|
|
66
|
+
|---|---|---|
|
|
67
|
+
| **Capture** | Rules and errors from each session, found by word patterns. | `SessionEnd` hook |
|
|
68
|
+
| **Recall** | Keyword search with no model or network call, and `--why` shows the match. | `hippo recall "why did the deploy break" --why` |
|
|
69
|
+
| **Outcomes** | A bad mark weakens a memory, and each recall keeps it longer. | `hippo outcome --bad` |
|
|
70
|
+
| **Mirror** | A markdown copy of every memory to grep, commit and review. | `.hippo/` |
|
|
71
|
+
| **Dashboard** | Memory health by project, on localhost. | `hippo dashboard` |
|
|
72
|
+
| **Git history** | Learns from the last 30 days of commits, then each day's. | `hippo learn --git` |
|
|
73
|
+
| **Sleep** | Merges related memories and stores faded ones, ready to restore. | `hippo sleep` |
|
|
74
|
+
| **Import** | Brings in the notes your agents already keep, unchanged. | `hippo import --agents` |
|
|
45
75
|
|
|
46
|
-
|
|
76
|
+
---
|
|
47
77
|
|
|
48
|
-
|
|
78
|
+
## Repeat mistakes per 100 chances
|
|
79
|
+
|
|
80
|
+
Real sessions on 29 repositories, with the plan registered before the run.
|
|
81
|
+
|
|
82
|
+
| Claude Code with | Repeats per 100 chances |
|
|
83
|
+
|---|--:|
|
|
84
|
+
| No memory | 83 |
|
|
85
|
+
| Built-in memory | 21 |
|
|
86
|
+
| Built-in plus hippo | **11** |
|
|
87
|
+
|
|
88
|
+
Cost with hippo: 1.006 times, the same.
|
|
89
|
+
|
|
90
|
+
The gain was the same with lesson capture switched off, so this run cannot say hippo's stored lessons caused it. The test agent was Claude Code only, so there's no measured result for Codex or Cursor. The [result](docs/evals/2026-11-08-z0-round3-result.md) has every figure, and [Benchmarks](#benchmarks) has the rest of the test.
|
|
91
|
+
|
|
92
|
+
### Public memory tests
|
|
93
|
+
|
|
94
|
+
One answer model and scorer for every system we ran in a test.
|
|
95
|
+
|
|
96
|
+
| Test | Questions | hippo | Other system |
|
|
97
|
+
|---|--:|--:|---|
|
|
98
|
+
| DMR | 495 | 77.6 | full chat 83.6 |
|
|
99
|
+
| PersonaMem v1 | 584 | 72.4 | MemOS 61.2, published |
|
|
100
|
+
| LongMemEval-S | 495 | 88.5 | |
|
|
101
|
+
| LoCoMo | 395 | 89.4 | Mem0 94.9 |
|
|
102
|
+
|
|
103
|
+
hippo used its local embedder, plus its local reranker on DMR. MemOS's score is from its own setup. [How we test](#benchmarks).
|
|
49
104
|
|
|
50
105
|
---
|
|
51
106
|
|
|
52
|
-
##
|
|
107
|
+
## Zero model calls to store a memory
|
|
108
|
+
|
|
109
|
+
Other memory tools run what you save through a language model, an LLM on top of your LLM.
|
|
110
|
+
|
|
111
|
+
| Tool | LLM calls per memory | Unit |
|
|
112
|
+
|---|--:|---|
|
|
113
|
+
| hippo | 0 | per memory, default path |
|
|
114
|
+
| Mem0 | 1 | per add |
|
|
115
|
+
| Zep (Graphiti) | 2+ | per episode, more for duplicates and dates |
|
|
116
|
+
| Letta | 2+ | per turn that writes memory, V1 server |
|
|
117
|
+
| Supermemory | count not public | uses a model, per its docs (low confidence) |
|
|
118
|
+
| Cognee | 1 | per text chunk, with a key |
|
|
119
|
+
| LangMem | 1+ | per run |
|
|
120
|
+
| MemOS | 1 | per message window |
|
|
53
121
|
|
|
54
|
-
|
|
55
|
-
Every measurement we have ever published is indexed in [`docs/evals/`](docs/evals/README.md),
|
|
56
|
-
pre-registrations kept next to their results, including the runs that failed and the one
|
|
57
|
-
claim we retracted.
|
|
122
|
+
Storing the LoCoMo test chats took Mem0 5,882 language-model calls, one per turn, and hippo none ([the count](benchmarks/public/results/2026-11-12-answer-accuracy/mem0-locomo-ingest-gate.json)). Counts come from each product's public code and docs, 13 Nov 2026.
|
|
58
123
|
|
|
59
|
-
|
|
60
|
-
- **Sequential Learning Benchmark.** [benchmarks/sequential-learning/](benchmarks/sequential-learning/). 50 tasks, 10 buried traps. Measures whether agents learn from past mistakes, not just retrieve text. v0.11.0 informal magnitude RETRACTED v1.7.9; mechanism remains shipped. See [CHANGELOG.md](./CHANGELOG.md) v1.7.9 entry.
|
|
61
|
-
- **LongMemEval oracle split, all 500 questions in one pooled store, BM25 only, no embeddings, v0.11: R@5 = 74.0%** ([benchmarks/README.md](benchmarks/README.md), scripts in [benchmarks/longmemeval/](benchmarks/longmemeval/)). A different setup from the per-haystack results under [Benchmarks](#benchmarks), so the two are not a before and after.
|
|
62
|
-
- **On a private 300-query developer store, R@1 0.41 to 0.62 with `hippo recall "<query>" --reranker jev`**, the free local cross-encoder against Jev ([full eval](docs/evals/2026-09-19-jev-reranker.md)). The opt-in [TypeSafe Jev](https://typesafe.ai) reranker, off by default, about 0.0004 USD a recall. 2000-draw paired bootstrap; the margin held in 20 of 20 seeds and a permutation null reached it in 0 of 200 runs. Ranking only: three graded tests on one 150-question LongMemEval set did **not** show a better answer rate than the free local cross-encoder, and that negative result is in the same doc. What it buys today is a shorter context: on that set, 2 memories ranked by Jev answered as well as 5 ranked by the cross-encoder.
|
|
63
|
-
- **Staged Slack corpus, 10 incident scenarios: recall beat transcript replay in 10 of 10** ([benchmarks/e1.3/](benchmarks/e1.3/)). The answers sit mid-channel by design. In every scenario hippo's top 10 results held all the answer messages, and the channel's last 10 messages held none.
|
|
64
|
-
- **Slack connector, 1000-event ingestion smoke: 0 outbound HTTP** ([benchmarks/e1.3/](benchmarks/e1.3/)). Proven by a `globalThis.fetch` spy that throws on call, not a hardcoded zero. Recall makes no network call by default. One default does: `hippo sleep` sends memory text to Anthropic for fact extraction when `ANTHROPIC_API_KEY` is set. Sleep runs at the end of every Claude Code and OpenCode session and in the daily job, so with the key set the call happens without you asking. `{"extraction":{"enabled":false}}` in `.hippo/config.json` turns that off. Opt-in features such as the Jev reranker above, the LLM reranker and the API embedders also call out.
|
|
65
|
-
- **3,500+ tests on a real database.** No module mocks and no mocked store; only paid network calls are stubbed. Project rule. The one mocks-vs-prod divergence that bit us early is now the constraint that kept the next ten releases honest.
|
|
66
|
-
- **3-cluster fixture where BM25 alone cannot discriminate: dlPFC goal-conditioned cluster discrimination passes 3 of 3 queries.** Full goal stack with policy weighting and lifespan-windowed outcome propagation, one query per goal; deterministic test in [`benchmarks/micro/results/b3-depth.json`](benchmarks/micro/results/b3-depth.json).
|
|
124
|
+
The exception is `hippo sleep`, which sends text to Anthropic when `ANTHROPIC_API_KEY` is set, and `{"extraction":{"enabled":false}}` turns it off.
|
|
67
125
|
|
|
68
126
|
---
|
|
69
127
|
|
|
70
|
-
##
|
|
128
|
+
## One memory, three readers
|
|
71
129
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
- **
|
|
75
|
-
- **
|
|
76
|
-
- **
|
|
130
|
+
You, your team and your platform lead read the same store.
|
|
131
|
+
|
|
132
|
+
- **The developer.** Say a rule with never, always or must, and every session starts with it.
|
|
133
|
+
- **The team.** Commit the markdown copy and review a rule in a pull request.
|
|
134
|
+
- **The platform lead.** One team server gives everyone the same store and their own key.
|
|
135
|
+
|
|
136
|
+
| Agent | What hippo adds |
|
|
137
|
+
|---|---|
|
|
138
|
+
| Claude Code | `CLAUDE.md` + 8 hooks |
|
|
139
|
+
| Codex | `AGENTS.md` + 2 hooks |
|
|
140
|
+
| Cursor | reads `AGENTS.md` |
|
|
141
|
+
| Pi | reads `AGENTS.md` |
|
|
142
|
+
| OpenClaw | plugin |
|
|
143
|
+
| OpenCode | plugin |
|
|
144
|
+
| Any MCP client | `hippo mcp`, 13 tools |
|
|
145
|
+
|
|
146
|
+
`hippo dashboard` opens a local page with each project's memory health.
|
|
77
147
|
|
|
78
148
|
---
|
|
79
149
|
|
|
80
|
-
##
|
|
150
|
+
## Memory is infrastructure you own
|
|
81
151
|
|
|
82
|
-
|
|
152
|
+
Your agents' memory belongs in a file you can read, diff and delete, whichever agent you use next year.
|
|
83
153
|
|
|
84
|
-
|
|
85
|
-
|
|
154
|
+
- **Today:** a team server that shares one store over MCP, with API keys, roles, an audit log, and Slack and GitHub connectors.
|
|
155
|
+
- **Planned:** a commercial edition with SSO, SCIM, SIEM export, hosting and an SLA.
|
|
86
156
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
157
|
+
There's no team result yet, so [open an issue](https://github.com/kitfunso/hippo-memory/issues/new?title=Pilot%20request&body=Company%2C%20team%20size%2C%20agents%20in%20use%3A) for a holdout pilot. [Open source and commercial](#open-source-and-commercial) says what each edition holds.
|
|
158
|
+
|
|
159
|
+
---
|
|
90
160
|
|
|
91
|
-
|
|
161
|
+
## Setup in detail
|
|
162
|
+
|
|
163
|
+
**Many repos at once.** Read what it changes first. `hippo init --scan <folder>` looks for git repos in the folder and up to three levels below it, skipping dot-folders and `node_modules`. Each repo gets a `.hippo/` store, seeded with lessons from the last 365 days of its commits and with its [agent memories](#agent-memories), and is added to the daily run's list. For the agents it finds, it installs the same user-level hooks as `hippo init`: 8 Claude Code hook entries in `~/.claude/settings.json` and the OpenCode plugin. It also sets up the daily 6:15am run, a crontab line on Linux and macOS or a scheduled task on Windows. It adds no block to any repo's `CLAUDE.md` or `AGENTS.md`. `--no-hooks`, `--no-schedule` and `--no-learn` leave out the hooks, the daily run and the history import.
|
|
92
164
|
|
|
93
165
|
```bash
|
|
94
166
|
hippo init --scan ~
|
|
@@ -123,6 +195,7 @@ Run `hippo init` inside one project. This is everything it writes, in the projec
|
|
|
123
195
|
- **OpenCode,** when the project has `.opencode/` or `opencode.json`: a plugin at `~/.config/opencode/plugins/hippo.ts`.
|
|
124
196
|
- **Codex,** when the project has `AGENTS.md` or `.codex` and Codex is installed (`$CODEX_HOME`, else `~/.codex`, exists): 2 hook entries in Codex's `hooks.json`, one on UserPromptSubmit that sends your pinned memories plus up to 5 that match the prompt with every prompt and one on SessionStart after a compaction. **Codex runs them only after you trust them once in `/hooks`.** Init also prints `hippo hook install codex`, the opt-in that wraps the Codex launcher to capture sessions; `hippo hook uninstall codex` removes hippo's hooks and the wrapper.
|
|
125
197
|
- **A daily run at 6:15am,** one per machine: a crontab line on Linux and macOS, a scheduled task named `hippo-daily-runner` on Windows. It runs `hippo learn --git --days 1` and then `hippo sleep` in every project listed in `~/.hippo/workspaces.json`, and init adds this project to that list.
|
|
198
|
+
- **The local embedder,** when hippo is a global npm install and no embedder is installed yet: `@huggingface/transformers@4.3.2` in the same global folder, about 480 MB. It goes in before anything else, can take a few minutes, and runs with `--ignore-scripts`, so no install script in that package tree runs. Its model (`Xenova/all-MiniLM-L6-v2`, 87 MB) downloads from Hugging Face the first time hippo embeds. Under npx, pnpm, yarn, bun or as a project dependency, init prints the command for that setup instead. `npm uninstall -g hippo-memory` leaves the embedder in place; `npm uninstall -g @huggingface/transformers` removes it.
|
|
126
199
|
- **Agent memories.** On every run, the notes your coding agents keep about this project go into its store, and the ones about you go into the global store. [Agent memories](#agent-memories) lists what is read.
|
|
127
200
|
|
|
128
201
|
```bash
|
|
@@ -138,7 +211,7 @@ hippo init
|
|
|
138
211
|
# Scheduled machine-level daily runner (6:15am) via crontab
|
|
139
212
|
```
|
|
140
213
|
|
|
141
|
-
To leave parts out: `--no-hooks` skips the instruction files and hooks, `--no-schedule` the daily run, and `--no-learn` the git history and agent memory import. `HIPPO_SKIP_AUTO_INTEGRATIONS=1` skips the same files and hooks that `--no-hooks` does,
|
|
214
|
+
To leave parts out: `--no-hooks` skips the instruction files and hooks, `--no-schedule` the daily run, `--no-embeddings` the embedder install, and `--no-learn` the git history and agent memory import. `HIPPO_SKIP_AUTO_INTEGRATIONS=1` skips the same files and hooks that `--no-hooks` does, `HIPPO_SKIP_SCHEDULE=1` skips the daily run as `--no-schedule` does, and `HIPPO_SKIP_EMBEDDER_INSTALL=1` skips the embedder as `--no-embeddings` does.
|
|
142
215
|
|
|
143
216
|
### Agent memories
|
|
144
217
|
|
|
@@ -165,7 +238,7 @@ Not read: Windsurf (the file format is not documented, and Cascade reached end o
|
|
|
165
238
|
|
|
166
239
|
## Cross-Tool Import
|
|
167
240
|
|
|
168
|
-
|
|
241
|
+
Hippo imports the notes and rules you already keep in other tools. `hippo import --agents` reads what your coding agents remember ([Agent memories](#agent-memories)); these commands read files:
|
|
169
242
|
|
|
170
243
|
```bash
|
|
171
244
|
# ChatGPT memory export
|
|
@@ -188,7 +261,7 @@ All import commands support `--dry-run` (preview without writing), `--global` (w
|
|
|
188
261
|
|
|
189
262
|
### Conversation Capture
|
|
190
263
|
|
|
191
|
-
|
|
264
|
+
Store memories from conversation text. Word patterns, not a model, find decisions, rules, errors and preferences.
|
|
192
265
|
|
|
193
266
|
```bash
|
|
194
267
|
# Pipe a conversation in
|
|
@@ -227,9 +300,11 @@ Backfill an existing channel: `SLACK_BOT_TOKEN=xoxb-... hippo slack backfill --c
|
|
|
227
300
|
|
|
228
301
|
Multi-workspace deployments populate `slack_workspaces (team_id, tenant_id)` to route events per tenant; single-workspace falls back to `HIPPO_TENANT`.
|
|
229
302
|
|
|
303
|
+
GitHub webhooks land the same way at `POST /v1/connectors/github/events`. Backfill a repository's issues and comments with `hippo github backfill --repo <owner/name>`. Every memory carries `kind`, `scope`, `owner` and `artifact_ref`, and one API call deletes a memory.
|
|
304
|
+
|
|
230
305
|
### Active task snapshots
|
|
231
306
|
|
|
232
|
-
|
|
307
|
+
Hippo can save the task in progress, so a later session can pick it up where this one stopped.
|
|
233
308
|
|
|
234
309
|
```bash
|
|
235
310
|
hippo snapshot save \
|
|
@@ -246,7 +321,7 @@ hippo snapshot clear
|
|
|
246
321
|
|
|
247
322
|
### Session event trails
|
|
248
323
|
|
|
249
|
-
|
|
324
|
+
Hippo can store short session events and link them to the active snapshot, so context output shows the latest steps as well as the last summary. The trail prints for the calling session's own snapshot; for another session's, the command in the pointer line prints it.
|
|
250
325
|
|
|
251
326
|
```bash
|
|
252
327
|
hippo session log \
|
|
@@ -296,28 +371,30 @@ hippo wm push --scope repo \
|
|
|
296
371
|
|
|
297
372
|
hippo wm read --scope repo # show current working notes
|
|
298
373
|
hippo wm clear --scope repo # wipe the scratchpad
|
|
299
|
-
hippo wm flush --scope repo #
|
|
374
|
+
hippo wm flush --scope repo # same as clear; nothing runs it at session end
|
|
300
375
|
```
|
|
301
376
|
|
|
302
377
|
The buffer holds a maximum of 20 entries per scope. When full, the lowest-importance entry is evicted.
|
|
303
378
|
|
|
304
379
|
### Explainable recall
|
|
305
380
|
|
|
306
|
-
|
|
381
|
+
`--why` shows, for each memory recall returns, the words of your query it matched, the embedding similarity when an embedder is on, and where the memory came from:
|
|
307
382
|
|
|
308
383
|
```bash
|
|
309
384
|
hippo recall "data pipeline" --why --limit 5
|
|
310
|
-
|
|
311
|
-
# --- mem_a1b2c3 [episodic] [observed] [local] score=0.847
|
|
312
|
-
# BM25: matched [data, pipeline]; cosine: 0.82
|
|
313
|
-
# ...memory content...
|
|
314
385
|
```
|
|
315
386
|
|
|
387
|
+
`hippo explain "<query>"` prints the full score breakdown for each result.
|
|
388
|
+
|
|
316
389
|
---
|
|
317
390
|
|
|
318
391
|
## How It Works
|
|
319
392
|
|
|
320
|
-
|
|
393
|
+
Every memory has a strength. It fades on a half-life, grows each time the memory is recalled, and drops when you mark it wrong. The hippocampus inspired the design, and we tested each part on its own. On synthetic tests, outcome marks and recall strengthening helped and decay made no measurable difference. Sleep lowered recall on one scorer, and the rest is unmeasured. The design notes are in [PLAN.md](PLAN.md#core-principles) and [RESEARCH.md](RESEARCH.md).
|
|
394
|
+
|
|
395
|
+
- **Buffer.** New information lands here for the session.
|
|
396
|
+
- **Episodic.** Timestamped memories that fade unless they are used.
|
|
397
|
+
- **Semantic.** Patterns merged from related episodes during sleep, by word overlap.
|
|
321
398
|
|
|
322
399
|
The store is SQLite (`.hippo/hippo.db`). The markdown files are mirrors written after each change. `index.json` is no longer refreshed by writes, deletes or recalls, so a copy an older version left on disk goes stale. Read the store through the CLI, the MCP server or the HTTP API.
|
|
323
400
|
|
|
@@ -362,7 +439,7 @@ sequenceDiagram
|
|
|
362
439
|
|
|
363
440
|
### Decay by default
|
|
364
441
|
|
|
365
|
-
|
|
442
|
+
A memory's strength halves each time a half-life passes without use, and the default half-life is 365 days. The default used to be 7 days, which turned out to be a bug. On a [test we registered first](docs/evals/2026-09-24-decay-default-result.md), the current version of a fact was in the top five 29.2% of the time at 7 days and 74.7% at 365. We did not tune 365. It tied with 730 days and with decay switched off, so on that test decay made no measurable difference. `hippo sleep` moves memories still on the old 7-day base to the new one, once, and records each move in the audit log. Set `defaultHalfLifeDays` in `.hippo/config.json` to choose your own, or pin a memory so it never decays.
|
|
366
443
|
|
|
367
444
|
```bash
|
|
368
445
|
hippo remember "always check cache contents after refresh"
|
|
@@ -375,9 +452,9 @@ hippo inspect mem_a1b2c3
|
|
|
375
452
|
|
|
376
453
|
---
|
|
377
454
|
|
|
378
|
-
###
|
|
455
|
+
### Recall makes a memory last longer
|
|
379
456
|
|
|
380
|
-
|
|
457
|
+
Each recall adds 2 days to a memory's half-life, so 365 days becomes 367 and then 369. Recall ranks memories by relevance, strength and recency, and the strongest matches fill the token budget first. In the first round of our audit this helped on the synthetic test, in its best case.
|
|
381
458
|
|
|
382
459
|
```bash
|
|
383
460
|
hippo recall "cache issues"
|
|
@@ -388,14 +465,13 @@ hippo recall "cache issues"
|
|
|
388
465
|
hippo recall "cache issues" # again next week
|
|
389
466
|
# retrieval_count: 2 -> 3
|
|
390
467
|
# half_life: 367d -> 369d
|
|
391
|
-
# this memory is learning to survive
|
|
392
468
|
```
|
|
393
469
|
|
|
394
470
|
---
|
|
395
471
|
|
|
396
472
|
### Active invalidation
|
|
397
473
|
|
|
398
|
-
When you
|
|
474
|
+
When you move from one tool to another, old memories about the replaced tool should stop showing up at once. Hippo detects migration and breaking-change commits during `hippo learn --git` and actively weakens matching memories.
|
|
399
475
|
|
|
400
476
|
```bash
|
|
401
477
|
hippo learn --git
|
|
@@ -433,17 +509,15 @@ hippo decide "Use CockroachDB for global services" \
|
|
|
433
509
|
|
|
434
510
|
---
|
|
435
511
|
|
|
436
|
-
###
|
|
512
|
+
### Errors last twice as long
|
|
437
513
|
|
|
438
|
-
|
|
514
|
+
A memory stored with `--error`, or captured from a failed tool call, gets a 730-day half-life against the usual 365. We haven't measured whether that helps.
|
|
439
515
|
|
|
440
516
|
```bash
|
|
441
517
|
hippo remember "deployment failed: forgot to run migrations" --error
|
|
442
518
|
# half_life: 730d instead of 365d
|
|
443
519
|
# emotional_valence: negative
|
|
444
520
|
# strength formula applies 2.0x multiplier (HIPPO_LOSS_AVERSION_RATIO=0.75 to keep v1.13.4 1.5x)
|
|
445
|
-
|
|
446
|
-
# production incidents don't fade quietly
|
|
447
521
|
```
|
|
448
522
|
|
|
449
523
|
---
|
|
@@ -466,13 +540,11 @@ When context is generated, confidence is shown inline:
|
|
|
466
540
|
[inferred] The flaky test might be a race condition
|
|
467
541
|
```
|
|
468
542
|
|
|
469
|
-
Agents can see at a glance what's established fact vs. a pattern worth questioning.
|
|
470
|
-
|
|
471
543
|
A memory not recalled for 30 days is shown as aged when it is read, and recalling it clears that. Pinned and `verified` memories are exempt. Nothing in the store changes.
|
|
472
544
|
|
|
473
545
|
### Conflict tracking
|
|
474
546
|
|
|
475
|
-
|
|
547
|
+
When two memories overlap in content and contradict each other, hippo records a conflict that `hippo conflicts` lists. Shared tags alone do not count; the statements themselves need to overlap in content. `hippo supersede <id> "<new fact>"` stores the new version of a fact and links the old one to it, `hippo recall --as-of <date>` shows what was current on a given day, and `hippo reject <id> --reason "<why>"` stops a value coming back at all. None of this is measured yet.
|
|
476
548
|
|
|
477
549
|
```bash
|
|
478
550
|
hippo sleep # refreshes open conflicts
|
|
@@ -518,9 +590,7 @@ hippo sleep
|
|
|
518
590
|
# New semantic: 2
|
|
519
591
|
```
|
|
520
592
|
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
Sleep keeps the store tidy. It has not been shown to improve recall. In round 2 of the mechanism audit, a slept LongMemEval store scored 3.6 points lower at hit@5 than the same store never slept, and no scorer showed sleep helping ([PR #232](https://github.com/kitfunso/hippo-memory/pull/232)).
|
|
593
|
+
Sleep merges two or more related episodes into one pattern, and the originals fade. It keeps the store tidy. It has not been shown to improve recall, and on one scorer a slept store scored 3.6 points lower than the same store never slept ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)).
|
|
524
594
|
|
|
525
595
|
**Experimental: learned memory-value rescue (opt-in, default off).** With
|
|
526
596
|
`{"memoryValue":{"enabled":true}}` in `.hippo/config.json`, sleep consults a learned
|
|
@@ -529,9 +599,8 @@ top 30% of its tenant by learned value is kept ("rescued") even though its stren
|
|
|
529
599
|
below the decay threshold. The scorer can only rescue, never delete: with the flag on,
|
|
530
600
|
sleep deletes a strict subset of what it would delete with the flag off. Every rescue is
|
|
531
601
|
recorded in the audit log (`hippo audit list --op mv_rescue`). The weights were learned
|
|
532
|
-
on
|
|
533
|
-
|
|
534
|
-
usage, NOT real usage value, so treat the flag as an experiment, not a recommendation.
|
|
602
|
+
on a LongMemEval retention test with simulated usage, so they do not measure real usage
|
|
603
|
+
value: treat the flag as an experiment.
|
|
535
604
|
Tenants with fewer than 10 non-pinned memories never rescue (rank statistics are noise at
|
|
536
605
|
tiny scale).
|
|
537
606
|
|
|
@@ -619,7 +688,7 @@ are not recorded. Set the rate to 0 only after the pilot window closes, because
|
|
|
619
688
|
|
|
620
689
|
### Outcome feedback
|
|
621
690
|
|
|
622
|
-
|
|
691
|
+
`hippo outcome --good` or `--bad` marks the memories from the last recall.
|
|
623
692
|
|
|
624
693
|
```bash
|
|
625
694
|
hippo recall "why is the gold model broken"
|
|
@@ -636,16 +705,16 @@ hippo outcome --bad
|
|
|
636
705
|
|
|
637
706
|
Outcomes are cumulative. A memory with 5 positive outcomes and 0 negative has a reward factor of ~1.42, making its effective half-life 42% longer. A memory with 0 positive and 3 negative has a factor of 0.625, so it decays 1.6 times as fast; each bad mark past the good ones also halves its strength, up to three times, and recall stops strengthening it. Mixed outcomes converge toward neutral (1.0).
|
|
638
707
|
|
|
639
|
-
|
|
708
|
+
On a synthetic test where every mark was correct, a bad mark cut how often a wrong memory stayed in the top five from 71.9% to 0.0% ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)). Real marks are noisier, because `--bad` marks the whole recall batch.
|
|
640
709
|
|
|
641
710
|
---
|
|
642
711
|
|
|
643
712
|
### Token budgets
|
|
644
713
|
|
|
645
|
-
Recall only what fits
|
|
714
|
+
Recall prints only what fits the token budget.
|
|
646
715
|
|
|
647
716
|
```bash
|
|
648
|
-
#
|
|
717
|
+
# a small block for task context
|
|
649
718
|
hippo recall "deployment checklist" --budget 2000
|
|
650
719
|
|
|
651
720
|
# need more for a big task
|
|
@@ -712,6 +781,7 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
712
781
|
| `hippo init` | Create `.hippo/`, install agent hooks and the daily run ([what it changes](#what-hippo-init-changes)) |
|
|
713
782
|
| `hippo init --global` | Create global store at `~/.hippo/` |
|
|
714
783
|
| `hippo init --no-hooks` | Create `.hippo/` without auto-installing hooks |
|
|
784
|
+
| `hippo setup` | Detect installed AI tools, install their hooks and import each agent's user-level memories |
|
|
715
785
|
| `hippo remember "<text>"` | Store a memory |
|
|
716
786
|
| `hippo remember "<text>" --tag <t>` | Store with tag (repeatable) |
|
|
717
787
|
| `hippo remember "<text>" --error` | Store as error (2x half-life) |
|
|
@@ -720,12 +790,14 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
720
790
|
| `hippo remember "<text>" --observed` | Set confidence: observed |
|
|
721
791
|
| `hippo remember "<text>" --inferred` | Set confidence: inferred |
|
|
722
792
|
| `hippo remember "<text>" --global` | Store in global `~/.hippo/` store |
|
|
793
|
+
| `hippo supersede <id> "<text>"` | Replace a memory with a new version; the old one leaves recall and points at it |
|
|
723
794
|
| `hippo recall "<query>"` | Retrieve relevant memories (local + global) |
|
|
724
795
|
| `hippo recall "<query>" --budget <n>` | Recall within token limit (default: 4000) |
|
|
725
796
|
| `hippo recall "<query>" --limit <n>` | Cap result count |
|
|
726
797
|
| `hippo recall "<query>" --why` | Show match reasons and source buckets |
|
|
727
798
|
| `hippo recall "<query>" --hops <n>` | Also surface memories N hops away in the entity/relation graph (0..3, default off) |
|
|
728
799
|
| `hippo recall "<query>" --json` | Output as JSON |
|
|
800
|
+
| `hippo recall "<query>" --as-of <date>` | Return the memories that were current on that date |
|
|
729
801
|
| `hippo context --auto` | Smart context injection (auto-detects task from git) |
|
|
730
802
|
| `hippo context "<query>" --budget <n>` | Context injection with explicit query (default: 1500) |
|
|
731
803
|
| `hippo context --limit <n>` | Cap memory count in context |
|
|
@@ -739,6 +811,8 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
739
811
|
| `hippo import --file <path>` | Import from any text file |
|
|
740
812
|
| `hippo import --dry-run` | Preview import without writing |
|
|
741
813
|
| `hippo import --global` | Write imported memories to `~/.hippo/` |
|
|
814
|
+
| `hippo import --agents [--dry-run]` | Bring in every coding agent's own memories now ([Agent memories](#agent-memories)) |
|
|
815
|
+
| `hippo export [file] [--format markdown]` | Export all memories as JSON (default) or markdown |
|
|
742
816
|
| `hippo capture --stdin` | Extract memories from piped conversation text |
|
|
743
817
|
| `hippo capture --file <path>` | Extract memories from a file |
|
|
744
818
|
| `hippo capture --dry-run` | Preview extraction without writing |
|
|
@@ -750,6 +824,7 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
750
824
|
| `hippo outcome --id <id> --good` | Target a specific memory |
|
|
751
825
|
| `hippo inspect <id>` | Full detail on one memory |
|
|
752
826
|
| `hippo forget <id>` | Force remove a memory |
|
|
827
|
+
| `hippo reject <id> --reason "<why>"` | Stop a value coming back: it refuses re-ingestion (`hippo rejections` lists them) |
|
|
753
828
|
| `hippo dormant [<query>]` | List faded memories sleep kept instead of deleting |
|
|
754
829
|
| `hippo dormant restore <id>` | Bring a dormant memory back to active memory |
|
|
755
830
|
| `hippo dormant forget <id>` | Delete a dormant memory permanently |
|
|
@@ -859,7 +934,7 @@ For Claude Code, it also adds 8 hook entries to `~/.claude/settings.json`:
|
|
|
859
934
|
- a `SessionEnd` hook that runs `hippo sleep` and then `hippo capture` when the session exits. Capture matches the last 20 user and 10 assistant messages of the transcript against word patterns for decisions, rules, errors and preferences. It uses no model unless you turn on `capture.followUpRules` (see Conversation Capture). The patterns do not read earlier turns, so a correction older than that window becomes a pinned rule only with `hippo remember --pin` by hand. Capture also saves each message you typed whole, so recall still finds the older correction (see What you type).
|
|
860
935
|
- a `SessionStart` hook that prints the previous session's consolidation log on stderr, where `claude --debug` shows it. When the last sleep failed or a compaction summary was set aside, it shows you one line that points to `hippo doctor`.
|
|
861
936
|
- a second `SessionStart` hook that runs `hippo session-rules`, which prints the rules you taught under `## Rules from the user` and nothing else. It has no matcher, so a resumed or cleared session and a compaction get the rules again.
|
|
862
|
-
- a `UserPromptSubmit` hook that runs `hippo context --pinned-only --include-recent 5 --format additional-context` every turn. It re-injects pinned memories (`hippo remember <text> --pin`) plus up to 5 memories that share words with your prompt. When nothing matches, it adds only the pinned ones. Since 1.55.0 this replaces the 5 newest memories
|
|
937
|
+
- a `UserPromptSubmit` hook that runs `hippo context --pinned-only --include-recent 5 --format additional-context` every turn. It re-injects pinned memories (`hippo remember <text> --pin`) plus up to 5 memories that share words with your prompt. When nothing matches, it adds only the pinned ones. Since 1.55.0 this replaces the 5 newest memories. `{"pinnedInject":{"promptRecall":false}}` brings back the 5 newest, so fresh same-session lessons appear on the next prompt whatever you ask. The block is rendered without live strength percentages, so it stays byte-identical while its memories do not change, and it is sent only when it changed since the session's last prompt: an unchanged block is skipped, resent every 10 skips (`pinnedInject.refreshTurns`, `0` never resends) and resent after compaction. The prompt-matched memories go in a separate block that is never skipped, so they are sent on every prompt they match. `{"pinnedInject":{"skipUnchanged":false}}` sends the pinned block every turn as before. `{"pinnedInject":{"excludeTags":["session-digest"]}}` leaves out every memory that carries a listed tag, whether it is pinned, recent or matched to the prompt; the list is empty by default. Opt out entirely with `{"pinnedInject":{"enabled":false}}` in `.hippo/config.json`.
|
|
863
938
|
- a `PreCompact` hook that runs `hippo pre-compact` before the transcript gets summarized. It records the compaction in the store, saves a working-state snapshot (task/summary/next step) so mid-session compaction can't drop it, and asks the summariser to end its summary with a "Memories for hippo" list: the lessons, decisions and corrections from the session that should outlive it. The `SessionEnd` hook still owns extracting durable memories from the transcript.
|
|
864
939
|
- a second `SessionStart` hook (matcher `compact`) that runs `hippo compact-resume`, printing that snapshot back into context right after compaction, if it is under 15 minutes old.
|
|
865
940
|
- a `PostCompact` hook that runs `hippo post-compact`. It keeps the summary in the store with secrets scrubbed, and saves each item of that list as a memory that sleep never deletes: at most 10 per compaction, skipping an item an earlier compaction already saved and any item that looks like a secret. An item over 500 characters stays in the compaction's record only. It then prints one line, such as "Hippo saved 3 memories from this compaction and restored your task snapshot." If the store is busy, the summary waits in the store's `compactions-spool/` folder and `hippo sleep` finishes the save; `hippo doctor` names any compaction left unfinished for over 10 minutes. It also counts spool files that are waiting, stale (left by a replay that stopped) or set aside as `.bad`. The session that compacted does not have those memories injected back into its own prompts, since it just read them in the summary; `hippo recall` still finds them. The hook prints nothing when there is no store to save to.
|
|
@@ -930,7 +1005,7 @@ The headers are a filter the caller picks, not access control: any member key ca
|
|
|
930
1005
|
|
|
931
1006
|
Native plugin with auto-context injection, workspace-aware memory lookup, and
|
|
932
1007
|
tool hooks for auto-learn / auto-sleep. When `autoSleep` is enabled, the
|
|
933
|
-
OpenClaw plugin
|
|
1008
|
+
OpenClaw plugin launches `hippo sleep` in a detached background worker at
|
|
934
1009
|
session end so the live session can exit immediately.
|
|
935
1010
|
|
|
936
1011
|
Query-time retrieval still uses the active workspace store plus the shared
|
|
@@ -952,64 +1027,33 @@ Full integration details: [integrations/](integrations/)
|
|
|
952
1027
|
|
|
953
1028
|
---
|
|
954
1029
|
|
|
955
|
-
## The Neuroscience
|
|
956
|
-
|
|
957
|
-
Hippo's design borrows seven properties of the human hippocampus. This section is design inspiration, not measured benefit. For what we measured, see the Receipts and [PR #232](https://github.com/kitfunso/hippo-memory/pull/232).
|
|
958
|
-
|
|
959
|
-
**Why two stores?** The brain uses a fast hippocampal buffer + a slow neocortical store (Complementary Learning Systems theory, McClelland et al. 1995). If the neocortex learned fast, new information would overwrite old knowledge. The buffer absorbs new episodes; the neocortex extracts patterns over time.
|
|
960
|
-
|
|
961
|
-
**Why decay at all?** In the brain, new neurons born in the dentate gyrus disrupt old memory traces (Frankland et al. 2013), which may reduce interference from outdated information. That is why hippo has decay. In hippo's own tests, age-based decay made no measurable difference to recall: at the 365-day default it tied with decay switched off. A bad outcome mark is the forgetting that measured helpful. Supersession, a newer fact replacing an old one, has not been measured.
|
|
962
|
-
|
|
963
|
-
**Why do errors stick?** The amygdala modulates hippocampal consolidation based on emotional significance. Fear and error signals boost encoding. Your first production incident is burned into memory. Your 200th uneventful deploy isn't.
|
|
964
|
-
|
|
965
|
-
**Why does retrieval strengthen?** Recalled memories undergo "reconsolidation" (Nader et al. 2000). The act of retrieval destabilizes the trace, then re-encodes it stronger. This is the testing effect. Hippo borrows the idea: each recall adds 2 days to a memory's half-life.
|
|
966
|
-
|
|
967
|
-
**Why does sleep consolidate?** During sleep, the hippocampus replays compressed versions of recent episodes and "teaches" the neocortex by repeatedly activating the same patterns. Hippo's `sleep` command borrows the idea for a consolidation pass. In hippo's own audit it lowered recall ([Sleep consolidation](#sleep-consolidation) has the numbers).
|
|
968
|
-
|
|
969
|
-
The 7 mechanisms in full: [PLAN.md#core-principles](PLAN.md#core-principles)
|
|
970
|
-
|
|
971
|
-
For how these mechanisms connect to LLM training, continual learning, and open research problems: **[RESEARCH.md](RESEARCH.md)**
|
|
972
|
-
|
|
973
|
-
**Why does reward modulate decay?** In spiking neural networks, reward-modulated STDP strengthens synapses that contribute to positive outcomes and weakens those that don't. Hippo's reward-proportional decay (v0.11.0) borrows this idea: memories with consistent positive outcomes decay slower, negatives decay faster, with no fixed deltas. Inspired by [MH-FLOCKE](https://github.com/MarcHesse/mhflocke)'s R-STDP architecture for quadruped locomotion, where the same mechanism produces stable learning with 11.6x lower variance than PPO.
|
|
974
|
-
|
|
975
|
-
**Prior art in agent memory simulation.** The idea that human-like memory produces human-like behavior as an emergent property was explored in IEEE research from 2010-2011 ([5952114](https://ieeexplore.ieee.org/document/5952114), [5548405](https://ieeexplore.ieee.org/document/5548405), [5953964](https://ieeexplore.ieee.org/document/5953964)). Walking between rooms and forgetting why you went there doesn't need direct simulation; it emerges naturally from a memory system with capacity limits and decay. Hippo takes the mechanisms as design ideas. Whether better agent behavior follows from them has not been shown.
|
|
976
|
-
|
|
977
|
-
**Related work:** [HippoRAG](https://arxiv.org/abs/2405.14831) (Gutierrez et al., 2024) applies hippocampal indexing to RAG via knowledge graphs. [MemPalace](https://github.com/milla-jovovich/mempalace) (Sigman & Jovovich, 2026) organizes memory spatially (wings/halls/rooms) with AAAK compression, achieving 100% on [LongMemEval](https://arxiv.org/abs/2410.10813). [MH-FLOCKE](https://github.com/MarcHesse/mhflocke) (Hesse, 2026) uses spiking neurons with R-STDP for embodied cognition. Each system tackles a different facet: HippoRAG optimizes retrieval quality, MemPalace optimizes retrieval organization, MH-FLOCKE optimizes embodied learning, and Hippo works on the memory lifecycle.
|
|
978
|
-
|
|
979
|
-
---
|
|
980
|
-
|
|
981
1030
|
## Comparison
|
|
982
1031
|
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
| Feature | Hippo | [MemPalace](https://github.com/milla-jovovich/mempalace) | [Mem0](https://github.com/mem0ai/mem0) | [Basic Memory](https://github.com/basicmachines-co/basic-memory) | [
|
|
986
|
-
|
|
987
|
-
| Decay by default | Yes | No | No | No | No | No | No | No | No |
|
|
988
|
-
| Retrieval strengthening | Yes | No | No | No | No | No |
|
|
989
|
-
| Reward-proportional decay | Yes | No | No | No | No | No | No | No | No |
|
|
990
|
-
| Hybrid search (BM25 + embeddings) | Yes | Embeddings + spatial | Yes (semantic + BM25 + entity) | No | Yes (
|
|
991
|
-
| Schema acceleration / knowledge graph | Yes (schema) | No | Partial (entity linking; graph memory on Pro) | No | Yes (
|
|
992
|
-
| Conflict detection + resolution | Yes | No | Partial (hosted platform marks superseded facts) | No | Yes (
|
|
993
|
-
| Multi-agent shared memory | Yes | No | No | No | Yes
|
|
994
|
-
| Transfer scoring | Yes | No | No | No | No | No | No | No | No |
|
|
995
|
-
| Outcome tracking | Yes | No | No | No | No | No | No | No |
|
|
996
|
-
| Confidence tiers | Yes | No | No | No | No
|
|
997
|
-
| Spatial organization | No | Yes (wings/halls/rooms) | No | No | No | No | No | No | No |
|
|
998
|
-
| Lossless compression | No | Yes (AAAK, 30x) | No | No | No | No | No | No | No |
|
|
999
|
-
| Cross-tool import (ChatGPT/Claude/Cursor) | Yes | No | No | No |
|
|
1000
|
-
| Auto-hook install | Yes | No | No | No | No | No | No | No | No |
|
|
1001
|
-
| MCP server | Yes | Yes | Yes (hosted, needs an account) | Yes | Yes (
|
|
1002
|
-
| Zero runtime deps | Yes | No (ChromaDB) | No | No | No (
|
|
1003
|
-
|
|
|
1004
|
-
|
|
|
1005
|
-
|
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
\* Hippo's figures are on `longmemeval_s_cleaned` with a per-question haystack, each the best of five retrieval settings in the benchmark scripts, not `hippo recall`. Any-evidence R@5 counts a hit when any answer session is in the top 5, over all 500 questions: 98.0% with the free local MiniLM embedder (an optional install) and 99.8% with voyage-3-large (measured 2026-06-09, not re-run). All-evidence R@5 counts a hit only when every answer session is in the top 5, over the 470 questions that have an answer: 86.8 to 88.5% with MiniLM. gbrain first published 97.6%, an any-evidence score over all 500; its [report](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-05-07-longmemeval-s.md) now leads with all-evidence, 95.53% (449 of 470) with the paid Voyage rerank-2.5 reranker and 93.19% without it. On all-evidence recall gbrain is ahead. The June 2026 build scored 98.6 any-evidence; [`docs/evals/2026-09-23-longmemeval-reproduction.md`](docs/evals/2026-09-23-longmemeval-reproduction.md) has both runs. An older hippo number, 86.8% R@5 on `longmemeval_oracle` under pooled (non-per-haystack) retrieval, is not comparable to per-haystack figures.
|
|
1009
|
-
|
|
1010
|
-
\*\* Different metric: these are end-to-end answer scores, not retrieval R@5. Mem0's 94.4 comes from its hosted platform, which its README says includes optimizations the open-source SDK lacks. Zep's 90.2% and 94.7% are accuracy figures from its homepage. Memoria's 88.78% and EverMind's 83% are overall accuracy with a reader LLM. Higher denominator + LLM helps. Not directly comparable to retrieval-only R@5 numbers above. The Mem0, Zep and Letta columns were last checked against each vendor's own pages on 2026-09-28.
|
|
1011
|
-
|
|
1012
|
-
Different tools answer different questions. Mem0 and Basic Memory implement "save everything, search later." MemPalace implements "store everything, organize spatially for retrieval." gbrain, Zep, and Cognee implement "extract typed entities and relationships into a knowledge graph." Letta implements "the agent edits its own memory blocks." Memoria implements "Git-style version control over the memory state itself." EverMind implements "self-evolving Skill Memory + multi-modal retrieval over hierarchical scopes." Hippo implements "learn what is wrong and stop repeating it." These are complementary takes, not a single-axis ranking: bio-lifecycle (Hippo) + GraphRAG (gbrain/Cognee/Zep) + agent-self-edit (Letta) + memory-VCS (Memoria) + skill-distillation (EverMind) cover different parts of the same problem.
|
|
1032
|
+
Hippo is for coding agents on machines you control. [Mem0](https://hippo-memory.com/vs/mem0/) fits better when the memory belongs to your app's end users, [Zep](https://hippo-memory.com/vs/zep/) when you want a managed knowledge graph built from business data, and [Letta Code](https://hippo-memory.com/vs/letta/) when you want an agent that edits its own memory. The table is a feature snapshot, not a verdict, and it holds no scores: [Benchmarks](#benchmarks) has those, split by setup.
|
|
1033
|
+
|
|
1034
|
+
| Feature\* | Hippo | [MemPalace](https://github.com/milla-jovovich/mempalace) | [Mem0](https://github.com/mem0ai/mem0) | [Basic Memory](https://github.com/basicmachines-co/basic-memory) | [Zep](https://www.getzep.com/) | [Letta](https://github.com/letta-ai/letta-code) | [Cognee](https://www.cognee.ai/) | [Memoria](https://github.com/matrixorigin/Memoria) | [EverMind](https://evermind.ai/) |
|
|
1035
|
+
|---------|-------|-----------|------|-------------|-----|-------|--------|---------|----------|
|
|
1036
|
+
| Decay by default | Yes | No | No | No | No | No | No | No | No |
|
|
1037
|
+
| Retrieval strengthening | Yes | No | No | No | No | No | Partial (recall tuning) | No | Partial (Skill Memory distills patterns) |
|
|
1038
|
+
| Reward-proportional decay | Yes | No | No | No | No | No | No | No | No |
|
|
1039
|
+
| Hybrid search (BM25 + embeddings) | Yes | Embeddings + spatial | Yes (semantic + BM25 + entity) | No | Yes (graph + vec) | ? | Yes (GraphRAG) | Yes (vector + full-text) | Yes (mRAG, multi-modal) |
|
|
1040
|
+
| Schema acceleration / knowledge graph | Yes (schema) | No | Partial (entity linking; graph memory on Pro) | No | Yes (temporal KG) | No | Yes (auto-ontologies) | No (typed claims) | Yes (hierarchical: user/group/agent) |
|
|
1041
|
+
| Conflict detection + resolution | Yes | No | Partial (hosted platform marks superseded facts) | No | Yes (auto-invalidate stale facts) | No | No | Yes (auto-detect + quarantine) | Partial (temporal tracking) |
|
|
1042
|
+
| Multi-agent shared memory | Yes | No | No | No | Yes | Yes (shared memory blocks) | Yes | Yes (branch/merge across sessions) | Yes (multi-agent coordination) |
|
|
1043
|
+
| Transfer scoring | Yes | No | No | No | No | No | No | No | No |
|
|
1044
|
+
| Outcome tracking | Yes | No | No | No | No | No | No | No | Partial (Cases: agent trajectories) |
|
|
1045
|
+
| Confidence tiers | Yes | No | No | No | No | No | No | No | No |
|
|
1046
|
+
| Spatial organization | No | Yes (wings/halls/rooms) | No | No | No | No | No | No | No |
|
|
1047
|
+
| Lossless compression | No | Yes (AAAK, 30x) | No | No | No | No | No | No | No |
|
|
1048
|
+
| Cross-tool import (ChatGPT/Claude/Cursor) | Yes | No | No | No | ? | No | Partial (28 data sources) | No (Git ops) | Partial (mRAG: PDFs/images/URLs) |
|
|
1049
|
+
| Auto-hook install | Yes | No | No | No | No | No | No | No | No |
|
|
1050
|
+
| MCP server | Yes | Yes | Yes (hosted, needs an account) | Yes | Yes (hosted, needs an account) | Yes (hosted, needs an API key) | Yes (first-party Claude/LangGraph) | Yes | ? |
|
|
1051
|
+
| Zero runtime deps | Yes | No (ChromaDB) | No | No | No (managed service) | No (npm deps) | No (Python deps) | Yes (single Rust binary) | No (managed + OSS) |
|
|
1052
|
+
| Git-friendly | Yes | No | No | Yes | No | Yes (memory tracked in git) | No | Yes (Git is the model) | ? |
|
|
1053
|
+
| Framework agnostic | Yes | Yes | Partial | Yes | Yes | Yes | Yes | Yes | Yes |
|
|
1054
|
+
| License | MIT | (open) | Apache-2.0 | (open) | Proprietary cloud (Graphiti: Apache-2.0) | Apache-2.0 | MIT (core) | Apache-2.0 | Apache-2.0 (OSS) + cloud |
|
|
1055
|
+
|
|
1056
|
+
\* The Mem0, Zep and Letta columns were last checked against each vendor's own pages on 2026-09-28.
|
|
1013
1057
|
|
|
1014
1058
|
Side-by-side pages on the website: [hippo vs mem0](https://hippo-memory.com/vs/mem0/) · [hippo vs Zep](https://hippo-memory.com/vs/zep/) · [hippo vs Letta](https://hippo-memory.com/vs/letta/) · [CLAUDE.md vs MEMORY.md](https://hippo-memory.com/vs/claude-memory/).
|
|
1015
1059
|
|
|
@@ -1017,101 +1061,119 @@ Side-by-side pages on the website: [hippo vs mem0](https://hippo-memory.com/vs/m
|
|
|
1017
1061
|
|
|
1018
1062
|
## Benchmarks
|
|
1019
1063
|
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
### LongMemEval (retrieval accuracy)
|
|
1023
|
-
|
|
1024
|
-
[LongMemEval](https://arxiv.org/abs/2410.10813) (ICLR 2025) is the industry-standard benchmark: 500 questions across 5 memory abilities, embedded in 115k+ token chat histories.
|
|
1025
|
-
|
|
1026
|
-
**`hippo recall` (measured 2026-09-28 on hippo-memory 1.52.5, re-run 2026-10-06 on 1.64.0).** On a default install, `hippo recall` puts an answer session in its top 5 for 85.6% of the 500 `_s` questions (95% CI 82.4 to 88.6), and 87.8% with the optional MiniLM embedder (84.8 to 90.6; 87.6% on 1.52.5). Most of the gap to the scripts below is the default 4,000-token budget. Each memory in this test is a whole session, about 2,600 tokens at the median, so recall returns a median of 2 sessions. With the budget lifted, the same rankings score 96.8% (95.2 to 98.2) and 97.4% (96.0 to 98.6). A lifted call returns every candidate, a median of 47 sessions and 123,491 tokens per question, so those two figures measure the ranking, not an amount of text an agent could take in. Result: [`docs/evals/2026-09-28-recall-cli-longmemeval-result.md`](docs/evals/2026-09-28-recall-cli-longmemeval-result.md); re-run: [`docs/evals/2026-10-06-recall-cli-longmemeval-rerun-1.64.0.md`](docs/evals/2026-10-06-recall-cli-longmemeval-rerun-1.64.0.md).
|
|
1064
|
+
We built hippo and ran these tests, so read them as a vendor's numbers. We ran rivals on the same answer model where we could. Scores marked published come from each team's own setup, so we give no ranks. The same results are on [hippo-memory.com/benchmarks](https://hippo-memory.com/benchmarks/).
|
|
1027
1065
|
|
|
1028
|
-
|
|
1066
|
+
### Repeat mistakes in real Claude Code sessions
|
|
1029
1067
|
|
|
1030
|
-
|
|
1031
|
-
|----------|----------------|-----------------|-----|
|
|
1032
|
-
| MiniLM-L6 (local, optional install) | 96.8 | 98.0 | 88.4 |
|
|
1033
|
-
| voyage-3-large (opt-in, paid) | 99.8 | 99.8 | 94.6 |
|
|
1068
|
+
Each run corrects Claude Code once and then, 2 to 4 tasks later, gives it a task where the same mistake is possible. We ran it on 29 repositories, [registered first](docs/evals/2026-11-08-z0-round3-result.md).
|
|
1034
1069
|
|
|
1035
|
-
|
|
1070
|
+
| Claude Code with | Repeats in 100 |
|
|
1071
|
+
|---|--:|
|
|
1072
|
+
| No memory | 83 |
|
|
1073
|
+
| Claude Code's built-in memory | 21 |
|
|
1074
|
+
| Built-in memory plus hippo | **11** |
|
|
1036
1075
|
|
|
1037
|
-
The
|
|
1076
|
+
The chart counts first chances 2 to 4 tasks after the correction. The registered measure, the first chance in each mistake family over 148 units, gives 14.9 points fewer repeats with hippo [5.5, 24.3], at the same cost, a ratio of 1.006 [0.964, 1.049]. On tasks with no lesson to apply, neither the cost nor the share solved changed.
|
|
1038
1077
|
|
|
1039
|
-
|
|
1040
|
-
- **Sleep lowered LongMemEval recall.** The slept store loses 3.6 points of hit@5 [-5.8, -1.4] to the never-slept one under the audit's declared scorer, and no scorer there shows sleep helping recall ([mechanism audit, round 2](docs/evals/2026-09-23-mechanism-audit-round2-result.md)).
|
|
1041
|
-
- **Outcome marks helped on the synthetic test.** Plain BM25 plus the fast outcome nudge drops trap persistence from 71.9% to 0.0% (round 2). Round 1 found outcome feedback and retrieval strengthening each help there, in the test's best case: every outcome mark is right, and every scheduled recall repeats the probe's query ([round 1](docs/evals/2026-09-23-mechanism-audit-result.md)). Supersession is not measured yet.
|
|
1078
|
+
What this run can't tell us is why. With lesson capture removed the result was the same, so we can't say hippo's lessons caused it. Round 4 tests that. The corrections were scripted, the agent was Claude Code only, and the test code and run records are not in the public repo. The round before was inconclusive, at 8.9 points fewer on 16 repositories.
|
|
1042
1079
|
|
|
1043
|
-
|
|
1080
|
+
### Official-setup rows side by side
|
|
1044
1081
|
|
|
1045
|
-
|
|
1082
|
+
Hippo 3.5.1 run with the same answer model, judge and settings as the published rows.
|
|
1046
1083
|
|
|
1047
|
-
|
|
|
1048
|
-
|
|
1049
|
-
|
|
|
1050
|
-
|
|
|
1051
|
-
|
|
|
1052
|
-
|
|
|
1053
|
-
|
|
|
1084
|
+
| Benchmark | hippo | Mem0 | Zep | MemOS | Supermemory | Honcho | Memobase | Other |
|
|
1085
|
+
|---|---|---|---|---|---|---|---|---|
|
|
1086
|
+
| **LoCoMo, Mem0 paper setup**<br><sub>Long chat recall, gpt-4o-mini answers; official setup</sub> | **69.22**<br><sub>embedder, 30 per speaker, all 1,540</sub> | [66.88](https://arxiv.org/html/2504.19413)<br><sub>gpt-4o-mini</sub> | [65.99](https://arxiv.org/html/2504.19413)<br><sub>gpt-4o-mini</sub> | - | - | - | - | [68.44](https://arxiv.org/html/2504.19413)<br><sub>Mem0g · gpt-4o-mini</sub> |
|
|
1087
|
+
| **LongMemEval-S, gpt-4o reader**<br><sub>Long chat recall, gpt-4o answers; official setup</sub> | **81.8**<br><sub>reranker, top 10 (embedder 78.2), official gpt-4o judge</sub> | - | [71.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o</sub> | - | [81.6](https://arxiv.org/html/2512.12818v1)<br><sub>gpt-4o</sub> | - | - | [86.0](https://www.emergence.ai/blog/sota-on-longmemeval-with-rag)<br><sub>EmergenceMem Internal · gpt-4o-2024-08-06</sub> |
|
|
1088
|
+
| **PersonaMem v1, MemOS suite setup**<br><sub>Personalisation, gpt-4o-mini answers; official setup</sub> | **61.97**<br><sub>embedder, top 20, mean of 3 runs</sub> | [43.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [57.8](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [61.2](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [53.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | [58.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.8](https://arxiv.org/html/2507.03724v4)<br><sub>MemU · gpt-4o-mini</sub> |
|
|
1089
|
+
| **EverMemBench, GPT-4.1-mini answers**<br><sub>Changing facts, GPT-4.1-mini answers; official setup, with a caveat</sub> | **44.20**<br><sub>embedder, top 10, mean of 9 categories</sub> | [37.09](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [39.97](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [42.55](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | - | - | [34.27](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [37.44](https://arxiv.org/html/2602.01313v3)<br><sub>Full Context · GPT-4.1-mini</sub> |
|
|
1090
|
+
| **HaluMem-Medium**<br><sub>Hallucination control, GPT-4o answers; official setup, with a caveat</sub> | **70.23**<br><sub>embedder, top 20, GPT-4o answers and judge</sub> | [53.02](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [55.47](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [67.23](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [54.07](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | - | [35.33](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [54.66](https://arxiv.org/html/2511.03506v3)<br><sub>Mem0-Graph · GPT-4o</sub> |
|
|
1091
|
+
| **HaluMem-Long**<br><sub>Hallucination control, GPT-4o answers; official setup, with a caveat</sub> | *paused* | [28.11](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [50.19](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [64.44](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [53.77](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | - | [33.60](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [32.44](https://arxiv.org/html/2511.03506v3)<br><sub>Mem0-Graph · GPT-4o</sub> |
|
|
1092
|
+
| **DolphinBench, official Claude Code setup**<br><sub>Agent memory, Claude Code with Sonnet 5; official setup</sub> | *in build* | [32.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/mem0/README.md)<br><sub>Claude Code, Sonnet 5 · 194/600</sub> | - | - | - | [35.8](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/honcho/README.md)<br><sub>Claude Code, Sonnet 5 · 215/600</sub> | - | [26.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/builtin/README.md)<br><sub>Built-in · Claude Code, Sonnet 5 · 158/600</sub> |
|
|
1054
1093
|
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
| single-session-assistant | 56 | 100.0% | 100.0% |
|
|
1058
|
-
| knowledge-update | 78 | 89.7% | 96.2% |
|
|
1059
|
-
| multi-session | 133 | 72.2% | 82.0% |
|
|
1060
|
-
| temporal-reasoning | 133 | 72.9% | 78.9% |
|
|
1061
|
-
| single-session-user | 70 | 62.9% | 71.4% |
|
|
1062
|
-
| single-session-preference | 30 | 20.0% | 33.3% |
|
|
1063
|
-
|
|
1064
|
-
For context: MemPalace scores 96.6% (raw) using ChromaDB embeddings + spatial indexing. Hippo v0.28 achieves 73.8% R@5 with hybrid BM25 + cosine. Hybrid scoring trades a little R@1 accuracy for better top-5 content relevance (answer_in_content@5 +3pp vs v0.11).
|
|
1065
|
-
|
|
1066
|
-
Hippo's strongest categories (single-session-assistant 100% R@5, knowledge-update 89.7%) are where keyword overlap between question and stored content is highest. The weakest (preference 20%) involves indirect references that need deeper semantic understanding.
|
|
1094
|
+
<details>
|
|
1095
|
+
<summary>How each row ran</summary>
|
|
1067
1096
|
|
|
1068
|
-
|
|
1097
|
+
- **LoCoMo, Mem0 paper setup.** Run as the Mem0 paper ran it, with Mem0's released evaluation code (mem0ai/mem0 aae5989e78): all 1,540 questions in categories 1 to 4, 30 memories per speaker, gpt-4o-mini answers under the paper's prompt, and the code's gpt-4o-mini judge. The paper itself does not name its judge. One run. The paper's full-context baseline scores 72.9, above every memory row. Compare the overall score only: our category labels follow our own mapping.
|
|
1098
|
+
- **LongMemEval-S, gpt-4o reader.** The answer model and the judge match the published gpt-4o rows: gpt-4o-2024-08-06 reads hippo's top 10 memories, and the official script's prompts on gpt-4o-2024-08-06 judge all 500 questions. Each system keeps its own retrieval, as in the published rows. The cell shows the reranker setup, 81.8; the embedder setup scores 78.2. We chose the reranker after seeing its Sonnet scores and recorded that before its run. EmergenceMem does not name its judge.
|
|
1099
|
+
- **PersonaMem v1, MemOS suite setup.** Run as MemOS's evaluation suite runs every vendor row (MemTensor/MemOS 3a16e5fe): the same data files and top 20, PM_ANSWER_PROMPT as one system message to gpt-4o-mini-2024-07-18 at temperature 0, 3 runs per question, scored as the mean of the run accuracies over all 589 rows. Each system keeps its own retrieval and memory text.
|
|
1100
|
+
- **EverMemBench, GPT-4.1-mini answers.** The answer model, prompts, top 10, multiple-choice scorer and the 9-category mean match the paper's Table 4 (EverMind-AI/EverMemBench e10b3d5): gpt-4.1-mini-2025-04-14 at temperature 0 answers all 2,400 items. One setting differs: open answers are judged by gpt-5.6-sol through codex with the authors' prompts, and the repository's default judge is gemini-3-flash-preview; the paper names no judge. A second judge, gpt-4o-2024-08-06 with the same prompts, agrees with codex on 96.9% of open verdicts and scores hippo 44.75. The default setup scores 41.66 with codex and 42.50 with gpt-4o.
|
|
1101
|
+
- **HaluMem-Medium.** Question answering only, with the paper's GPT-4o answers and judge (gpt-4o-2024-08-06, temperature 0) and the same data, question set, top 20 and prompts. We do not produce the extraction and update scores, so we cannot enter the HaluMem board yet.
|
|
1102
|
+
- **HaluMem-Long.** Question answering only, with the paper's GPT-4o answers and judge (gpt-4o-2024-08-06, temperature 0), the same setup as the HaluMem-Medium row. The run paused when the OpenAI credit ran out on 16 Nov, so the row has no score yet. We do not produce the extraction and update scores, so we cannot enter the HaluMem board yet.
|
|
1103
|
+
- **DolphinBench, official Claude Code setup.** The same setup as the published Claude Code rows: one Claude Code turn with the hippo plugin for each of the 13,539 messages, each test in a fresh sandbox with memory writes off, Sonnet 5 and the gpt-5.6-sol judge. The run is in build, so there is no score yet.
|
|
1069
1104
|
|
|
1070
|
-
|
|
1071
|
-
cd benchmarks/longmemeval
|
|
1072
|
-
python ingest_direct.py --data data/longmemeval_oracle.json --store-dir ./store
|
|
1073
|
-
python retrieve_fast.py --data data/longmemeval_oracle.json --store-dir ./store --output results/retrieval.jsonl
|
|
1074
|
-
python evaluate_retrieval.py --retrieval results/retrieval.jsonl --data data/longmemeval_oracle.json
|
|
1075
|
-
```
|
|
1105
|
+
</details>
|
|
1076
1106
|
|
|
1077
|
-
###
|
|
1107
|
+
### Our own setup
|
|
1108
|
+
|
|
1109
|
+
Every other benchmark we ran. Our setup differs from the published rows, so these scores do not sit beside theirs.
|
|
1110
|
+
|
|
1111
|
+
| Benchmark | hippo | Mem0 | Zep | MemOS | Supermemory | Honcho | Hindsight | EverMemOS | Memobase | Other |
|
|
1112
|
+
|---|---|---|---|---|---|---|---|---|---|---|
|
|
1113
|
+
| **LoCoMo**<br><sub>Long chat recall, Sonnet 5 answers</sub> | **91.5**<br><sub>embedder, top 200, gpt-4o-mini judge, 400 sample</sub> | [92.5](https://mem0.ai/research-3)<br><sub>not stated · unit not stated</sub> | [75.14](https://www.getzep.com/blog/lies-damn-lies-statistics-is-mem0-really-sota-in-agent-memory/)<br><sub>not stated</sub> | - | - | [89.9](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>Claude Haiku 4.5</sub> | [89.61](https://arxiv.org/html/2512.12818v1)<br><sub>Gemini-3 Pro</sub> | [93.05](https://evermind.ai/blogs/everos-sota-results-across-four-memory-benchmarks-and-what-it-means-for-llm-agents)<br><sub>not stated</sub> | [75.78](https://arxiv.org/html/2512.12818v1)<br><sub>not stated</sub> | [90.0](https://backboard.io/blog/best-ai-memory-score-in-the-world)<br><sub>Backboard · not stated</sub> |
|
|
1114
|
+
| **LongMemEval-S**<br><sub>Long chat recall, Sonnet 5 answers</sub> | **88.4**<br><sub>embedder, top 200, official gpt-4o judge</sub> | [94.4](https://mem0.ai/blog/locomo-benchmark)<br><sub>not stated · unit not stated</sub> | [71.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o</sub> | - | [85.2](https://arxiv.org/html/2512.12818v1)<br><sub>gemini-3-pro</sub> | [92.6](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>Gemini 3 Pro</sub> | [91.4](https://arxiv.org/html/2512.12818v1)<br><sub>Gemini-3 Pro</sub> | [83.00](https://evermind.ai/blogs/everos-sota-results-across-four-memory-benchmarks-and-what-it-means-for-llm-agents)<br><sub>not stated</sub> | - | [94.87](https://mastra.ai/research/observational-memory)<br><sub>Mastra Observational Memory · gpt-5-mini</sub> |
|
|
1115
|
+
| **LongMemEval-M**<br><sub>Very long chat recall, Sonnet 5 answers</sub> | **78.4**<br><sub>default, top 200, official gpt-4o judge</sub> | - | - | - | - | - | - | - | - | - |
|
|
1116
|
+
| **DMR**<br><sub>Multi-session recall, Sonnet 5 answers</sub> | **77.6**<br><sub>reranker, top 20</sub> | - | [98.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o-mini</sub> | - | - | - | - | - | - | [98.0](https://arxiv.org/html/2501.13956v1)<br><sub>Full-conversation baseline · gpt-4o-mini</sub> |
|
|
1117
|
+
| **DMR, gpt-4o-mini answers**<br><sub>Multi-session recall, gpt-4o-mini answers</sub> | **85.60**<br><sub>reranker, top 20, Speaker 1 role</sub> | - | [98.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o-mini</sub> | - | - | - | - | - | - | [98.0](https://arxiv.org/html/2501.13956v1)<br><sub>Full-conversation baseline · gpt-4o-mini</sub> |
|
|
1118
|
+
| **PersonaMem v1**<br><sub>Personalisation, Sonnet 5 answers</sub> | **72.4**<br><sub>embedder, top 20</sub> | [43.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [57.8](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [61.2](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [53.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | - | - | [58.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.8](https://arxiv.org/html/2507.03724v4)<br><sub>MemU · gpt-4o-mini</sub> |
|
|
1119
|
+
| **PersonaMem v2**<br><sub>Personalisation, Sonnet 5 answers</sub> | **50.1**<br><sub>embedder, top 20</sub> | - | - | [40.58](https://raw.githubusercontent.com/MemTensor/MemOS/main/README.md)<br><sub>not stated</sub> | - | - | - | [53.25](https://evermind.ai/blogs/top-ai-memory-systems-benchmarked-in-2026)<br><sub>not stated</sub> | - | [55.2](https://arxiv.org/html/2512.06688)<br><sub>Qwen3-4B agentic memory · Qwen3-4B-Instruct-2507</sub> |
|
|
1120
|
+
| **PrefEval**<br><sub>Preference following, Sonnet 5 answers</sub> | **83.0**<br><sub>default, top 5</sub> | [63.7](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [39.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [71.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.7](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | - | - | [34.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [92.15](https://arxiv.org/html/2502.09597)<br><sub>Claude 3 Sonnet + RAG · Claude 3 Sonnet</sub> |
|
|
1121
|
+
| **EverMemBench**<br><sub>Changing facts, Sonnet 5 answers</sub> | **56.4**<br><sub>default, top 10, mean of 9 categories</sub> | [37.09](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [39.97](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [42.55](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | - | - | - | - | [34.27](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [72.61](https://arxiv.org/html/2602.01313v3)<br><sub>Full Context · Gemini-3-Flash</sub> |
|
|
1122
|
+
| **BEAM-100K**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **72.4**<br><sub>embedder</sub> | - | - | - | - | [63.0](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [73.4](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [79.0](https://www.cognee.ai/benchmarking-cognee-on-beam)<br><sub>cognee · not stated</sub> |
|
|
1123
|
+
| **BEAM-500K**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **70.6**<br><sub>embedder</sub> | - | - | - | - | [64.9](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [71.1](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [35.9](https://arxiv.org/html/2510.27246)<br><sub>LIGHT · Llama-4-Maverick</sub> |
|
|
1124
|
+
| **BEAM-1M**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **69.9**<br><sub>default</sub> | [64.1](https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm)<br><sub>gpt-5</sub> | - | - | - | [63.1](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [73.9](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [33.6](https://arxiv.org/html/2510.27246)<br><sub>LIGHT · Llama-4-Maverick</sub> |
|
|
1125
|
+
| **BEAM-10M**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **54.3**<br><sub>embedder</sub> | [48.6](https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm)<br><sub>gpt-5</sub> | - | - | - | [40.6](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [64.1](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [up to 67.0](https://www.cognee.ai/benchmarking-cognee-on-beam)<br><sub>cognee · not stated</sub> |
|
|
1126
|
+
| **ConvoMem**<br><sub>Conversation memory, Sonnet 5 answers</sub> | **59.6**<br><sub>default, top 20</sub> | - | - | - | - | - | - | - | - | [92.9](https://vectorize.io/articles/mempalace-benchmarks)<br><sub>MemPalace · not stated</sub> |
|
|
1127
|
+
| **HotPotQA-24**<br><sub>Multi-hop questions, F1, Sonnet 5 answers</sub> | **91.0**<br><sub>reranker, top 15, all 24 questions, one run</sub> | [12.0](https://www.cognee.ai/knowledge-graph-memory-benchmarks)<br><sub>not stated</sub> | - | - | - | - | - | - | - | [84.1](https://www.cognee.ai/knowledge-graph-memory-benchmarks)<br><sub>Cognee · not stated</sub> |
|
|
1128
|
+
| **DolphinBench, shortcut run**<br><sub>Agent memory, Claude Code with Sonnet 5</sub> | **45.0**<br><sub>270/600 · Claude Code, Sonnet 5</sub> | [32.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/mem0/README.md)<br><sub>Claude Code, Sonnet 5 · 194/600</sub> | - | - | - | [35.8](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/honcho/README.md)<br><sub>Claude Code, Sonnet 5 · 215/600</sub> | - | - | - | [26.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/builtin/README.md)<br><sub>Built-in · Claude Code, Sonnet 5 · 158/600</sub> |
|
|
1129
|
+
| **Terminal-Bench 2.0**<br><sub>Agent coding tasks, Claude Code with Sonnet 5</sub> | **8/10**<br><sub>first 10 of 89 tasks, 1 trial; Claude Code with no memory 9/10</sub> | - | - | - | - | - | - | - | - | - |
|
|
1078
1130
|
|
|
1079
|
-
|
|
1131
|
+
<details>
|
|
1132
|
+
<summary>How each row ran</summary>
|
|
1133
|
+
|
|
1134
|
+
- **LoCoMo.** Judged again with the Mem0 paper's prompt on gpt-4o-mini, the judge in Mem0's released code. This row still scores a 400-question sample with Sonnet 5 answers, and the published rows use all 1,540 questions. The "LoCoMo, Mem0 paper setup" row runs that setup in full.
|
|
1135
|
+
- **LongMemEval-S.** Judged with the official script's prompt for each question type and its judge, gpt-4o-2024-08-06, over all 500 questions. One setting differs: the answer model is Sonnet 5, and each published row uses its own. The Mem0 cell used its runner's lenient judge prompt, so it stays grey. The "LongMemEval-S, gpt-4o reader" row uses a gpt-4o reader.
|
|
1136
|
+
- **LongMemEval-M.** Judged with the official script's prompts and its gpt-4o-2024-08-06 judge, over all 500 questions. The answer model is Sonnet 5. No product publishes an answer score for M.
|
|
1137
|
+
- **DMR.** Our harness caps the score: the whole chat scores 83.6% here, against 94% to 98% in the Zep paper.
|
|
1138
|
+
- **DMR, gpt-4o-mini answers.** gpt-4o-mini answers, as in Zep's rows, and the prompt now says the answerer is Speaker 1. The harness check failed: the whole chat scores 92.4 here, against 98.0 for the paper's full-conversation row with the same model, so our setup still caps the score. Compare each system with the whole chat in its own setup only: Zep +0.2 points, hippo's reranker setup -6.8 points. Neither paper names its judge.
|
|
1139
|
+
- **PersonaMem v1.** The vendor rows all answer with gpt-4o-mini under MemOS's prompt, one model for every system. Against the authors' own board of full-context models (best 52%), our score compares with a caveat. A rule that picks the longest option scores 48.9%.
|
|
1140
|
+
- **PersonaMem v2.** The paper's row uses the same text split and scorer with another answer model. The MemOS and EverMemOS rows do not say how they ran. A rule that picks the longest option scores 45.0%.
|
|
1141
|
+
- **PrefEval.** The paper's RAG rows keep the whole chat in the prompt; ours shows 5 retrieved turns. The vendor rows use another data split, top 10 and gpt-4o-mini.
|
|
1142
|
+
- **EverMemBench.** Every published row answers with GPT-4.1-mini, one model for every system, and ours answers with Sonnet 5.
|
|
1143
|
+
- **BEAM-100K.** Hindsight uses another answer prompt and Gemini models with a budget it does not state. Honcho scores event ordering another way. cognee ran one conversation. Mem0 publishes no 100K score.
|
|
1144
|
+
- **BEAM-500K.** Hindsight uses another answer prompt and Gemini models with a budget it does not state. Honcho scores event ordering another way. The LIGHT row comes from the paper's scorer.
|
|
1145
|
+
- **BEAM-1M.** Mem0's row uses the same runner code, prompts and scorer. Mem0 answered and judged with gpt-5 and showed 200 memories; we use Sonnet 5, gpt-5.6-sol and 100. The other rows use other scorers or setups.
|
|
1146
|
+
- **BEAM-10M.** Mem0's row uses the same runner code, prompts and scorer. Mem0 answered and judged with gpt-5 and showed 200 memories; we use Sonnet 5, gpt-5.6-sol and 100. The other rows use other scorers or setups.
|
|
1147
|
+
- **ConvoMem.** The sample design is ours, and no published number exists at 50 conversations. The MemPalace row is a 300-item sample with no stated model or judge.
|
|
1148
|
+
- **HotPotQA-24.** F1 over all 24 questions with Cognee's own scorer, from one run with Sonnet 5 answers. Cognee averages 45 runs and names no answer model, and one question moves the score by about 4 points.
|
|
1149
|
+
- **DolphinBench, shortcut run.** Our run wrote each message straight to the store with no agent turn, used a recall hook we wrote, and judged through codex, which the official validator rejects.
|
|
1150
|
+
- **Terminal-Bench 2.0.** The first 10 of 89 tasks, 1 trial each at medium effort; the rule is 5 trials, and the matching Claude Code row used max effort. The 2.0 board is closed, and the open board is 4.1.
|
|
1080
1151
|
|
|
1081
|
-
|
|
1152
|
+
</details>
|
|
1082
1153
|
|
|
1083
|
-
|
|
1084
|
-
|----------|--:|------------------:|
|
|
1085
|
-
| single-hop | 282 | 0.239 |
|
|
1086
|
-
| multi-hop | 321 | 0.491 |
|
|
1087
|
-
| temporal-reasoning | 92 | 0.169 |
|
|
1088
|
-
| open-domain | 841 | 0.450 |
|
|
1089
|
-
| adversarial | 446 | 0.226 |
|
|
1090
|
-
| **overall** | 1,982 | **0.363** |
|
|
1154
|
+
### Finding the right session
|
|
1091
1155
|
|
|
1092
|
-
|
|
1156
|
+
On LongMemEval-S, hippo's benchmark scripts put an answer session in the top five for 98.0% of 500 questions with the free local MiniLM embedder. That is the best of five settings, chosen after the run, on a measure close to its ceiling. Requiring every answer session, the same runs score 86.8 to 88.5%. The `hippo recall` command on a default install scores 85.6% [82.4, 88.6]. On one store of all 19,195 sessions, recall fell to 47.2% in June 2026.
|
|
1093
1157
|
|
|
1094
|
-
###
|
|
1158
|
+
### What the two rerankers bought
|
|
1095
1159
|
|
|
1096
|
-
|
|
1160
|
+
`--reranker cross-encoder` reorders the top 50 memories with a free model on your machine, and is off by default. [On DMR](benchmarks/public/results/2026-11-12-answer-accuracy/dmr-best.txt) it lifted hippo's answers from 73.1 with the embedder alone to 77.6, by 4.4 points [0.6, 8.3]. On PersonaMem v1 and HotPotQA-24 the change was within the noise.
|
|
1097
1161
|
|
|
1098
|
-
|
|
1162
|
+
`--reranker jev` sends the top candidates to the hosted TypeSafe Jev API, and is off by default. On 300 developer queries from a private store, the right memory ranked first for 62% of them, against 41% with the free local cross-encoder. [Three graded-answer tests](docs/evals/2026-09-19-jev-reranker.md) showed no better answer rate. It buys a shorter context for about 0.0004 USD a recall, and it sends your query and candidate text to a third party.
|
|
1099
1163
|
|
|
1100
|
-
|
|
1164
|
+
### What didn't work
|
|
1101
1165
|
|
|
1102
|
-
|
|
1103
|
-
|
|
1166
|
+
- Sleep lowered recall by 3.6 points [1.4, 5.8] on one scorer and raised it a little on another, so we claim no benefit for it.
|
|
1167
|
+
- Decay made no measurable difference, on a synthetic test that gives a 365-day half-life little time to act.
|
|
1168
|
+
- The old 7-day default was a bug that cost recall, so the default moved to 365 days.
|
|
1169
|
+
- Seven smaller experiments failed, stalled or were thrown out, among them a correction detector at 0.82 precision against a 0.90 bar.
|
|
1170
|
+
- One early headline number did not reproduce, and [we retracted it](docs/RETRACTION.md).
|
|
1104
1171
|
|
|
1105
|
-
|
|
1172
|
+
### Check our work
|
|
1106
1173
|
|
|
1107
|
-
|
|
1174
|
+
The score files for five of these public tests and their write-ups are in the repo, under `benchmarks/public`.
|
|
1108
1175
|
|
|
1109
|
-
|
|
1110
|
-
|
|
1111
|
-
```bash
|
|
1112
|
-
cd benchmarks/sequential-learning
|
|
1113
|
-
node run.mjs --adapter all
|
|
1114
|
-
```
|
|
1176
|
+
[Benchmark results](benchmarks/public) · [Eval documents](docs/evals/README.md) · [Report a mistake](https://github.com/kitfunso/hippo-memory/issues/new)
|
|
1115
1177
|
|
|
1116
1178
|
---
|
|
1117
1179
|
|
|
@@ -1180,7 +1242,7 @@ On repeat mistakes, yes, in [one test of our own](https://github.com/kitfunso/hi
|
|
|
1180
1242
|
Issues and PRs welcome. Before contributing, run `hippo status` in the repo root to see the project's own memory.
|
|
1181
1243
|
|
|
1182
1244
|
The interesting problems:
|
|
1183
|
-
- **
|
|
1245
|
+
- **Recall as one store grows.** On LongMemEval-S, `hippo recall` on a default install scores 85.6% [82.4, 88.6] ([result](docs/evals/2026-09-28-recall-cli-longmemeval-result.md)), and on one store of all 19,195 sessions recall fell to 47.2% in June 2026. Requiring every answer session in the top five, the benchmark scripts score 86.8 to 88.5%. The lifecycle stress eval (ROADMAP Part III) is the next measurement.
|
|
1184
1246
|
- Better consolidation heuristics (LLM-powered merge vs current text overlap)
|
|
1185
1247
|
- Web UI / dashboard for visualizing decay curves and memory health
|
|
1186
1248
|
- Optimal decay parameter tuning from real usage data
|
|
@@ -1190,9 +1252,11 @@ The interesting problems:
|
|
|
1190
1252
|
|
|
1191
1253
|
## Open source and commercial
|
|
1192
1254
|
|
|
1193
|
-
Hippo is open core. This repository
|
|
1255
|
+
Hippo is open core. This repository is the MIT core for individual developers and self-hosted teams: the CLI, the MCP server, the supported hooks and adapters, and the dashboard. It includes an HTTP server with tenants, API keys, admin and member roles, per-key scope grants and an audit log on every change. Keys are stored scrypt-hashed, and one API call deletes a memory. Slack and GitHub connectors bring in what your team has already written down, with Jira and Notion connectors next. Automatic capture and delivery depend on the configured integration. Code published here stays MIT and stays here.
|
|
1256
|
+
|
|
1257
|
+
The fair test for a team is a holdout, where a random share of sessions runs without memory and you compare cost per merged PR and repeat errors between the two groups. We have no team result to show. If your team wants to try, [open an issue](https://github.com/kitfunso/hippo-memory/issues/new) and tell us what you'd need. More on [hippo-memory.com/teams](https://hippo-memory.com/teams/).
|
|
1194
1258
|
|
|
1195
|
-
The commercial edition is planned; its private repository is a scaffold, not a released enterprise product. It is intended for larger companies as a separate package under a commercial licence from KITFUNSO LTD. Planned capabilities include SSO (OIDC and SAML sign-in), SCIM, organisation/team/project policy, an org admin view, the pilot report and telemetry join, SIEM export of the audit log, offline licence keys, serving agent hooks to laptops that keep no store of their own, hosted SaaS, and support with an SLA. Pull requests implementing those commercial features belong there; see [CONTRIBUTING.md](CONTRIBUTING.md). Release availability and independently verified benefit are separate milestones in the [roadmap](https://github.com/kitfunso/hippo-memory/blob/master/ROADMAP.md#current-execution-index).
|
|
1259
|
+
The commercial edition is planned; its private repository is a scaffold, not a released enterprise product. None of it is released. It is intended for larger companies as a separate package under a commercial licence from KITFUNSO LTD. Planned capabilities include SSO (OIDC and SAML sign-in), SCIM, organisation/team/project policy, an org admin view, the pilot report and telemetry join, SIEM export of the audit log, offline licence keys, serving agent hooks to laptops that keep no store of their own, hosted SaaS, and support with an SLA. Pull requests implementing those commercial features belong there; see [CONTRIBUTING.md](CONTRIBUTING.md). Release availability and independently verified benefit are separate milestones in the [roadmap](https://github.com/kitfunso/hippo-memory/blob/master/ROADMAP.md#current-execution-index).
|
|
1196
1260
|
|
|
1197
1261
|
## License
|
|
1198
1262
|
|