hippo-memory 3.7.0 → 3.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +253 -190
- package/dist/base/util/version.d.ts +1 -1
- package/dist/base/util/version.js +1 -1
- package/dist/surface/cli/usage.d.ts +1 -1
- package/dist/surface/cli/usage.js +1 -1
- package/extensions/openclaw-plugin/README.md +5 -13
- package/extensions/openclaw-plugin/openclaw.plugin.json +2 -2
- package/extensions/openclaw-plugin/package.json +2 -2
- package/openclaw.plugin.json +2 -2
- package/package.json +4 -5
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
#
|
|
1
|
+
# <img src="https://hippo-memory.com/logo.svg" alt="" width="48" height="42"> Hippo
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**One memory for every agent you'll ever use.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Inspired by the human hippocampus, hippo saves what you teach one coding agent in your repo, and your other agents read it, so switching agents stops meaning starting over.
|
|
6
6
|
|
|
7
7
|
[](https://npmjs.com/package/hippo-memory)
|
|
8
8
|
[](https://npmjs.com/package/hippo-memory)
|
|
@@ -14,81 +14,153 @@ Hippo is open-source memory that runs on your machine and gives the rules it cap
|
|
|
14
14
|
<img src="https://raw.githubusercontent.com/kitfunso/hippo-memory/master/assets/hippo-init.svg" alt="hippo init adding memory to one project" width="720">
|
|
15
15
|
</p>
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
```
|
|
18
|
+
Works with: Claude Code, Codex, Cursor, Pi, OpenClaw, OpenCode, any MCP client
|
|
19
|
+
Imports from: ChatGPT, Claude (CLAUDE.md), Cursor (.cursorrules), Slack, markdown
|
|
20
|
+
Storage: SQLite with markdown mirrors you can read and commit
|
|
21
|
+
Dependencies: Zero runtime deps. Node.js 22.16+. Optional embeddings: bring-your-own local Transformers.js (`npm i @huggingface/transformers`, or legacy `@xenova/transformers`) or an opt-in API embedder (OpenAI/Voyage/Cohere). Nothing is auto-installed.
|
|
22
|
+
```
|
|
18
23
|
|
|
19
|
-
Install
|
|
24
|
+
**Contents:** [Install](#install) · [Follow one memory](#follow-one-memory) · [What hippo does](#what-hippo-does) · [Results](#repeat-mistakes-per-100-chances) · [Setup in detail](#setup-in-detail) · [Agent setup](#framework-integrations) · [MCP server](#mcp-server) · [How it works](#how-it-works) · [CLI](#cli-reference) · [Comparison](#comparison) · [Benchmarks](#benchmarks) · [FAQ](#faq) · [Contributing](#contributing)
|
|
20
25
|
|
|
21
|
-
|
|
22
|
-
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## Install
|
|
29
|
+
|
|
30
|
+
Paste one prompt into your agent:
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
Install hippo-memory for me by following https://github.com/kitfunso/hippo-memory/blob/master/llms-install.md step by step. Run every command yourself and finish with hippo doctor.
|
|
23
34
|
```
|
|
24
35
|
|
|
25
|
-
|
|
36
|
+
Or by hand:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
npm install -g hippo-memory
|
|
40
|
+
|
|
41
|
+
# then, in your project
|
|
42
|
+
hippo init
|
|
43
|
+
```
|
|
26
44
|
|
|
27
45
|
Installing the package alone captures nothing, and what runs after init depends on the agent.
|
|
28
46
|
|
|
29
|
-
|
|
47
|
+
`hippo init` creates a `.hippo/` store in the project, adds a short block to an existing `CLAUDE.md` or `AGENTS.md`, installs Claude Code's hooks and schedules a daily run at 6:15am that stores lessons from the day's commits. `--no-hooks` and `--no-schedule` skip those parts. Codex runs hippo's hooks only once you trust them in `/hooks`. `hippo doctor` checks the install. [What hippo init changes](#what-hippo-init-changes) lists every file it writes, and [llms-install.md](llms-install.md) is the guide the prompt above follows.
|
|
30
48
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
Storage: SQLite backbone with markdown mirrors. Git-trackable, human-readable.
|
|
35
|
-
Dependencies: Zero runtime deps. Node.js 22.16+. Optional embeddings: bring-your-own local Transformers.js (`npm i @huggingface/transformers`, or legacy `@xenova/transformers`) or an opt-in API embedder (OpenAI/Voyage/Cohere). Nothing is auto-installed.
|
|
36
|
-
```
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Follow one memory
|
|
37
52
|
|
|
38
|
-
**
|
|
53
|
+
1. **You correct your agent once.** Say always, never or must, and hippo's session-end hook pins it.
|
|
54
|
+
2. **Next session, it's already there.** Your rules load before your first prompt.
|
|
55
|
+
3. **Every agent reads it.** All your agents read one store, so switching loses nothing.
|
|
56
|
+
4. **Each prompt gets what's relevant.** Your pinned rules, plus up to 5 memories that share its words.
|
|
57
|
+
5. **A wrong memory gets weaker.** After a bad answer, `hippo outcome --bad` weakens the memories behind it.
|
|
58
|
+
6. **A new fact replaces the old.** With `hippo supersede`, the old one leaves recall but stays on record.
|
|
59
|
+
7. **Unused fades, pinned stays.** Unused memories weaken over a year or two, and pinned ones never fade.
|
|
39
60
|
|
|
40
61
|
---
|
|
41
62
|
|
|
42
|
-
##
|
|
63
|
+
## What hippo does
|
|
43
64
|
|
|
44
|
-
|
|
65
|
+
| | What it does | Where |
|
|
66
|
+
|---|---|---|
|
|
67
|
+
| **Capture** | Rules and errors from each session, found by word patterns. | `SessionEnd` hook |
|
|
68
|
+
| **Recall** | Keyword search with no model or network call, and `--why` shows the match. | `hippo recall "why did the deploy break" --why` |
|
|
69
|
+
| **Outcomes** | A bad mark weakens a memory, and each recall keeps it longer. | `hippo outcome --bad` |
|
|
70
|
+
| **Mirror** | A markdown copy of every memory to grep, commit and review. | `.hippo/` |
|
|
71
|
+
| **Dashboard** | Memory health by project, on localhost. | `hippo dashboard` |
|
|
72
|
+
| **Git history** | Learns from the last 30 days of commits, then each day's. | `hippo learn --git` |
|
|
73
|
+
| **Sleep** | Merges related memories and stores faded ones, ready to restore. | `hippo sleep` |
|
|
74
|
+
| **Import** | Brings in the notes your agents already keep, unchanged. | `hippo import --agents` |
|
|
45
75
|
|
|
46
|
-
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## Repeat mistakes per 100 chances
|
|
79
|
+
|
|
80
|
+
Real sessions on 29 repositories, with the plan registered before the run.
|
|
81
|
+
|
|
82
|
+
| Claude Code with | Repeats per 100 chances |
|
|
83
|
+
|---|--:|
|
|
84
|
+
| No memory | 83 |
|
|
85
|
+
| Built-in memory | 21 |
|
|
86
|
+
| Built-in plus hippo | **11** |
|
|
87
|
+
|
|
88
|
+
Cost with hippo: 1.006 times, the same.
|
|
47
89
|
|
|
48
|
-
|
|
90
|
+
The gain was the same with lesson capture switched off, so this run cannot say hippo's stored lessons caused it. The test agent was Claude Code only, so there's no measured result for Codex or Cursor. The [result](docs/evals/2026-11-08-z0-round3-result.md) has every figure, and [Benchmarks](#benchmarks) has the rest of the test.
|
|
91
|
+
|
|
92
|
+
### Public memory tests
|
|
93
|
+
|
|
94
|
+
One answer model and scorer for every system we ran in a test.
|
|
95
|
+
|
|
96
|
+
| Test | Questions | hippo | Other system |
|
|
97
|
+
|---|--:|--:|---|
|
|
98
|
+
| DMR | 495 | 77.6 | full chat 83.6 |
|
|
99
|
+
| PersonaMem v1 | 584 | 72.4 | MemOS 61.2, published |
|
|
100
|
+
| LongMemEval-S | 495 | 88.5 | |
|
|
101
|
+
| LoCoMo | 395 | 89.4 | Mem0 94.9 |
|
|
102
|
+
|
|
103
|
+
hippo used its local embedder, plus its local reranker on DMR. MemOS's score is from its own setup. [How we test](#benchmarks).
|
|
49
104
|
|
|
50
105
|
---
|
|
51
106
|
|
|
52
|
-
##
|
|
107
|
+
## Zero model calls to store a memory
|
|
108
|
+
|
|
109
|
+
Other memory tools run what you save through a language model, an LLM on top of your LLM.
|
|
53
110
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
111
|
+
| Tool | LLM calls per memory | Unit |
|
|
112
|
+
|---|--:|---|
|
|
113
|
+
| hippo | 0 | per memory, default path |
|
|
114
|
+
| Mem0 | 1 | per add |
|
|
115
|
+
| Zep (Graphiti) | 2+ | per episode, more for duplicates and dates |
|
|
116
|
+
| Letta | 2+ | per turn that writes memory, V1 server |
|
|
117
|
+
| Supermemory | count not public | uses a model, per its docs (low confidence) |
|
|
118
|
+
| Cognee | 1 | per text chunk, with a key |
|
|
119
|
+
| LangMem | 1+ | per run |
|
|
120
|
+
| MemOS | 1 | per message window |
|
|
58
121
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
- **On a private 300-query developer store, R@1 0.41 to 0.62 with `hippo recall "<query>" --reranker jev`**, the free local cross-encoder against Jev ([full eval](docs/evals/2026-09-19-jev-reranker.md)). The opt-in [TypeSafe Jev](https://typesafe.ai) reranker, off by default, about 0.0004 USD a recall. 2000-draw paired bootstrap; the margin held in 20 of 20 seeds and a permutation null reached it in 0 of 200 runs. Ranking only: three graded tests on one 150-question LongMemEval set did **not** show a better answer rate than the free local cross-encoder, and that negative result is in the same doc. What it buys today is a shorter context: on that set, 2 memories ranked by Jev answered as well as 5 ranked by the cross-encoder.
|
|
63
|
-
- **Staged Slack corpus, 10 incident scenarios: recall beat transcript replay in 10 of 10** ([benchmarks/e1.3/](benchmarks/e1.3/)). The answers sit mid-channel by design. In every scenario hippo's top 10 results held all the answer messages, and the channel's last 10 messages held none.
|
|
64
|
-
- **Slack connector, 1000-event ingestion smoke: 0 outbound HTTP** ([benchmarks/e1.3/](benchmarks/e1.3/)). Proven by a `globalThis.fetch` spy that throws on call, not a hardcoded zero. Recall makes no network call by default. One default does: `hippo sleep` sends memory text to Anthropic for fact extraction when `ANTHROPIC_API_KEY` is set. Sleep runs at the end of every Claude Code and OpenCode session and in the daily job, so with the key set the call happens without you asking. `{"extraction":{"enabled":false}}` in `.hippo/config.json` turns that off. Opt-in features such as the Jev reranker above, the LLM reranker and the API embedders also call out.
|
|
65
|
-
- **3,500+ tests on a real database.** No module mocks and no mocked store; only paid network calls are stubbed. Project rule. The one mocks-vs-prod divergence that bit us early is now the constraint that kept the next ten releases honest.
|
|
66
|
-
- **3-cluster fixture where BM25 alone cannot discriminate: dlPFC goal-conditioned cluster discrimination passes 3 of 3 queries.** Full goal stack with policy weighting and lifespan-windowed outcome propagation, one query per goal; deterministic test in [`benchmarks/micro/results/b3-depth.json`](benchmarks/micro/results/b3-depth.json).
|
|
122
|
+
Storing the LoCoMo test chats took Mem0 5,882 language-model calls, one per turn, and hippo none ([the count](benchmarks/public/results/2026-11-12-answer-accuracy/mem0-locomo-ingest-gate.json)). Counts come from each product's public code and docs, 13 Nov 2026.
|
|
123
|
+
|
|
124
|
+
The exception is `hippo sleep`, which sends text to Anthropic when `ANTHROPIC_API_KEY` is set, and `{"extraction":{"enabled":false}}` turns it off.
|
|
67
125
|
|
|
68
126
|
---
|
|
69
127
|
|
|
70
|
-
##
|
|
128
|
+
## One memory, three readers
|
|
129
|
+
|
|
130
|
+
You, your team and your platform lead read the same store.
|
|
131
|
+
|
|
132
|
+
- **The developer.** Say a rule with never, always or must, and every session starts with it.
|
|
133
|
+
- **The team.** Commit the markdown copy and review a rule in a pull request.
|
|
134
|
+
- **The platform lead.** One team server gives everyone the same store and their own key.
|
|
71
135
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
136
|
+
| Agent | What hippo adds |
|
|
137
|
+
|---|---|
|
|
138
|
+
| Claude Code | `CLAUDE.md` + 8 hooks |
|
|
139
|
+
| Codex | `AGENTS.md` + 2 hooks |
|
|
140
|
+
| Cursor | reads `AGENTS.md` |
|
|
141
|
+
| Pi | reads `AGENTS.md` |
|
|
142
|
+
| OpenClaw | plugin |
|
|
143
|
+
| OpenCode | plugin |
|
|
144
|
+
| Any MCP client | `hippo mcp`, 13 tools |
|
|
145
|
+
|
|
146
|
+
`hippo dashboard` opens a local page with each project's memory health.
|
|
77
147
|
|
|
78
148
|
---
|
|
79
149
|
|
|
80
|
-
##
|
|
150
|
+
## Memory is infrastructure you own
|
|
81
151
|
|
|
82
|
-
|
|
152
|
+
Your agents' memory belongs in a file you can read, diff and delete, whichever agent you use next year.
|
|
83
153
|
|
|
84
|
-
|
|
85
|
-
|
|
154
|
+
- **Today:** a team server that shares one store over MCP, with API keys, roles, an audit log, and Slack and GitHub connectors.
|
|
155
|
+
- **Planned:** a commercial edition with SSO, SCIM, SIEM export, hosting and an SLA.
|
|
86
156
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
157
|
+
There's no team result yet, so [open an issue](https://github.com/kitfunso/hippo-memory/issues/new?title=Pilot%20request&body=Company%2C%20team%20size%2C%20agents%20in%20use%3A) for a holdout pilot. [Open source and commercial](#open-source-and-commercial) says what each edition holds.
|
|
158
|
+
|
|
159
|
+
---
|
|
90
160
|
|
|
91
|
-
|
|
161
|
+
## Setup in detail
|
|
162
|
+
|
|
163
|
+
**Many repos at once.** Read what it changes first. `hippo init --scan <folder>` looks for git repos in the folder and up to three levels below it, skipping dot-folders and `node_modules`. Each repo gets a `.hippo/` store, seeded with lessons from the last 365 days of its commits and with its [agent memories](#agent-memories), and is added to the daily run's list. For the agents it finds, it installs the same user-level hooks as `hippo init`: 8 Claude Code hook entries in `~/.claude/settings.json` and the OpenCode plugin. It also sets up the daily 6:15am run, a crontab line on Linux and macOS or a scheduled task on Windows. It adds no block to any repo's `CLAUDE.md` or `AGENTS.md`. `--no-hooks`, `--no-schedule` and `--no-learn` leave out the hooks, the daily run and the history import.
|
|
92
164
|
|
|
93
165
|
```bash
|
|
94
166
|
hippo init --scan ~
|
|
@@ -165,7 +237,7 @@ Not read: Windsurf (the file format is not documented, and Cascade reached end o
|
|
|
165
237
|
|
|
166
238
|
## Cross-Tool Import
|
|
167
239
|
|
|
168
|
-
|
|
240
|
+
Hippo imports the notes and rules you already keep in other tools. `hippo import --agents` reads what your coding agents remember ([Agent memories](#agent-memories)); these commands read files:
|
|
169
241
|
|
|
170
242
|
```bash
|
|
171
243
|
# ChatGPT memory export
|
|
@@ -188,7 +260,7 @@ All import commands support `--dry-run` (preview without writing), `--global` (w
|
|
|
188
260
|
|
|
189
261
|
### Conversation Capture
|
|
190
262
|
|
|
191
|
-
|
|
263
|
+
Store memories from conversation text. Word patterns, not a model, find decisions, rules, errors and preferences.
|
|
192
264
|
|
|
193
265
|
```bash
|
|
194
266
|
# Pipe a conversation in
|
|
@@ -227,9 +299,11 @@ Backfill an existing channel: `SLACK_BOT_TOKEN=xoxb-... hippo slack backfill --c
|
|
|
227
299
|
|
|
228
300
|
Multi-workspace deployments populate `slack_workspaces (team_id, tenant_id)` to route events per tenant; single-workspace falls back to `HIPPO_TENANT`.
|
|
229
301
|
|
|
302
|
+
GitHub webhooks land the same way at `POST /v1/connectors/github/events`. Backfill a repository's issues and comments with `hippo github backfill --repo <owner/name>`. Every memory carries `kind`, `scope`, `owner` and `artifact_ref`, and one API call deletes a memory.
|
|
303
|
+
|
|
230
304
|
### Active task snapshots
|
|
231
305
|
|
|
232
|
-
|
|
306
|
+
Hippo can save the task in progress, so a later session can pick it up where this one stopped.
|
|
233
307
|
|
|
234
308
|
```bash
|
|
235
309
|
hippo snapshot save \
|
|
@@ -246,7 +320,7 @@ hippo snapshot clear
|
|
|
246
320
|
|
|
247
321
|
### Session event trails
|
|
248
322
|
|
|
249
|
-
|
|
323
|
+
Hippo can store short session events and link them to the active snapshot, so context output shows the latest steps as well as the last summary. The trail prints for the calling session's own snapshot; for another session's, the command in the pointer line prints it.
|
|
250
324
|
|
|
251
325
|
```bash
|
|
252
326
|
hippo session log \
|
|
@@ -296,28 +370,30 @@ hippo wm push --scope repo \
|
|
|
296
370
|
|
|
297
371
|
hippo wm read --scope repo # show current working notes
|
|
298
372
|
hippo wm clear --scope repo # wipe the scratchpad
|
|
299
|
-
hippo wm flush --scope repo #
|
|
373
|
+
hippo wm flush --scope repo # same as clear; nothing runs it at session end
|
|
300
374
|
```
|
|
301
375
|
|
|
302
376
|
The buffer holds a maximum of 20 entries per scope. When full, the lowest-importance entry is evicted.
|
|
303
377
|
|
|
304
378
|
### Explainable recall
|
|
305
379
|
|
|
306
|
-
|
|
380
|
+
`--why` shows, for each memory recall returns, the words of your query it matched, the embedding similarity when an embedder is on, and where the memory came from:
|
|
307
381
|
|
|
308
382
|
```bash
|
|
309
383
|
hippo recall "data pipeline" --why --limit 5
|
|
310
|
-
|
|
311
|
-
# --- mem_a1b2c3 [episodic] [observed] [local] score=0.847
|
|
312
|
-
# BM25: matched [data, pipeline]; cosine: 0.82
|
|
313
|
-
# ...memory content...
|
|
314
384
|
```
|
|
315
385
|
|
|
386
|
+
`hippo explain "<query>"` prints the full score breakdown for each result.
|
|
387
|
+
|
|
316
388
|
---
|
|
317
389
|
|
|
318
390
|
## How It Works
|
|
319
391
|
|
|
320
|
-
|
|
392
|
+
Every memory has a strength. It fades on a half-life, grows each time the memory is recalled, and drops when you mark it wrong. The hippocampus inspired the design, and we tested each part on its own. On synthetic tests, outcome marks and recall strengthening helped and decay made no measurable difference. Sleep lowered recall on one scorer, and the rest is unmeasured. The design notes are in [PLAN.md](PLAN.md#core-principles) and [RESEARCH.md](RESEARCH.md).
|
|
393
|
+
|
|
394
|
+
- **Buffer.** New information lands here for the session.
|
|
395
|
+
- **Episodic.** Timestamped memories that fade unless they are used.
|
|
396
|
+
- **Semantic.** Patterns merged from related episodes during sleep, by word overlap.
|
|
321
397
|
|
|
322
398
|
The store is SQLite (`.hippo/hippo.db`). The markdown files are mirrors written after each change. `index.json` is no longer refreshed by writes, deletes or recalls, so a copy an older version left on disk goes stale. Read the store through the CLI, the MCP server or the HTTP API.
|
|
323
399
|
|
|
@@ -362,7 +438,7 @@ sequenceDiagram
|
|
|
362
438
|
|
|
363
439
|
### Decay by default
|
|
364
440
|
|
|
365
|
-
|
|
441
|
+
A memory's strength halves each time a half-life passes without use, and the default half-life is 365 days. The default used to be 7 days, which turned out to be a bug. On a [test we registered first](docs/evals/2026-09-24-decay-default-result.md), the current version of a fact was in the top five 29.2% of the time at 7 days and 74.7% at 365. We did not tune 365. It tied with 730 days and with decay switched off, so on that test decay made no measurable difference. `hippo sleep` moves memories still on the old 7-day base to the new one, once, and records each move in the audit log. Set `defaultHalfLifeDays` in `.hippo/config.json` to choose your own, or pin a memory so it never decays.
|
|
366
442
|
|
|
367
443
|
```bash
|
|
368
444
|
hippo remember "always check cache contents after refresh"
|
|
@@ -375,9 +451,9 @@ hippo inspect mem_a1b2c3
|
|
|
375
451
|
|
|
376
452
|
---
|
|
377
453
|
|
|
378
|
-
###
|
|
454
|
+
### Recall makes a memory last longer
|
|
379
455
|
|
|
380
|
-
|
|
456
|
+
Each recall adds 2 days to a memory's half-life, so 365 days becomes 367 and then 369. Recall ranks memories by relevance, strength and recency, and the strongest matches fill the token budget first. In the first round of our audit this helped on the synthetic test, in its best case.
|
|
381
457
|
|
|
382
458
|
```bash
|
|
383
459
|
hippo recall "cache issues"
|
|
@@ -388,14 +464,13 @@ hippo recall "cache issues"
|
|
|
388
464
|
hippo recall "cache issues" # again next week
|
|
389
465
|
# retrieval_count: 2 -> 3
|
|
390
466
|
# half_life: 367d -> 369d
|
|
391
|
-
# this memory is learning to survive
|
|
392
467
|
```
|
|
393
468
|
|
|
394
469
|
---
|
|
395
470
|
|
|
396
471
|
### Active invalidation
|
|
397
472
|
|
|
398
|
-
When you
|
|
473
|
+
When you move from one tool to another, old memories about the replaced tool should stop showing up at once. Hippo detects migration and breaking-change commits during `hippo learn --git` and actively weakens matching memories.
|
|
399
474
|
|
|
400
475
|
```bash
|
|
401
476
|
hippo learn --git
|
|
@@ -433,17 +508,15 @@ hippo decide "Use CockroachDB for global services" \
|
|
|
433
508
|
|
|
434
509
|
---
|
|
435
510
|
|
|
436
|
-
###
|
|
511
|
+
### Errors last twice as long
|
|
437
512
|
|
|
438
|
-
|
|
513
|
+
A memory stored with `--error`, or captured from a failed tool call, gets a 730-day half-life against the usual 365. We haven't measured whether that helps.
|
|
439
514
|
|
|
440
515
|
```bash
|
|
441
516
|
hippo remember "deployment failed: forgot to run migrations" --error
|
|
442
517
|
# half_life: 730d instead of 365d
|
|
443
518
|
# emotional_valence: negative
|
|
444
519
|
# strength formula applies 2.0x multiplier (HIPPO_LOSS_AVERSION_RATIO=0.75 to keep v1.13.4 1.5x)
|
|
445
|
-
|
|
446
|
-
# production incidents don't fade quietly
|
|
447
520
|
```
|
|
448
521
|
|
|
449
522
|
---
|
|
@@ -466,13 +539,11 @@ When context is generated, confidence is shown inline:
|
|
|
466
539
|
[inferred] The flaky test might be a race condition
|
|
467
540
|
```
|
|
468
541
|
|
|
469
|
-
Agents can see at a glance what's established fact vs. a pattern worth questioning.
|
|
470
|
-
|
|
471
542
|
A memory not recalled for 30 days is shown as aged when it is read, and recalling it clears that. Pinned and `verified` memories are exempt. Nothing in the store changes.
|
|
472
543
|
|
|
473
544
|
### Conflict tracking
|
|
474
545
|
|
|
475
|
-
|
|
546
|
+
When two memories overlap in content and contradict each other, hippo records a conflict that `hippo conflicts` lists. Shared tags alone do not count; the statements themselves need to overlap in content. `hippo supersede <id> "<new fact>"` stores the new version of a fact and links the old one to it, `hippo recall --as-of <date>` shows what was current on a given day, and `hippo reject <id> --reason "<why>"` stops a value coming back at all. None of this is measured yet.
|
|
476
547
|
|
|
477
548
|
```bash
|
|
478
549
|
hippo sleep # refreshes open conflicts
|
|
@@ -518,9 +589,7 @@ hippo sleep
|
|
|
518
589
|
# New semantic: 2
|
|
519
590
|
```
|
|
520
591
|
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
Sleep keeps the store tidy. It has not been shown to improve recall. In round 2 of the mechanism audit, a slept LongMemEval store scored 3.6 points lower at hit@5 than the same store never slept, and no scorer showed sleep helping ([PR #232](https://github.com/kitfunso/hippo-memory/pull/232)).
|
|
592
|
+
Sleep merges two or more related episodes into one pattern, and the originals fade. It keeps the store tidy. It has not been shown to improve recall, and on one scorer a slept store scored 3.6 points lower than the same store never slept ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)).
|
|
524
593
|
|
|
525
594
|
**Experimental: learned memory-value rescue (opt-in, default off).** With
|
|
526
595
|
`{"memoryValue":{"enabled":true}}` in `.hippo/config.json`, sleep consults a learned
|
|
@@ -529,9 +598,8 @@ top 30% of its tenant by learned value is kept ("rescued") even though its stren
|
|
|
529
598
|
below the decay threshold. The scorer can only rescue, never delete: with the flag on,
|
|
530
599
|
sleep deletes a strict subset of what it would delete with the flag off. Every rescue is
|
|
531
600
|
recorded in the audit log (`hippo audit list --op mv_rescue`). The weights were learned
|
|
532
|
-
on
|
|
533
|
-
|
|
534
|
-
usage, NOT real usage value, so treat the flag as an experiment, not a recommendation.
|
|
601
|
+
on a LongMemEval retention test with simulated usage, so they do not measure real usage
|
|
602
|
+
value: treat the flag as an experiment.
|
|
535
603
|
Tenants with fewer than 10 non-pinned memories never rescue (rank statistics are noise at
|
|
536
604
|
tiny scale).
|
|
537
605
|
|
|
@@ -619,7 +687,7 @@ are not recorded. Set the rate to 0 only after the pilot window closes, because
|
|
|
619
687
|
|
|
620
688
|
### Outcome feedback
|
|
621
689
|
|
|
622
|
-
|
|
690
|
+
`hippo outcome --good` or `--bad` marks the memories from the last recall.
|
|
623
691
|
|
|
624
692
|
```bash
|
|
625
693
|
hippo recall "why is the gold model broken"
|
|
@@ -636,16 +704,16 @@ hippo outcome --bad
|
|
|
636
704
|
|
|
637
705
|
Outcomes are cumulative. A memory with 5 positive outcomes and 0 negative has a reward factor of ~1.42, making its effective half-life 42% longer. A memory with 0 positive and 3 negative has a factor of 0.625, so it decays 1.6 times as fast; each bad mark past the good ones also halves its strength, up to three times, and recall stops strengthening it. Mixed outcomes converge toward neutral (1.0).
|
|
638
706
|
|
|
639
|
-
|
|
707
|
+
On a synthetic test where every mark was correct, a bad mark cut how often a wrong memory stayed in the top five from 71.9% to 0.0% ([mechanism audit, round 2](https://github.com/kitfunso/hippo-memory/pull/232)). Real marks are noisier, because `--bad` marks the whole recall batch.
|
|
640
708
|
|
|
641
709
|
---
|
|
642
710
|
|
|
643
711
|
### Token budgets
|
|
644
712
|
|
|
645
|
-
Recall only what fits
|
|
713
|
+
Recall prints only what fits the token budget.
|
|
646
714
|
|
|
647
715
|
```bash
|
|
648
|
-
#
|
|
716
|
+
# a small block for task context
|
|
649
717
|
hippo recall "deployment checklist" --budget 2000
|
|
650
718
|
|
|
651
719
|
# need more for a big task
|
|
@@ -712,6 +780,7 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
712
780
|
| `hippo init` | Create `.hippo/`, install agent hooks and the daily run ([what it changes](#what-hippo-init-changes)) |
|
|
713
781
|
| `hippo init --global` | Create global store at `~/.hippo/` |
|
|
714
782
|
| `hippo init --no-hooks` | Create `.hippo/` without auto-installing hooks |
|
|
783
|
+
| `hippo setup` | Detect installed AI tools, install their hooks and import each agent's user-level memories |
|
|
715
784
|
| `hippo remember "<text>"` | Store a memory |
|
|
716
785
|
| `hippo remember "<text>" --tag <t>` | Store with tag (repeatable) |
|
|
717
786
|
| `hippo remember "<text>" --error` | Store as error (2x half-life) |
|
|
@@ -720,12 +789,14 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
720
789
|
| `hippo remember "<text>" --observed` | Set confidence: observed |
|
|
721
790
|
| `hippo remember "<text>" --inferred` | Set confidence: inferred |
|
|
722
791
|
| `hippo remember "<text>" --global` | Store in global `~/.hippo/` store |
|
|
792
|
+
| `hippo supersede <id> "<text>"` | Replace a memory with a new version; the old one leaves recall and points at it |
|
|
723
793
|
| `hippo recall "<query>"` | Retrieve relevant memories (local + global) |
|
|
724
794
|
| `hippo recall "<query>" --budget <n>` | Recall within token limit (default: 4000) |
|
|
725
795
|
| `hippo recall "<query>" --limit <n>` | Cap result count |
|
|
726
796
|
| `hippo recall "<query>" --why` | Show match reasons and source buckets |
|
|
727
797
|
| `hippo recall "<query>" --hops <n>` | Also surface memories N hops away in the entity/relation graph (0..3, default off) |
|
|
728
798
|
| `hippo recall "<query>" --json` | Output as JSON |
|
|
799
|
+
| `hippo recall "<query>" --as-of <date>` | Return the memories that were current on that date |
|
|
729
800
|
| `hippo context --auto` | Smart context injection (auto-detects task from git) |
|
|
730
801
|
| `hippo context "<query>" --budget <n>` | Context injection with explicit query (default: 1500) |
|
|
731
802
|
| `hippo context --limit <n>` | Cap memory count in context |
|
|
@@ -739,6 +810,8 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
739
810
|
| `hippo import --file <path>` | Import from any text file |
|
|
740
811
|
| `hippo import --dry-run` | Preview import without writing |
|
|
741
812
|
| `hippo import --global` | Write imported memories to `~/.hippo/` |
|
|
813
|
+
| `hippo import --agents [--dry-run]` | Bring in every coding agent's own memories now ([Agent memories](#agent-memories)) |
|
|
814
|
+
| `hippo export [file] [--format markdown]` | Export all memories as JSON (default) or markdown |
|
|
742
815
|
| `hippo capture --stdin` | Extract memories from piped conversation text |
|
|
743
816
|
| `hippo capture --file <path>` | Extract memories from a file |
|
|
744
817
|
| `hippo capture --dry-run` | Preview extraction without writing |
|
|
@@ -750,6 +823,7 @@ The command runs with no shell, so each argument reaches the program as written.
|
|
|
750
823
|
| `hippo outcome --id <id> --good` | Target a specific memory |
|
|
751
824
|
| `hippo inspect <id>` | Full detail on one memory |
|
|
752
825
|
| `hippo forget <id>` | Force remove a memory |
|
|
826
|
+
| `hippo reject <id> --reason "<why>"` | Stop a value coming back: it refuses re-ingestion (`hippo rejections` lists them) |
|
|
753
827
|
| `hippo dormant [<query>]` | List faded memories sleep kept instead of deleting |
|
|
754
828
|
| `hippo dormant restore <id>` | Bring a dormant memory back to active memory |
|
|
755
829
|
| `hippo dormant forget <id>` | Delete a dormant memory permanently |
|
|
@@ -859,7 +933,7 @@ For Claude Code, it also adds 8 hook entries to `~/.claude/settings.json`:
|
|
|
859
933
|
- a `SessionEnd` hook that runs `hippo sleep` and then `hippo capture` when the session exits. Capture matches the last 20 user and 10 assistant messages of the transcript against word patterns for decisions, rules, errors and preferences. It uses no model unless you turn on `capture.followUpRules` (see Conversation Capture). The patterns do not read earlier turns, so a correction older than that window becomes a pinned rule only with `hippo remember --pin` by hand. Capture also saves each message you typed whole, so recall still finds the older correction (see What you type).
|
|
860
934
|
- a `SessionStart` hook that prints the previous session's consolidation log on stderr, where `claude --debug` shows it. When the last sleep failed or a compaction summary was set aside, it shows you one line that points to `hippo doctor`.
|
|
861
935
|
- a second `SessionStart` hook that runs `hippo session-rules`, which prints the rules you taught under `## Rules from the user` and nothing else. It has no matcher, so a resumed or cleared session and a compaction get the rules again.
|
|
862
|
-
- a `UserPromptSubmit` hook that runs `hippo context --pinned-only --include-recent 5 --format additional-context` every turn. It re-injects pinned memories (`hippo remember <text> --pin`) plus up to 5 memories that share words with your prompt. When nothing matches, it adds only the pinned ones. Since 1.55.0 this replaces the 5 newest memories
|
|
936
|
+
- a `UserPromptSubmit` hook that runs `hippo context --pinned-only --include-recent 5 --format additional-context` every turn. It re-injects pinned memories (`hippo remember <text> --pin`) plus up to 5 memories that share words with your prompt. When nothing matches, it adds only the pinned ones. Since 1.55.0 this replaces the 5 newest memories. `{"pinnedInject":{"promptRecall":false}}` brings back the 5 newest, so fresh same-session lessons appear on the next prompt whatever you ask. The block is rendered without live strength percentages, so it stays byte-identical while its memories do not change, and it is sent only when it changed since the session's last prompt: an unchanged block is skipped, resent every 10 skips (`pinnedInject.refreshTurns`, `0` never resends) and resent after compaction. The prompt-matched memories go in a separate block that is never skipped, so they are sent on every prompt they match. `{"pinnedInject":{"skipUnchanged":false}}` sends the pinned block every turn as before. `{"pinnedInject":{"excludeTags":["session-digest"]}}` leaves out every memory that carries a listed tag, whether it is pinned, recent or matched to the prompt; the list is empty by default. Opt out entirely with `{"pinnedInject":{"enabled":false}}` in `.hippo/config.json`.
|
|
863
937
|
- a `PreCompact` hook that runs `hippo pre-compact` before the transcript gets summarized. It records the compaction in the store, saves a working-state snapshot (task/summary/next step) so mid-session compaction can't drop it, and asks the summariser to end its summary with a "Memories for hippo" list: the lessons, decisions and corrections from the session that should outlive it. The `SessionEnd` hook still owns extracting durable memories from the transcript.
|
|
864
938
|
- a second `SessionStart` hook (matcher `compact`) that runs `hippo compact-resume`, printing that snapshot back into context right after compaction, if it is under 15 minutes old.
|
|
865
939
|
- a `PostCompact` hook that runs `hippo post-compact`. It keeps the summary in the store with secrets scrubbed, and saves each item of that list as a memory that sleep never deletes: at most 10 per compaction, skipping an item an earlier compaction already saved and any item that looks like a secret. An item over 500 characters stays in the compaction's record only. It then prints one line, such as "Hippo saved 3 memories from this compaction and restored your task snapshot." If the store is busy, the summary waits in the store's `compactions-spool/` folder and `hippo sleep` finishes the save; `hippo doctor` names any compaction left unfinished for over 10 minutes. It also counts spool files that are waiting, stale (left by a replay that stopped) or set aside as `.bad`. The session that compacted does not have those memories injected back into its own prompts, since it just read them in the summary; `hippo recall` still finds them. The hook prints nothing when there is no store to save to.
|
|
@@ -930,7 +1004,7 @@ The headers are a filter the caller picks, not access control: any member key ca
|
|
|
930
1004
|
|
|
931
1005
|
Native plugin with auto-context injection, workspace-aware memory lookup, and
|
|
932
1006
|
tool hooks for auto-learn / auto-sleep. When `autoSleep` is enabled, the
|
|
933
|
-
OpenClaw plugin
|
|
1007
|
+
OpenClaw plugin launches `hippo sleep` in a detached background worker at
|
|
934
1008
|
session end so the live session can exit immediately.
|
|
935
1009
|
|
|
936
1010
|
Query-time retrieval still uses the active workspace store plus the shared
|
|
@@ -952,37 +1026,11 @@ Full integration details: [integrations/](integrations/)
|
|
|
952
1026
|
|
|
953
1027
|
---
|
|
954
1028
|
|
|
955
|
-
## The Neuroscience
|
|
956
|
-
|
|
957
|
-
Hippo's design borrows seven properties of the human hippocampus. This section is design inspiration, not measured benefit. For what we measured, see the Receipts and [PR #232](https://github.com/kitfunso/hippo-memory/pull/232).
|
|
958
|
-
|
|
959
|
-
**Why two stores?** The brain uses a fast hippocampal buffer + a slow neocortical store (Complementary Learning Systems theory, McClelland et al. 1995). If the neocortex learned fast, new information would overwrite old knowledge. The buffer absorbs new episodes; the neocortex extracts patterns over time.
|
|
960
|
-
|
|
961
|
-
**Why decay at all?** In the brain, new neurons born in the dentate gyrus disrupt old memory traces (Frankland et al. 2013), which may reduce interference from outdated information. That is why hippo has decay. In hippo's own tests, age-based decay made no measurable difference to recall: at the 365-day default it tied with decay switched off. A bad outcome mark is the forgetting that measured helpful. Supersession, a newer fact replacing an old one, has not been measured.
|
|
962
|
-
|
|
963
|
-
**Why do errors stick?** The amygdala modulates hippocampal consolidation based on emotional significance. Fear and error signals boost encoding. Your first production incident is burned into memory. Your 200th uneventful deploy isn't.
|
|
964
|
-
|
|
965
|
-
**Why does retrieval strengthen?** Recalled memories undergo "reconsolidation" (Nader et al. 2000). The act of retrieval destabilizes the trace, then re-encodes it stronger. This is the testing effect. Hippo borrows the idea: each recall adds 2 days to a memory's half-life.
|
|
966
|
-
|
|
967
|
-
**Why does sleep consolidate?** During sleep, the hippocampus replays compressed versions of recent episodes and "teaches" the neocortex by repeatedly activating the same patterns. Hippo's `sleep` command borrows the idea for a consolidation pass. In hippo's own audit it lowered recall ([Sleep consolidation](#sleep-consolidation) has the numbers).
|
|
968
|
-
|
|
969
|
-
The 7 mechanisms in full: [PLAN.md#core-principles](PLAN.md#core-principles)
|
|
970
|
-
|
|
971
|
-
For how these mechanisms connect to LLM training, continual learning, and open research problems: **[RESEARCH.md](RESEARCH.md)**
|
|
972
|
-
|
|
973
|
-
**Why does reward modulate decay?** In spiking neural networks, reward-modulated STDP strengthens synapses that contribute to positive outcomes and weakens those that don't. Hippo's reward-proportional decay (v0.11.0) borrows this idea: memories with consistent positive outcomes decay slower, negatives decay faster, with no fixed deltas. Inspired by [MH-FLOCKE](https://github.com/MarcHesse/mhflocke)'s R-STDP architecture for quadruped locomotion, where the same mechanism produces stable learning with 11.6x lower variance than PPO.
|
|
974
|
-
|
|
975
|
-
**Prior art in agent memory simulation.** The idea that human-like memory produces human-like behavior as an emergent property was explored in IEEE research from 2010-2011 ([5952114](https://ieeexplore.ieee.org/document/5952114), [5548405](https://ieeexplore.ieee.org/document/5548405), [5953964](https://ieeexplore.ieee.org/document/5953964)). Walking between rooms and forgetting why you went there doesn't need direct simulation; it emerges naturally from a memory system with capacity limits and decay. Hippo takes the mechanisms as design ideas. Whether better agent behavior follows from them has not been shown.
|
|
976
|
-
|
|
977
|
-
**Related work:** [HippoRAG](https://arxiv.org/abs/2405.14831) (Gutierrez et al., 2024) applies hippocampal indexing to RAG via knowledge graphs. [MemPalace](https://github.com/milla-jovovich/mempalace) (Sigman & Jovovich, 2026) organizes memory spatially (wings/halls/rooms) with AAAK compression, achieving 100% on [LongMemEval](https://arxiv.org/abs/2410.10813). [MH-FLOCKE](https://github.com/MarcHesse/mhflocke) (Hesse, 2026) uses spiking neurons with R-STDP for embodied cognition. Each system tackles a different facet: HippoRAG optimizes retrieval quality, MemPalace optimizes retrieval organization, MH-FLOCKE optimizes embodied learning, and Hippo works on the memory lifecycle.
|
|
978
|
-
|
|
979
|
-
---
|
|
980
|
-
|
|
981
1029
|
## Comparison
|
|
982
1030
|
|
|
983
|
-
|
|
1031
|
+
Hippo is for coding agents on machines you control. [Mem0](https://hippo-memory.com/vs/mem0/) fits better when the memory belongs to your app's end users, [Zep](https://hippo-memory.com/vs/zep/) when you want a managed knowledge graph built from business data, and [Letta Code](https://hippo-memory.com/vs/letta/) when you want an agent that edits its own memory. The table is a feature snapshot, not a verdict, and it holds no scores: [Benchmarks](#benchmarks) has those, split by setup.
|
|
984
1032
|
|
|
985
|
-
| Feature | Hippo | [MemPalace](https://github.com/milla-jovovich/mempalace) | [Mem0](https://github.com/mem0ai/mem0) | [Basic Memory](https://github.com/basicmachines-co/basic-memory) | [gbrain](https://hermesatlas.com/projects/garrytan/gbrain) | [Zep](https://www.getzep.com/) | [Letta](https://github.com/letta-ai/letta-code) | [Cognee](https://www.cognee.ai/) | [Memoria](https://github.com/matrixorigin/Memoria) | [EverMind](https://evermind.ai/) |
|
|
1033
|
+
| Feature\* | Hippo | [MemPalace](https://github.com/milla-jovovich/mempalace) | [Mem0](https://github.com/mem0ai/mem0) | [Basic Memory](https://github.com/basicmachines-co/basic-memory) | [gbrain](https://hermesatlas.com/projects/garrytan/gbrain) | [Zep](https://www.getzep.com/) | [Letta](https://github.com/letta-ai/letta-code) | [Cognee](https://www.cognee.ai/) | [Memoria](https://github.com/matrixorigin/Memoria) | [EverMind](https://evermind.ai/) |
|
|
986
1034
|
|---------|-------|-----------|------|-------------|--------|-----|-------|--------|---------|----------|
|
|
987
1035
|
| Decay by default | Yes | No | No | No | No | No | No | No | No | No |
|
|
988
1036
|
| Retrieval strengthening | Yes | No | No | No | No | No | No | Partial (recall tuning) | No | Partial (Skill Memory distills patterns) |
|
|
@@ -1000,16 +1048,11 @@ The AI-memory category matured fast in 2026. Hippo's specific take (bio-decay, s
|
|
|
1000
1048
|
| Auto-hook install | Yes | No | No | No | No | No | No | No | No | No |
|
|
1001
1049
|
| MCP server | Yes | Yes | Yes (hosted, needs an account) | Yes | Yes (stdio + HTTP/OAuth) | Yes (hosted, needs an account) | Yes (hosted, needs an API key) | Yes (first-party Claude/LangGraph) | Yes | ? |
|
|
1002
1050
|
| Zero runtime deps | Yes | No (ChromaDB) | No | No | No (PGLite or PG+pgvector) | No (managed service) | No (npm deps) | No (Python deps) | Yes (single Rust binary) | No (managed + OSS) |
|
|
1003
|
-
| LongMemEval (best published) | 98.0% any / 88.5% all R@5\* (local MiniLM; 99.8% any-evidence with voyage-3-large; s_cleaned, per-haystack) | 96.6% raw / 100% reranked R@5 | 94.4 (hosted platform)\*\* | N/A | 95.53% all R@5\* (paid reranker; 93.19% without; s_cleaned) | 90.2% accuracy\*\* (LoCoMo 94.7%) | N/A | N/A | 88.78% accuracy\*\* (overall, with a reader) | 83.00% accuracy\*\* (overall; LoCoMo 93.05%, HaluMem 93.04%) |
|
|
1004
1051
|
| Git-friendly | Yes | No | No | Yes | Yes | No | Yes (memory tracked in git) | No | Yes (Git is the model) | ? |
|
|
1005
1052
|
| Framework agnostic | Yes | Yes | Partial | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
|
|
1006
1053
|
| License | MIT | (open) | Apache-2.0 | (open) | MIT | Proprietary cloud (Graphiti: Apache-2.0) | Apache-2.0 | MIT (core) | Apache-2.0 | Apache-2.0 (OSS) + cloud |
|
|
1007
1054
|
|
|
1008
|
-
\*
|
|
1009
|
-
|
|
1010
|
-
\*\* Different metric: these are end-to-end answer scores, not retrieval R@5. Mem0's 94.4 comes from its hosted platform, which its README says includes optimizations the open-source SDK lacks. Zep's 90.2% and 94.7% are accuracy figures from its homepage. Memoria's 88.78% and EverMind's 83% are overall accuracy with a reader LLM. Higher denominator + LLM helps. Not directly comparable to retrieval-only R@5 numbers above. The Mem0, Zep and Letta columns were last checked against each vendor's own pages on 2026-09-28.
|
|
1011
|
-
|
|
1012
|
-
Different tools answer different questions. Mem0 and Basic Memory implement "save everything, search later." MemPalace implements "store everything, organize spatially for retrieval." gbrain, Zep, and Cognee implement "extract typed entities and relationships into a knowledge graph." Letta implements "the agent edits its own memory blocks." Memoria implements "Git-style version control over the memory state itself." EverMind implements "self-evolving Skill Memory + multi-modal retrieval over hierarchical scopes." Hippo implements "learn what is wrong and stop repeating it." These are complementary takes, not a single-axis ranking: bio-lifecycle (Hippo) + GraphRAG (gbrain/Cognee/Zep) + agent-self-edit (Letta) + memory-VCS (Memoria) + skill-distillation (EverMind) cover different parts of the same problem.
|
|
1055
|
+
\* The Mem0, Zep and Letta columns were last checked against each vendor's own pages on 2026-09-28.
|
|
1013
1056
|
|
|
1014
1057
|
Side-by-side pages on the website: [hippo vs mem0](https://hippo-memory.com/vs/mem0/) · [hippo vs Zep](https://hippo-memory.com/vs/zep/) · [hippo vs Letta](https://hippo-memory.com/vs/letta/) · [CLAUDE.md vs MEMORY.md](https://hippo-memory.com/vs/claude-memory/).
|
|
1015
1058
|
|
|
@@ -1017,101 +1060,119 @@ Side-by-side pages on the website: [hippo vs mem0](https://hippo-memory.com/vs/m
|
|
|
1017
1060
|
|
|
1018
1061
|
## Benchmarks
|
|
1019
1062
|
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
### LongMemEval (retrieval accuracy)
|
|
1023
|
-
|
|
1024
|
-
[LongMemEval](https://arxiv.org/abs/2410.10813) (ICLR 2025) is the industry-standard benchmark: 500 questions across 5 memory abilities, embedded in 115k+ token chat histories.
|
|
1063
|
+
We built hippo and ran these tests, so read them as a vendor's numbers. We ran rivals on the same answer model where we could. Scores marked published come from each team's own setup, so we give no ranks. The same results are on [hippo-memory.com/benchmarks](https://hippo-memory.com/benchmarks/).
|
|
1025
1064
|
|
|
1026
|
-
|
|
1065
|
+
### Repeat mistakes in real Claude Code sessions
|
|
1027
1066
|
|
|
1028
|
-
|
|
1067
|
+
Each run corrects Claude Code once and then, 2 to 4 tasks later, gives it a task where the same mistake is possible. We ran it on 29 repositories, [registered first](docs/evals/2026-11-08-z0-round3-result.md).
|
|
1029
1068
|
|
|
1030
|
-
|
|
|
1031
|
-
|
|
1032
|
-
|
|
|
1033
|
-
|
|
|
1069
|
+
| Claude Code with | Repeats in 100 |
|
|
1070
|
+
|---|--:|
|
|
1071
|
+
| No memory | 83 |
|
|
1072
|
+
| Claude Code's built-in memory | 21 |
|
|
1073
|
+
| Built-in memory plus hippo | **11** |
|
|
1034
1074
|
|
|
1035
|
-
|
|
1075
|
+
The chart counts first chances 2 to 4 tasks after the correction. The registered measure, the first chance in each mistake family over 148 units, gives 14.9 points fewer repeats with hippo [5.5, 24.3], at the same cost, a ratio of 1.006 [0.964, 1.049]. On tasks with no lesson to apply, neither the cost nor the share solved changed.
|
|
1036
1076
|
|
|
1037
|
-
|
|
1077
|
+
What this run can't tell us is why. With lesson capture removed the result was the same, so we can't say hippo's lessons caused it. Round 4 tests that. The corrections were scripted, the agent was Claude Code only, and the test code and run records are not in the public repo. The round before was inconclusive, at 8.9 points fewer on 16 repositories.
|
|
1038
1078
|
|
|
1039
|
-
|
|
1040
|
-
- **Sleep lowered LongMemEval recall.** The slept store loses 3.6 points of hit@5 [-5.8, -1.4] to the never-slept one under the audit's declared scorer, and no scorer there shows sleep helping recall ([mechanism audit, round 2](docs/evals/2026-09-23-mechanism-audit-round2-result.md)).
|
|
1041
|
-
- **Outcome marks helped on the synthetic test.** Plain BM25 plus the fast outcome nudge drops trap persistence from 71.9% to 0.0% (round 2). Round 1 found outcome feedback and retrieval strengthening each help there, in the test's best case: every outcome mark is right, and every scheduled recall repeats the probe's query ([round 1](docs/evals/2026-09-23-mechanism-audit-result.md)). Supersession is not measured yet.
|
|
1079
|
+
### Official-setup rows side by side
|
|
1042
1080
|
|
|
1043
|
-
|
|
1081
|
+
Hippo 3.5.1 run with the same answer model, judge and settings as the published rows.
|
|
1044
1082
|
|
|
1045
|
-
|
|
1083
|
+
| Benchmark | hippo | Mem0 | Zep | MemOS | Supermemory | Honcho | Memobase | Other |
|
|
1084
|
+
|---|---|---|---|---|---|---|---|---|
|
|
1085
|
+
| **LoCoMo, Mem0 paper setup**<br><sub>Long chat recall, gpt-4o-mini answers; official setup</sub> | **69.22**<br><sub>embedder, 30 per speaker, all 1,540</sub> | [66.88](https://arxiv.org/html/2504.19413)<br><sub>gpt-4o-mini</sub> | [65.99](https://arxiv.org/html/2504.19413)<br><sub>gpt-4o-mini</sub> | - | - | - | - | [68.44](https://arxiv.org/html/2504.19413)<br><sub>Mem0g · gpt-4o-mini</sub> |
|
|
1086
|
+
| **LongMemEval-S, gpt-4o reader**<br><sub>Long chat recall, gpt-4o answers; official setup</sub> | **81.8**<br><sub>reranker, top 10 (embedder 78.2), official gpt-4o judge</sub> | - | [71.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o</sub> | - | [81.6](https://arxiv.org/html/2512.12818v1)<br><sub>gpt-4o</sub> | - | - | [86.0](https://www.emergence.ai/blog/sota-on-longmemeval-with-rag)<br><sub>EmergenceMem Internal · gpt-4o-2024-08-06</sub> |
|
|
1087
|
+
| **PersonaMem v1, MemOS suite setup**<br><sub>Personalisation, gpt-4o-mini answers; official setup</sub> | **61.97**<br><sub>embedder, top 20, mean of 3 runs</sub> | [43.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [57.8](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [61.2](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [53.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | [58.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.8](https://arxiv.org/html/2507.03724v4)<br><sub>MemU · gpt-4o-mini</sub> |
|
|
1088
|
+
| **EverMemBench, GPT-4.1-mini answers**<br><sub>Changing facts, GPT-4.1-mini answers; official setup, with a caveat</sub> | **44.20**<br><sub>embedder, top 10, mean of 9 categories</sub> | [37.09](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [39.97](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [42.55](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | - | - | [34.27](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [37.44](https://arxiv.org/html/2602.01313v3)<br><sub>Full Context · GPT-4.1-mini</sub> |
|
|
1089
|
+
| **HaluMem-Medium**<br><sub>Hallucination control, GPT-4o answers; official setup, with a caveat</sub> | **70.23**<br><sub>embedder, top 20, GPT-4o answers and judge</sub> | [53.02](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [55.47](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [67.23](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [54.07](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | - | [35.33](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [54.66](https://arxiv.org/html/2511.03506v3)<br><sub>Mem0-Graph · GPT-4o</sub> |
|
|
1090
|
+
| **HaluMem-Long**<br><sub>Hallucination control, GPT-4o answers; official setup, with a caveat</sub> | *paused* | [28.11](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [50.19](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [64.44](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [53.77](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | - | [33.60](https://arxiv.org/html/2511.03506v3)<br><sub>GPT-4o</sub> | [32.44](https://arxiv.org/html/2511.03506v3)<br><sub>Mem0-Graph · GPT-4o</sub> |
|
|
1091
|
+
| **DolphinBench, official Claude Code setup**<br><sub>Agent memory, Claude Code with Sonnet 5; official setup</sub> | *in build* | [32.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/mem0/README.md)<br><sub>Claude Code, Sonnet 5 · 194/600</sub> | - | - | - | [35.8](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/honcho/README.md)<br><sub>Claude Code, Sonnet 5 · 215/600</sub> | - | [26.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/builtin/README.md)<br><sub>Built-in · Claude Code, Sonnet 5 · 158/600</sub> |
|
|
1046
1092
|
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
| Recall@1 | 46.6% | 50.4% |
|
|
1050
|
-
| Recall@3 | **67.0%** | 66.6% |
|
|
1051
|
-
| Recall@5 | 73.8% | 74.0% |
|
|
1052
|
-
| Recall@10 | 81.0% | 82.6% |
|
|
1053
|
-
| Answer in content@5 | **49.6%** | 46.6% |
|
|
1054
|
-
|
|
1055
|
-
| Question Type | Count | R@5 | R@10 |
|
|
1056
|
-
|---------------|-------|-----|------|
|
|
1057
|
-
| single-session-assistant | 56 | 100.0% | 100.0% |
|
|
1058
|
-
| knowledge-update | 78 | 89.7% | 96.2% |
|
|
1059
|
-
| multi-session | 133 | 72.2% | 82.0% |
|
|
1060
|
-
| temporal-reasoning | 133 | 72.9% | 78.9% |
|
|
1061
|
-
| single-session-user | 70 | 62.9% | 71.4% |
|
|
1062
|
-
| single-session-preference | 30 | 20.0% | 33.3% |
|
|
1063
|
-
|
|
1064
|
-
For context: MemPalace scores 96.6% (raw) using ChromaDB embeddings + spatial indexing. Hippo v0.28 achieves 73.8% R@5 with hybrid BM25 + cosine. Hybrid scoring trades a little R@1 accuracy for better top-5 content relevance (answer_in_content@5 +3pp vs v0.11).
|
|
1065
|
-
|
|
1066
|
-
Hippo's strongest categories (single-session-assistant 100% R@5, knowledge-update 89.7%) are where keyword overlap between question and stored content is highest. The weakest (preference 20%) involves indirect references that need deeper semantic understanding.
|
|
1067
|
-
|
|
1068
|
-
> Note: v0.28 R@10 is 1.6pp below v0.11's BM25-only result. The earlier v0.27 benchmark showed an apparent 35pp regression; that was a methodology bug (budget-limited retrieval vs unlimited), fixed in v0.28 with the `minResults` option. See [`evals/README.md`](evals/README.md) for the full investigation and per-type breakdown.
|
|
1093
|
+
<details>
|
|
1094
|
+
<summary>How each row ran</summary>
|
|
1069
1095
|
|
|
1070
|
-
|
|
1071
|
-
|
|
1072
|
-
|
|
1073
|
-
|
|
1074
|
-
|
|
1075
|
-
|
|
1096
|
+
- **LoCoMo, Mem0 paper setup.** Run as the Mem0 paper ran it, with Mem0's released evaluation code (mem0ai/mem0 aae5989e78): all 1,540 questions in categories 1 to 4, 30 memories per speaker, gpt-4o-mini answers under the paper's prompt, and the code's gpt-4o-mini judge. The paper itself does not name its judge. One run. The paper's full-context baseline scores 72.9, above every memory row. Compare the overall score only: our category labels follow our own mapping.
|
|
1097
|
+
- **LongMemEval-S, gpt-4o reader.** The answer model and the judge match the published gpt-4o rows: gpt-4o-2024-08-06 reads hippo's top 10 memories, and the official script's prompts on gpt-4o-2024-08-06 judge all 500 questions. Each system keeps its own retrieval, as in the published rows. The cell shows the reranker setup, 81.8; the embedder setup scores 78.2. We chose the reranker after seeing its Sonnet scores and recorded that before its run. EmergenceMem does not name its judge.
|
|
1098
|
+
- **PersonaMem v1, MemOS suite setup.** Run as MemOS's evaluation suite runs every vendor row (MemTensor/MemOS 3a16e5fe): the same data files and top 20, PM_ANSWER_PROMPT as one system message to gpt-4o-mini-2024-07-18 at temperature 0, 3 runs per question, scored as the mean of the run accuracies over all 589 rows. Each system keeps its own retrieval and memory text.
|
|
1099
|
+
- **EverMemBench, GPT-4.1-mini answers.** The answer model, prompts, top 10, multiple-choice scorer and the 9-category mean match the paper's Table 4 (EverMind-AI/EverMemBench e10b3d5): gpt-4.1-mini-2025-04-14 at temperature 0 answers all 2,400 items. One setting differs: open answers are judged by gpt-5.6-sol through codex with the authors' prompts, and the repository's default judge is gemini-3-flash-preview; the paper names no judge. A second judge, gpt-4o-2024-08-06 with the same prompts, agrees with codex on 96.9% of open verdicts and scores hippo 44.75. The default setup scores 41.66 with codex and 42.50 with gpt-4o.
|
|
1100
|
+
- **HaluMem-Medium.** Question answering only, with the paper's GPT-4o answers and judge (gpt-4o-2024-08-06, temperature 0) and the same data, question set, top 20 and prompts. We do not produce the extraction and update scores, so we cannot enter the HaluMem board yet.
|
|
1101
|
+
- **HaluMem-Long.** Question answering only, with the paper's GPT-4o answers and judge (gpt-4o-2024-08-06, temperature 0), the same setup as the HaluMem-Medium row. The run paused when the OpenAI credit ran out on 16 Nov, so the row has no score yet. We do not produce the extraction and update scores, so we cannot enter the HaluMem board yet.
|
|
1102
|
+
- **DolphinBench, official Claude Code setup.** The same setup as the published Claude Code rows: one Claude Code turn with the hippo plugin for each of the 13,539 messages, each test in a fresh sandbox with memory writes off, Sonnet 5 and the gpt-5.6-sol judge. The run is in build, so there is no score yet.
|
|
1076
1103
|
|
|
1077
|
-
|
|
1104
|
+
</details>
|
|
1078
1105
|
|
|
1079
|
-
|
|
1106
|
+
### Our own setup
|
|
1107
|
+
|
|
1108
|
+
Every other benchmark we ran. Our setup differs from the published rows, so these scores do not sit beside theirs.
|
|
1109
|
+
|
|
1110
|
+
| Benchmark | hippo | Mem0 | Zep | MemOS | Supermemory | Honcho | Hindsight | EverMemOS | Memobase | Other |
|
|
1111
|
+
|---|---|---|---|---|---|---|---|---|---|---|
|
|
1112
|
+
| **LoCoMo**<br><sub>Long chat recall, Sonnet 5 answers</sub> | **91.5**<br><sub>embedder, top 200, gpt-4o-mini judge, 400 sample</sub> | [92.5](https://mem0.ai/research-3)<br><sub>not stated · unit not stated</sub> | [75.14](https://www.getzep.com/blog/lies-damn-lies-statistics-is-mem0-really-sota-in-agent-memory/)<br><sub>not stated</sub> | - | - | [89.9](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>Claude Haiku 4.5</sub> | [89.61](https://arxiv.org/html/2512.12818v1)<br><sub>Gemini-3 Pro</sub> | [93.05](https://evermind.ai/blogs/everos-sota-results-across-four-memory-benchmarks-and-what-it-means-for-llm-agents)<br><sub>not stated</sub> | [75.78](https://arxiv.org/html/2512.12818v1)<br><sub>not stated</sub> | [90.0](https://backboard.io/blog/best-ai-memory-score-in-the-world)<br><sub>Backboard · not stated</sub> |
|
|
1113
|
+
| **LongMemEval-S**<br><sub>Long chat recall, Sonnet 5 answers</sub> | **88.4**<br><sub>embedder, top 200, official gpt-4o judge</sub> | [94.4](https://mem0.ai/blog/locomo-benchmark)<br><sub>not stated · unit not stated</sub> | [71.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o</sub> | - | [85.2](https://arxiv.org/html/2512.12818v1)<br><sub>gemini-3-pro</sub> | [92.6](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>Gemini 3 Pro</sub> | [91.4](https://arxiv.org/html/2512.12818v1)<br><sub>Gemini-3 Pro</sub> | [83.00](https://evermind.ai/blogs/everos-sota-results-across-four-memory-benchmarks-and-what-it-means-for-llm-agents)<br><sub>not stated</sub> | - | [94.87](https://mastra.ai/research/observational-memory)<br><sub>Mastra Observational Memory · gpt-5-mini</sub> |
|
|
1114
|
+
| **LongMemEval-M**<br><sub>Very long chat recall, Sonnet 5 answers</sub> | **78.4**<br><sub>default, top 200, official gpt-4o judge</sub> | - | - | - | - | - | - | - | - | - |
|
|
1115
|
+
| **DMR**<br><sub>Multi-session recall, Sonnet 5 answers</sub> | **77.6**<br><sub>reranker, top 20</sub> | - | [98.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o-mini</sub> | - | - | - | - | - | - | [98.0](https://arxiv.org/html/2501.13956v1)<br><sub>Full-conversation baseline · gpt-4o-mini</sub> |
|
|
1116
|
+
| **DMR, gpt-4o-mini answers**<br><sub>Multi-session recall, gpt-4o-mini answers</sub> | **85.60**<br><sub>reranker, top 20, Speaker 1 role</sub> | - | [98.2](https://arxiv.org/html/2501.13956v1)<br><sub>gpt-4o-mini</sub> | - | - | - | - | - | - | [98.0](https://arxiv.org/html/2501.13956v1)<br><sub>Full-conversation baseline · gpt-4o-mini</sub> |
|
|
1117
|
+
| **PersonaMem v1**<br><sub>Personalisation, Sonnet 5 answers</sub> | **72.4**<br><sub>embedder, top 20</sub> | [43.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [57.8](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [61.2](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [53.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | - | - | [58.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.8](https://arxiv.org/html/2507.03724v4)<br><sub>MemU · gpt-4o-mini</sub> |
|
|
1118
|
+
| **PersonaMem v2**<br><sub>Personalisation, Sonnet 5 answers</sub> | **50.1**<br><sub>embedder, top 20</sub> | - | - | [40.58](https://raw.githubusercontent.com/MemTensor/MemOS/main/README.md)<br><sub>not stated</sub> | - | - | - | [53.25](https://evermind.ai/blogs/top-ai-memory-systems-benchmarked-in-2026)<br><sub>not stated</sub> | - | [55.2](https://arxiv.org/html/2512.06688)<br><sub>Qwen3-4B agentic memory · Qwen3-4B-Instruct-2507</sub> |
|
|
1119
|
+
| **PrefEval**<br><sub>Preference following, Sonnet 5 answers</sub> | **83.0**<br><sub>default, top 5</sub> | [63.7](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [39.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [71.9](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [56.7](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | - | - | - | [34.1](https://arxiv.org/html/2507.03724v4)<br><sub>gpt-4o-mini</sub> | [92.15](https://arxiv.org/html/2502.09597)<br><sub>Claude 3 Sonnet + RAG · Claude 3 Sonnet</sub> |
|
|
1120
|
+
| **EverMemBench**<br><sub>Changing facts, Sonnet 5 answers</sub> | **56.4**<br><sub>default, top 10, mean of 9 categories</sub> | [37.09](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [39.97](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [42.55](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | - | - | - | - | [34.27](https://arxiv.org/html/2602.01313v3)<br><sub>GPT-4.1-mini</sub> | [72.61](https://arxiv.org/html/2602.01313v3)<br><sub>Full Context · Gemini-3-Flash</sub> |
|
|
1121
|
+
| **BEAM-100K**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **72.4**<br><sub>embedder</sub> | - | - | - | - | [63.0](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [73.4](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [79.0](https://www.cognee.ai/benchmarking-cognee-on-beam)<br><sub>cognee · not stated</sub> |
|
|
1122
|
+
| **BEAM-500K**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **70.6**<br><sub>embedder</sub> | - | - | - | - | [64.9](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [71.1](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [35.9](https://arxiv.org/html/2510.27246)<br><sub>LIGHT · Llama-4-Maverick</sub> |
|
|
1123
|
+
| **BEAM-1M**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **69.9**<br><sub>default</sub> | [64.1](https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm)<br><sub>gpt-5</sub> | - | - | - | [63.1](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [73.9](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [33.6](https://arxiv.org/html/2510.27246)<br><sub>LIGHT · Llama-4-Maverick</sub> |
|
|
1124
|
+
| **BEAM-10M**<br><sub>Long-horizon memory, Sonnet 5 answers</sub> | **54.3**<br><sub>embedder</sub> | [48.6](https://mem0.ai/blog/mem0-the-token-efficient-memory-algorithm)<br><sub>gpt-5</sub> | - | - | - | [40.6](https://espirai-blog.pages.dev/research/Benchmarking-Honcho)<br><sub>claude-haiku-4-5</sub> | [64.1](https://hindsight.vectorize.io/blog/2026/04/02/beam-sota)<br><sub>not stated</sub> | - | - | [up to 67.0](https://www.cognee.ai/benchmarking-cognee-on-beam)<br><sub>cognee · not stated</sub> |
|
|
1125
|
+
| **ConvoMem**<br><sub>Conversation memory, Sonnet 5 answers</sub> | **59.6**<br><sub>default, top 20</sub> | - | - | - | - | - | - | - | - | [92.9](https://vectorize.io/articles/mempalace-benchmarks)<br><sub>MemPalace · not stated</sub> |
|
|
1126
|
+
| **HotPotQA-24**<br><sub>Multi-hop questions, F1, Sonnet 5 answers</sub> | **91.0**<br><sub>reranker, top 15, all 24 questions, one run</sub> | [12.0](https://www.cognee.ai/knowledge-graph-memory-benchmarks)<br><sub>not stated</sub> | - | - | - | - | - | - | - | [84.1](https://www.cognee.ai/knowledge-graph-memory-benchmarks)<br><sub>Cognee · not stated</sub> |
|
|
1127
|
+
| **DolphinBench, shortcut run**<br><sub>Agent memory, Claude Code with Sonnet 5</sub> | **45.0**<br><sub>270/600 · Claude Code, Sonnet 5</sub> | [32.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/mem0/README.md)<br><sub>Claude Code, Sonnet 5 · 194/600</sub> | - | - | - | [35.8](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/honcho/README.md)<br><sub>Claude Code, Sonnet 5 · 215/600</sub> | - | - | - | [26.3](https://github.com/mem0ai/dolphinbench/blob/81cb6f8405b40a9e76089cef650806a80af06ea2/results/claude-code/claude-sonnet-5/builtin/README.md)<br><sub>Built-in · Claude Code, Sonnet 5 · 158/600</sub> |
|
|
1128
|
+
| **Terminal-Bench 2.0**<br><sub>Agent coding tasks, Claude Code with Sonnet 5</sub> | **8/10**<br><sub>first 10 of 89 tasks, 1 trial; Claude Code with no memory 9/10</sub> | - | - | - | - | - | - | - | - | - |
|
|
1080
1129
|
|
|
1081
|
-
|
|
1130
|
+
<details>
|
|
1131
|
+
<summary>How each row ran</summary>
|
|
1132
|
+
|
|
1133
|
+
- **LoCoMo.** Judged again with the Mem0 paper's prompt on gpt-4o-mini, the judge in Mem0's released code. This row still scores a 400-question sample with Sonnet 5 answers, and the published rows use all 1,540 questions. The "LoCoMo, Mem0 paper setup" row runs that setup in full.
|
|
1134
|
+
- **LongMemEval-S.** Judged with the official script's prompt for each question type and its judge, gpt-4o-2024-08-06, over all 500 questions. One setting differs: the answer model is Sonnet 5, and each published row uses its own. The Mem0 cell used its runner's lenient judge prompt, so it stays grey. The "LongMemEval-S, gpt-4o reader" row uses a gpt-4o reader.
|
|
1135
|
+
- **LongMemEval-M.** Judged with the official script's prompts and its gpt-4o-2024-08-06 judge, over all 500 questions. The answer model is Sonnet 5. No product publishes an answer score for M.
|
|
1136
|
+
- **DMR.** Our harness caps the score: the whole chat scores 83.6% here, against 94% to 98% in the Zep paper.
|
|
1137
|
+
- **DMR, gpt-4o-mini answers.** gpt-4o-mini answers, as in Zep's rows, and the prompt now says the answerer is Speaker 1. The harness check failed: the whole chat scores 92.4 here, against 98.0 for the paper's full-conversation row with the same model, so our setup still caps the score. Compare each system with the whole chat in its own setup only: Zep +0.2 points, hippo's reranker setup -6.8 points. Neither paper names its judge.
|
|
1138
|
+
- **PersonaMem v1.** The vendor rows all answer with gpt-4o-mini under MemOS's prompt, one model for every system. Against the authors' own board of full-context models (best 52%), our score compares with a caveat. A rule that picks the longest option scores 48.9%.
|
|
1139
|
+
- **PersonaMem v2.** The paper's row uses the same text split and scorer with another answer model. The MemOS and EverMemOS rows do not say how they ran. A rule that picks the longest option scores 45.0%.
|
|
1140
|
+
- **PrefEval.** The paper's RAG rows keep the whole chat in the prompt; ours shows 5 retrieved turns. The vendor rows use another data split, top 10 and gpt-4o-mini.
|
|
1141
|
+
- **EverMemBench.** Every published row answers with GPT-4.1-mini, one model for every system, and ours answers with Sonnet 5.
|
|
1142
|
+
- **BEAM-100K.** Hindsight uses another answer prompt and Gemini models with a budget it does not state. Honcho scores event ordering another way. cognee ran one conversation. Mem0 publishes no 100K score.
|
|
1143
|
+
- **BEAM-500K.** Hindsight uses another answer prompt and Gemini models with a budget it does not state. Honcho scores event ordering another way. The LIGHT row comes from the paper's scorer.
|
|
1144
|
+
- **BEAM-1M.** Mem0's row uses the same runner code, prompts and scorer. Mem0 answered and judged with gpt-5 and showed 200 memories; we use Sonnet 5, gpt-5.6-sol and 100. The other rows use other scorers or setups.
|
|
1145
|
+
- **BEAM-10M.** Mem0's row uses the same runner code, prompts and scorer. Mem0 answered and judged with gpt-5 and showed 200 memories; we use Sonnet 5, gpt-5.6-sol and 100. The other rows use other scorers or setups.
|
|
1146
|
+
- **ConvoMem.** The sample design is ours, and no published number exists at 50 conversations. The MemPalace row is a 300-item sample with no stated model or judge.
|
|
1147
|
+
- **HotPotQA-24.** F1 over all 24 questions with Cognee's own scorer, from one run with Sonnet 5 answers. Cognee averages 45 runs and names no answer model, and one question moves the score by about 4 points.
|
|
1148
|
+
- **DolphinBench, shortcut run.** Our run wrote each message straight to the store with no agent turn, used a recall hook we wrote, and judged through codex, which the official validator rejects.
|
|
1149
|
+
- **Terminal-Bench 2.0.** The first 10 of 89 tasks, 1 trial each at medium effort; the rule is 5 trials, and the matching Claude Code row used max effort. The 2.0 board is closed, and the open board is 4.1.
|
|
1082
1150
|
|
|
1083
|
-
|
|
1084
|
-
|----------|--:|------------------:|
|
|
1085
|
-
| single-hop | 282 | 0.239 |
|
|
1086
|
-
| multi-hop | 321 | 0.491 |
|
|
1087
|
-
| temporal-reasoning | 92 | 0.169 |
|
|
1088
|
-
| open-domain | 841 | 0.450 |
|
|
1089
|
-
| adversarial | 446 | 0.226 |
|
|
1090
|
-
| **overall** | 1,982 | **0.363** |
|
|
1151
|
+
</details>
|
|
1091
1152
|
|
|
1092
|
-
|
|
1153
|
+
### Finding the right session
|
|
1093
1154
|
|
|
1094
|
-
|
|
1155
|
+
On LongMemEval-S, hippo's benchmark scripts put an answer session in the top five for 98.0% of 500 questions with the free local MiniLM embedder. That is the best of five settings, chosen after the run, on a measure close to its ceiling. Requiring every answer session, the same runs score 86.8 to 88.5%, and gbrain is ahead at 95.53% with a paid reranker. The `hippo recall` command on a default install scores 85.6% [82.4, 88.6]. On one store of all 19,195 sessions, recall fell to 47.2% in June 2026.
|
|
1095
1156
|
|
|
1096
|
-
|
|
1157
|
+
### What the two rerankers bought
|
|
1097
1158
|
|
|
1098
|
-
50
|
|
1159
|
+
`--reranker cross-encoder` reorders the top 50 memories with a free model on your machine, and is off by default. [On DMR](benchmarks/public/results/2026-11-12-answer-accuracy/dmr-best.txt) it lifted hippo's answers from 73.1 with the embedder alone to 77.6, by 4.4 points [0.6, 8.3]. On PersonaMem v1 and HotPotQA-24 the change was within the noise.
|
|
1099
1160
|
|
|
1100
|
-
|
|
1161
|
+
`--reranker jev` sends the top candidates to the hosted TypeSafe Jev API, and is off by default. On 300 developer queries from a private store, the right memory ranked first for 62% of them, against 41% with the free local cross-encoder. [Three graded-answer tests](docs/evals/2026-09-19-jev-reranker.md) showed no better answer rate. It buys a shorter context for about 0.0004 USD a recall, and it sends your query and candidate text to a third party.
|
|
1101
1162
|
|
|
1102
|
-
|
|
1103
|
-
<summary>Original v0.11.0 informal numbers (RETRACTED, preserved as audit trail in git, not reproduced here)</summary>
|
|
1163
|
+
### What didn't work
|
|
1104
1164
|
|
|
1105
|
-
|
|
1165
|
+
- Sleep lowered recall by 3.6 points [1.4, 5.8] on one scorer and raised it a little on another, so we claim no benefit for it.
|
|
1166
|
+
- Decay made no measurable difference, on a synthetic test that gives a 365-day half-life little time to act.
|
|
1167
|
+
- The old 7-day default was a bug that cost recall, so the default moved to 365 days.
|
|
1168
|
+
- Seven smaller experiments failed, stalled or were thrown out, among them a correction detector at 0.82 precision against a 0.90 bar.
|
|
1169
|
+
- One early headline number did not reproduce, and [we retracted it](docs/RETRACTION.md).
|
|
1106
1170
|
|
|
1107
|
-
|
|
1171
|
+
### Check our work
|
|
1108
1172
|
|
|
1109
|
-
The
|
|
1173
|
+
The score files for five of these public tests and their write-ups are in the repo, under `benchmarks/public`.
|
|
1110
1174
|
|
|
1111
|
-
|
|
1112
|
-
cd benchmarks/sequential-learning
|
|
1113
|
-
node run.mjs --adapter all
|
|
1114
|
-
```
|
|
1175
|
+
[Benchmark results](benchmarks/public) · [Eval documents](docs/evals/README.md) · [Report a mistake](https://github.com/kitfunso/hippo-memory/issues/new)
|
|
1115
1176
|
|
|
1116
1177
|
---
|
|
1117
1178
|
|
|
@@ -1180,7 +1241,7 @@ On repeat mistakes, yes, in [one test of our own](https://github.com/kitfunso/hi
|
|
|
1180
1241
|
Issues and PRs welcome. Before contributing, run `hippo status` in the repo root to see the project's own memory.
|
|
1181
1242
|
|
|
1182
1243
|
The interesting problems:
|
|
1183
|
-
- **
|
|
1244
|
+
- **Recall as one store grows.** On LongMemEval-S, `hippo recall` on a default install scores 85.6% [82.4, 88.6] ([result](docs/evals/2026-09-28-recall-cli-longmemeval-result.md)), and on one store of all 19,195 sessions recall fell to 47.2% in June 2026. Requiring every answer session in the top five, the benchmark scripts score 86.8 to 88.5%, and gbrain is ahead at 95.53%. The lifecycle stress eval (ROADMAP Part III) is the next measurement.
|
|
1184
1245
|
- Better consolidation heuristics (LLM-powered merge vs current text overlap)
|
|
1185
1246
|
- Web UI / dashboard for visualizing decay curves and memory health
|
|
1186
1247
|
- Optimal decay parameter tuning from real usage data
|
|
@@ -1190,9 +1251,11 @@ The interesting problems:
|
|
|
1190
1251
|
|
|
1191
1252
|
## Open source and commercial
|
|
1192
1253
|
|
|
1193
|
-
Hippo is open core. This repository
|
|
1254
|
+
Hippo is open core. This repository is the MIT core for individual developers and self-hosted teams: the CLI, the MCP server, the supported hooks and adapters, and the dashboard. It includes an HTTP server with tenants, API keys, admin and member roles, per-key scope grants and an audit log on every change. Keys are stored scrypt-hashed, and one API call deletes a memory. Slack and GitHub connectors bring in what your team has already written down, with Jira and Notion connectors next. Automatic capture and delivery depend on the configured integration. Code published here stays MIT and stays here.
|
|
1255
|
+
|
|
1256
|
+
The fair test for a team is a holdout, where a random share of sessions runs without memory and you compare cost per merged PR and repeat errors between the two groups. We have no team result to show. If your team wants to try, [open an issue](https://github.com/kitfunso/hippo-memory/issues/new) and tell us what you'd need. More on [hippo-memory.com/teams](https://hippo-memory.com/teams/).
|
|
1194
1257
|
|
|
1195
|
-
The commercial edition is planned; its private repository is a scaffold, not a released enterprise product. It is intended for larger companies as a separate package under a commercial licence from KITFUNSO LTD. Planned capabilities include SSO (OIDC and SAML sign-in), SCIM, organisation/team/project policy, an org admin view, the pilot report and telemetry join, SIEM export of the audit log, offline licence keys, serving agent hooks to laptops that keep no store of their own, hosted SaaS, and support with an SLA. Pull requests implementing those commercial features belong there; see [CONTRIBUTING.md](CONTRIBUTING.md). Release availability and independently verified benefit are separate milestones in the [roadmap](https://github.com/kitfunso/hippo-memory/blob/master/ROADMAP.md#current-execution-index).
|
|
1258
|
+
The commercial edition is planned; its private repository is a scaffold, not a released enterprise product. None of it is released. It is intended for larger companies as a separate package under a commercial licence from KITFUNSO LTD. Planned capabilities include SSO (OIDC and SAML sign-in), SCIM, organisation/team/project policy, an org admin view, the pilot report and telemetry join, SIEM export of the audit log, offline licence keys, serving agent hooks to laptops that keep no store of their own, hosted SaaS, and support with an SLA. Pull requests implementing those commercial features belong there; see [CONTRIBUTING.md](CONTRIBUTING.md). Release availability and independently verified benefit are separate milestones in the [roadmap](https://github.com/kitfunso/hippo-memory/blob/master/ROADMAP.md#current-execution-index).
|
|
1196
1259
|
|
|
1197
1260
|
## License
|
|
1198
1261
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
* Single source of truth for the package version, written by scripts/sync-version.mjs; never
|
|
3
3
|
* edit by hand. Read by the db rollback guard, /health, MCP. A constant, not a runtime
|
|
4
4
|
* package.json read: the published bundle may lack package.json on a path ESM can resolve. */
|
|
5
|
-
export declare const PACKAGE_VERSION = "3.7.
|
|
5
|
+
export declare const PACKAGE_VERSION = "3.7.1";
|
|
6
6
|
/** The floor a store takes on its first expiring key: the first release with schema v53, set by
|
|
7
7
|
* hand in that release; scripts/checks/check-expiring-keys-floor.mjs gates it. */
|
|
8
8
|
export declare const EXPIRING_KEYS_MIN_BINARY = "1.64.0";
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
* Single source of truth for the package version, written by scripts/sync-version.mjs; never
|
|
3
3
|
* edit by hand. Read by the db rollback guard, /health, MCP. A constant, not a runtime
|
|
4
4
|
* package.json read: the published bundle may lack package.json on a path ESM can resolve. */
|
|
5
|
-
export const PACKAGE_VERSION = '3.7.
|
|
5
|
+
export const PACKAGE_VERSION = '3.7.1';
|
|
6
6
|
/** The floor a store takes on its first expiring key: the first release with schema v53, set by
|
|
7
7
|
* hand in that release; scripts/checks/check-expiring-keys-floor.mjs gates it. */
|
|
8
8
|
export const EXPIRING_KEYS_MIN_BINARY = '1.64.0';
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
export declare function printAuditPruneUsage(): void;
|
|
2
2
|
export declare function printSlackBackfillUsage(): void;
|
|
3
3
|
export declare function printSlackWorkspacesUsage(): void;
|
|
4
|
-
export declare const USAGE_HEADER = "\nHippo -
|
|
4
|
+
export declare const USAGE_HEADER = "\nHippo - One memory for every agent you'll ever use.\n\nUsage: hippo <command> [options]\n\nCommands:";
|
|
5
5
|
export declare const USAGE_EXAMPLES = "\n\nExamples:\n hippo init\n hippo remember \"FRED cache can silently drop series\" --tag error\n hippo recall \"data pipeline issues\" --budget 2000\n hippo context --auto --budget 1500\n hippo conflicts\n hippo reject mem_abc123 --reason \"leaked credential\"\n hippo reject --value \"never store my key again\" --reason \"secret\"\n hippo rejections\n hippo unreject a1b2c3d4e5f6\n hippo dormant \"staging hostname\"\n hippo dormant restore mem_abc123\n hippo tokens --days 7\n hippo session log --id sess_123 --task \"Ship feature\" --type progress --content \"Build is green, next step is docs\"\n hippo session latest --json\n hippo session resume\n hippo snapshot save --task \"Ship feature\" --summary \"Tests are green\" --next-step \"Open the PR\" --session sess_123\n hippo handoff create --summary \"PR is open, tests green\" --next \"Merge after review\" --session sess_123 --artifact src/foo.ts\n hippo handoff create --summary s --constraint a --constraint b --outcome partial --target-runtime codex --card-id c1 --tests pass\n hippo card create --title \"Add cards table\" --repo hippo --depends-on card_abc\n hippo card claim card_abc --runtime codex\n hippo card complete card_abc --outcome success\n hippo card heartbeat card_abc --run 7\n hippo card reclaim\n hippo embed --status\n hippo watch -- npm run build\n hippo learn --git --days 30\n hippo promote mem_abc123\n hippo sync\n hippo setup\n hippo hook install claude-code\n hippo decide \"Use PostgreSQL for new services\" --context \"JSONB support\"\n hippo incident \"Prod outage: DB connection pool exhausted\" --context \"spike at 14:00\"\n hippo process new \"Release\" --step \"run tests\" --step \"bump version\" --step \"publish\"\n hippo policy new \"Data retention\" --text \"Delete logs after 90 days\" --from 2026-01-01\n hippo policy asof 2026-03-01 --name \"Data retention\"\n hippo skill new \"Run tests\" --instructions \"npm test before every commit\" --trigger \"before commit\"\n hippo skill export\n hippo brief new \"hippo\" --summary \"Agent-memory library; E2 first-class objects in progress\"\n hippo brief refresh \"hippo\"\n hippo note new \"Acme Corp\" --text \"Renewal call: wants SSO before Q3; champion is the VP Eng\"\n hippo note list --customer \"Acme Corp\" --status active\n hippo graph extract\n hippo invalidate \"REST API\" --dry-run\n hippo invalidate \"REST API\" --reason \"migrated to GraphQL\"\n hippo invalidate --id mem_a1b2c3d4e5f6 --reason \"superseded by new policy\"\n hippo invalidate --churn --dry-run\n hippo export memories.json\n hippo export --format markdown memories.md\n hippo sleep --dry-run\n hippo outcome --good\n hippo status\n";
|
|
6
6
|
//# sourceMappingURL=usage.d.ts.map
|
|
@@ -19,7 +19,7 @@ export function printSlackWorkspacesUsage() {
|
|
|
19
19
|
console.log(' remove --team <T> Remove a workspace registration');
|
|
20
20
|
}
|
|
21
21
|
export const USAGE_HEADER = `
|
|
22
|
-
Hippo -
|
|
22
|
+
Hippo - One memory for every agent you'll ever use.
|
|
23
23
|
|
|
24
24
|
Usage: hippo <command> [options]
|
|
25
25
|
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Hippo Memory - OpenClaw Plugin
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
One memory for every agent you'll ever use. Hippo writes what you teach a coding agent into a store in your repo, and your other agents read that same store.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
The plugin puts your project's memories in front of the OpenClaw agent at session start, saves the errors its tools hit and gives it hippo's memory tools. Claude Code, Codex, Cursor and any MCP client read the same store.
|
|
6
6
|
|
|
7
7
|
## Install
|
|
8
8
|
|
|
@@ -101,17 +101,9 @@ When `autoLearn` is enabled, the plugin captures tool errors as memories. To pre
|
|
|
101
101
|
|
|
102
102
|
Errors that pass these three filters are stored with `hippo remember`.
|
|
103
103
|
|
|
104
|
-
###
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|---|---|---|
|
|
108
|
-
| Decay | Yes, 365-day half-life by default | No, saves everything |
|
|
109
|
-
| Retrieval strengthening | Yes | No |
|
|
110
|
-
| Outcome feedback | Yes | No |
|
|
111
|
-
| Cross-tool | Claude Code, Codex, Cursor, OpenClaw | Claude Code only |
|
|
112
|
-
| API calls | None by default. If `ANTHROPIC_API_KEY` is set, `hippo sleep` sends memory text to Anthropic; `{"extraction":{"enabled":false}}` stops it | Uses Claude API for compression |
|
|
113
|
-
| Token cost | ~1500 tokens/session (configurable) | Variable |
|
|
114
|
-
| Memecoin | No | Yes ($CMEM on Solana) |
|
|
104
|
+
### Model calls
|
|
105
|
+
|
|
106
|
+
Storing a memory makes no model call. The exception is `hippo sleep`, which sends text to Anthropic when `ANTHROPIC_API_KEY` is set, and `{"extraction":{"enabled":false}}` turns it off.
|
|
115
107
|
|
|
116
108
|
## Requirements
|
|
117
109
|
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"id": "hippo-memory",
|
|
3
3
|
"name": "Hippo Memory",
|
|
4
|
-
"description": "
|
|
5
|
-
"version": "3.7.
|
|
4
|
+
"description": "One memory for every agent you'll ever use. Hippo writes what you teach a coding agent into a store in your repo, and your other agents read that same store.",
|
|
5
|
+
"version": "3.7.1",
|
|
6
6
|
|
|
7
7
|
"configSchema": {
|
|
8
8
|
"type": "object",
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "hippo-memory",
|
|
3
|
-
"version": "3.7.
|
|
3
|
+
"version": "3.7.1",
|
|
4
4
|
"type": "module",
|
|
5
|
-
"description": "
|
|
5
|
+
"description": "One memory for every agent you'll ever use. Hippo writes what you teach a coding agent into a store in your repo, and your other agents read that same store.",
|
|
6
6
|
"main": "index.ts",
|
|
7
7
|
"openclaw": {
|
|
8
8
|
"extensions": [
|
package/openclaw.plugin.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"id": "hippo-memory",
|
|
3
3
|
"name": "Hippo Memory",
|
|
4
|
-
"description": "
|
|
5
|
-
"version": "3.7.
|
|
4
|
+
"description": "One memory for every agent you'll ever use. Hippo writes what you teach a coding agent into a store in your repo, and your other agents read that same store.",
|
|
5
|
+
"version": "3.7.1",
|
|
6
6
|
"configSchema": {
|
|
7
7
|
"type": "object",
|
|
8
8
|
"additionalProperties": false,
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "hippo-memory",
|
|
3
|
-
"version": "3.7.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "3.7.1",
|
|
4
|
+
"description": "One memory for every agent you'll ever use. Hippo writes what you teach a coding agent into a store in your repo, and your other agents read that same store.",
|
|
5
5
|
"mcpName": "io.github.kitfunso/hippo-memory",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"main": "./dist/index.js",
|
|
@@ -52,7 +52,7 @@
|
|
|
52
52
|
"fmt": "oxfmt src tests scripts bin",
|
|
53
53
|
"fmt:check": "oxfmt --check src tests scripts bin",
|
|
54
54
|
"sbom": "node scripts/sbom.mjs",
|
|
55
|
-
"audit:security": "
|
|
55
|
+
"audit:security": "node scripts/checks/check-audit.mjs",
|
|
56
56
|
"smoke:pack": "node scripts/smoke-pack.mjs",
|
|
57
57
|
"smoke:openclaw-install": "node scripts/smoke-openclaw-install.mjs",
|
|
58
58
|
"build:ui": "cd ui && npm ci && npm run build",
|
|
@@ -81,12 +81,11 @@
|
|
|
81
81
|
"coding-agent",
|
|
82
82
|
"agents",
|
|
83
83
|
"llm",
|
|
84
|
-
"rag",
|
|
85
84
|
"retrieval",
|
|
86
|
-
"hybrid-search",
|
|
87
85
|
"embeddings",
|
|
88
86
|
"sqlite",
|
|
89
87
|
"local-first",
|
|
88
|
+
"open-source",
|
|
90
89
|
"cli",
|
|
91
90
|
"context",
|
|
92
91
|
"recall"
|