agent-working-memory 0.14.1 → 0.14.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/README.md +240 -653
  2. package/dist/adapters/claude-code.d.ts.map +1 -1
  3. package/dist/adapters/claude-code.js +234 -131
  4. package/dist/adapters/claude-code.js.map +1 -1
  5. package/dist/adapters/common.d.ts +44 -5
  6. package/dist/adapters/common.d.ts.map +1 -1
  7. package/dist/adapters/common.js +111 -41
  8. package/dist/adapters/common.js.map +1 -1
  9. package/dist/adapters/hook-scripts.d.ts +54 -0
  10. package/dist/adapters/hook-scripts.d.ts.map +1 -0
  11. package/dist/adapters/hook-scripts.js +340 -0
  12. package/dist/adapters/hook-scripts.js.map +1 -0
  13. package/dist/adapters/types.d.ts +13 -3
  14. package/dist/adapters/types.d.ts.map +1 -1
  15. package/dist/cli.js +21 -5
  16. package/dist/cli.js.map +1 -1
  17. package/dist/core/agent-id.d.ts +19 -0
  18. package/dist/core/agent-id.d.ts.map +1 -0
  19. package/dist/core/agent-id.js +28 -0
  20. package/dist/core/agent-id.js.map +1 -0
  21. package/dist/core/whoami.d.ts +9 -1
  22. package/dist/core/whoami.d.ts.map +1 -1
  23. package/dist/core/whoami.js +12 -2
  24. package/dist/core/whoami.js.map +1 -1
  25. package/dist/engine/activation.d.ts +2 -0
  26. package/dist/engine/activation.d.ts.map +1 -1
  27. package/dist/engine/activation.js +18 -3
  28. package/dist/engine/activation.js.map +1 -1
  29. package/dist/engine/eval.d.ts +15 -0
  30. package/dist/engine/eval.d.ts.map +1 -1
  31. package/dist/engine/eval.js +23 -0
  32. package/dist/engine/eval.js.map +1 -1
  33. package/dist/hooks/prime.d.ts +10 -4
  34. package/dist/hooks/prime.d.ts.map +1 -1
  35. package/dist/hooks/prime.js +16 -6
  36. package/dist/hooks/prime.js.map +1 -1
  37. package/dist/hooks/sidecar.d.ts +18 -2
  38. package/dist/hooks/sidecar.d.ts.map +1 -1
  39. package/dist/hooks/sidecar.js +30 -4
  40. package/dist/hooks/sidecar.js.map +1 -1
  41. package/dist/mcp.js +48 -19
  42. package/dist/mcp.js.map +1 -1
  43. package/dist/storage/pglite.d.ts +7 -0
  44. package/dist/storage/pglite.d.ts.map +1 -1
  45. package/dist/storage/pglite.js +17 -3
  46. package/dist/storage/pglite.js.map +1 -1
  47. package/dist/storage/postgres.d.ts +7 -0
  48. package/dist/storage/postgres.d.ts.map +1 -1
  49. package/dist/storage/postgres.js +17 -3
  50. package/dist/storage/postgres.js.map +1 -1
  51. package/dist/storage/sqlite.d.ts +12 -0
  52. package/dist/storage/sqlite.d.ts.map +1 -1
  53. package/dist/storage/sqlite.js +24 -3
  54. package/dist/storage/sqlite.js.map +1 -1
  55. package/dist/types/engram.d.ts +16 -0
  56. package/dist/types/engram.d.ts.map +1 -1
  57. package/dist/types/eval.d.ts +2 -0
  58. package/dist/types/eval.d.ts.map +1 -1
  59. package/package.json +94 -72
  60. package/src/adapters/claude-code.ts +223 -125
  61. package/src/adapters/common.ts +117 -42
  62. package/src/adapters/hook-scripts.ts +370 -0
  63. package/src/adapters/types.ts +14 -3
  64. package/src/cli.ts +19 -5
  65. package/src/core/agent-id.ts +28 -0
  66. package/src/core/whoami.ts +11 -1
  67. package/src/engine/activation.ts +18 -3
  68. package/src/engine/eval.ts +34 -0
  69. package/src/hooks/prime.ts +22 -7
  70. package/src/hooks/sidecar.ts +50 -6
  71. package/src/mcp.ts +50 -18
  72. package/src/storage/pglite.ts +22 -4
  73. package/src/storage/postgres.ts +22 -4
  74. package/src/storage/sqlite.ts +26 -4
  75. package/src/types/engram.ts +16 -0
  76. package/src/types/eval.ts +3 -1
package/README.md CHANGED
@@ -1,653 +1,240 @@
1
- # AgentWorkingMemory (AWM)
2
-
3
- **Persistent working memory for AI agents.**
4
-
5
- AWM helps agents retain important project knowledge across conversations and sessions. Instead of storing everything and retrieving by similarity alone, it filters for salience, builds associative links between related memories, and periodically consolidates useful knowledge while letting noise fade.
6
-
7
- Use it through Claude Code via MCP or as a local HTTP service for custom agents. Everything runs locally: SQLite + ONNX models + Node.js. No cloud, no API keys.
8
-
9
- ### Without AWM
10
- - Agent forgets earlier architecture decision
11
- - Suggests Redux after project standardized on Zustand
12
- - Repeats discussion already settled three days ago
13
- - Every new conversation starts from scratch
14
-
15
- ### With AWM
16
- - Recalls prior state-management decision and rationale
17
- - Surfaces related implementation patterns from past sessions
18
- - Continues work without re-asking for context
19
- - Gets more consistent the longer you use it
20
-
21
- ---
22
-
23
- ## Quick Start
24
-
25
- **Node.js 22 LTS+** required — check with `node --version`. (Node 20 reached EOL 2026-04-30; AWM 0.8.6+ requires 22.)
26
-
27
- ```bash
28
- npm install -g agent-working-memory
29
- awm setup --global
30
- ```
31
-
32
- Restart Claude Code. That's it — 19 tools appear automatically (17 memory + 2 onboarding).
33
-
34
- ### Upgrading
35
-
36
- ```bash
37
- npm install -g agent-working-memory@latest
38
- awm setup --global # Updates MCP config, CLAUDE.md instructions, and hooks
39
- ```
40
-
41
- Restart Claude Code after upgrading. Your existing memory database is preserved — all upgrades are backward compatible. New features (metadata tags, workspace recall, synthesis) are opt-in.
42
-
43
- > **From v0.6.x → v0.7.x:** The `memory_write` tool now accepts optional metadata parameters (`project`, `topic`, `session_id`, etc.) that improve recall quality. Re-running `awm setup --global` updates your CLAUDE.md with instructions for the agent to use them.
44
-
45
- First conversation will be ~30 seconds slower while ML models download (~200MB total, cached locally). After that, everything runs on your machine.
46
-
47
- > For isolated memory per folder, see [Separate Memory Pools](#separate-memory-pools). For team onboarding, see [docs/quickstart.md](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/quickstart.md).
48
-
49
- > **Starting on an existing project?** Warm-start the store from its own docs so recall is
50
- > useful immediately: `awm onboard ./docs --repo . --project <name>` → review the pack →
51
- > `awm import <pack> --db <path> --dedupe`. See [What's New in v0.11.0](#whats-new-in-v0110).
52
-
53
- ---
54
-
55
- ## Who this is for
56
-
57
- - **Long-running coding agents** that need cross-session project knowledge
58
- - **Multi-agent workflows** where specialized agents share a common memory
59
- - **Local-first setups** where cloud memory is not acceptable
60
- - **Teams using Claude Code** who want persistent context without manual notes
61
-
62
- ## What this is not
63
-
64
- - Not a chatbot UI
65
- - Not a hosted SaaS
66
- - Not a generic vector database
67
- - Not a replacement for your source of truth (code, docs, tickets)
68
-
69
- ---
70
-
71
- ## Why it's different
72
-
73
- > **New to the vocabulary?** Terms like *engram, salience, activation, Hebbian, staging*
74
- > are defined plainly, one paragraph each, in
75
- > [`docs/onboarding-vocabulary.md`](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/onboarding-vocabulary.md) — a 5-minute read if any of the table below is unfamiliar.
76
-
77
- Most "memory for AI" projects are vector databases with a retrieval wrapper. AWM goes further:
78
-
79
- | | Typical RAG / Vector Store | AWM |
80
- |---|---|---|
81
- | **Storage** | Everything | Salience-filtered with low-confidence fallback (novel events go active, borderline enter staging, low-salience stored at reduced confidence) |
82
- | **Retrieval** | Cosine similarity | 10-phase pipeline: dual BM25 (keyword + expanded) + vectors + reranking + graph walk + decay + coref expansion |
83
- | **Named things** | Vocabulary-dependent — misses if the query doesn't lexically match | Entity inverted index: exact lookup on named entities ("ticket 19252", a person's name), immune to phrasing mismatch |
84
- | **Connections** | None | Hebbian edges that strengthen when memories co-activate |
85
- | **Over time** | Grows forever, gets noisier | Consolidation: diameter-enforced clustering, cross-topic bridges, synaptic-tagged decay |
86
- | **Forgetting** | Manual cleanup | Cognitive forgetting: unused memories fade, reinforced knowledge persists (access-count modulated) |
87
- | **Feedback** | None | Useful/not-useful signals tune confidence and retrieval rank |
88
- | **Correction** | Delete and re-insert | Retraction: wrong memories invalidated, corrections linked, penalties propagate (depth 2, decaying) |
89
- | **Graph** | None or single graph | Multi-graph: semantic, temporal, causal, entity — independent traversal with fused scoring |
90
- | **Learning** | Unconditional co-activation | Validation-gated: edges strengthen only on positive feedback (Kairos-inspired) |
91
- | **Noise rejection** | None | Multi-channel agreement gate: requires 2+ retrieval channels to agree before returning results |
92
- | **Duplicates** | Stored repeatedly | Reinforce-on-duplicate: near-exact matches boost existing memory instead of creating copies |
93
-
94
- The design is based on cognitive science — ACT-R activation decay, Hebbian learning, complementary learning systems, synaptic homeostasis, and synaptic tagging — rather than ad-hoc heuristics. See [How It Works](#how-it-works) and [docs/cognitive-model.md](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/cognitive-model.md) for details.
95
-
96
- > **New to AWM?** [`docs/pipeline-walkthrough.html`](https://completeideas.github.io/agent-working-memory/pipeline-walkthrough.html) is a visual, plain-language walkthrough (no background required) — what happens when AWM learns and recalls a fact, why it's built this way, and how it differs from a plain vector store. Open it in a browser.
97
-
98
- > **Build an agent on it:** the [AWM-Native Agent Harness pattern](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/patterns/awm-native-harness.md) shows how to use AWM as an always-on cognitive *substrate* (not a tool the model calls) so the agent learns automatically by working — letting a cheap model perform at a high level and get cheaper + better over time. Measured: gpt-5.4-mini + AWM beat a frontier model on a domain workload at ~1/40th the cost.
99
-
100
- > **For builders & researchers:** [`docs/awm-for-agents.html`](https://completeideas.github.io/agent-working-memory/awm-for-agents.html) is the agent playbook — why AWM exists (the context-window wall), the PRIME→ACT→VERIFY→LEARN harness, the full agent feature surface (workspace, session IDs, bearer-token hooks, supersede/feedback), how multi-hop is solved in the harness, and the honest gauntlet findings (where AWM wins, ties, and what isn't measured yet). Open it in a browser.
101
-
102
- ---
103
-
104
- ## Why it matters at scale
105
-
106
- The reason AWM exists: **past roughly half a million tokens, you can no longer keep a large project's context alive by carrying it.** The codebase, the docs, the decision history, and the meeting/work transcripts outgrow every model's window — and summarizing to fit silently drops the fact you needed next.
107
-
108
- These figures come from real-world use on a large software platform project, where a single work agent has accumulated **20,000+ memories** over a multi-million-token codebase and documentation set:
109
-
110
- | To answer one question, carry… | tokens | AWM scoped recall |
111
- |---|---|---|
112
- | the accumulated memory (~20K memories) | ~1.3M | **~630, flat** |
113
- | the project's notes & transcript docs | ~2M | **~630, flat** |
114
- | the whole system (code + docs) | ~29M — fits in no window, any tier | **~630, flat** |
115
-
116
- A scoped recall answers from the relevant *slice*, independent of how large the store grows. Measured consequences on real questions against the real project:
117
-
118
- - **~2,000× fewer tokens per query** than carrying the memory store — and **~5× fewer** than opening the single best-matching documentation file (a floor; agents usually open several and still miss cross-file facts).
119
- - **At scale, "carry everything" isn't an option.** At ~20K memories no context window holds it, so retrieval is not an optimization — it is the only door. A static notes file or long-context approach is forced to truncate, which silently drops facts.
120
-
121
- Two structural advantages a file or a flat vector store cannot match:
122
-
123
- - **Staleness is tracked.** When a fact changes, `memory_supersede` retires the old value and recall stops returning it — the system *knows* what changed. A notes file or repo goes stale silently; you would re-scan everything to find out. (This work agent has superseded and retracted dozens of facts as the project moved.)
124
- - **Dead weight costs nothing.** ~90% of accumulated memories are never recalled for a given task — a notes file pays for all of them in every prompt; recall pays for ≈zero.
125
-
126
- ### Honest about the trade-offs
127
-
128
- - AWM does **not** win on small, one-shot tasks — write/recall overhead exceeds the savings until knowledge is reused or the corpus grows past what fits in context.
129
- - Recall is not free: a few seconds of latency per query buys the token reduction.
130
- - Recall accuracy is bounded by what was written — write quality matters (lead with the fact; tag with identifiers like file, table, ticket).
131
- - It does not replace your source of truth. The intended pattern is: **recall first, read/grep the code for ground truth on a miss, and supersede when reality differs.**
132
-
133
- ---
134
-
135
- ## Benchmarks
136
-
137
- Two kinds of tests, both reproducible (see [Testing & Evaluation](#testing--evaluation)).
138
- First, **recall quality** — does the pipeline return the right memory? Second,
139
- **behavior under stress** — does it stay honest, filter noise, and hold up as the
140
- store grows and ages?
141
-
142
- ### 1 · The 0.13.x retrieval wins
143
-
144
- Three changes shipped in 0.13.4–0.13.6. Each row names **the corpus it was measured
145
- on** — they are not the same, and the difference matters. Enable all three together:
146
-
147
- ```bash
148
- AWM_RERANK2=1 AWM_RERANK_WINDOW=query AWM_RERANK_TAGS=1
149
- ```
150
-
151
- | Change | Flag | Measured result | Measured on |
152
- |---|---|---|---|
153
- | **Second-stage rerank** — let the cross-encoder's own score decide final order, instead of a blend that capped its vote at 70% | `AWM_RERANK2=1` | **+9.7pp success@1** (37.8 → 47.6), p<0.001, paired McNemar. Costs no extra inference — the scores already existed and were being partly discarded. | 616 LoCoMo probes — ⚠ the benchmark retired below |
154
- | **Query-aware rerank window** — spend the same 400-char budget on the window densest in query terms instead of the prefix | `AWM_RERANK_WINDOW=query` | **25.0% → 87.5%** long-memory success@1 (3.5×), at **+0.07%** CPU and **zero** added tokens. | Generated long-memory corpus, calibrated to real-store statistics, answer planted at a controlled offset |
155
- | **Tags into the rerank passage** — put words that exist only as tags in front of the component that decides | `AWM_RERANK_TAGS=1` | **+7.4pp success@1** (56.4 → 63.8) on category queries. | 450 probes on a **frozen real-store snapshot** |
156
-
157
- **Combined, on the real store** (450 category probes, frozen 11,294-engram snapshot):
158
- **s@1 56.4 → 63.8%**, **s@5 66.2 → 68.4%**, **MRR 60.6 → 66.0%** — with adversarial
159
- abstention held at **90.0%** in every arm. Selectivity was not traded away to buy accuracy.
160
-
161
- > **On the +9.7pp figure.** It comes from LoCoMo, which this page retires two sections
162
- > below. Reported as-measured rather than quietly dropped, because the provenance is part
163
- > of the story: LoCoMo's short passages are exactly why it could not see the 400-char
164
- > truncation, and that blind spot is what made the standalone `AWM_RERANK2`
165
- > recommendation wrong. The combined real-store number above is the one to trust.
166
-
167
- > **End-to-end, this did not move the acceptance test.** See the gauntlet row below.
168
-
169
- > **⚠ Enable them together.** `AWM_RERANK2` *alone* regresses long-memory s@5 from
170
- > 91.7% to 25.0%. BM25 over full content had been quietly compensating for the
171
- > reranker's 400-char blindness; making a blind reranker authoritative removes that
172
- > cover. Ship both, or neither.
173
-
174
- The finding underneath all three: **a memory is unreachable when a word it needs was
175
- never written into its body.** On this store, 66.2% of topical tag terms never appear
176
- in the text at all. Three other approaches to the same defect were tested and
177
- rejected — re-embedding with tags (+0.3pp), mined dialect aliases (−0.2pp), and a
178
- larger 768d embedder (+0.7pp alone, −1.1pp combined). You cannot recover a word that
179
- was never written; you can only put the word that *is* recorded in front of the ranker.
180
- That is also why 0.13.6 rewrote the **writing guidance**, not just the ranker.
181
-
182
- Full evidence, protocol, and the rejected arms: [`docs/archive/`](docs/archive/README.md).
183
-
184
- ### 2 · Everything else, in one table
185
-
186
- | What | Result | Detail |
187
- |---|---|---|
188
- | **Eval harness** (retrieval / associative / redundancy / temporal) | Recall@5 **0.980** · success@10 **1.000** · dedup F1 **0.966** · Spearman **0.932** — all four above threshold | [`docs/benchmarks.md`](docs/benchmarks.md) |
189
- | **Unit + subsystem** | `test:run` **715/715** · `test:self` **93.9%** · `test:edge` **~32/34** · `test:mcp` **5/5** | [`docs/benchmarks.md`](docs/benchmarks.md) |
190
- | **Adversarial / noise rejection** | `test:pilot` **14/15** (5/5 distractors rejected) · `test:ab` **AWM 10/11 vs keyword 8/11** | [`docs/benchmarks.md`](docs/benchmarks.md) |
191
- | **End-to-end ablation** (the gauntlet) | **74%±5pp memory-dependent vs 0% no-memory control** (0.11.x baseline); only the memory substrate varies. Both arms complete at k=10: baseline **74.0%** vs **81.0%** with the flags (**+7.0pp**, Fisher p=0.31 — not significant, but directionally matching the +7.4pp fixture result). **All 10 probes flip between identical runs**, and `multihop` has never passed at any k | [`gauntlet-baseline`](docs/archive/gauntlet-baseline-2026-07-30.md) |
192
- | **Consolidation under stress** | Recall **holds 90–100%** across 100 cycles; edges grow to ~2,300 then self-prune to ~1,500 | [`docs/benchmarks.md`](docs/benchmarks.md) |
193
- | **Token economics** | **9.8× lower** aggregate cost than the Read/Grep/Glob rediscovery it replaces | [`docs/benchmarks.md`](docs/benchmarks.md) |
194
-
195
- **The retrieval gains above have not yet shown up end-to-end, and the reason is now
196
- understood.** Both arms now run complete at k=10: baseline **74.0%** against **81.0%**
197
- with the flags — **+7.0pp**, Fisher exact **p = 0.31**. Not significant, but the direction
198
- and magnitude match the fixture-level +7.4pp, so this is consistent with the gains
199
- converting rather than evidence that they do. Resolving it is hard because **all 10 probes
200
- flip between identical runs**, with `composite` passing 5/10 under an unchanged configuration, and `multihop` moving 2/10 to 6/10 with the flags — half of those passes at 2 steps or fewer, i.e. better ranking rather than the agent chaining. Raising k narrows the interval
201
- around an unstable mean; it does not make the suite able to resolve a few-point
202
- difference. The next step is probe determinism, not more repetitions. See
203
- [`docs/benchmarks.md`](docs/benchmarks.md).
204
-
205
- Two other numbers are easy to misread, so they are stated plainly:
206
-
207
- - **`test:sleep` = 78.6% is a consolidation-*quality* score**, not recall falling to
208
- 78.6%. It asks "after the maintenance pass, is recall at least as good and the
209
- structure better?" Recall is held flat across three cycles while the graph reorganizes.
210
- - **Token savings depend entirely on the baseline you pick.** vs carrying the full
211
- history (what a memoryless agent must actually do): **+67% at 97.5% accuracy**. vs an
212
- oracle that pre-scoped context to the exactly-relevant task: **≈ −13%** — a
213
- deliberately brutal bar that hands the baseline the very scoping retrieval exists to
214
- do. 0.13.x added a third and stricter measure, **sufficiency**: does the delivered
215
- text actually *contain* the answer, or merely point at it?
216
-
217
- > **LoCoMo was retired in 0.13.x.** It was useful for learning how to benchmark this
218
- > product but does not represent it: median 115-char passages against a real store's
219
- > 1,965; seeded in one shot, so decay, Hebbian weights and salience contribute nothing;
220
- > no supersession, no cross-session use; it *rewards* indiscriminate retention, so the
221
- > salience filter — the product — caps its score regardless of ranking quality; and it
222
- > is structurally blind to the 400-char truncation that turned out to affect 79% of real
223
- > ground-truth identifiers. `tests/realstore-eval/` replaces it.
224
-
225
- ---
226
-
227
- ## Features
228
-
229
- ### Memory Tools (17 + 2 onboarding = 19)
230
-
231
- | Tool | Purpose |
232
- |------|---------|
233
- | `memory_write` | Store a memory (salience filter + reinforce-on-duplicate) |
234
- | `memory_recall` | Retrieve relevant memories by context (dual BM25 + coref expansion) |
235
- | `memory_feedback` | Report whether a recalled memory was useful |
236
- | `memory_retract` | Invalidate a wrong memory with optional correction |
237
- | `memory_supersede` | Replace outdated memory with current version |
238
- | `memory_stats` | View memory health metrics and activity |
239
- | `memory_whoami` | Identify the instance — agent id, workspace, backend, store path, sibling agent spaces |
240
- | `memory_checkpoint` | Save execution state (survives context compaction) |
241
- | `memory_restore` | Recover state + relevant context at session start |
242
- | `memory_task_add` | Create a prioritized task |
243
- | `memory_task_update` | Change task status/priority |
244
- | `memory_task_list` | List tasks by status |
245
- | `memory_task_next` | Get the highest-priority actionable task |
246
- | `memory_task_begin` | Start a task — auto-checkpoints and recalls context |
247
- | `memory_task_end` | End a task — writes summary and checkpoints |
248
- | `compress_output` | Encode a structured tool output as TOON — ~50-65% fewer tokens, lossless, output-only |
249
- | `retrieve_original` | Get the verbatim source back for a `compress_output` ref |
250
-
251
- ### Onboarding Tools (2)
252
-
253
- For warm-starting a cold store from a project's own docs/repo — see [What's New in v0.11.0](#whats-new-in-v0110).
254
-
255
- | Tool | Purpose |
256
- |------|---------|
257
- | `onboard_scan` | Extract candidate memories from a project's docs/repo for review |
258
- | `onboard_questions` | Anchored interview questions to refine what a cold store should know |
259
-
260
- ### Separate Memory Pools
261
-
262
- By default, all projects share one memory pool. For isolated pools per folder, place a `.mcp.json` in each parent folder with a different `AWM_AGENT_ID`:
263
-
264
- ```
265
- C:\Users\you\work\.mcp.json -> AWM_AGENT_ID: "work"
266
- C:\Users\you\personal\.mcp.json -> AWM_AGENT_ID: "personal"
267
- ```
268
-
269
- Claude Code uses the closest `.mcp.json` ancestor. Same database, isolation by agent ID.
270
-
271
- ### Incognito Mode
272
-
273
- ```bash
274
- AWM_INCOGNITO=1 claude
275
- ```
276
-
277
- Registers zero tools — Claude doesn't see memory at all. All other tools and MCP servers work normally.
278
-
279
- ### Auto-Checkpoint Hooks
280
-
281
- Installed by `awm setup --global`:
282
-
283
- - **Stop** — reminds Claude to write/recall after each response
284
- - **PreCompact** — auto-checkpoints before context compression
285
- - **SessionEnd** — auto-checkpoints and consolidates on close
286
- - **15-min timer** — silent auto-checkpoint while session is active
287
-
288
- ### Auto-Backup
289
-
290
- The HTTP server automatically copies the database to a `backups/` directory on startup with a timestamp. Cheap insurance against data loss.
291
-
292
- ### Activity Log
293
-
294
- ```bash
295
- tail -f "$(npm root -g)/agent-working-memory/data/awm.log"
296
- ```
297
-
298
- Real-time: writes, recalls, reinforcements, checkpoints, consolidation, hook events.
299
-
300
- ### Activity Stats
301
-
302
- ```bash
303
- curl http://127.0.0.1:8401/stats
304
- ```
305
-
306
- Returns daily counts: `{"writes": 8, "recalls": 9, "hooks": 3, "total": 25}`
307
-
308
- ---
309
-
310
- ## Memory Invocation Strategy
311
-
312
- AWM combines deterministic hooks for guaranteed memory operations at lifecycle transitions with agent-directed usage during active work.
313
-
314
- ### Deterministic triggers (always happen)
315
-
316
- | Event | Action |
317
- |-------|--------|
318
- | Session start | `memory_restore` — recover state + recall context |
319
- | Pre-compaction | Auto-checkpoint via hook sidecar |
320
- | Session end | Auto-checkpoint + full consolidation |
321
- | Every 15 min | Silent auto-checkpoint (if active) |
322
- | Task start | `memory_task_begin` — checkpoint + recall |
323
- | Task end | `memory_task_end` — summary + checkpoint |
324
-
325
- ### Agent-directed triggers (when these situations occur)
326
-
327
- **Write memory when:**
328
- - A project decision is made or changed
329
- - A root cause is discovered
330
- - A reusable implementation pattern is established
331
- - A preference, constraint, or requirement is clarified
332
- - A prior assumption is found to be wrong
333
-
334
- **Recall memory when:**
335
- - Starting work on a new task or subsystem
336
- - Re-entering code you haven't touched recently
337
- - After context compaction
338
- - After a failed attempt (check if there's prior knowledge)
339
- - Before refactoring or making architectural changes
340
-
341
- **Retract when:**
342
- - A stored memory turns out to be wrong or outdated
343
-
344
- **Feedback when:**
345
- - A recalled memory was used (useful) or irrelevant (not useful)
346
-
347
- ---
348
-
349
- ## HTTP API
350
-
351
- For custom agents, scripts, or non-Claude-Code workflows:
352
-
353
- ```bash
354
- awm serve # From npm install
355
- npx tsx src/index.ts # From source
356
- ```
357
-
358
- ```bash
359
- # Write
360
- curl -X POST http://localhost:8400/memory/write -H "Content-Type: application/json" -d '{
361
- "agentId": "my-agent",
362
- "concept": "Express error handling",
363
- "content": "Use centralized error middleware as the last app.use()",
364
- "eventType": "causal", "surprise": 0.5, "causalDepth": 0.7
365
- }'
366
-
367
- # Recall
368
- curl -X POST http://localhost:8400/memory/activate -H "Content-Type: application/json" -d '{
369
- "agentId": "my-agent",
370
- "context": "How should I handle errors in my Express API?"
371
- }'
372
- ```
373
-
374
- **Substrate primitives (0.8+)** — for long-running structured projects (novels,
375
- codebases, investigations) where an agent tracks typed state across hundreds of writes
376
- without polluting cognitive retrieval: `/memory/latest-by-tag` (latest per tag key),
377
- `/memory/top-by` (native filter + sort), `/memory/supersede` (atomic write-and-supersede
378
- by concept match), `/memory/sequence/:agentId/next` (race-free chronology). The
379
- `memory_class: "structural"` class keeps high-volume system-written records out of
380
- cognitive `/activate` while preserving them at canonical salience.
381
-
382
- Every endpoint, with request/response schemas and worked examples:
383
- [`docs/reference.md`](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/reference.md).
384
-
385
- ---
386
-
387
- ## How It Works
388
-
389
- ### The Memory Lifecycle
390
-
391
- 1. **Write** — Salience scoring evaluates novelty, surprise, causal depth, and effort. High-salience memories go active; borderline ones enter staging; low-salience stored at reduced confidence for recall fallback. Near-duplicates reinforce existing memories instead of creating copies.
392
-
393
- 2. **Connect** — Vector embedding (BGE-small-en-v1.5, 384d). Temporal edges link to recent memories. Hebbian edges form between co-retrieved memories. Coref expansion resolves pronouns to entity names.
394
-
395
- 3. **Retrieve** — 10-phase pipeline: coref expansion + query expansion + dual BM25 (keyword-stripped + expanded) + semantic vectors + Rocchio pseudo-relevance feedback + ACT-R temporal decay (synaptic-tagged) + Hebbian boost + entity-bridge boost + graph walk + cross-encoder reranking + multi-channel agreement gate.
396
-
397
- 4. **Consolidate** — 7-phase sleep cycle: diameter-enforced clustering (prevents chaining), edge strengthening (access-weighted), cross-topic bridge formation (direct closest-pair), confidence-modulated decay (synaptic tagging extends half-life), synaptic homeostasis, cognitive forgetting, staging sweep. Embedding backfill ensures all memories are clusterable.
398
-
399
- 5. **Feedback** — Useful/not-useful signals adjust confidence, affecting retrieval rank and forgetting resistance.
400
-
401
- ### Cognitive Foundations
402
-
403
- - **ACT-R activation decay** (Anderson 1993) — memories decay with time, strengthen with use. Synaptic tagging: heavily-accessed memories decay slower (log-scaled).
404
- - **Hebbian learning** — co-retrieved memories form stronger associative edges
405
- - **Complementary Learning Systems** — fast capture (salience + staging) + slow consolidation (sleep cycle)
406
- - **Synaptic homeostasis** — edge weight normalization prevents hub domination
407
- - **Forgetting as feature** — noise removal improves signal-to-noise for connected memories
408
- - **Diameter-enforced clustering** — prevents semantic chaining (e.g., physics->biophysics->cooking = 1 cluster)
409
- - **Multi-channel agreement** — OOD detection requires multiple retrieval channels to agree
410
-
411
- ---
412
-
413
- ## Architecture
414
-
415
- ```
416
- src/
417
- core/ # Cognitive primitives
418
- embeddings.ts - Local vector embeddings (BGE-small-en-v1.5, 384d)
419
- reranker.ts - Cross-encoder passage scoring (ms-marco-MiniLM)
420
- query-expander.ts - Synonym expansion (flan-t5-small)
421
- salience.ts - Write-time importance scoring (novelty + salience + reinforce-on-duplicate)
422
- decay.ts - ACT-R temporal activation decay
423
- hebbian.ts - Association strengthening/weakening
424
- logger.ts - Append-only activity log (data/awm.log)
425
- engine/ # Processing pipelines
426
- activation.ts - 10-phase retrieval pipeline (dual BM25, coref, agreement gate)
427
- consolidation.ts - 7-phase sleep cycle (diameter clustering, direct bridging, synaptic tagging)
428
- connections.ts - Discover links between memories
429
- staging.ts - Weak signal buffer (promote or discard)
430
- retraction.ts - Negative memory / corrections
431
- eviction.ts - Capacity enforcement
432
- hooks/
433
- sidecar.ts - Hook HTTP server (auto-checkpoint, stats, timer)
434
- storage/
435
- sqlite.ts - SQLite + FTS5 persistence layer
436
- api/
437
- routes.ts - HTTP endpoints (memory + task + system)
438
- mcp.ts - MCP server (19 tools: 17 memory + 2 onboarding, incognito support)
439
- cli.ts - CLI (setup, serve, hook config)
440
- index.ts - HTTP server entry point (auto-backup on startup)
441
- ```
442
-
443
- For detailed architecture including pipeline phases, database schema, and system diagrams, see [docs/architecture.md](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/architecture.md).
444
-
445
- ---
446
-
447
- ## Testing & Evaluation
448
-
449
- ```bash
450
- npx vitest run # Unit: salience, decay, hebbian, supersession
451
- npm run eval # 4 benchmark suites
452
- npm run eval -- --suite=retrieval # One suite
453
- npm run eval -- --bm25-only # Ablation: isolate a channel's contribution
454
- ```
455
-
456
- ### Real-store benchmark (0.13.x — replaces LoCoMo)
457
-
458
- Measures the pipeline against a **frozen snapshot of a real store**, so passage lengths,
459
- decay, supersession and Hebbian weights are all real rather than synthetic. Ground truth
460
- is a unique-identifier hold-out verified through FTS, so it needs no hand labeling. And
461
- correct **abstention scores positively** — selectivity is the product, so a benchmark
462
- that punishes silence is measuring the wrong system.
463
-
464
- ```bash
465
- node tests/realstore-eval/snapshot.mjs # freeze a copy of the live store
466
- npx tsx tests/realstore-eval/runner.ts # identifier fixture (regression guard)
467
- REALSTORE_FIXTURE=fixture-category.json \
468
- npx tsx tests/realstore-eval/runner.ts # category fixture (retrievability)
469
- bash tests/realstore-eval/campaign/full-comparison.sh # baseline vs recommended, all suites
470
- ```
471
-
472
- Each run works on a **copy** — activation mutates access counts, and a benchmark must
473
- not drift the thing it measures. The runner prints the active flag fingerprint, so every
474
- result records which configuration produced it.
475
-
476
- Per-suite methodology, scoring, and the remaining `test:*` scripts:
477
- [`docs/benchmarks.md`](docs/benchmarks.md).
478
-
479
- ---
480
-
481
- ## Environment Variables
482
-
483
- **Recommended recall configuration (0.13.x)** — default-OFF, but measured wins. Enable
484
- all three together (see [Benchmarks](#benchmarks)):
485
-
486
- ```bash
487
- AWM_RERANK2=1 AWM_RERANK_WINDOW=query AWM_RERANK_TAGS=1
488
- ```
489
-
490
- | Variable | Effect |
491
- |---|---|
492
- | `AWM_RERANK2=1` | Second-stage reorder of the returned window by cross-encoder score alone, after the abstention gate. **+9.7pp s@1.** Must be paired with `AWM_RERANK_WINDOW=query` |
493
- | `AWM_RERANK_WINDOW=query` | Spend the rerank char budget on the window densest in query terms instead of the first N chars. **25% → 87.5%** long-memory s@1 |
494
- | `AWM_RERANK_TAGS=1` | Append structured tags to the rerank passage, so category words that exist only as tags reach the deciding stage. **+7.4pp s@1** |
495
-
496
- Verify what a running process actually has — `memory_whoami` prints a `Recall config:`
497
- line and `GET /health` reports the same fingerprint. A submodule bump can report a new
498
- version while the flags never reached the process; on version alone that looks like success.
499
-
500
- **Core settings:**
501
-
502
- | Variable | Default | Purpose |
503
- |----------|---------|---------|
504
- | `AWM_DB_PATH` | `memory.db` (SQLite) / `./memory-pglite` (PGlite) | Storage path — file for SQLite, directory for PGlite |
505
- | `AWM_STORE_BACKEND` | `sqlite` | `sqlite` (WAL, multi-process safe) · `pglite` (single-process) · `postgres` (networked, **experimental**) |
506
- | `AWM_AGENT_ID` | `claude-code` | Agent id — the memory namespace. Pin it explicitly; an unpinned session lands in a per-directory UUID space nothing else can recall |
507
- | `AWM_WORKSPACE` | *(unset)* | Default workspace for cross-agent recall in hive setups |
508
- | `AWM_PORT` / `AWM_HOOK_PORT` | `8400` / `8401` | HTTP server and hook sidecar ports |
509
- | `AWM_API_KEY` / `AWM_HOOK_SECRET` | *(none)* | Bearer tokens. Binding beyond loopback without an API key fails closed |
510
- | `AWM_INCOGNITO` | *(unset)* | `1` disables all tools |
511
- | `AWM_EMBED_MODEL` / `AWM_EMBED_DIMS` | `Xenova/bge-small-en-v1.5` / `384` | ⚠ `cosineSimilarity` returns **0** on dimension mismatch — migrate the whole corpus or not at all |
512
-
513
- Every variable — including the salience, decay, fade, confidence, granularity and
514
- diagnostic knobs, each with its measured effect and the **rejected** experiments and why
515
- they lost — is documented in [`docs/reference.md`](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/reference.md).
516
-
517
- ---
518
-
519
- ## Tech Stack
520
-
521
- | Component | Technology |
522
- |-----------|-----------|
523
- | Language | TypeScript (ES2022, strict) |
524
- | Database | SQLite via better-sqlite3 + FTS5 |
525
- | HTTP | Fastify 5 |
526
- | MCP | @modelcontextprotocol/sdk |
527
- | ML Runtime | @huggingface/transformers (local ONNX) |
528
- | Embeddings | BGE-small-en-v1.5 (BAAI, retrieval-optimized, 384d) |
529
- | Reranker | ms-marco-MiniLM-L-6-v2 (cross-encoder) |
530
- | Query Expansion | flan-t5-small (synonym generation) |
531
- | Tests | Vitest 4 |
532
- | Validation | Zod 4 |
533
-
534
- All three ML models run locally via ONNX. No external API calls for retrieval. The entire system is a single SQLite file + a Node.js process.
535
-
536
- ## What's New in v0.14.0 (latest)
537
-
538
- Retrieval-quality releases, all additive. Corpus provenance differs per result and is
539
- named in [Benchmarks](#benchmarks) — only the tags result and the combined figure come
540
- from a real-store snapshot.
541
-
542
- - **Second-stage rerank (`AWM_RERANK2`)** — final order was a blend that capped the
543
- cross-encoder at 70% of the vote. It disagrees with that blend about rank 1 on 38.6%
544
- of queries, and where the disagreement is decidable **the cross-encoder is right 77%
545
- of the time**. Re-sorting by its score: **+9.7pp s@1**, no added inference. Placed
546
- after the abstention gate so it cannot affect selectivity — predicted 0 broken / 0
547
- fixed on adversarial, and measured exactly that.
548
- - **Query-aware rerank window (`AWM_RERANK_WINDOW=query`)** — truncating passages to
549
- the first 400 chars is necessary (cross-encoders pad to the longest passage in a
550
- batch), but a *prefix* is the wrong 400. Real canonical memories are median 1,965
551
- chars, 98.7% exceed 400, and **99.9%** of long ones carry their identifiers only
552
- past char 400. Same budget, densest window: **25% → 87.5%**.
553
- - **`memory_whoami` reports the effective recall config** — version alone does not
554
- answer "what am I actually running". This caught a real deployment failure the day
555
- it shipped: a project-level `.mcp.json` was overriding the config being edited, so
556
- the new version reported success while the flags never reached the process.
557
- - **Tags into the rerank passage (`AWM_RERANK_TAGS`)** — **+7.4pp s@1**, with no
558
- re-embed, no new model and no write-path change; existing corpora benefit immediately
559
- because the tags are already stored.
560
- - **Writing guidance corrected at the source** — the shipped advice was *causing* the
561
- problem it warned about. "Pick the most specific topic" pushed authors away from
562
- category words, reliably producing memories that are maximally specific and
563
- categorically anonymous. Two new rules: **name the CATEGORY as well as the
564
- specifics**, and **tags are not a substitute for body text** (only BM25 indexes tags —
565
- the embedding and the rerank passage are both built from `concept + content`, so a
566
- tag-only word is invisible to two of three channels, including the one that now
567
- decides ordering).
568
-
569
- ### Previously, in v0.12.x
570
-
571
- `memory_whoami` instance identity · entity inverted index (`AWM_ENTITY_INDEX_FETCH=1`,
572
- opt-in) · local-first security defaults (loopback bind, fail-closed without an API key)
573
- · write-path slow-write telemetry · memory-spine provenance (`origin_class`,
574
- `valid_from`/`valid_to`) · cognition recipes at `memory_task_end` · engram ids in
575
- recall results · eager warm at MCP startup + sidecar warm recall.
576
-
577
- Full version-by-version history — every release back to v0.6.0, including the
578
- 0.7.6→0.7.14 latency work (11s→300ms) and the 0.8.5 recall-quality hardening pass —
579
- lives in [CHANGELOG.md](https://github.com/CompleteIdeas/agent-working-memory/blob/master/CHANGELOG.md).
580
-
581
- ## Integrations
582
-
583
- AWM is a standard MCP server, so it plugs into any MCP-capable agent host with
584
- **no adapter code** — the same server Claude Code uses. Point two hosts at the
585
- same `AWM_DB_PATH` (with a shared `AWM_AGENT_ID`/`AWM_WORKSPACE`) and they share
586
- one cognitive memory.
587
-
588
- ### Hermes Agent (Nous Research)
589
-
590
- 1. Make AWM available where Hermes runs (e.g. a derived Docker image — the
591
- Hermes image already bundles Node):
592
-
593
- ```dockerfile
594
- FROM hermes-agent:local
595
- USER root
596
- RUN npm install -g agent-working-memory@latest
597
- ENV HF_HOME=/opt/data/.cache/huggingface
598
- ```
599
-
600
- 2. Register it in `~/.hermes/config.yaml`:
601
-
602
- ```yaml
603
- mcp_servers:
604
- awm:
605
- command: node
606
- args: ["/usr/local/lib/node_modules/agent-working-memory/dist/mcp.js"]
607
- env:
608
- AWM_AGENT_ID: hermes
609
- AWM_DB_PATH: /opt/data/awm/hermes.db # on a persistent volume
610
- HF_HOME: /opt/data/.cache/huggingface
611
- timeout: 600 # first call downloads the embedder
612
- ```
613
-
614
- 3. AWM's tools appear to the agent as `mcp_awm_memory_write`,
615
- `mcp_awm_memory_recall`, etc. Works with any Hermes model provider
616
- (verified on Anthropic and Azure `gpt-5-4-mini`).
617
-
618
- Full recipe — model-provider examples, the Azure GPT-5.x `/openai/v1` note, and
619
- gotchas (incl. the Windows CRLF/s6 clone fix) — is in
620
- [docs/integrations/hermes.md](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/integrations/hermes.md).
621
-
622
- ## Project Status
623
-
624
- AWM is in active development (v0.14.0). The core memory pipeline, consolidation
625
- system, multi-agent coordination, and MCP integration are stable and used
626
- daily in production coding workflows.
627
-
628
- - Core retrieval and consolidation: **stable**
629
- - MCP tools and Claude Code integration: **stable** (19 tools: 17 memory + 2 onboarding)
630
- - Other MCP hosts (e.g. [Hermes Agent](https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/integrations/hermes.md)): **supported** — AWM drops in as an MCP memory server with no adapter code
631
- - Multi-agent coordination: **stable** (v0.8.1 hardening)
632
- - Task management: **stable**
633
- - Hook sidecar and auto-checkpoint: **stable** — plus `POST /memory/activate` warm recall for hooks (v0.12.2)
634
- - HTTP API: **stable** (for custom agents)
635
- - Eval harness: **stable** (v0.6.0, extended through 0.8.x); gauntlet acceptance test added (v0.12.0)
636
- - Recall confidence + opt-in abstention (PR-1, PR-2): **stable** (v0.8.5)
637
- - Coherence-weighted retraction + counter-narrative inheritance: **stable** (v0.8.5)
638
- - Content fade stage + adaptive output granularity: **stable** (v0.8.5)
639
- - PGlite backend (alternative to SQLite, with pgvector + ivfflat): **stable** (v0.8.x)
640
- - Networked Postgres backend (`pg` + pgvector, multi-connection): **experimental** (v0.10.0)
641
- - Backend-agnostic `import`/`export` (embeddings included, cross-backend port): **stable** (v0.10.0)
642
- - Instance identity (`memory_whoami`), local-first security defaults, write-path telemetry, memory-spine provenance (`origin_class`/`valid_from`/`valid_to`), cognition recipes: **stable** (v0.12.0)
643
- - Entity inverted index + guarded index-backed retrieval: **stable, opt-in** (`AWM_ENTITY_INDEX_FETCH=1`, default off pending broader eval) (v0.12.0)
644
- - Second-stage rerank, query-aware rerank window, tags-into-rerank: **stable, opt-in** (`AWM_RERANK2=1 AWM_RERANK_WINDOW=query AWM_RERANK_TAGS=1` — enable together) (v0.13.4-0.13.6)
645
- - Real-store benchmark (`tests/realstore-eval/`), replacing LoCoMo: **stable** (v0.13.x)
646
-
647
- See [CHANGELOG.md](https://github.com/CompleteIdeas/agent-working-memory/blob/master/CHANGELOG.md) for version history.
648
-
649
- ---
650
-
651
- ## License
652
-
653
- Apache 2.0 — see [LICENSE](https://github.com/CompleteIdeas/agent-working-memory/blob/master/LICENSE) and [NOTICE](https://github.com/CompleteIdeas/agent-working-memory/blob/master/NOTICE).
1
+ # AgentWorkingMemory (AWM)
2
+
3
+ **Give your AI coding agent a memory that survives the conversation — and knows when to stay quiet.**
4
+
5
+ Every session with an AI assistant starts blank. It has forgotten what your team decided last
6
+ week, which approach was tried and rejected, and which table actually holds the thing it needs.
7
+ So it re-derives all of it — reading files, running searches, asking you — and on a large project
8
+ it will confidently rebuild something that was already decided against.
9
+
10
+ AWM fixes that with one local process and one SQLite file. The agent writes short notes as it
11
+ learns; AWM decides which are worth keeping, hands back the two or three that matter when asked,
12
+ and says **nothing** when nothing fits.
13
+
14
+ ```bash
15
+ npm install -g agent-working-memory && awm setup --global
16
+ ```
17
+
18
+ Restart Claude Code. 19 tools appear. No cloud, no API keys, nothing leaves your machine.
19
+
20
+ <p align="center">
21
+ <a href="https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/for-decision-makers.md"><b>Deciding whether to adopt it? →</b></a> &nbsp;·&nbsp;
22
+ <a href="https://github.com/CompleteIdeas/agent-working-memory/blob/master/docs/walkthrough.md"><b>Want the mechanism walked through? →</b></a> &nbsp;·&nbsp;
23
+ <a href="https://completeideas.github.io/agent-working-memory/"><b>Docs site →</b></a>
24
+ </p>
25
+
26
+ ---
27
+
28
+ ## What it does, measured
29
+
30
+ Reproducible from this repository against a frozen copy of a **real 30,000-memory store**, not
31
+ synthetic data. Current numbers, stamped with the commit that produced them:
32
+ [`docs/benchmarks-current.md`](docs/benchmarks-current.md).
33
+
34
+ | | |
35
+ |---|---|
36
+ | **Returns the right memory first** | **92.7%** of identifier queries · **92.0%** of topic queries · **96.7%** in the top five |
37
+ | **Stays silent when it should** | **90.0%** correct abstention on questions about facts never stored |
38
+ | **Costs the same at any scale** | Scoped recall answers in **~630 tokens flat**. Carrying the store instead: ~1.3M tokens |
39
+ | **Lets a cheap model punch up** | Small model + AWM beat a frontier model on a 15-task domain workload: **14/15 vs 7/15**, at ~1/40th the cost |
40
+
41
+ Two independently built fixtures agree within a point. Warm recall takes about half a second.
42
+
43
+ <details>
44
+ <summary>Two more, from real work rather than fixtures</summary>
45
+
46
+ **Halves the digging.** Same agent, same tools, four real support tickets: with memory, 2× the
47
+ specific facts the real answer needed and half the database queries (49 vs 100). On one ticket
48
+ the memoryless arm ran 25 queries, exhausted its budget and answered nothing; the memory arm
49
+ answered in 4.
50
+
51
+ **Holds decisions for months.** On a six-month application project, recorded decisions were
52
+ recalled a median of 30 days after being written — some after 167. 41% of the technical
53
+ identifiers the agent used had entered the conversation *only* through a recall.
54
+ </details>
55
+
56
+ ---
57
+
58
+ ## Is this for you?
59
+
60
+ The honest version, because the overhead is real and it does not pay back for everyone.
61
+
62
+ | If you are… | What you actually get | It pays back when |
63
+ |---|---|---|
64
+ | **A solo developer on one long-lived project** | The agent stops re-asking what you already told it, and stops re-proposing approaches you rejected | The project outgrows the context window — roughly, when you start saying "I have explained this before" |
65
+ | **A team sharing conventions** | One pool where a decision written by anyone is recalled by everyone, with corrections superseding rather than accumulating | More than one person is steering agents at the same codebase |
66
+ | **Running support or operations agents** | Ticket-scoped recall by real identifiers, so the second agent to touch a ticket knows what the first one did | Work arrives as a queue that several sessions or people share |
67
+ | **Building multi-agent pipelines** | A shared substrate with per-agent scoping, so specialists cooperate without sharing one context window | You have specialised agents that need the same facts |
68
+ | **Cost-constrained** | A small local model with good recall beats a frontier model without it on domain work | Your workload is narrow and repetitive rather than novel each time |
69
+ | **Somewhere cloud memory is not allowed** | One SQLite file on the machine. No account, no telemetry, no egress | Compliance makes hosted memory a non-starter |
70
+
71
+ **Skip it if:** your tasks are short and one-off with nothing worth reusing; you want a hosted
72
+ service someone else operates; you need memory synced across machines out of the box; or you were
73
+ hoping to replace your code, tickets and docs as the source of truth. AWM is a fast path back to
74
+ what was already decided — recall first, verify against the source when it matters, supersede when
75
+ reality differs.
76
+
77
+ It is also **not** a chatbot, a generic vector database, or a RAG pipeline over your files.
78
+
79
+ ---
80
+
81
+ ## Get started
82
+
83
+ ```bash
84
+ npm install -g agent-working-memory
85
+ awm setup --global # MCP config, CLAUDE.md guidance, hooks
86
+ ```
87
+
88
+ Requires **Node.js 22+**. Restart Claude Code.
89
+
90
+ **What to expect.** The first conversation is ~30 s slower while three small ONNX models download
91
+ (~200 MB, once). After that the agent has 19 memory tools, and the hooks save state on compaction
92
+ and session end without you doing anything. For the first day or two recall will be thin — it only
93
+ knows what has been written. That is the normal shape of it, not a fault.
94
+
95
+ **Starting on an existing project?** Warm-start the store from its own docs so recall is useful on
96
+ day one:
97
+
98
+ ```bash
99
+ awm onboard ./docs --repo . --project <name> # review the pack, then
100
+ awm import <pack> --db <path> --dedupe
101
+ ```
102
+
103
+ **Turn on the three retrieval improvements.** They ship default-off and belong together:
104
+
105
+ ```bash
106
+ AWM_RERANK2=1 AWM_RERANK_WINDOW=query AWM_RERANK_TAGS=1
107
+ ```
108
+
109
+ Second-stage rerank by the cross-encoder's own score; a 400-character rerank window placed on the
110
+ densest query-term region rather than the prefix (**25% → 87.5%** on long memories); and tags fed
111
+ into the rerank passage (**+7.4pp**). `awm setup` writes them on a fresh install. `memory_whoami`
112
+ prints the active fingerprint so you can confirm a running process has them.
113
+
114
+ **Check it worked:** `awm doctor claude-code` reports the live sidecars, which agent each serves,
115
+ and whether anything needs a re-run of setup.
116
+
117
+ | Next | |
118
+ |---|---|
119
+ | Install, first write, first recall | [`docs/quickstart.md`](docs/quickstart.md) |
120
+ | Separate pools per project, incognito mode, hooks | [`docs/claude-code-setup.md`](docs/claude-code-setup.md) |
121
+ | Teams and multi-agent | [`docs/team-setup-guide.md`](docs/team-setup-guide.md) |
122
+ | Custom agents over HTTP | [`docs/reference.md`](docs/reference.md) |
123
+
124
+ ---
125
+
126
+ ## Why it works when a vector store doesn't
127
+
128
+ Most "memory for AI" stores everything and retrieves by similarity. AWM makes the opposite bet:
129
+ **a memory is only useful if it is selective.**
130
+
131
+ **It refuses most of what it sees.** Every write is scored for importance before storage — is it
132
+ new, is it a decision or a root cause, does it name its subject? About a third of what the agent
133
+ offers is kept at full confidence. The rest never competes, so recall stays sharp as the store
134
+ grows.
135
+
136
+ **It forgets gracefully.** Memories that keep getting used stay strong; ones nobody touches fade
137
+ from ranking without being deleted. The model is borrowed from cognitive science (ACT-R), and it
138
+ means no one curates the store.
139
+
140
+ **It tells you when it doesn't know.** When the top candidates are only marginally better than the
141
+ tenth, AWM returns nothing and says how many it withheld. A confident wrong memory is more
142
+ expensive than an admitted gap — the benchmark scores the silence as a win.
143
+
144
+ **It knows what changed.** When a fact is corrected, the old memory is marked superseded and
145
+ carries a visible warning if it ever surfaces. A notes file has two paragraphs that read with equal
146
+ confidence; AWM knows which one won. One project has 108 such supersessions — an agent resuming it
147
+ gets one current state, not five.
148
+
149
+ **It stays local and per-person.** One SQLite file, three small ONNX models. Work and personal
150
+ memories are separate pools; several sessions share a pool safely.
151
+
152
+ <details>
153
+ <summary>Against a typical RAG / vector store, feature by feature</summary>
154
+
155
+ | | Typical vector store | AWM |
156
+ |---|---|---|
157
+ | Storage | Everything | Salience-filtered: active / staging / low-confidence fallback |
158
+ | Retrieval | Cosine similarity | Keyword + vector candidates → liveness scoring → cross-encoder rerank → abstention gate |
159
+ | Forgetting | Manual cleanup | ACT-R decay; reinforced knowledge persists |
160
+ | Correction | Delete and re-insert | Supersede with a visible pointer; retract with confidence propagation to neighbours |
161
+ | Duplicates | Stored again | Reinforce the existing memory instead |
162
+ | Wrong answers | Best of a bad set | Returns nothing, says why |
163
+ | Named things | Vocabulary-dependent | Entity index on tickets, people, tables, files |
164
+ | Feedback | None | Useful / not-useful adjusts confidence and rank |
165
+ | Multi-agent | Per-instance | Shared store, per-agent scoping, opt-in shared workspace |
166
+ </details>
167
+
168
+ ---
169
+
170
+ ## How it works, in one paragraph
171
+
172
+ A write is scored for salience — novelty against the existing store, event type, whether it names
173
+ identifiers — and lands active, staged, or low-confidence. A recall casts a wide net with keyword
174
+ and vector search, scores each candidate for relevance and liveness, lets linked memories vote,
175
+ hands the shortlist to a cross-encoder that actually reads the text, then checks whether the score
176
+ distribution justifies answering at all. A maintenance pass on session end clusters, decays unused
177
+ links, and archives what has gone cold. Corrections supersede rather than overwrite.
178
+
179
+ The whole thing, one memory followed end to end with every threshold sourced:
180
+ [`docs/walkthrough.md`](docs/walkthrough.md). The mental model an engineer needs to predict its
181
+ behaviour — proposers, deciders, and when it stays silent:
182
+ [`How AWM decides what to say`](https://completeideas.github.io/agent-working-memory/pipeline-walkthrough.html).
183
+ The theory and its citations: [`docs/cognitive-model.md`](docs/cognitive-model.md).
184
+
185
+ ---
186
+
187
+ ## Honest limits
188
+
189
+ - No help on small one-off tasks — the overhead pays back when knowledge is reused or the project
190
+ outgrows the context window.
191
+ - Recall is bounded by what was written. A memory that never names its subject cannot be found by
192
+ it. The writing guidance exists for this reason.
193
+ - The association graph, as of this release, rarely changes a final answer; the reranker does.
194
+ - It is 0.x. Benchmark figures have been corrected upward more than once after the *instrument*
195
+ turned out to be wrong; every correction is documented in place rather than revised away.
196
+
197
+ Everything else, with evidence and workarounds: [`docs/known-limitations.md`](docs/known-limitations.md).
198
+
199
+ ---
200
+
201
+ ## Reference
202
+
203
+ The README points outward; it does not duplicate. Everything below is the authoritative source.
204
+
205
+ | | |
206
+ |---|---|
207
+ | **All 19 MCP tools**, every HTTP endpoint with schemas, every environment variable with its measured effect | [`docs/reference.md`](docs/reference.md) |
208
+ | Architecture, pipelines, schema, backends (SQLite · PGlite · Postgres) | [`docs/architecture.md`](docs/architecture.md) · [`docs/pglite-feature-parity.md`](docs/pglite-feature-parity.md) |
209
+ | Current benchmark numbers, generated and stamped | [`docs/benchmarks-current.md`](docs/benchmarks-current.md) |
210
+ | What each eval suite measures, and the measurement caveats | [`docs/benchmarks.md`](docs/benchmarks.md) |
211
+ | Building an agent on AWM as a substrate (PRIME → ACT → VERIFY → LEARN) | [`docs/patterns/awm-native-harness.md`](docs/patterns/awm-native-harness.md) · [agent playbook](https://completeideas.github.io/agent-working-memory/awm-for-agents.html) |
212
+ | Running it as a service; backup, restore, migration | [`docs/deployment.md`](docs/deployment.md) |
213
+ | Behaviour as the store grows | [`docs/using-awm-at-scale.md`](docs/using-awm-at-scale.md) |
214
+ | The vocabulary — engram, salience, activation, Hebbian, staging | [`docs/onboarding-vocabulary.md`](docs/onboarding-vocabulary.md) |
215
+ | When something is wrong | [`docs/troubleshooting.md`](docs/troubleshooting.md) |
216
+ | Full index | [`docs/README.md`](docs/README.md) |
217
+
218
+ **Stack:** TypeScript · SQLite + FTS5 (or PGlite / Postgres) · Fastify · `@modelcontextprotocol/sdk` ·
219
+ local ONNX via `@huggingface/transformers` — bge-small-en-v1.5 embeddings, ms-marco-MiniLM cross-encoder,
220
+ flan-t5-small expansion. Node 22+.
221
+
222
+ ```bash
223
+ npx vitest run # 760 tests
224
+ npm run bench # benchmark suites
225
+ npm run test:docker # clean-room install of the packed tarball
226
+ npm run test:linux # build + full suite on Linux (760/760)
227
+ ```
228
+
229
+ ---
230
+
231
+ ## Status
232
+
233
+ Active development, v0.14.6. Core retrieval, consolidation, MCP integration, hooks, task
234
+ management, and the HTTP API are stable and in daily production use. PGlite backend stable;
235
+ networked Postgres experimental.
236
+
237
+ Release notes, and every benchmark correction recorded against the release that made it:
238
+ [CHANGELOG.md](CHANGELOG.md).
239
+
240
+ **License:** Apache 2.0 — [LICENSE](LICENSE) · [NOTICE](NOTICE)