agent-memory-integrity 0.5.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,410 @@
1
+ Metadata-Version: 2.4
2
+ Name: agent-memory-integrity
3
+ Version: 0.5.0
4
+ Summary: Agent Memory Integrity (agmi): a conformance test suite that measures whether AI agent memory and checkpoint stores notice tampering
5
+ Author-email: Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
6
+ Maintainer-email: Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
7
+ License: MIT License
8
+
9
+ Copyright (c) 2026 Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
10
+
11
+ Permission is hereby granted, free of charge, to any person obtaining a copy
12
+ of this software and associated documentation files (the "Software"), to deal
13
+ in the Software without restriction, including without limitation the rights
14
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
15
+ copies of the Software, and to permit persons to whom the Software is
16
+ furnished to do so, subject to the following conditions:
17
+
18
+ The above copyright notice and this permission notice shall be included in all
19
+ copies or substantial portions of the Software.
20
+
21
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
22
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
23
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
24
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
25
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
26
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
27
+ SOFTWARE.
28
+
29
+ Project-URL: Homepage, https://github.com/tech4biz-yasha/agmi
30
+ Project-URL: Issues, https://github.com/tech4biz-yasha/agmi/issues
31
+ Keywords: ai-agents,agent-memory,integrity,security,langgraph,letta,mem0
32
+ Classifier: Development Status :: 3 - Alpha
33
+ Classifier: Intended Audience :: Developers
34
+ Classifier: License :: OSI Approved :: MIT License
35
+ Classifier: Programming Language :: Python :: 3
36
+ Classifier: Topic :: Security
37
+ Classifier: Topic :: Software Development :: Testing
38
+ Requires-Python: >=3.10
39
+ Description-Content-Type: text/markdown
40
+ License-File: LICENSE
41
+ Provides-Extra: dev
42
+ Requires-Dist: pytest; extra == "dev"
43
+ Provides-Extra: langgraph
44
+ Requires-Dist: langgraph-checkpoint-sqlite; extra == "langgraph"
45
+ Provides-Extra: letta
46
+ Requires-Dist: letta; extra == "letta"
47
+ Requires-Dist: asyncpg; extra == "letta"
48
+ Requires-Dist: psycopg2-binary; extra == "letta"
49
+ Requires-Dist: pgvector; extra == "letta"
50
+ Requires-Dist: pgserver; extra == "letta"
51
+ Provides-Extra: mem0
52
+ Requires-Dist: mem0ai; extra == "mem0"
53
+ Requires-Dist: qdrant-client; extra == "mem0"
54
+ Provides-Extra: inspeximus
55
+ Requires-Dist: inspeximus[crypto]>=2.38.0; extra == "inspeximus"
56
+ Dynamic: license-file
57
+
58
+ <p align="center">
59
+ <img src="docs/logo.svg" width="96" alt="agmi">
60
+ </p>
61
+
62
+ <h1 align="center">agmi</h1>
63
+ <p align="center"><strong>Agent Memory Integrity</strong><br>
64
+ A conformance test suite that measures whether AI agent memory and checkpoint stores notice when they are tampered with.</p>
65
+
66
+ <p align="center">
67
+ <a href="https://github.com/tech4biz-yasha/agmi/actions/workflows/scorecard.yml"><img src="https://github.com/tech4biz-yasha/agmi/actions/workflows/scorecard.yml/badge.svg" alt="scorecard"></a>
68
+ <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT"></a>
69
+ <img src="https://img.shields.io/badge/python-3.10%2B-blue.svg" alt="python">
70
+ <img src="https://img.shields.io/badge/real%20targets-3-green.svg" alt="targets">
71
+ </p>
72
+
73
+ [![Preprint DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.22765627.svg)](https://doi.org/10.5281/zenodo.22765627)
74
+ [![SSRN](https://img.shields.io/badge/SSRN-7461118-blue)](https://ssrn.com/abstract=7461118)
75
+ [![Software DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.22860887.svg)](https://doi.org/10.5281/zenodo.22860887)
76
+
77
+ ---
78
+
79
+ ## The result in one table
80
+
81
+ Three of the most used agent memory layers were seeded through their own APIs, edited behind their backs, and asked to read their memory again. None of them noticed.
82
+
83
+ | Target | Version | tamper | truncate | delete_middle | reorder | forge |
84
+ |---|---|:-:|:-:|:-:|:-:|:-:|
85
+ | LangGraph `SqliteSaver` | langgraph-checkpoint-sqlite 3.1.1 | accepted | accepted | accepted | accepted | accepted |
86
+ | Letta core memory checkpoint history | letta 0.16.8 | accepted | accepted | accepted | accepted | accepted |
87
+ | Mem0 local Qdrant store | mem0ai 2.0.20 | accepted | accepted | accepted | accepted | accepted |
88
+ | inspeximus, receipts off (default), read path | inspeximus 2.38.0 | accepted | accepted | accepted | accepted | accepted |
89
+ | inspeximus, receipts on with a key, attacker holds the store's directory | inspeximus 2.38.0 | reported | reported | reported | reported | reported |
90
+ | inspeximus, receipts on with a key, attacker also holds the user's config home | inspeximus 2.38.0 | reported | accepted | reported | reported | reported |
91
+
92
+ "Accepted" means the tool loaded the altered store, raised nothing, and the agent carried on from the altered memory as if it were true. "Reported" means the tool's own integrity check named the problem after a reload, and only that. After any of the five attacks the store still loads and the read path (`recall()` for inspeximus) answers from the altered store, so a reported cell says a separate audit call (`verify_writes()` in the inspeximus rows) caught it, not that the agent was protected at read time. `full_runner` names the detection point in a checkedAt column: "read" when verify() is the read path, "audit" when it is a call the operator has to make. This table has no such column; every reported cell in it is an audit detection. Every row is a measurement of the real library at the version shown, reproducible in under a minute, and pinned by a test that fails the day that library adds a check.
93
+
94
+ This is a design gap, not a bug. LangGraph, Letta and Mem0 do not claim their stores are tamper evident. inspeximus makes that claim for its receipts mode, and the table shows what that buys and where it stops. The point of agmi is that nobody had measured the gap with one yardstick, and that the gap matters the moment agent memory is used as a record.
95
+
96
+ ## Quick start
97
+
98
+ ```bash
99
+ git clone https://github.com/tech4biz-yasha/agmi && cd agmi
100
+ python3 -m venv .venv && source .venv/bin/activate
101
+ pip install -e ".[dev,langgraph,letta,mem0,inspeximus]"
102
+ PYTHONPATH=. python3 agmi/full_runner.py 2>/dev/null | grep "|"
103
+ ```
104
+
105
+ Everything runs offline. No API keys, no model downloads, no Docker. The Letta row starts an embedded Postgres through `pgserver`; set `LETTA_PG_URI` if you would rather point it at your own.
106
+
107
+ ## Contents
108
+
109
+ 1. [Why this exists](#why-this-exists)
110
+ 2. [Threat model](#threat-model)
111
+ 3. [How one measurement works](#how-one-measurement-works)
112
+ 4. [Architecture](#architecture)
113
+ 5. [Attack catalogue](#attack-catalogue)
114
+ 6. [Targets and what each measurement means](#targets-and-what-each-measurement-means)
115
+ 7. [Reading the scorecard honestly](#reading-the-scorecard-honestly)
116
+ 8. [Writing an adapter](#writing-an-adapter)
117
+ 9. [Roadmap](#roadmap)
118
+ 10. [Contributing, security, citation](#contributing-security-citation)
119
+
120
+ ## Why this exists
121
+
122
+ Agent frameworks persist two kinds of state so an agent survives a restart: execution checkpoints (where the graph was, what each channel held) and long term memory (facts about the user, past decisions, retrieved context). Both are written to a database or a file and read back later as truth.
123
+
124
+ These stores were built for recovery. Recovery asks "can I load this?". Integrity asks "is this what was written?". Almost every store answers the first question and never asks the second. That is fine while the store is as trusted as the process. It stops being fine when:
125
+
126
+ - the memory is the audit trail (finance, healthcare, compliance, anything with a regulator);
127
+ - the store is shared infrastructure reachable by more than the agent (multi tenant hosting, a Postgres several services share, a mounted volume);
128
+ - an agent's past decisions are replayed to justify its next one;
129
+ - a lower privileged process, backup job or migration script can write where the agent reads.
130
+
131
+ agmi exists to give one common yardstick for that second question, across tools, with numbers a maintainer can reproduce and a buyer can compare.
132
+
133
+ ## Threat model
134
+
135
+ The attacker has write access to the backing store and nothing else.
136
+
137
+ ```mermaid
138
+ flowchart LR
139
+ subgraph trusted["Tool process (trusted)"]
140
+ A[Agent] --> T[Memory / checkpoint library]
141
+ end
142
+ T -- "put() / add()" --> S[(Backing store<br/>SQLite file, Postgres rows,<br/>pickled blobs)]
143
+ S -- "get() / list() / search()" --> T
144
+ X((Attacker)) -. "direct write:<br/>UPDATE, DELETE, INSERT,<br/>edit file bytes" .-> S
145
+ style X fill:#fee,stroke:#c00
146
+ ```
147
+
148
+ Concretely the attacker can:
149
+
150
+ - run SQL against the tool's database;
151
+ - rewrite bytes inside a file the tool reads;
152
+ - insert rows that look like the tool wrote them.
153
+
154
+ The attacker cannot:
155
+
156
+ - run code inside the tool's process;
157
+ - see or use keys the tool holds only in memory;
158
+ - change the tool's source.
159
+
160
+ This is the "database compromise or privileged write at rest" model. It is the model behind the real checkpointer deserialization CVEs, and it is the model a compliance reviewer assumes when they ask whether a log can be rewritten.
161
+
162
+ ## How one measurement works
163
+
164
+ Every cell in the scorecard is produced by the same four steps. The tool's own API is used on both sides of the tampering, so the result is the tool's answer, never ours.
165
+
166
+ ```mermaid
167
+ sequenceDiagram
168
+ participant R as Runner
169
+ participant Ad as Adapter
170
+ participant Tool as Target library
171
+ participant Store as Backing store
172
+
173
+ R->>Ad: setup()
174
+ Ad->>Tool: create fresh store
175
+ R->>Ad: seed(5)
176
+ Ad->>Tool: put() / add() x5 through the normal API
177
+ Tool->>Store: writes 5 entries
178
+
179
+ R->>Ad: attack.tamper()
180
+ Ad->>Store: raw edit that bypasses the tool
181
+
182
+ R->>Ad: reload()
183
+ Ad->>Tool: reopen the store
184
+ R->>Ad: verify()
185
+ Ad->>Tool: get() / list() / search() / undo()
186
+ Tool-->>Ad: loaded fine, or raised
187
+ Ad-->>R: True = accepted, False = detected
188
+ ```
189
+
190
+ The pass/fail rule is deliberately narrow. A tool is **detected** only if it raises, refuses or reports the problem itself on reload. A tool that loads the altered store and answers normally is **accepted**. We never infer detection from the content coming back different, because the tool did not say anything.
191
+
192
+ ## Architecture
193
+
194
+ Attacks are written once against a small adapter interface. Each target gets one adapter that knows where its data lives and how to edit it raw. Adding a tool is one file; the attacks do not change.
195
+
196
+ ```mermaid
197
+ flowchart TB
198
+ subgraph attacks["agmi/attacks"]
199
+ AR["at_rest.py<br/>tamper, truncate, delete_middle,<br/>reorder, forge"]
200
+ MS["memory_specific.py<br/>injection, cross_session_bleed,<br/>retrieval_hijack, indirect_prompt_injection"]
201
+ end
202
+
203
+ subgraph iface["agmi/adapters/base.py"]
204
+ MA["MemoryAdapter<br/>setup, seed, read_all_raw,<br/>write_raw, delete_raw, reload, verify<br/>mutate_payload, forge_record"]
205
+ SA["SemanticMemoryAdapter<br/>add, search per user"]
206
+ end
207
+
208
+ subgraph adapters["agmi/adapters"]
209
+ LG["langgraph_sqlite.py"]
210
+ LT["letta_block_history.py"]
211
+ M0["mem0_at_rest.py"]
212
+ OF["openfang.py (model)"]
213
+ NM["naive_memory.py (baseline)"]
214
+ end
215
+
216
+ subgraph targets["Real libraries"]
217
+ LGL["langgraph-checkpoint-sqlite"]
218
+ LTL["letta + Postgres"]
219
+ M0L["mem0ai + qdrant-client"]
220
+ end
221
+
222
+ RUN["full_runner.py<br/>runs every attack on every adapter<br/>and prints the scorecard"] --> AR
223
+ RUN --> MS
224
+ AR --> MA
225
+ MS --> SA
226
+ MA --> LG
227
+ MA --> LT
228
+ MA --> M0
229
+ MA --> OF
230
+ SA --> NM
231
+ LG --> LGL
232
+ LT --> LTL
233
+ M0 --> M0L
234
+ ```
235
+
236
+ Folder map:
237
+
238
+ ```
239
+ agmi/
240
+ attacks/
241
+ base.py Attack contract and AttackResult (detected / accepted / n/a / error)
242
+ at_rest.py The five at-rest attacks, written once for every adapter
243
+ memory_specific.py The four retrieval attacks for user-scoped semantic memory
244
+ adapters/
245
+ base.py MemoryAdapter interface plus the two payload hooks
246
+ semantic_base.py SemanticMemoryAdapter interface for retrieval tools
247
+ langgraph_sqlite.py Real LangGraph SqliteSaver
248
+ letta_block_history.py Real Letta core memory checkpoint history (Postgres)
249
+ mem0_at_rest.py Real Mem0 on its local Qdrant store, offline
250
+ openfang.py Python model of OpenFang's hash-chained audit log
251
+ naive_memory.py Deliberately undefended retrieval baseline
252
+ full_runner.py Builds the matrix and prints the scorecard
253
+ tests/ One pinned test module per real target
254
+ .github/workflows/ Scorecard on every push, plus weekly re-measurement
255
+ ```
256
+
257
+ ## Attack catalogue
258
+
259
+ Each attack has one precise rule. There are no heuristics and no scoring thresholds in the at-rest set.
260
+
261
+ | Attack | What the attacker does to the store | Detected means | Why it matters |
262
+ |---|---|---|---|
263
+ | `tamper` | Changes the content of one entry in the middle, without breaking its encoding | Tool refuses or flags the entry on reload | Silent rewriting of a past memory or decision |
264
+ | `truncate` | Deletes the newest two entries | Tool notices the chain ends early | Rolling an agent back to an older state and erasing recent actions from the record |
265
+ | `delete_middle` | Removes one entry from the middle | Tool notices a hole in the sequence | Erasing one inconvenient event from a history that still looks continuous |
266
+ | `reorder` | Swaps the content of two entries | Tool notices the sequence is out of order | Changing what happened before what |
267
+ | `forge` | Inserts a fabricated entry after the tip, with a valid looking id and parent | Tool rejects the unsigned or unchained entry | Planting a memory or checkpoint the agent then resumes from |
268
+
269
+ The memory-specific set asks a different question, "did attacker content reach the agent or cross a user boundary", and applies only to user-scoped retrieval tools:
270
+
271
+ | Attack | Question it answers |
272
+ |---|---|
273
+ | `memory_injection` | Does a planted memory later retrieve as fact for an innocent query? |
274
+ | `cross_session_bleed` | Can user B retrieve what user A stored? |
275
+ | `retrieval_hijack` | Can an entry be crafted to surface for unrelated queries? |
276
+ | `indirect_prompt_injection` | Does instruction-shaped stored content get delivered into retrieved context? |
277
+
278
+ `indirect_prompt_injection` measures delivery into context, not whether a model obeys it. A portable suite cannot drive every tool's live model; delivery is the property the tool owns.
279
+
280
+ ## Targets and what each measurement means
281
+
282
+ ### LangGraph `SqliteSaver`
283
+
284
+ | | |
285
+ |---|---|
286
+ | Measured on | langgraph-checkpoint-sqlite 3.1.1, langgraph-checkpoint 4.2.0 |
287
+ | What is targeted | The `checkpoints` table: one row per checkpoint, msgpack blob, parent id, time-ordered UUID |
288
+ | Seeded through | `SqliteSaver.put()` |
289
+ | Read back through | `SqliteSaver.get()` and `list()` |
290
+ | verify() | True if the thread loads and every row deserializes |
291
+
292
+ There is no integrity logic on the store. The only thing that can fail on reload is deserialization, so a tamper that keeps the msgpack valid is invisible. After `forge` the agent resumes from the attacker's checkpoint.
293
+
294
+ ### Letta core memory checkpoint history
295
+
296
+ | | |
297
+ |---|---|
298
+ | Measured on | letta 0.16.8 (Postgres only since 0.13; embedded via pgserver here) |
299
+ | What is targeted | `block_history`: one row per checkpoint of a core memory block with a `sequence_number`, plus `block.current_history_entry_id` |
300
+ | Seeded through | `BlockManager.create_or_update_block_async`, `update_block_async`, `checkpoint_block_async` |
301
+ | Read back through | `get_block_by_id_async`, then `undo_checkpoint_block` to the start and `redo_checkpoint_block` back |
302
+ | verify() | True if Letta raises nothing during the full undo and redo walk |
303
+
304
+ Letta's undo and redo are written to tolerate missing sequence numbers, so a holed or truncated history is invisible by design. After `truncate` the agent's core memory silently rewinds two checkpoints and every call succeeds. After `forge` the agent's core memory is the attacker's text.
305
+
306
+ ### Mem0 local Qdrant store
307
+
308
+ | | |
309
+ |---|---|
310
+ | Measured on | mem0ai 2.0.20, qdrant-client local mode |
311
+ | What is targeted | `points` table of the on-disk Qdrant collection (one pickled `PointStruct` per memory) and Mem0's `history` SQLite table |
312
+ | Seeded through | `Memory.add(infer=False)` with a deterministic offline embedder |
313
+ | Read back through | `get_all()`, `search()`, `history()` |
314
+ | verify() | True if all three succeed |
315
+
316
+ Mem0 writes an ADD event to `history` for every memory and stores an md5 of each memory's text. Neither is checked: the hash is for de-duplication and the history is never reconciled with the vector store. After `truncate` the history still lists five memories while the agent can see three, and Mem0 reports nothing.
317
+
318
+ The memory-specific columns stay `n/a` for Mem0 because retrieval ranking under the offline embedder would measure our embedder, not Mem0.
319
+
320
+ ### inspeximus
321
+
322
+ | | |
323
+ |---|---|
324
+ | Measured on | inspeximus 2.38.0, submitted by the inspeximus maintainer; reproduced independently by agmi on inspeximus 3.0.0 (macOS, Python 3.12). Receipts rows: `Inspeximus(path, receipts=True, receipt_key=sk)` with a fresh Ed25519 key. Default row: `Inspeximus(path)` |
325
+ | What is targeted | The `records` table of the SQLite store, one JSON document per memory, and `<store>.receipts.json`, the signed hash chain of write receipts, both in the store's directory. In the third row also the chain head the store keeps in the user's config home |
326
+ | Seeded through | `remember(text, key=...)` |
327
+ | Read back through | Receipts rows: `verify_writes(expected_pubkey=pk)`, the store's own audit method (also its `verify_writes` MCP tool). Default row: the store loads, `recall()` answers, `history()` answers |
328
+ | verify() | Receipts rows: True if the receipt chain recomputes, every stored record matches its receipt, and the chain is not shorter than the head kept outside the directory. Default row: True if the read path raises nothing |
329
+
330
+ Three rows, because the answer depends on the configuration and on what the attacker holds. Receipts are off on a fresh store. Off, nothing checks the rows and the store reads like LangGraph: five accepted. Off, `verify_writes()` also refuses to vouch for any store, touched or not, which would score every attack "reported" for the wrong reason, so the receipts rows seed with receipts on and a fresh key.
331
+
332
+ With receipts on, each write gets a receipt that commits to the record's text, key, type and attribution, chained by hash to the previous receipt and signed, and the store writes the chain's head (first receipt, count, tip) to the user's config home after every receipt. `verify_writes()` recomputes the chain, compares each stored record with its receipt, and compares the chain on disk with that head; the named-tamper test shows the altered row's id in the problems list. The second row is the README's attacker, write access to the backing store: the SQLite file and the receipts sidecar. Tamper, reorder and forge are reported because the receipts are signed and the attacker has no key. `delete_middle` is reported because the receipt after the gap names the missing one as its predecessor, and that link is inside the signed payload. `truncate` is reported because the chain is shorter than the head, and the agent's own later writes do not lower the head.
333
+
334
+ The third row gives the attacker the config home as well, so the head goes with the cut. Four stay reported; `truncate` is accepted: a tail cut with its receipts leaves a shorter chain that is internally consistent, and no file outside the attacker's reach records the earlier length. Any anchor the same user account can write, wherever it sits, shares that limit; only an anchor off the machine does not. The remedy inspeximus offers for it is `anchor()` handed to a witness plus `verify_consistency()`; a test in `tests/test_inspeximus_rows.py` shows an anchor taken earlier reporting `write log shrank: 3 < anchored 5`. agmi does not model an anchor off the machine, so the cell stays accepted.
335
+
336
+ Two limits to read the receipts rows by. Detection is the audit call: after any of the five attacks the store still loads and `recall()` serves the altered record, as with the other targets. And a receipt commits to text, key, type and attribution; an at-rest edit to a field outside that set, such as the timestamp, verifies clean.
337
+
338
+ The adapter is `agmi/adapters/inspeximus_rows.py`; `pip install -e ".[inspeximus]"` (the extra pulls `inspeximus[crypto]`, since Ed25519 signing needs the `cryptography` package).
339
+
340
+ ### Reference rows
341
+
342
+ `openfang(model,fixed)` is a Python re-implementation of OpenFang's hash-chained audit log, including the tip persistence fix from [openfang PR #1287](https://github.com/RightNow-AI/openfang/pull/1287). It proves the five attacks are detectable by a chained store. It is not a measurement of the Rust binary.
343
+
344
+ `naive-mem` is a deliberately undefended retriever. A "safe" from it is a weak signal and exists only so the memory-specific attacks have something to run against until real retrieval adapters land.
345
+
346
+ ## Reading the scorecard honestly
347
+
348
+ - **`n/a` is information.** Which attacks apply depends on what a tool claims to be. An audit log cannot be memory-injected; a bare vector store has no chain to truncate. No tool faces all nine. The map of which cells apply is part of the finding.
349
+ - **"Accepted" is not "vulnerable to remote attack".** The attacker already has store access. The question is only whether the tool can tell.
350
+ - **Model rows are labelled.** Anything not measured against the real library says `(model)` in its name and is excluded from the headline table.
351
+ - **Versions are pinned.** Each real target has a test asserting the measured result at the measured version. When a maintainer adds a check the test fails, the CI goes red, and the scorecard gets updated with the new version and a note. The weekly CI run does this against the latest release without anyone needing to remember.
352
+
353
+ ## Writing an adapter
354
+
355
+ One file. Implement `MemoryAdapter` from `agmi/adapters/base.py`:
356
+
357
+ ```python
358
+ class MyToolAdapter(MemoryAdapter):
359
+ name = "mytool-store"
360
+
361
+ def setup(self): ... # fresh, isolated store in a temp dir or scratch db
362
+ def teardown(self): ...
363
+ def seed(self, n): ... # write n entries through the TOOL'S OWN API
364
+ def read_all_raw(self): ... # list[Record] in chain order, raw fields, bypassing the tool
365
+ def write_raw(self, rec): ... # write one Record back, raw
366
+ def delete_raw(self, seq): ...
367
+ def reload(self): ... # reopen the store the way a restart would
368
+ def verify(self): ... # the TOOL'S answer: True loaded fine, False it complained
369
+
370
+ # Optional hooks for blob-based stores
371
+ def mutate_payload(self, rec): ... # change meaning without breaking encoding
372
+ def forge_record(self, tmpl): ... # a plausible new tip with a valid-looking id
373
+ ```
374
+
375
+ Rules that keep a row honest:
376
+
377
+ 1. Seed and verify through the tool's public API, never through the raw store.
378
+ 2. `verify()` reports what the tool says. Do not compare content and call a difference "detected".
379
+ 3. Pin the version in a test, as `tests/test_langgraph_sqlite.py` does.
380
+ 4. If the target needs a hosted model or an API key to run, nobody can reproduce it; find an offline path or mark the cell `n/a` with a reason.
381
+ 5. Label anything that is not the real library `(model)`.
382
+
383
+ Then add the adapter to `full_runner.py` and open a PR with the new scorecard row.
384
+
385
+ ## Roadmap
386
+
387
+ | Version | Scope | Status |
388
+ |---|---|---|
389
+ | 0.1 | Attack catalogue, adapter interface, OpenFang model, naive baseline | done |
390
+ | 0.2 to 0.5 | Real at-rest measurements for LangGraph, Letta and Mem0; licensing; CI | done |
391
+ | 0.6 | Memory-specific attacks on real retrieval tools with real embedders (Mem0, Graphiti) | next |
392
+ | 0.7 | Deserialization safety: crafted stored payloads that execute on load. Both LangGraph and Mem0 still unpickle from their stores, and one has patched a version of this before | planned |
393
+ | 0.8 | In-flow attacks: replay, cross-thread poison, rollback during execution | planned |
394
+ | 0.9 | Reference integrity layer: a hash chain in checkpoint metadata, offered upstream as an optional mode | planned |
395
+ | 1.0 | Stable adapter interface, published conformance levels, monthly report cadence | planned |
396
+
397
+ ## Contributing, security, citation
398
+
399
+ - Contributions: see [CONTRIBUTING.md](CONTRIBUTING.md). New real-library adapters are the most valuable thing you can send.
400
+ - Security: agmi finds design gaps and discusses them in public. If you find an actual vulnerability in a target using this suite, see [SECURITY.md](SECURITY.md) and do not open a public issue.
401
+ - Changes: [CHANGELOG.md](CHANGELOG.md).
402
+
403
+ If you use agmi in a paper, a review or a procurement decision, cite it:
404
+
405
+ ```
406
+ Yasha Khandelwal (2026). agmi: Agent Memory Integrity, a conformance test suite for
407
+ tamper evidence in AI agent memory and checkpoint stores. https://github.com/tech4biz-yasha/agmi
408
+ ```
409
+
410
+ MIT licensed. Copyright (c) 2026 Yasha Khandelwal, yasha.khandelwal@tech4biz.io.
@@ -0,0 +1,24 @@
1
+ agent_memory_integrity-0.5.0.dist-info/licenses/LICENSE,sha256=6-KsO8_X7MZp6N0I-kMZE6DUkqYn3AgnJj1-guTrSks,1104
2
+ agmi/__init__.py,sha256=-D5xMIPOk9Vin44KIaGwbhLsv6WEbli0nH2VMUkJvpI,142
3
+ agmi/full_runner.py,sha256=gwBGFKjjZ-5vMAwwhSG4O8Mkokx-0cT03ORfNWe5XY4,4801
4
+ agmi/mem0_runner.py,sha256=Ng58gKYPyHaRHl8CLhdZ2IGloc1NneXoEvBtvSZvU98,5236
5
+ agmi/runner.py,sha256=ZG3rxuut9xE_FuvyxSsrwHtt83JihrED8SEJOFxdoIk,2420
6
+ agmi/adapters/__init__.py,sha256=-D5xMIPOk9Vin44KIaGwbhLsv6WEbli0nH2VMUkJvpI,142
7
+ agmi/adapters/base.py,sha256=eYm8VAXo7jIHNccAjeDww4V1IlmQfxNT7gJCjh5Cd1M,4298
8
+ agmi/adapters/inspeximus_rows.py,sha256=MaG_7Z8d0n3r3dJ8b375hzERVQ1sO40ou-vXttqgrCI,12096
9
+ agmi/adapters/langgraph_sqlite.py,sha256=85LadZ3LvnhEASVxJPdc1NGJVPeK0R13-QfGc135OQY,8021
10
+ agmi/adapters/letta_block_history.py,sha256=INYl1AdUTD2Yh2HPN2S7OGBH-4i2zpoyLlnVnjsKw6w,11492
11
+ agmi/adapters/mem0_adapter.py,sha256=n7n9TbeoOgww-VPZJUeuCQl-gXxNJdejPKd4cqd2D34,3983
12
+ agmi/adapters/mem0_at_rest.py,sha256=EL0CxEQWiV6AJ3Ps-pBa56qqsXfMpTdkVxm-1prZ4u8,8169
13
+ agmi/adapters/naive_memory.py,sha256=5gO6GjEo0ZFGkHJaXqgcObV0EgQ4ywu6dPrg93ZjJCE,2081
14
+ agmi/adapters/openfang.py,sha256=1rv1T8_bg6Nb47ZolpcFp5TROjZ7DzNYkXEEb8SN0UQ,6868
15
+ agmi/adapters/semantic_base.py,sha256=NaFbufk0bu4ydjXhu4O-TFOKAipIatrCH4JuowvWAgo,2464
16
+ agmi/attacks/__init__.py,sha256=-D5xMIPOk9Vin44KIaGwbhLsv6WEbli0nH2VMUkJvpI,142
17
+ agmi/attacks/at_rest.py,sha256=4Yn-YKsHdbLZt2FPt5mreYPT1D0hfZzIz34ISBYXu3E,3706
18
+ agmi/attacks/base.py,sha256=TTUd8GiBmO4F88FbKEbnIInk9MTeJ5B0VIsZef1Cr1A,3243
19
+ agmi/attacks/memory_specific.py,sha256=kr3X_Tf1md6sTGHbT_1IiZXl5I0ClcO0kNcSCmiCFMg,9106
20
+ agent_memory_integrity-0.5.0.dist-info/METADATA,sha256=bskwrvPuY4MBcDZ8BqgpoteSRTewK_eUDLNa8TDDu3Q,25410
21
+ agent_memory_integrity-0.5.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
22
+ agent_memory_integrity-0.5.0.dist-info/entry_points.txt,sha256=jpspniVV0Fihw0a7_-Kh2ugkepUpf1FxSD10oNjduoo,42
23
+ agent_memory_integrity-0.5.0.dist-info/top_level.txt,sha256=bVSt4aZR3vtJyPp5FP2zk6Cu2q-ktUvHHANmhWayPvI,5
24
+ agent_memory_integrity-0.5.0.dist-info/RECORD,,
@@ -0,0 +1,5 @@
1
+ Wheel-Version: 1.0
2
+ Generator: setuptools (84.0.0)
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
5
+
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ agmi = agmi.runner:main
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
agmi/__init__.py ADDED
@@ -0,0 +1,3 @@
1
+ # agmi: Agent Memory Integrity test suite
2
+ # Copyright (c) 2026 Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
3
+ # SPDX-License-Identifier: MIT
@@ -0,0 +1,3 @@
1
+ # agmi: Agent Memory Integrity test suite
2
+ # Copyright (c) 2026 Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
3
+ # SPDX-License-Identifier: MIT
agmi/adapters/base.py ADDED
@@ -0,0 +1,109 @@
1
+ # agmi: Agent Memory Integrity test suite
2
+ # Copyright (c) 2026 Yasha Khandelwal <yasha.khandelwal@tech4biz.io>
3
+ # SPDX-License-Identifier: MIT
4
+
5
+ """The contract every memory-store adapter must satisfy.
6
+
7
+ An adapter is the only thing that knows tool-specific details: where the
8
+ memory is stored, how to read a raw record, how to write one back, and how
9
+ to ask the tool whether its stored state is still trustworthy.
10
+
11
+ Attacks are written against this interface alone. That is the whole point:
12
+ one attack, written once, runs against every tool that has an adapter.
13
+ """
14
+
15
+ from __future__ import annotations
16
+
17
+ from abc import ABC, abstractmethod
18
+ from dataclasses import dataclass
19
+
20
+
21
+ @dataclass
22
+ class Record:
23
+ """One stored memory/audit entry, in a tool-neutral shape.
24
+
25
+ `seq` is the position in the chain/log (0-indexed). `fields` is the
26
+ tool's own column/key values for that entry, opaque to the attacks
27
+ except that they can be mutated and written back via the adapter.
28
+ """
29
+
30
+ seq: int
31
+ fields: dict
32
+
33
+
34
+ class MemoryAdapter(ABC):
35
+ """Wraps one memory/checkpoint tool so attacks can operate on it.
36
+
37
+ Lifecycle used by every attack:
38
+ setup() -> fresh, empty store
39
+ seed(n) -> tool writes n legitimate entries through its
40
+ own normal API (so hashes/signatures are real)
41
+ snapshot() -> record the tool's own "is this trustworthy?"
42
+ answer while the store is still clean
43
+ <attack mutates the raw store via read_raw / write_raw / delete_raw>
44
+ reload() -> simulate a restart: tool re-opens the store
45
+ verify() -> tool's own integrity answer after tampering
46
+
47
+ An attack PASSES (the tool is safe) when verify() reports a problem
48
+ after the raw store was tampered with. It FAILS when verify() still
49
+ says the tampered store is fine.
50
+ """
51
+
52
+ #: Short tool name for the scorecard, e.g. "openfang".
53
+ name: str
54
+
55
+ @abstractmethod
56
+ def setup(self) -> None:
57
+ """Create a fresh, empty store for one test run."""
58
+
59
+ @abstractmethod
60
+ def teardown(self) -> None:
61
+ """Dispose of the store and any temp files."""
62
+
63
+ @abstractmethod
64
+ def seed(self, n: int) -> None:
65
+ """Write `n` legitimate entries through the tool's normal API."""
66
+
67
+ @abstractmethod
68
+ def read_all_raw(self) -> list[Record]:
69
+ """Read every stored entry directly from the backing store,
70
+ bypassing the tool. This is how attacks see the raw bytes."""
71
+
72
+ @abstractmethod
73
+ def write_raw(self, record: Record) -> None:
74
+ """Write a (possibly mutated) record back to the raw store,
75
+ bypassing the tool's own write path and integrity hooks."""
76
+
77
+ @abstractmethod
78
+ def delete_raw(self, seq: int) -> None:
79
+ """Delete the entry at `seq` directly from the raw store."""
80
+
81
+ @abstractmethod
82
+ def reload(self) -> None:
83
+ """Re-open the store as the tool would on a restart, so any
84
+ load-time integrity check runs against the tampered data."""
85
+
86
+ @abstractmethod
87
+ def verify(self) -> bool:
88
+ """Return True if the tool reports its stored state is intact,
89
+ False if it detects tampering. Attacks invert this into pass/fail."""
90
+
91
+ # --- optional payload hooks --------------------------------------
92
+ # Content-level attacks (tamper, forge) need to change what an entry
93
+ # SAYS without corrupting how it is ENCODED. Only the adapter knows
94
+ # that. The defaults work for row-per-entry stores with plain string
95
+ # columns; blob-based stores override them.
96
+
97
+ def mutate_payload(self, record: Record) -> Record:
98
+ """Return `record` with its human-meaningful content altered."""
99
+ for key, value in record.fields.items():
100
+ if isinstance(value, str) and key not in ("hash", "prev_hash"):
101
+ record.fields[key] = value + " [TAMPERED]"
102
+ return record
103
+ raise RuntimeError("no mutable string field in record")
104
+
105
+ def forge_record(self, template: Record) -> Record:
106
+ """Return a new record that would sit after `template` and passes
107
+ as authentic on inspection."""
108
+ forged = Record(seq=template.seq + 1, fields=dict(template.fields))
109
+ return self.mutate_payload(forged)