@1mbrain/benchmarks 0.1.1

package/package.json

@@ -0,0 +1,22 @@
+{
+  "name": "@1mbrain/benchmarks",
+  "version": "0.1.1",
+  "description": "Provider-level benchmarks for 1MBrain and comparable memory providers.",
+  "license": "MIT",
+  "type": "module",
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "scripts": {
+    "bench": "tsx src/runner.ts",
+    "build": "tsc -p tsconfig.json",
+    "typecheck": "tsc --noEmit"
+  },
+  "dependencies": {
+    "@1mbrain/core": "*",
+    "mem0ai": "^3.0.9"
+  },
+  "devDependencies": {
+    "tsx": "^4.19.0",
+    "typescript": "^5.7.0"
+  }
+}

package/reports/benchmark_report.md

@@ -0,0 +1,48 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `graph-stress-hard` dataset (60 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 1 | 1 | 0.917 | 7.49ms | 9.552ms/case |
+| 1MBrain Graph Light | 1 | 1 | 0.917 | 6.65ms | 8.098ms/case |
+| 1MBrain Vector Only | 0.361 | 0.889 | 0.786 | 2.164ms | 7.73ms/case |
+| Vector Baseline (SQLite) | 0.322 | 0.883 | 0.752 | 0.673ms | 1.392ms/case |
+| Qdrant Vector | N/A | N/A | N/A | N/A | N/A (Unsupported) |
+| Mem0 (Cloud) | N/A | N/A | N/A | N/A | N/A (Unsupported) |
+| Zep/Graphiti | N/A | N/A | N/A | N/A | N/A (Unsupported) |
+| Letta | N/A | N/A | N/A | N/A | N/A (Unsupported) |
+| LangMem | N/A | N/A | N/A | N/A | N/A (Unsupported) |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **210.345%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **1** for Graph Full versus **0.602** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **7.49ms**, compared to **0.673ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **1** compared to **0.361** for 1MBrain Vector Only, a **176.923%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.602** to **1**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **1**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **6.817ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_adversarial.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-adversarial` dataset (60 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.789 | 0.906 | 0.732 | 2.859ms | 5.058ms/case |
+| 1MBrain Vector Only | 0.789 | 0.906 | 0.732 | 1.935ms | 4.899ms/case |
+| Vector Baseline (SQLite) | 0.789 | 0.906 | 0.732 | 0.713ms | 1.237ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.833** for Graph Full versus **0.833** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **2.859ms**, compared to **0.713ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.789** compared to **0.789** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.833** to **0.833**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.422**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **2.146ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_adversarial_adaptive.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-adversarial` dataset (60 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.789 | 0.906 | 0.731 | 5.144ms | 9.504ms/case |
+| 1MBrain Vector Only | 0.789 | 0.906 | 0.732 | 4.164ms | 10.152ms/case |
+| Vector Baseline (SQLite) | 0.789 | 0.906 | 0.732 | 1.301ms | 2.721ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.833** for Graph Full versus **0.833** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **5.144ms**, compared to **1.301ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.789** compared to **0.789** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.833** to **0.833**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.422**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **3.842ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_adversarial_adaptive2_fast.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-adversarial` dataset (60 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.789 | 0.906 | 0.731 | 2.264ms | 4.355ms/case |
+| 1MBrain Vector Only | 0.789 | 0.906 | 0.732 | 1.854ms | 4.179ms/case |
+| Vector Baseline (SQLite) | 0.789 | 0.906 | 0.732 | 0.492ms | 1.099ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.833** for Graph Full versus **0.833** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **2.264ms**, compared to **0.492ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.789** compared to **0.789** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.833** to **0.833**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.422**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **1.772ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_adversarial_adaptive_fast.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-adversarial` dataset (60 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.756 | 0.872 | 0.734 | 2.795ms | 4.394ms/case |
+| 1MBrain Vector Only | 0.789 | 0.906 | 0.732 | 1.99ms | 4.169ms/case |
+| Vector Baseline (SQLite) | 0.789 | 0.906 | 0.732 | 0.502ms | 1.092ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **-4.225%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.833** for Graph Full versus **0.833** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **2.795ms**, compared to **0.502ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.756** compared to **0.789** for 1MBrain Vector Only, a **-4.225%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.833** to **0.833**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.422**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **2.293ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_adversarial_rerank.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-adversarial` dataset (60 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.686 | 0.819 | 0.602 | 11.214ms | 10.703ms/case |
+| 1MBrain Vector Only | 0.789 | 0.906 | 0.732 | 6.429ms | 11.327ms/case |
+| Vector Baseline (SQLite) | 0.789 | 0.906 | 0.732 | 1.531ms | 2.597ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **-13.028%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.917** for Graph Full versus **0.833** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **11.214ms**, compared to **1.531ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.686** compared to **0.789** for 1MBrain Vector Only, a **-13.028%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.833** to **0.917**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.511**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **9.684ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_balanced_mini.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-balanced-mini` dataset (40 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.802 | 0.902 | 0.711 | 2.347ms | 3.656ms/case |
+| 1MBrain Vector Only | 0.802 | 0.902 | 0.711 | 1.406ms | 3.395ms/case |
+| Vector Baseline (SQLite) | 0.802 | 0.902 | 0.711 | 0.694ms | 0.883ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.646** for Graph Full versus **0.646** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **2.347ms**, compared to **0.694ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.802** compared to **0.802** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.646** to **0.646**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.65**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **1.654ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_balanced_mini_adaptive.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-balanced-mini` dataset (40 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.802 | 0.902 | 0.682 | 4.102ms | 4.766ms/case |
+| 1MBrain Vector Only | 0.802 | 0.902 | 0.711 | 3.356ms | 6.928ms/case |
+| Vector Baseline (SQLite) | 0.802 | 0.902 | 0.711 | 1.171ms | 2.004ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.646** for Graph Full versus **0.646** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **4.102ms**, compared to **1.171ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.802** compared to **0.802** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.646** to **0.646**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.65**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **2.931ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_balanced_mini_adaptive2_fast.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-balanced-mini` dataset (40 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.802 | 0.902 | 0.682 | 1.852ms | 3.095ms/case |
+| 1MBrain Vector Only | 0.802 | 0.902 | 0.711 | 1.638ms | 3.058ms/case |
+| Vector Baseline (SQLite) | 0.802 | 0.902 | 0.711 | 0.434ms | 0.896ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.646** for Graph Full versus **0.646** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **1.852ms**, compared to **0.434ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.802** compared to **0.802** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.646** to **0.646**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.65**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **1.418ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_balanced_mini_adaptive_fast.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-balanced-mini` dataset (40 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.821 | 0.921 | 0.682 | 2.078ms | 3.249ms/case |
+| 1MBrain Vector Only | 0.802 | 0.902 | 0.711 | 1.392ms | 2.865ms/case |
+| Vector Baseline (SQLite) | 0.802 | 0.902 | 0.711 | 0.448ms | 0.881ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **2.338%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.833** for Graph Full versus **0.646** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **2.078ms**, compared to **0.448ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.821** compared to **0.802** for 1MBrain Vector Only, a **2.338%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.646** to **0.833**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.65**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **1.63ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_balanced_mini_rerank.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-balanced-mini` dataset (40 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.788 | 0.888 | 0.641 | 8.7ms | 9.007ms/case |
+| 1MBrain Vector Only | 0.802 | 0.902 | 0.711 | 5.1ms | 9.156ms/case |
+| Vector Baseline (SQLite) | 0.802 | 0.902 | 0.711 | 1.545ms | 2.322ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **-1.818%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **1** for Graph Full versus **0.646** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **8.7ms**, compared to **1.545ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.788** compared to **0.802** for 1MBrain Vector Only, a **-1.818%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.646** to **1**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.8**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **7.155ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_realistic_medium.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-realistic-medium` dataset (120 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.619 | 0.757 | 0.557 | 2.823ms | 5.678ms/case |
+| 1MBrain Vector Only | 0.619 | 0.757 | 0.557 | 4.417ms | 8.086ms/case |
+| Vector Baseline (SQLite) | 0.619 | 0.757 | 0.557 | 1.587ms | 3.331ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **0%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.806** for Graph Full versus **0.806** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **2.823ms**, compared to **1.587ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.619** compared to **0.619** for 1MBrain Vector Only, a **0%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.806** to **0.806**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.2**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **1.236ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.

package/reports/benchmark_report_claude_realistic_medium_adaptive.md

@@ -0,0 +1,42 @@
+# 1MBrain Benchmark Final Report
+
+This report evaluates the performance of **1MBrain** against standard vector-only baselines and other providers using the `memory-bench-realistic-medium` dataset (120 cases).
+
+## Performance Leaderboard
+
+| Provider | Evidence Accuracy | Recall@5 | MRR | p95 Latency | Ingestion Rate |
+|---|---:|---:|---:|---:|---:|
+| 1MBrain Graph Full | 0.628 | 0.782 | 0.576 | 5.115ms | 10.215ms/case |
+| 1MBrain Vector Only | 0.619 | 0.757 | 0.557 | 4.559ms | 10.847ms/case |
+| Vector Baseline (SQLite) | 0.619 | 0.757 | 0.557 | 1.459ms | 2.908ms/case |
+
+## Key Evaluation Questions
+
+### 1. Where does 1MBrain outperform typical vector-only memory?
+1MBrain Graph Full outperforms the Vector Baseline by **1.345%** in evidence retrieval accuracy on this focused dataset. The clearest measurable advantage is in graph-aware scenarios: multi-hop evidence accuracy is **0.806** for Graph Full versus **0.806** for Vector Only.
+
+### 2. Where does 1MBrain underperform?
+The main weakness is not graph traversal cost; it is retrieval precision under paraphrase, stale preference conflicts, and noisy distractors. Graph Full p95 latency is **5.115ms**, compared to **1.459ms** for the raw SQLite vector baseline. This is still low in absolute terms, but quality improvements are modest because the benchmark currently uses a local keyword embedder rather than a stronger semantic embedder.
+
+### 3. Does association graph improve recall quality?
+Partially. 1MBrain Graph Full achieved evidence accuracy of **0.628** compared to **0.619** for 1MBrain Vector Only, a **1.345%** relative improvement. This shows graph links help, but the improvement is not yet large enough to claim the graph layer alone solves recall quality.
+
+### 4. Does spreading activation improve multi-hop reasoning?
+Yes, with caveats. Multi-hop evidence accuracy improved from **0.806** to **0.806**, but some required supporting memories were still missed. The failure cases indicate that graph traversal needs better seed recall and/or query expansion to consistently reach the correct neighboring nodes.
+
+### 5. Does decay/refresh help prevent stale memory pollution?
+Not convincingly in this run. Memory update evidence accuracy is **0.2**, but stale-memory failures are still present. This benchmark should be treated as evidence that explicit recency/conflict resolution needs more work before public claims about stale-memory handling.
+
+### 6. Is Memory Passport practically useful?
+Yes for the 1MBrain adapters tested here. Graph Full portability success rate is **0** on the focused portability cases. The vector baseline has no portability capability and is expected to fail those operation checks.
+
+### 7. What is the tradeoff between quality, latency, and cost?
+- **Quality:** Graph-enabled 1MBrain is the best local provider in this run, but only by a modest margin.
+- **Latency:** SQLite vector-only baseline is the fastest, while graph traversal adds roughly **3.656ms** p95 latency in this small dataset.
+- **Cost:** Since 1MBrain can run fully locally (SQLite + local embedder/Ollama), the running query cost is **$0.00** per 1,000 queries, compared to high cloud API vendor fees.
+
+### 8. What should be improved before public release?
+- Replace or complement the keyword embedder with a stronger local semantic embedder for paraphrase-heavy questions.
+- Add explicit recency/conflict ranking so newer preferences reliably beat stale memories.
+- Improve seed recall and query expansion before spreading activation so graph traversal starts from the right nodes.
+- Keep failure-case reporting in the public benchmark so claims remain reproducible and falsifiable.