memtrace 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -19
- package/package.json +4 -4
package/README.md
CHANGED
|
@@ -48,29 +48,33 @@ On top of that, the structural layer is comprehensive:
|
|
|
48
48
|
- **Relationships are edges** — `CALLS`, `IMPLEMENTS`, `IMPORTS`, `EXPORTS`, `CONTAINS`
|
|
49
49
|
- **Community detection** — Louvain algorithm identifies architectural modules automatically
|
|
50
50
|
- **Hybrid search** — Tantivy BM25 + vector embeddings + Reciprocal Rank Fusion, all on top of the graph
|
|
51
|
-
- **Rust-native** — compiled binary, no Python/JS runtime overhead, sub-
|
|
51
|
+
- **Rust-native** — compiled binary, no Python/JS runtime overhead, sub-8ms average query latency
|
|
52
52
|
|
|
53
53
|
The agent doesn't just search your code. It *remembers* it.
|
|
54
54
|
|
|
55
55
|
## Benchmarks
|
|
56
56
|
|
|
57
|
-
All
|
|
57
|
+
All four systems run on the same machine, same mempalace checkout, same 1,000 queries, same evaluator. Ground truth is extracted by Python's stdlib `ast` module — **not** from any tool's index — so no system gets a home-field advantage. Full reproduction scripts and raw results: [`benchmarks/fair/`](https://github.com/syncable-dev/memtrace-public/tree/main/benchmarks/fair).
|
|
58
58
|
|
|
59
|
-
|
|
59
|
+
<img alt="Benchmark overview: Memtrace 96.7% Acc@1, 100% Acc@10, 9.16ms latency, 195 tokens — vs ChromaDB, GitNexus, CodeGrapher" src="https://raw.githubusercontent.com/syncable-dev/memtrace-public/main/assets/benchmarks/benchmark-overview.svg" width="720"/>
|
|
60
60
|
|
|
61
|
-
|
|
61
|
+
### Results (1,000 Python symbol-lookup queries on mempalace)
|
|
62
62
|
|
|
63
|
-
|
|
63
|
+
| Tool | Coverage | Acc@1 | Acc@5 | Acc@10 | Avg lat | Tokens |
|
|
64
|
+
|:-----|---------:|------:|------:|-------:|--------:|-------:|
|
|
65
|
+
| **Memtrace** (ArcadeDB) | **100.0%** | **96.7%** | **100.0%** | **100.0%** | **9.16 ms** | 195 |
|
|
66
|
+
| ChromaDB (all-MiniLM-L6-v2) | 100.0% | 62.3% | 86.1% | 87.9% | 58.5 ms | 1,937 |
|
|
67
|
+
| GitNexus (eval-server) | 99.5% | 27.1% | 89.7% | 89.9% | 191.2 ms | 213 |
|
|
68
|
+
| CodeGrapherContext (CLI) | 67.2% | 6.4% | 66.4% | 66.7% | 1627.2 ms | 221 |
|
|
64
69
|
|
|
65
|
-
|
|
70
|
+
**What the numbers say, read fairly:**
|
|
66
71
|
|
|
67
|
-
|
|
72
|
+
- **Memtrace** is exact-symbol lookup's sweet spot: 100% coverage, rank-1 hit in 96.7% of queries, and the correct file is in the top-10 every single time. 9 ms per query, 195 tokens per response.
|
|
73
|
+
- **ChromaDB** shows what semantic embeddings look like for this workload — 88% top-10 but rank-1 is probabilistic, and the response is 10× larger because it returns 800-char chunks rather than symbol metadata.
|
|
74
|
+
- **GitNexus** finds the right file 90% of the time — its response leads with execution *flows*, pushing standalone definitions down the list, which costs it rank-1 but not top-10.
|
|
75
|
+
- **CodeGrapherContext**'s 67.2% coverage means its parser extracted two-thirds of the symbols Python's AST finds. Among symbols it did index, top-10 hit rate is excellent (~99%). Latency is dominated by CLI re-initialising FalkorDB per call.
|
|
68
76
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
### How long to set up?
|
|
72
|
-
|
|
73
|
-
<img alt="Indexing: Memtrace 1.5s vs Graphiti 6h vs Mem0 31m" src="https://raw.githubusercontent.com/syncable-dev/memtrace-public/main/assets/benchmarks/indexing-speed.svg" width="720"/>
|
|
77
|
+
**Where each tool shines** — this benchmark measures exact-symbol lookup only. Different workloads produce different rankings: ChromaDB wins on natural-language queries, GitNexus on execution-flow traces, Memtrace on exact lookup / typo tolerance / temporal queries / cross-service API topology. See [`benchmarks/fair/README.md`](https://github.com/syncable-dev/memtrace-public/tree/main/benchmarks/fair/README.md) for a per-workload breakdown.
|
|
74
78
|
|
|
75
79
|
<details>
|
|
76
80
|
<summary><strong>Memtrace vs. general memory systems (Mem0, Graphiti)</strong></summary>
|
|
@@ -85,7 +89,7 @@ Mem0 and Graphiti are strong conversational memory engines designed for tracking
|
|
|
85
89
|
|
|
86
90
|
**Both** accumulate $10–50+ in API costs for large codebases because every relationship is inferred rather than parsed.
|
|
87
91
|
|
|
88
|
-
**Memtrace takes a different approach:** it indexes
|
|
92
|
+
**Memtrace takes a different approach:** it indexes files in ~4 seconds for $0.00 — no LLM calls, no API costs, no rate limits. Native Tree-sitter AST parsers resolve deterministic symbol references (`CALLS`, `IMPLEMENTS`, `IMPORTS`) locally. The tradeoff is that Memtrace is purpose-built for code — it doesn't handle conversational entity memory the way Mem0 and Graphiti do.
|
|
89
93
|
|
|
90
94
|
</details>
|
|
91
95
|
|
|
@@ -105,14 +109,16 @@ GitNexus and CodeGrapherContext both build AST-based code graphs with structural
|
|
|
105
109
|
| Community detection (Louvain) | **Yes** | Yes | No |
|
|
106
110
|
| Hybrid search (BM25 + vector + RRF) | **Yes — Tantivy + embeddings** | No | BM25 + optional embeddings |
|
|
107
111
|
| Language | **Rust (compiled binary)** | JavaScript | Python |
|
|
108
|
-
|
|
|
109
|
-
|
|
|
110
|
-
|
|
|
111
|
-
|
|
|
112
|
+
| Coverage (1K queries) | **100%** | 99.5% | 67.2% |
|
|
113
|
+
| Acc@1 (1K queries) | **96.7%** | 27.1% | 6.4% |
|
|
114
|
+
| Acc@10 (1K queries) | **100%** | 89.9% | 66.7% |
|
|
115
|
+
| Query latency (1K queries) | **9.16 ms avg** | 191.2 ms avg | 1627.2 ms avg |
|
|
116
|
+
| Tokens per query | **195 avg** | 213 avg | 221 avg |
|
|
117
|
+
| Index time (~250 files / 2.3K nodes / 5.8K edges) | **~4 sec** (≈500 ms of real work + ~3 s Docker / Bolt / schema DDL startup on first run) | ~6 sec | ~1 sec (cached) |
|
|
112
118
|
|
|
113
|
-
|
|
119
|
+
All numbers from [the fair benchmark](https://github.com/syncable-dev/memtrace-public/tree/main/benchmarks/fair) on the same machine, same mempalace checkout, same 1,000 queries. Ground truth is extracted by Python's stdlib `ast` — not from any tool's index — so no system is advantaged in the dataset itself.
|
|
114
120
|
|
|
115
|
-
The latency difference is primarily Rust vs. interpreted runtimes, and
|
|
121
|
+
The latency difference is primarily Rust vs. interpreted runtimes, and ArcadeDB's Graph-OLAP engine (native CSR projections, PageRank/betweenness as in-database procedures) vs. HTTP/embedding pipelines. The feature difference is temporal memory and API topology — dimensions Memtrace adds on top of the shared AST-graph foundation.
|
|
116
122
|
|
|
117
123
|
</details>
|
|
118
124
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "memtrace",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.1",
|
|
4
4
|
"description": "Code intelligence graph — MCP server + AI agent skills + visualization UI",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"mcp",
|
|
@@ -36,9 +36,9 @@
|
|
|
36
36
|
"fs-extra": "^11.0.0"
|
|
37
37
|
},
|
|
38
38
|
"optionalDependencies": {
|
|
39
|
-
"@memtrace/darwin-arm64": "0.2.
|
|
40
|
-
"@memtrace/linux-x64": "0.2.
|
|
41
|
-
"@memtrace/win32-x64": "0.2.
|
|
39
|
+
"@memtrace/darwin-arm64": "0.2.1",
|
|
40
|
+
"@memtrace/linux-x64": "0.2.1",
|
|
41
|
+
"@memtrace/win32-x64": "0.2.1"
|
|
42
42
|
},
|
|
43
43
|
"engines": {
|
|
44
44
|
"node": ">=18"
|