rust-kgdb 0.6.45 → 0.6.47

package/CHANGELOG.md

@@ -2,6 +2,57 @@
 
 All notable changes to the rust-kgdb TypeScript SDK will be documented in this file.
 
+## [0.6.47] - 2025-12-17
+
+### Memory Retrieval Depth Benchmark Added
+
+#### New Benchmark Documentation
+Added Memory Retrieval Depth Benchmark to README, based on academic benchmarks:
+- MemQ (arXiv 2503.05193)
+- mKGQAgent (Text2SPARQL 2025)
+- MTEB (Massive Text Embedding Benchmark)
+
+**Results** (50 queries per depth, HNSW index):
+
+| Depth | P50 Latency | Recall@5 | Recall@10 | MRR |
+|-------|-------------|----------|-----------|-----|
+| 10 | 0.06 ms | 78% | 100% | 0.68 |
+| 100 | 0.50 ms | 88% | 98% | 0.42 |
+| 1,000 | 1.59 ms | 80% | 94% | 0.50 |
+| 10,000 | 16.71 ms | 76% | 94% | 0.54 |
+
+**Key insight**: Even at 10K stored queries, Recall@10 stays at 94% with sub-17ms latency.
+
+Reproduce: `node memory-retrieval-benchmark.js`
+
+---
+
+## [0.6.46] - 2025-12-17
+
+### Honest Comparison Fix
+
+#### Fixed Misleading "Before & After" Section
+- **Old (misleading)**: Implied vanilla LLMs CAN'T use schema/context
+- **New (honest)**: Shows both approaches work, difference is integration effort
+
+The "Before & After" section now honestly shows:
+- **Manual Approach**: Works (~71% accuracy), but requires 5-8 manual integration steps
+  - Write schema manually
+  - Pass to LLM
+  - Parse SPARQL from response
+  - Find external database
+  - Connect, execute, parse results
+  - Build audit trail yourself
+
+- **HyperMind Approach**: Same accuracy (~71%), but integrated
+  - Schema auto-extracted from your data
+  - Built-in database executes queries
+  - Audit trail included automatically
+
+**Key insight**: We don't claim better accuracy than manual approach with schema. We provide integration convenience.
+
+---
+
 ## [0.6.45] - 2025-12-17
 
 ### ARCADE Pipeline Documentation & Benchmark Methodology

package/README.md

@@ -135,6 +135,31 @@ db.loadTtl(':Provider123 :hasRiskScore "0.87" .', null)
 └─────────────────────────────────────────────────────────────────────────────┘
 ```
 
+### Memory Retrieval Depth Benchmark
+
+Based on academic benchmarks: MemQ (arXiv 2503.05193), mKGQAgent (Text2SPARQL 2025), MTEB.
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│  BENCHMARK: Memory Retrieval at Depth (50 queries per depth)                │
+│  METHODOLOGY: LUBM schema-driven queries, HNSW index, random seed 42        │
+├─────────────────────────────────────────────────────────────────────────────┤
+│                                                                             │
+│  DEPTH     │ P50 LATENCY  │ P95 LATENCY  │ Recall@5 │ Recall@10 │   MRR    │
+│  ──────────────────────────────────────────────────────────────────────────│
+│      10    │    0.06 ms   │    0.26 ms   │   78%    │   100%    │  0.68    │
+│     100    │    0.50 ms   │    0.75 ms   │   88%    │    98%    │  0.42    │
+│   1,000    │    1.59 ms   │    5.03 ms   │   80%    │    94%    │  0.50    │
+│  10,000    │   16.71 ms   │   17.37 ms   │   76%    │    94%    │  0.54    │
+│  ──────────────────────────────────────────────────────────────────────────│
+│                                                                             │
+│  KEY INSIGHT: Even at 10,000 stored queries, Recall@10 stays at 94%        │
+│  Sub-17ms retrieval from 10K query pool = practical for production use     │
+│                                                                             │
+│  Reproduce: node memory-retrieval-benchmark.js                              │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
 ### Where We Actually Outperform (Database Performance)
 
 ```
@@ -188,47 +213,60 @@ db.loadTtl(':Provider123 :hasRiskScore "0.87" .', null)
 
 ---
 
-## The Difference: Before & After
+## The Difference: Manual vs Integrated
 
-### Before: Vanilla LLM (Unreliable)
+### Manual Approach (Works, But Tedious)
 
 ```javascript
-// Ask LLM to query your database
+// STEP 1: Manually write your schema (takes hours for large ontologies)
+const LUBM_SCHEMA = `
+PREFIX ub: <http://swat.cse.lehigh.edu/onto/univ-bench.owl#>
+Classes: University, Department, Professor, Student, Course, Publication
+Properties: teacherOf(Faculty→Course), worksFor(Faculty→Department)
+`;
+
+// STEP 2: Pass schema to LLM
 const answer = await openai.chat.completions.create({
   model: 'gpt-4o',
-  messages: [{ role: 'user', content: 'Find suspicious providers in my database' }]
+  messages: [
+    { role: 'system', content: `${LUBM_SCHEMA}\nOutput raw SPARQL only.` },
+    { role: 'user', content: 'Find suspicious providers' }
+  ]
 });
 
-console.log(answer.choices[0].message.content);
-// "Based on my analysis, Provider P001 appears suspicious because..."
-//
-// PROBLEMS:
-// ❌ Did it actually query your database? No - it's guessing
-// ❌ Where's the evidence? None - it made up "Provider P001"
-// ❌ Will this answer be the same tomorrow? No - probabilistic
-// ❌ Can you audit this for regulators? No - black box
+// STEP 3: Parse out the SPARQL (handle markdown, explanations, etc.)
+const sparql = extractSPARQL(answer.choices[0].message.content);
+
+// STEP 4: Find a SPARQL database (Jena? RDFox? Virtuoso?)
+// STEP 5: Connect to database
+// STEP 6: Execute query
+// STEP 7: Parse results
+// STEP 8: No audit trail - you'd have to build that yourself
+
+// RESULT: ~71% accuracy (same as HyperMind with schema)
+// BUT: 5-8 manual integration steps
 ```
 
-### After: HyperMind (Verifiable)
+### HyperMind Approach (Integrated)
 
 ```javascript
-// Ask HyperMind to query your database
+// ONE-TIME SETUP: Load your data
 const { HyperMindAgent, GraphDB } = require('rust-kgdb');
 
 const db = new GraphDB('http://insurance.org/');
-db.loadTtl(yourActualData, null);  // Your real data
+db.loadTtl(yourActualData, null);  // Schema auto-extracted from data
 
 const agent = new HyperMindAgent({ kg: db, model: 'gpt-4o' });
 const result = await agent.call('Find suspicious providers');
 
 console.log(result.answer);
 // "Provider PROV001 has risk score 0.87 with 47 claims over $50,000"
-//
-// VERIFIED:
-// ✅ Queried your actual database (SPARQL executed)
-// ✅ Evidence included (47 real claims found)
-// ✅ Reproducible (same hash every time)
-// ✅ Full audit trail for regulators
+
+// WHAT YOU GET (ALL AUTOMATIC):
+// ✅ Schema auto-extracted (no manual prompt engineering)
+// ✅ Query executed on built-in database (no external DB needed)
+// ✅ Full audit trail included
+// ✅ Reproducible hash for compliance
 
 console.log(result.reasoningTrace);
 // [
@@ -240,7 +278,9 @@ console.log(result.hash);
 // "sha256:8f3a2b1c..." - Same question = Same answer = Same hash
 ```
 
-**The key insight**: The LLM plans WHAT to look for. The database finds EXACTLY that. Every answer traces back to your actual data.
+**Honest comparison**: Both approaches achieve ~71% accuracy on LUBM benchmark. The difference is integration effort:
+- **Manual**: Write schema, integrate database, build audit trail yourself
+- **HyperMind**: Database + schema extraction + audit trail built-in
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "rust-kgdb",
-  "version": "0.6.45",
+  "version": "0.6.47",
   "description": "High-performance RDF/SPARQL database with AI agent framework. GraphDB (449ns lookups, 35x faster than RDFox), GraphFrames analytics (PageRank, motifs), Datalog reasoning, HNSW vector embeddings. HyperMindAgent for schema-aware query generation with audit trails. W3C SPARQL 1.1 compliant. Native performance via Rust + NAPI-RS.",
   "main": "index.js",
   "types": "index.d.ts",