rust-kgdb 0.6.43 → 0.6.44

package/CHANGELOG.md

@@ -2,6 +2,24 @@
 
 All notable changes to the rust-kgdb TypeScript SDK will be documented in this file.
 
+## [0.6.44] - 2025-12-17
+
+### Honest Documentation (All Numbers Verified)
+
+#### Fixed All Misleading Claims
+- **Removed ALL 85.7% claims**: Our verified benchmark shows 71.4% with schema for ALL frameworks
+- **Honest comparison**: Schema injection helps everyone equally (~71%)
+- **Clear positioning**: We beat databases (RDFox), not LLM frameworks (different category)
+
+#### Verified Benchmark Results (from `verified_benchmark_results.json`)
+| Framework | No Schema | With Schema |
+|-----------|-----------|-------------|
+| Vanilla OpenAI | 0.0% | 71.4% |
+| LangChain | 0.0% | 71.4% |
+| DSPy | 14.3% | 71.4% |
+
+---
+
 ## [0.6.43] - 2025-12-17
 
 ### Clearer Honest Benchmarks

package/README.md

@@ -233,9 +233,9 @@ console.log(result.hash);
 │                                                                           │
 │  TRADITIONAL (Code Gen)          OUR APPROACH (Proxy Layer)               │
 │  • 2-5 seconds per query         • <100ms per query (20-50x FASTER)       │
-│  • 20-40% accuracy               • 85.7% accuracy                         │
+│  • 0-14% accuracy (no schema)    • 71% accuracy (schema auto-injected)    │
 │  • Retry loops on errors         • No retries needed                      │
-│  • $0.01-0.05 per query          • <$0.001 per query (no LLM)             │
+│  • $0.01-0.05 per query          • <$0.001 per query (cached patterns)    │
 │                                                                           │
 ├───────────────────────────────────────────────────────────────────────────┤
 │  WHY NO CODE GENERATION:                                                  │
@@ -286,7 +286,7 @@ OUR APPROACH:       User → Proxied Objects → WASM Sandbox → RPC → Real S
                         └── Every answer has derivation chain
                         └── Deterministic hash for reproducibility
 
-                    (85.7% accuracy, <100ms/query, <$0.001/query)
+                    (71% accuracy with schema, <100ms/query, <$0.001/query)
 ```
 
 **The Three Pillars** (all as OBJECTS, not strings):
@@ -362,7 +362,7 @@ The following code snippets show EXACTLY how each framework was tested. All test
 
 **Reproduce yourself**: `python3 benchmark-frameworks.py` (included in package)
 
-### Vanilla OpenAI (0% → 85.7% with schema)
+### Vanilla OpenAI (0% → 71.4% with schema)
 
 ```python
 # WITHOUT SCHEMA: 0% accuracy
@@ -378,7 +378,7 @@ response = client.chat.completions.create(
 ```
 
 ```python
-# WITH SCHEMA: 85.7% accuracy (+85.7 pp improvement)
+# WITH SCHEMA: 71.4% accuracy (+71.4 pp improvement)
 LUBM_SCHEMA = """
 PREFIX ub: <http://swat.cse.lehigh.edu/onto/univ-bench.owl#>
 Classes: University, Department, Professor, Student, Course, Publication
@@ -399,7 +399,7 @@ response = client.chat.completions.create(
 # WORKS: Valid SPARQL using correct ontology terms
 ```
 
-### LangChain (0% → 85.7% with schema)
+### LangChain (0% → 71.4% with schema)
 
 ```python
 # WITHOUT SCHEMA: 0% accuracy
@@ -419,7 +419,7 @@ result = chain.invoke({"question": "Find all teachers"})
 ```
 
 ```python
-# WITH SCHEMA: 85.7% accuracy (+85.7 pp improvement)
+# WITH SCHEMA: 71.4% accuracy (+71.4 pp improvement)
 template = PromptTemplate(
     input_variables=["question", "schema"],
     template="""You are a SPARQL query generator.
@@ -434,7 +434,7 @@ result = chain.invoke({"question": "Find all teachers", "schema": LUBM_SCHEMA})
 # WORKS: Schema injection guides correct predicate selection
 ```
 
-### DSPy (14.3% → 85.7% with schema)
+### DSPy (14.3% → 71.4% with schema)
 
 ```python
 # WITHOUT SCHEMA: 14.3% accuracy (best without schema!)
@@ -456,7 +456,7 @@ result = generator(question="Find all teachers")
 ```
 
 ```python
-# WITH SCHEMA: 85.7% accuracy (+71.4 pp improvement)
+# WITH SCHEMA: 71.4% accuracy (+57.1 pp improvement)
 class SchemaSPARQLGenerator(dspy.Signature):
     """Generate SPARQL query using the provided schema."""
     schema = dspy.InputField(desc="Database schema with classes and properties")
@@ -495,7 +495,7 @@ console.log(result.hash);
 // "sha256:a7b2c3..." - Reproducible answer
 ```
 
-**Key Insight**: All frameworks achieve the SAME accuracy (85.7%) when given schema. HyperMind's value is that it extracts and injects schema AUTOMATICALLY from your data—no manual prompt engineering required.
+**Key Insight**: All frameworks achieve the SAME accuracy (~71%) when given schema. HyperMind's value is that it extracts and injects schema AUTOMATICALLY from your data—no manual prompt engineering required. Plus it includes the database to actually execute queries.
 
 ---
 
@@ -1072,15 +1072,15 @@ console.log('Supersteps:', result.supersteps)  // 5
 
 ### AI Agent Accuracy (Verified December 2025)
 
-| Framework | No Schema | With Schema | With HyperMind |
-|-----------|-----------|-------------|----------------|
-| **Vanilla OpenAI** | 0.0% | 71.4% | 85.7% (+14.3 pp) |
-| **LangChain** | 0.0% | 71.4% | 85.7% (+14.3 pp) |
-| **DSPy** | 14.3% | 71.4% | 85.7% (+14.3 pp) |
+| Framework | No Schema | With Schema |
+|-----------|-----------|-------------|
+| **Vanilla OpenAI** | 0.0% | 71.4% |
+| **LangChain** | 0.0% | 71.4% |
+| **DSPy** | 14.3% | 71.4% |
 
-*HyperMind's predicate resolver adds +14.3 pp over schema injection alone.*
+*Schema injection improves ALL frameworks equally. See `verified_benchmark_results.json` for raw data.*
 
-*Tested: GPT-4o, 7 LUBM queries, real API calls. See `framework_benchmark_*.json` for raw data.*
+*Tested: GPT-4o, 7 LUBM queries, real API calls.*
 
 ### AI Framework Architectural Comparison
 
@@ -1469,7 +1469,7 @@ Result: ❌ PARSER ERROR - Invalid SPARQL syntax
 3. LLM hallucinates class names → `ub:Faculty` doesn't exist (it's `ub:Professor`)
 4. LLM has no schema awareness → guesses predicates and classes
 
-**HyperMind fixes all of this** with schema injection and typed tools, achieving **85.7% accuracy** vs **0% for vanilla LLMs**.
+**HyperMind fixes all of this** with schema injection and typed tools, achieving **71% accuracy** vs **0% for vanilla LLMs without schema**.
 
 ### Competitive Landscape
 
@@ -1497,7 +1497,7 @@ Result: ❌ PARSER ERROR - Invalid SPARQL syntax
 | LangChain | ❌ No | ❌ No | ❌ No | ❌ No |
 | DSPy | ⚠️ Partial | ❌ No | ❌ No | ❌ No |
 
-**Note**: This compares architectural features. Benchmark (Dec 2025): Schema injection brings all frameworks to 71.4%. HyperMind's predicate resolver adds +14.3 pp to reach 85.7%.
+**Note**: This compares architectural features. Benchmark (Dec 2025): Schema injection brings all frameworks to ~71% accuracy equally.
 
 ```
 ┌─────────────────────────────────────────────────────────────────┐

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "rust-kgdb",
-  "version": "0.6.43",
+  "version": "0.6.44",
   "description": "High-performance RDF/SPARQL database with AI agent framework. GraphDB (449ns lookups, 35x faster than RDFox), GraphFrames analytics (PageRank, motifs), Datalog reasoning, HNSW vector embeddings. HyperMindAgent for schema-aware query generation with audit trails. W3C SPARQL 1.1 compliant. Native performance via Rust + NAPI-RS.",
   "main": "index.js",
   "types": "index.d.ts",