rust-kgdb 0.6.32 → 0.6.33

package/CHANGELOG.md

@@ -2,6 +2,48 @@
 
 All notable changes to the rust-kgdb TypeScript SDK will be documented in this file.
 
+## [0.6.33] - 2025-12-16
+
+### Framework Comparison Code Snippets
+
+Added clear, reproducible benchmark setup with side-by-side code comparisons.
+
+#### Added
+- **Framework Comparison Section**: New section in README showing exact code for each framework
+  - Vanilla OpenAI: With and without schema (0% → 71.4%)
+  - LangChain: With and without schema (0% → 71.4%)
+  - DSPy: With and without schema (14.3% → 71.4%)
+  - HyperMind: Auto schema extraction
+- **Reproducible Examples**: All code snippets are copy-paste ready
+- **Clear Results Comments**: Each snippet shows expected output
+
+#### Key Insight Documented
+All frameworks achieve SAME accuracy (71.4%) when given schema. HyperMind's value = automatic schema extraction from your data.
+
+---
+
+## [0.6.32] - 2025-12-16
+
+### Verified Benchmark Results
+
+Real API testing with GPT-4o on LUBM dataset—no mocking.
+
+#### Added
+- `benchmark-frameworks.py`: Python benchmark comparing Vanilla/LangChain/DSPy
+- `verified_benchmark_results.json`: Raw results from real API calls
+- Updated README with verified accuracy numbers
+- Updated HYPERMIND_BENCHMARK_REPORT.md with complete code snippets
+
+#### Verified Results
+| Framework | No Schema | With Schema | Improvement |
+|-----------|-----------|-------------|-------------|
+| Vanilla OpenAI | 0.0% | 71.4% | +71.4 pp |
+| LangChain | 0.0% | 71.4% | +71.4 pp |
+| DSPy | 14.3% | 71.4% | +57.1 pp |
+| Average | 4.8% | 71.4% | +66.7 pp |
+
+---
+
 ## [0.6.25] - 2025-12-16
 
 ### Documentation Cleanup

package/README.md

@@ -275,6 +275,149 @@ console.log(result.reasoningTrace)  // Full audit trail
 
 ---
 
+## Framework Comparison (Verified Benchmark Setup)
+
+The following code snippets show EXACTLY how each framework was tested. All tests use the same LUBM dataset (3,272 triples) and GPT-4o model with real API calls—no mocking.
+
+**Reproduce yourself**: `python3 benchmark-frameworks.py` (included in package)
+
+### Vanilla OpenAI (0% → 71.4% with schema)
+
+```python
+# WITHOUT SCHEMA: 0% accuracy
+from openai import OpenAI
+client = OpenAI()
+
+response = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": "Find all teachers"}]
+)
+# Returns: Long explanation with markdown code blocks
+# FAILS: No usable SPARQL query
+```
+
+```python
+# WITH SCHEMA: 71.4% accuracy (+71.4 pp improvement)
+LUBM_SCHEMA = """
+PREFIX ub: <http://swat.cse.lehigh.edu/onto/univ-bench.owl#>
+Classes: University, Department, Professor, Student, Course, Publication
+Properties: teacherOf(Faculty→Course), worksFor(Faculty→Department)
+"""
+
+response = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{
+        "role": "system",
+        "content": f"{LUBM_SCHEMA}\nOutput raw SPARQL only, no markdown."
+    }, {
+        "role": "user",
+        "content": "Find all teachers"
+    }]
+)
+# Returns: SELECT DISTINCT ?teacher WHERE { ?teacher a ub:Professor . }
+# WORKS: Valid SPARQL using correct ontology terms
+```
+
+### LangChain (0% → 71.4% with schema)
+
+```python
+# WITHOUT SCHEMA: 0% accuracy
+from langchain_openai import ChatOpenAI
+from langchain_core.prompts import PromptTemplate
+from langchain_core.output_parsers import StrOutputParser
+
+llm = ChatOpenAI(model="gpt-4o")
+template = PromptTemplate(
+    input_variables=["question"],
+    template="Generate SPARQL for: {question}"
+)
+chain = template | llm | StrOutputParser()
+result = chain.invoke({"question": "Find all teachers"})
+# Returns: Explanation + markdown code blocks
+# FAILS: Not executable SPARQL
+```
+
+```python
+# WITH SCHEMA: 71.4% accuracy (+71.4 pp improvement)
+template = PromptTemplate(
+    input_variables=["question", "schema"],
+    template="""You are a SPARQL query generator.
+{schema}
+TYPE CONTRACT: Output raw SPARQL only, NO markdown, NO explanation.
+Query: {question}
+Output raw SPARQL only:"""
+)
+chain = template | llm | StrOutputParser()
+result = chain.invoke({"question": "Find all teachers", "schema": LUBM_SCHEMA})
+# Returns: SELECT DISTINCT ?teacher WHERE { ?teacher a ub:Professor . }
+# WORKS: Schema injection guides correct predicate selection
+```
+
+### DSPy (14.3% → 71.4% with schema)
+
+```python
+# WITHOUT SCHEMA: 14.3% accuracy (best without schema!)
+import dspy
+from dspy import LM
+
+lm = LM("openai/gpt-4o")
+dspy.configure(lm=lm)
+
+class SPARQLGenerator(dspy.Signature):
+    """Generate SPARQL query."""
+    question = dspy.InputField()
+    sparql = dspy.OutputField(desc="Raw SPARQL query only")
+
+generator = dspy.Predict(SPARQLGenerator)
+result = generator(question="Find all teachers")
+# Returns: SELECT ?teacher WHERE { ?teacher a :Teacher . }
+# PARTIAL: Sometimes works due to DSPy's structured output
+```
+
+```python
+# WITH SCHEMA: 71.4% accuracy (+57.1 pp improvement)
+class SchemaSPARQLGenerator(dspy.Signature):
+    """Generate SPARQL query using the provided schema."""
+    schema = dspy.InputField(desc="Database schema with classes and properties")
+    question = dspy.InputField(desc="Natural language question")
+    sparql = dspy.OutputField(desc="Raw SPARQL query, no markdown")
+
+generator = dspy.Predict(SchemaSPARQLGenerator)
+result = generator(schema=LUBM_SCHEMA, question="Find all teachers")
+# Returns: SELECT DISTINCT ?teacher WHERE { ?teacher a ub:Professor . }
+# WORKS: Schema + DSPy structured output = reliable queries
+```
+
+### HyperMind (Built-in Schema Awareness)
+
+```javascript
+// HyperMind auto-extracts schema from your data
+const { HyperMindAgent, createSchemaAwareGraphDB } = require('rust-kgdb');
+
+const db = createSchemaAwareGraphDB('http://university.org/');
+db.loadTtl(lubmData, null);  // Load LUBM 3,272 triples
+
+const agent = new HyperMindAgent({
+  kg: db,
+  model: 'gpt-4o',
+  apiKey: process.env.OPENAI_API_KEY
+});
+
+const result = await agent.call('Find all teachers');
+// Schema auto-extracted: { classes: Set(30), properties: Map(23) }
+// Query generated: SELECT ?x WHERE { ?x ub:teacherOf ?course . }
+// Result: 39 faculty members who teach courses
+
+console.log(result.reasoningTrace);
+// [{ tool: 'kg.sparql.query', query: 'SELECT...', bindings: 39 }]
+console.log(result.hash);
+// "sha256:a7b2c3..." - Reproducible answer
+```
+
+**Key Insight**: All frameworks achieve the SAME accuracy (71.4%) when given schema. HyperMind's value is that it extracts and injects schema AUTOMATICALLY from your data—no manual prompt engineering required.
+
+---
+
 ## Use Cases
 
 ### Fraud Detection

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "rust-kgdb",
-  "version": "0.6.32",
+  "version": "0.6.33",
   "description": "Production-grade Neuro-Symbolic AI Framework with Schema-Aware GraphDB, Context Theory, and Memory Hypergraph: +86.4% accuracy over vanilla LLMs. Features Schema-Aware GraphDB (auto schema extraction), BYOO (Bring Your Own Ontology) for enterprise, cross-agent schema caching, LLM Planner for natural language to typed SPARQL, ProofDAG with Curry-Howard witnesses. High-performance (2.78µs lookups, 35x faster than RDFox). W3C SPARQL 1.1 compliant.",
   "main": "index.js",
   "types": "index.d.ts",