rust-kgdb 0.6.79 → 0.6.80

package/README.md

@@ -4,108 +4,24 @@
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![W3C](https://img.shields.io/badge/W3C-SPARQL%201.1%20%7C%20RDF%201.2-blue)](https://www.w3.org/TR/sparql11-query/)
 
-> **Native Graph Embeddings + Multi-Vector Search**: The only knowledge graph with built-in RDF2Vec, composite embeddings, and distributed SPARQL - all at native Rust speed.
+> **Enterprise Knowledge Graph with Native Graph Embeddings**: A production-grade RDF database featuring built-in RDF2Vec, multi-vector composite search, and distributed SPARQL execution—engineered for teams who need verifiable AI at scale.
 
 ---
 
-## RDF2Vec: Graph Embeddings That Blow Away The Competition
+## What's New in v0.6.79
 
-**Why wait for API calls when you can have 98 nanosecond lookups?**
+| Feature | Description | Performance |
+|---------|-------------|-------------|
+| **Rdf2VecEngine** | Native graph embeddings from random walks | 68 µs lookup (3,000x faster than APIs) |
+| **Composite Multi-Vector** | RRF fusion of RDF2Vec + OpenAI + domain | +26% recall improvement |
+| **Distributed SPARQL** | HDRF-partitioned Kubernetes clusters | 66-141ms across 3 executors |
+| **Auto-Embedding Triggers** | Vectors generated on graph insert/update | 37 µs incremental updates |
 
 ```javascript
 const { GraphDB, Rdf2VecEngine, EmbeddingService } = require('rust-kgdb')
-
-// Create graph and load your knowledge
-const db = new GraphDB('http://myapp/knowledge')
-db.loadTtl(myOntology, null)  // 130,923 triples/sec
-
-// RDF2Vec: Train embeddings from graph structure
-const rdf2vec = new Rdf2VecEngine()
-const walks = extractRandomWalks(db)  // Graph topology → training data
-rdf2vec.train(JSON.stringify(walks))  // 1,207 walks/sec → 384-dim vectors
-
-// Blazing fast similarity search
-const embedding = rdf2vec.getEmbedding('http://myapp/entity123')  // 68 µs
-const similar = rdf2vec.findSimilar(entity, candidates, 5)         // 303 µs
 ```
 
-### Performance Numbers That Matter
-
-| Metric | rust-kgdb | OpenAI API | Speedup |
-|--------|-----------|------------|---------|
-| **Embedding Lookup** | **68 µs** | 200-500 ms | **3,000-7,000x faster** |
-| **Similarity Search** | **303 µs** | 300-800 ms | **1,000-2,600x faster** |
-| **Training (1K walks)** | **829 ms** | N/A (no graph structure) | - |
-| **Batch Processing** | **In-process** | Rate-limited API | **No quotas** |
-
-**Why this matters**: OpenAI/Cohere embeddings require HTTP round-trips (200-500ms latency) and rate limits. RDF2Vec runs in your process at native speed. For real-time fraud detection or recommendation engines, this is the difference between catching fraud before payment clears vs. flagging it days later.
-
-### Multi-Vector Composite Embeddings (RRF Fusion)
-
-Combine multiple embedding sources for maximum recall:
-
-```javascript
-const service = new EmbeddingService()
-
-// Store embeddings from different providers
-service.storeComposite('CLM001', JSON.stringify({
-  rdf2vec: rdf2vec.getEmbedding('CLM001'),     // Graph structure (local)
-  openai: await openaiEmbed(claimDescription),  // Semantic text (API)
-  domain: customFraudEmbedding                  // Domain-specific
-}))
-
-// RRF (Reciprocal Rank Fusion) combines all sources
-const similar = service.findSimilarComposite('CLM001', 10, 0.7, 'rrf')
-// Formula: Score = Σ(1/(k+rank_i)), k=60
-// Result: Better recall than any single embedding source
-```
-
-| Pool Size | Single Embedding | RRF Composite | Improvement |
-|-----------|------------------|---------------|-------------|
-| 100 entities | 78% recall | **89% recall** | +14% |
-| 1K entities | 72% recall | **85% recall** | +18% |
-| 10K entities | 65% recall | **82% recall** | +26% |
-
-### Distributed Cluster Performance (Real LUBM Benchmark)
-
-Tested on Kubernetes: 1 coordinator + 3 executors via NodePort:
-
-| Query | Description | Results | Time |
-|-------|-------------|---------|------|
-| Q1 | GraduateStudent type | 150 | **66ms** |
-| Q4 | Advisor relationships | 150 | **101ms** |
-| Q6 | 2-way join (advisor+dept) | 46 | **75ms** |
-| Q7 | Course enrollment | 570 | **141ms** |
-
-**3,272 LUBM triples** distributed across 3 executors via HDRF partitioning. Multi-hop joins execute seamlessly across partition boundaries.
-
-### Graph → Embedding Pipeline (End-to-End)
-
-```javascript
-// 1. Insert triples (auto-distributed across executors)
-db.loadTtl(newData, null)  // Triggers auto-embedding if configured
-
-// 2. Extract walks from relationships (graph topology)
-const walks = [
-  ['Company1', 'employs', 'Person1'],
-  ['Person1', 'knows', 'Person2'],
-  ['Person2', 'worksFor', 'Company1']
-]
-
-// 3. Train on walks → 384-dimensional embeddings
-const result = JSON.parse(rdf2vec.train(JSON.stringify(walks)))
-// { vocabulary_size: 4, dimensions: 384, training_time_secs: 0.8 }
-
-// 4. Find similar entities in 303 µs
-const similar = rdf2vec.findSimilar('Person1', candidates, 5)
-```
-
-**Pipeline Throughput:**
-- Graph load: **130,923 triples/sec**
-- RDF2Vec training: **1,207 walks/sec**
-- Embedding lookup: **68 µs** (14,700/sec)
-- Similarity search: **303 µs** (3,300/sec)
-- Incremental update: **37 µs** (no full retrain)
+*See [Native Graph Embeddings](#native-graph-embeddings-rdf2vec-engine) for complete documentation and benchmarks.*
 
 ---
 
@@ -208,6 +124,26 @@ The math matters. When your fraud detection runs 35x faster, you catch fraud bef
 
 Most AI frameworks trust the LLM. We don't.
 
+### Core Capabilities
+
+| Layer | Feature | What It Does |
+|-------|---------|--------------|
+| **Database** | GraphDB | W3C SPARQL 1.1 compliant RDF store with 449ns lookups |
+| **Database** | Distributed SPARQL | HDRF partitioning across Kubernetes executors |
+| **Embeddings** | Rdf2VecEngine | Train 384-dim vectors from graph random walks |
+| **Embeddings** | EmbeddingService | Multi-provider composite vectors with RRF fusion |
+| **Embeddings** | HNSW Index | Approximate nearest neighbor search in 303µs |
+| **Analytics** | GraphFrames | PageRank, connected components, motif matching |
+| **Analytics** | Pregel API | Bulk synchronous parallel graph algorithms |
+| **Reasoning** | Datalog Engine | Recursive rule evaluation with fixpoint semantics |
+| **AI Agent** | HyperMindAgent | Schema-aware SPARQL generation from natural language |
+| **AI Agent** | Type System | Hindley-Milner type inference for query validation |
+| **AI Agent** | Proof DAG | SHA-256 audit trail for every AI decision |
+| **Security** | WASM Sandbox | Capability-based isolation with fuel metering |
+| **Security** | Schema Cache | Cross-agent ontology sharing with validation |
+
+### The Architecture Difference
+
 ```
 +===========================================================================+
 |                                                                           |
@@ -406,6 +342,97 @@ The difference? HyperMind treats tools as **typed morphisms** (category theory),
 
 ---
 
+## Native Graph Embeddings: RDF2Vec Engine
+
+Traditional embedding pipelines introduce significant latency: serialize your entity, make an HTTP request to OpenAI or Cohere, wait 200-500ms, parse the response. For applications requiring real-time similarity—fraud detection, recommendation engines, entity resolution—this latency model becomes a critical bottleneck.
+
+**RDF2Vec takes a fundamentally different approach.** Instead of treating entities as text to be embedded by external APIs, it learns vector representations directly from your graph's topology. The algorithm performs random walks across your knowledge graph, treating the resulting paths as "sentences" that capture structural relationships. These walks train a Word2Vec model in-process, producing embeddings that encode how entities relate to each other.
+
+```javascript
+const { GraphDB, Rdf2VecEngine } = require('rust-kgdb')
+
+// Load your knowledge graph
+const db = new GraphDB('http://enterprise/claims')
+db.loadTtl(claimsOntology, null)  // 130,923 triples/sec throughput
+
+// Initialize the RDF2Vec engine
+const rdf2vec = new Rdf2VecEngine()
+
+// Train embeddings from graph structure
+// Walks capture: Provider → submits → Claim → involves → Patient
+const walks = extractRandomWalks(db)
+rdf2vec.train(JSON.stringify(walks))  // 1,207 walks/sec → 384-dim vectors
+
+// Retrieve embeddings with microsecond latency
+const embedding = rdf2vec.getEmbedding('http://claims/provider/4521')  // 68 µs
+
+// Find structurally similar entities
+const similar = rdf2vec.findSimilar(provider, candidateProviders, 10)  // 303 µs
+```
+
+### Performance: Why Microseconds Matter
+
+| Operation | rust-kgdb (RDF2Vec) | External API (OpenAI) | Advantage |
+|-----------|---------------------|----------------------|-----------|
+| **Single Embedding Lookup** | 68 µs | 200-500 ms | **3,000-7,000x faster** |
+| **Similarity Search (k=10)** | 303 µs | 300-800 ms | **1,000-2,600x faster** |
+| **Batch Training (1K walks)** | 829 ms | N/A | Graph-native training |
+| **Rate Limits** | None (in-process) | Quota-restricted | Unlimited throughput |
+
+**Practical Impact**: When investigating a flagged claim, an analyst might check 50 similar providers. At 300ms per API call, that's 15 seconds of waiting. With RDF2Vec at 303µs per lookup, the same operation completes in 15 milliseconds—a 1,000x improvement that transforms the user experience from "waiting for AI" to "instant insight."
+
+### Multi-Vector Composite Embeddings with RRF
+
+Real-world similarity often requires multiple perspectives. A claim's structural relationships (RDF2Vec) tell a different story than its textual description (OpenAI) or domain-specific features (custom model). The `EmbeddingService` supports composite embeddings with Reciprocal Rank Fusion (RRF) to combine these views:
+
+```javascript
+const service = new EmbeddingService()
+
+// Store embeddings from multiple sources
+service.storeComposite('CLM-2024-0847', JSON.stringify({
+  rdf2vec: rdf2vec.getEmbedding('CLM-2024-0847'),   // Graph structure
+  openai: await openaiEmbed(claimNarrative),         // Semantic content
+  domain: fraudRiskEmbedding                         // Domain-specific signals
+}))
+
+// RRF fusion combines rankings from each source
+// Formula: Score = Σ(1 / (k + rank_i)), k=60
+const similar = service.findSimilarComposite('CLM-2024-0847', 10, 0.7, 'rrf')
+```
+
+| Candidate Pool | Single-Source Recall | RRF Composite Recall | Improvement |
+|----------------|---------------------|---------------------|-------------|
+| 100 entities | 78% | **89%** | +14% |
+| 1,000 entities | 72% | **85%** | +18% |
+| 10,000 entities | 65% | **82%** | +26% |
+
+### Distributed Cluster Benchmarks (Kubernetes)
+
+For deployments exceeding single-node capacity, rust-kgdb supports distributed execution across Kubernetes clusters. Verified benchmarks on the LUBM academic dataset:
+
+| Query | Pattern | Results | Latency |
+|-------|---------|---------|---------|
+| Q1 | Type lookup (GraduateStudent) | 150 | **66 ms** |
+| Q4 | Join (student → advisor) | 150 | **101 ms** |
+| Q6 | 2-hop join (advisor → department) | 46 | **75 ms** |
+| Q7 | Course enrollment scan | 570 | **141 ms** |
+
+**Configuration**: 1 coordinator + 3 executors, HDRF partitioning, NodePort access at `localhost:30080`. Triples distribute automatically across executors; multi-hop joins execute seamlessly across partition boundaries.
+
+### End-to-End Pipeline Throughput
+
+| Stage | Throughput | Notes |
+|-------|------------|-------|
+| Graph ingestion | **130,923 triples/sec** | Bulk load with indexing |
+| RDF2Vec training | **1,207 walks/sec** | Configurable walk length/count |
+| Embedding lookup | **68 µs** (14,700/sec) | In-memory, zero network |
+| Similarity search | **303 µs** (3,300/sec) | HNSW index |
+| Incremental update | **37 µs** | No full retrain required |
+
+*For detailed configuration options, see [Walk Configuration](#walk-configuration-tuning-rdf2vec-performance) and [Auto-Embedding Triggers](#auto-embedding-triggers-automatic-on-graph-insertupdate) below.*
+
+---
+
 ## The Deeper Problem: AI Agents Forget
 
 Fixing SPARQL syntax is table stakes. Here's what keeps enterprise architects up at night:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "rust-kgdb",
-  "version": "0.6.79",
+  "version": "0.6.80",
   "description": "High-performance RDF/SPARQL database with AI agent framework. GraphDB (449ns lookups, 35x faster than RDFox), GraphFrames analytics (PageRank, motifs), Datalog reasoning, HNSW vector embeddings. HyperMindAgent for schema-aware query generation with audit trails. W3C SPARQL 1.1 compliant. Native performance via Rust + NAPI-RS.",
   "main": "index.js",
   "types": "index.d.ts",