@ruvector/ruvllm 0.2.3 → 2.4.0

package/README.md

@@ -1,886 +1,526 @@
-# RuvLLM
+# @ruvector/ruvllm v2.4
 
-[![Rust](https://img.shields.io/badge/rust-1.77%2B-orange.svg)](https://www.rust-lang.org/)
-[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)](LICENSE)
-[![Tests](https://img.shields.io/badge/tests-62%20passing-brightgreen.svg)](#testing)
-[![CPU](https://img.shields.io/badge/platform-CPU%20SIMD-green.svg)](#architecture)
-[![HuggingFace](https://img.shields.io/badge/export-HuggingFace-yellow.svg)](#huggingface-export)
-[![npm](https://img.shields.io/npm/v/@ruvector/ruvllm.svg)](https://www.npmjs.com/package/@ruvector/ruvllm)
-[![TRM](https://img.shields.io/badge/TRM-7M%20params-purple.svg)](#trm-tiny-recursive-models)
+Self-learning LLM orchestration with SONA adaptive learning, HNSW memory, RLM recursive retrieval, and SIMD inference for Node.js.
 
-**Self-Optimizing Neural Architecture (SONA) with TRM Recursive Reasoning, LFM2 Cortex, Ruvector Memory, and Intelligent Routing**
+## Installation
 
-> *"The intelligence is not in one model anymore. It is in the loop."*
-
----
-
-## What is RuvLLM?
-
-RuvLLM is a **self-learning language model orchestration system** that combines frozen foundation models with adaptive memory and intelligent routing. Unlike traditional LLMs that rely solely on static parameters, RuvLLM continuously improves from every interaction through three temporal learning loops.
-
-**Key Innovation**: RuvLLM doesn't replace your LLM—it makes any LLM smarter over time by learning from experience, routing intelligently, and preventing catastrophic forgetting.
-
-```
-┌─────────────────────────────────────────────────────────────────────────┐
-│                         RuvLLM Architecture                              │
-├─────────────────────────────────────────────────────────────────────────┤
-│                                                                          │
-│    Query ──► Embedding ──► Memory Search ──► Router Decision            │
-│                               │                    │                     │
-│                               ▼                    ▼                     │
-│                         Graph Attention      Model Selection             │
-│                               │                    │                     │
-│                               └────────┬───────────┘                     │
-│                                        ▼                                 │
-│                              ┌─────────────────────┐                     │
-│                              │   LLM Inference    │                     │
-│                              │  (Any LLM Backend)  │                     │
-│                              └─────────────────────┘                     │
-│                                        │                                 │
-│                                        ▼                                 │
-│                    ┌───────────────────────────────────┐                │
-│                    │  SONA Learning (3 Temporal Loops) │                │
-│                    │  • Instant: Per-request MicroLoRA │                │
-│                    │  • Background: Hourly patterns    │                │
-│                    │  • Deep: Weekly EWC++ updates     │                │
-│                    └───────────────────────────────────┘                │
-│                                                                          │
-└─────────────────────────────────────────────────────────────────────────┘
+```bash
+npm install @ruvector/ruvllm
 ```
 
----
-
-## Features
-
-### Core Components
-
-| Component | Description | Implementation |
-|-----------|-------------|----------------|
-| **LFM2 Cortex** | Frozen reasoning engine (135M-2.6B params) | Mock, Candle, or external (llama.cpp/vLLM) |
-| **Ruvector Memory** | Adaptive synaptic mesh with HNSW indexing | Full CPU implementation with graph expansion |
-| **FastGRNN Router** | Intelligent model selection circuit | Sparse + low-rank matrices with EWC learning |
-| **Graph Attention** | Multi-head attention with edge features | 8-head attention, layer normalization |
-| **SONA Engine** | Self-optimizing neural architecture | LoRA + EWC++ + ReasoningBank |
-| **TRM Engine** | Tiny Recursive Models (7M params) | Recursive latent refinement with SONA bridge |
-
-### TRM (Tiny Recursive Models)
-
-RuvLLM v0.2.3 introduces **TRM** - Samsung SAIL Montreal's parameter-efficient recursive reasoning approach. TRM achieves strong reasoning performance with only **7M parameters** through iterative latent refinement.
+## Quick Start
 
-```
-┌─────────────────────────────────────────────────────────────────────────┐
-│                         TRM Architecture                                 │
-├─────────────────────────────────────────────────────────────────────────┤
-│                                                                          │
-│    Question ──┬► Latent Update (n times) ──► Answer Refine ──┐          │
-│               │                                               │          │
-│               └───────────────────────────────────────────────┘          │
-│                            (repeat K times)                              │
-│                                                                          │
-│    Components:                                                           │
-│    • MLP Latent Updater - Fast feed-forward updates                     │
-│    • Attention Latent Updater - Multi-head attention refinement         │
-│    • Confidence Scorer - Early stopping based on convergence            │
-│    • Answer Refiner - Residual-based answer improvement                 │
-│    • SONA Bridge - Integration with learning loops                      │
-│                                                                          │
-└─────────────────────────────────────────────────────────────────────────┘
-```
+```typescript
+import { RuvLLM, RuvLLMConfig } from '@ruvector/ruvllm';
 
-**Key Features:**
-- **7M parameters** - Achieves 83% on GSM8K with minimal compute
-- **Recursive refinement** - Iteratively improves answers through K iterations
-- **Adaptive K** - SONA routing determines optimal iteration count
-- **Early stopping** - Confidence-based termination for efficiency
-- **NaN-safe** - Robust numerical guards prevent gradient explosions
-- **Buffer reuse** - Optimized memory allocation for production use
+// Initialize with default configuration
+const llm = new RuvLLM();
 
-### SONA: Self-Optimizing Neural Architecture
+// Or with custom configuration
+const llm = new RuvLLM({
+  modelPath: './models/ruvltra-small-q4km.gguf',
+  sonaEnabled: true,
+  flashAttention: true,
+  maxTokens: 256,
+});
 
-RuvLLM introduces **SONA**, a three-tier temporal learning system:
+// Generate text
+const response = await llm.query('Explain quantum computing');
+console.log(response.text);
 
-```
-┌──────────────────────────────────────────────────────────────────────────┐
-│  Loop A: Instant (Per-Request)                           Latency: <100μs │
-│  ──────────────────────────────────────                                  │
-│  • Records query trajectories with activation patterns                   │
-│  • MicroLoRA adaptation (rank 1-2) for immediate improvement             │
-│  • SIMD-optimized: 2,236 ops/sec throughput                              │
-├──────────────────────────────────────────────────────────────────────────┤
-│  Loop B: Background (Hourly)                                             │
-│  ─────────────────────────────                                           │
-│  • K-means++ clustering extracts patterns (100 clusters = 1.3ms search)  │
-│  • Base LoRA updates (rank 4-16) from successful patterns                │
-│  • ReasoningBank stores learned strategies                               │
-├──────────────────────────────────────────────────────────────────────────┤
-│  Loop C: Deep (Weekly)                                                   │
-│  ─────────────────────                                                   │
-│  • Dream consolidation across all memory                                 │
-│  • EWC++ prevents catastrophic forgetting (λ=2000 optimal)               │
-│  • Concept hierarchies created, old nodes archived                       │
-└──────────────────────────────────────────────────────────────────────────┘
+// Stream generation
+for await (const token of llm.stream('Write a haiku about Rust')) {
+  process.stdout.write(token);
+}
 ```
 
-### Advanced Features
+## What's New in v2.4
 
 | Feature | Description |
 |---------|-------------|
-| **SIMD Inference** | Native AVX2/AVX512/SSE4.1 operations for CPU optimization |
-| **Q4 Quantization** | 4-bit weight quantization for memory efficiency |
-| **MicroLoRA** | Per-request adaptation with rank 1-2 (benchmark: rank-2 is 5% faster) |
-| **EWC++** | Enhanced elastic weight consolidation with online Fisher estimation |
-| **ReasoningBank** | Pattern storage with K-means++ clustering |
-| **HuggingFace Export** | Export LoRA weights, patterns, and preference pairs |
-| **Real Inference** | Candle-based inference with HuggingFace model support |
-| **Multi-Model Routing** | Automatic selection between SmolLM, Qwen2, TinyLlama |
-| **Federated Learning** | Distributed learning across ephemeral agents with central coordinator |
-| **WASM Support** | Run SONA in browsers and edge devices |
-| **Training Pipelines** | Templated training for code, chat, reasoning, and custom agents |
-| **Agent Factory** | Create and manage multiple specialized learning agents |
-| **TRM Reasoning** | Recursive reasoning with only 7M parameters (83% GSM8K) |
-| **Adaptive K Routing** | SONA-driven iteration count for optimal compute |
-| **NaN Guards** | Robust numerical stability for production deployment |
-
-### Federated Learning Architecture
-
-RuvLLM supports **federated learning** where ephemeral agents collect trajectories and export to a central coordinator:
-
-```
-┌─────────────┐     ┌─────────────┐     ┌─────────────┐
-│  Agent A    │     │  Agent B    │     │  Agent C    │
-│ (ephemeral) │     │ (ephemeral) │     │ (ephemeral) │
-└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
-       │                   │                   │
-       │    export()       │    export()       │    export()
-       ▼                   ▼                   ▼
-  ┌────────────────────────────────────────────────┐
-  │            Federated Coordinator               │
-  │         (persistent, large capacity)           │
-  │  • Aggregates trajectories from all agents     │
-  │  • Quality-filtered acceptance (threshold)     │
-  │  • Auto-consolidation every N agents           │
-  │  • Shares patterns with new agents             │
-  └────────────────────────────────────────────────┘
-```
+| **RLM (Recursive Language Model)** | Query decomposition with recursive retrieval and synthesis |
+| **100% Routing Accuracy** | Hybrid keyword-first strategy achieves 100% on Claude Code tasks |
+| **145 Tests Passing** | Comprehensive test coverage across all modules |
+| **Contrastive Fine-tuning** | LoRA-based training with 793 contrastive pairs |
+| **Training Scripts** | Generate routing datasets and fine-tune models |
+| **HuggingFace Models** | Pre-trained RuvLTRA models available |
 
-**Key Components**:
-- **EphemeralAgent**: Short-lived agents that process tasks and export learned state
-- **FederatedCoordinator**: Central aggregator with 50K trajectory capacity
-- **AgentExport**: Serializable state containing trajectories, stats, and patterns
-- **Quality Filtering**: Only high-quality trajectories (>0.4 score) are aggregated
+## RLM (Recursive Language Model)
 
----
+RLM provides recursive retrieval-augmented generation that breaks down complex queries into sub-queries and synthesizes answers from retrieved context.
 
-## Performance Benchmarks
+### Basic Usage
 
-### Orchestration Latency (CPU-Only)
+```typescript
+import { RlmController } from '@ruvector/ruvllm';
 
-| Metric | Value | Notes |
-|--------|-------|-------|
-| **Initialization** | 3.71ms | Full system startup |
-| **Average Query** | 0.09ms | Single query latency |
-| **Session Query** | 0.04ms | With context reuse |
-| **Throughput** | ~38,000 q/s | 8 concurrent queries |
-| **Memory Footprint** | ~50MB | Base system |
+const rlm = new RlmController({
+  maxDepth: 5,
+  retrievalTopK: 10,
+  enableCache: true,
+});
 
-### Latency Breakdown
+// Add knowledge to memory
+await rlm.addMemory('TypeScript adds static typing to JavaScript.');
+await rlm.addMemory('React is a library for building user interfaces.');
 
+// Query with recursive retrieval
+const answer = await rlm.query('What are causes and solutions for type errors in React?');
+console.log(answer.text);
+console.log('Sources:', answer.sources);
+console.log('Quality Score:', answer.qualityScore);
+console.log('Confidence:', answer.confidence);
 ```
-Embedding:    ~0.02ms  ████░░░░░░  (20%)
-Retrieval:    ~0.01ms  ██░░░░░░░░  (10%)
-Routing:      ~0.01ms  ██░░░░░░░░  (10%)
-Attention:    ~0.02ms  ████░░░░░░  (20%)
-Generation:   ~0.04ms  ████████░░  (40%)
-```
-
-### SONA Learning Performance
-
-| Component | Metric | Value |
-|-----------|--------|-------|
-| MicroLoRA | Throughput | 2,236 ops/sec |
-| MicroLoRA | Batch-32 Latency | 0.447ms |
-| ReasoningBank | Pattern Search | 1.3ms (100 clusters) |
-| EWC++ | Fisher Update | <1ms |
-
-### Comparison with Traditional Systems
-
-| System | P50 (ms) | P95 (ms) | vs GPT-4o |
-|--------|----------|----------|-----------|
-| GPT-4o (API) | 450.00 | 585.00 | 1.0x (baseline) |
-| Claude 3.5 Sonnet | 380.00 | 456.00 | 1.2x |
-| Gemini 2.0 Flash | 180.00 | 234.00 | 2.5x |
-| Llama 3.3 70B (vLLM) | 120.00 | 168.00 | 3.8x |
-| **RuvLLM Orchestration** | **0.06** | **0.08** | **~7,500x** |
-
-> **Note**: RuvLLM orchestration latency measures memory retrieval, routing, and context preparation—NOT LLM generation. Actual response quality depends on your LLM backend.
-
----
-
-## Feature Comparison
-
-| Feature | GPT-4o | Claude | RAG | vLLM | RuvLLM |
-|---------|--------|--------|-----|------|--------|
-| On-device Inference | ✗ | ✗ | ✗ | ✓ | ✓ |
-| Continuous Learning | ✗ | ✗ | ✗ | ✗ | ✓ |
-| Graph-based Memory | ✗ | ✗ | △ | ✗ | ✓ |
-| Adaptive Model Routing | ✗ | ✗ | ✗ | ✗ | ✓ |
-| EWC Anti-Forgetting | ✗ | ✗ | ✗ | ✗ | ✓ |
-| LoRA Adaptation | ✗ | ✗ | ✗ | ✗ | ✓ |
-| Pattern Extraction | ✗ | ✗ | ✗ | ✗ | ✓ |
-| HuggingFace Export | ✗ | ✗ | ✗ | ✗ | ✓ |
-| SIMD Optimization | ✗ | ✗ | ✗ | △ | ✓ |
-| Sub-ms Orchestration | ✗ | ✗ | ✗ | ✗ | ✓ |
-| Federated Learning | ✗ | ✗ | ✗ | ✗ | ✓ |
-| WASM/Browser Support | ✗ | ✗ | ✗ | ✗ | ✓ |
-| Training Pipelines | ✗ | ✗ | ✗ | ✗ | ✓ |
-| Works with ANY LLM | ✗ | ✗ | ✓ | ✗ | ✓ |
-| **TRM Recursive Reasoning** | ✗ | ✗ | ✗ | ✗ | ✓ |
-| **7M Param Efficiency** | ✗ | ✗ | ✗ | ✗ | ✓ |
-
-*Legend: ✓ = Full Support, △ = Partial, ✗ = Not Supported*
-
----
-
-## Quick Start
 
-### Prerequisites
+### Streaming
 
-- Rust 1.77+
-- Cargo
+```typescript
+import { RlmController } from '@ruvector/ruvllm';
 
-### Installation
+const rlm = new RlmController();
 
-```bash
-# Clone the repository
-git clone https://github.com/ruvnet/ruvector.git
-cd ruvector/examples/ruvLLM
-
-# Build in release mode
-cargo build --release
+for await (const event of rlm.queryStream('Explain machine learning')) {
+  if (event.type === 'token') {
+    process.stdout.write(event.text);
+  } else {
+    console.log('\n\nQuality:', event.answer.qualityScore);
+  }
+}
 ```
 
-### Run the Demo
-
-```bash
-# Interactive demo with mock inference
-cargo run --bin ruvllm-demo --release
-
-# SIMD capabilities demo
-cargo run --bin ruvllm-simd-demo --release
-
-# Quick benchmark
-cargo run --bin ruvllm-bench --release
-
-# Full benchmark suite
-cargo run --bin ruvllm-benchmark-suite --release
+### With Reflection
 
-# HTTP server (requires 'server' feature)
-cargo run --bin ruvllm-server --release --features server
+```typescript
+import { RlmController } from '@ruvector/ruvllm';
 
-# Pretraining pipeline
-cargo run --bin ruvllm-pretrain --release
+const rlm = new RlmController({
+  enableReflection: true,
+  maxReflectionIterations: 2,
+  minQualityScore: 0.8,
+});
 
-# HuggingFace export (requires 'hf-export' feature)
-cargo run --bin ruvllm-export --release --features hf-export -- help
+// Answers will be iteratively refined until quality >= 0.8
+const answer = await rlm.query('Complex multi-part technical question...');
 ```
 
-### Library Usage
+### RLM Configuration
 
-```rust
-use ruvllm::{Config, RuvLLM, Result};
-
-#[tokio::main]
-async fn main() -> Result<()> {
-    // Configure the system
-    let config = Config::builder()
-        .embedding_dim(768)
-        .router_hidden_dim(128)
-        .hnsw_params(32, 200, 64)  // M, ef_construction, ef_search
-        .learning_enabled(true)
-        .build()?;
-
-    // Initialize
-    let llm = RuvLLM::new(config).await?;
+```typescript
+interface RlmConfig {
+  maxDepth?: number;              // Max recursion depth (default: 3)
+  maxSubQueries?: number;         // Max sub-queries per level (default: 5)
+  tokenBudget?: number;           // Token budget (default: 4096)
+  enableCache?: boolean;          // Enable caching (default: true)
+  cacheTtl?: number;              // Cache TTL in ms (default: 300000)
+  retrievalTopK?: number;         // Memory spans to retrieve (default: 10)
+  minQualityScore?: number;       // Min quality threshold (default: 0.7)
+  enableReflection?: boolean;     // Enable self-reflection (default: false)
+  maxReflectionIterations?: number; // Max reflection loops (default: 2)
+}
+```
 
-    // Create a session for multi-turn conversation
-    let session = llm.new_session();
+## Exports
 
-    // Query with session context
-    let response = llm.query_session(&session, "What is machine learning?").await?;
+```typescript
+import {
+  // Core
+  RuvLLM,
+  RuvLLMConfig,
 
-    println!("Response: {}", response.text);
-    println!("Model: {:?}", response.routing_info.model);
-    println!("Confidence: {:.2}%", response.confidence * 100.0);
+  // RLM - Recursive Language Model
+  RlmController,
+  RlmConfig,
+  RlmAnswer,
+  MemorySpan,
+  StreamToken,
 
-    // Provide feedback for learning
-    llm.feedback(Feedback {
-        request_id: response.request_id,
-        rating: Some(5),
-        correction: None,
-        task_success: Some(true),
-    }).await?;
+  // RLM Training
+  RlmTrainer,
+  RlmTrainingConfig,
+  RlmTrainingExample,
+  createRlmTrainer,
+  DEFAULT_RLM_CONFIG,
+  FAST_RLM_CONFIG,
+  THOROUGH_RLM_CONFIG,
+  ROUTING_FOCUSED_CONFIG,
 
-    Ok(())
-}
-```
+  // SONA Learning
+  SonaCoordinator,
+  TrajectoryBuilder,
 
-### SIMD Inference Engine
+  // Federated Learning
+  EphemeralAgent,
+  FederatedCoordinator,
 
-```rust
-use ruvllm::{SimdInferenceEngine, SimdGenerationConfig, SimdOps};
+  // LoRA Adapters
+  LoraAdapter,
+  LoraManager,
 
-// Create SIMD-optimized engine
-let engine = SimdInferenceEngine::new(256, 128, 4, 4)?;
+  // Sessions
+  SessionManager,
 
-// Configure generation
-let config = SimdGenerationConfig {
-    max_tokens: 50,
-    temperature: 0.7,
-    top_p: 0.9,
-    ..Default::default()
-};
+  // Contrastive Training
+  ContrastiveTrainer,
 
-// Generate with SIMD acceleration
-let result = engine.generate("Once upon a time", &config)?;
+  // Benchmarks
+  ModelComparisonBenchmark,
+  RoutingBenchmark,
+  EmbeddingBenchmark,
+} from '@ruvector/ruvllm';
 ```
 
-### SONA Learning Loops
+## Performance
 
-```rust
-use ruvllm::sona::{LoopCoordinator, SonaConfig, InstantLoop, BackgroundLoop};
+### Test Results
 
-// Initialize SONA coordinator
-let config = SonaConfig {
-    hidden_dim: 256,
-    embedding_dim: 256,
-    pattern_clusters: 100,
-    ..Default::default()
-};
-
-let coordinator = LoopCoordinator::new(config);
+```
+145 tests passing
+  - RLM Controller: 24 tests
+  - Routing Accuracy: 18 tests
+  - Contrastive Training: 15 tests
+  - SIMD Operations: 22 tests
+  - SONA Learning: 19 tests
+  - Memory/HNSW: 21 tests
+  - Benchmarks: 26 tests
+```
 
-// Instant learning (per-request)
-coordinator.instant_loop().record_trajectory(query, response, quality);
+### Routing Accuracy (Claude Code Tasks)
 
-// Background learning (hourly)
-coordinator.background_loop().extract_patterns().await;
+| Strategy | RuvLTRA | Qwen Base |
+|----------|---------|-----------|
+| Embedding Only | 45% | 40% |
+| Keyword-First (Hybrid) | **100%** | 95% |
 
-// Deep learning (weekly) - automatically handles EWC++
-coordinator.deep_consolidation().await;
-```
+### Inference Performance (M4 Pro)
 
-### TRM Recursive Reasoning
+| Operation | Performance |
+|-----------|-------------|
+| Inference | 88-135 tok/s |
+| Flash Attention | 320us (seq=2048) |
+| HNSW Search | 17-62us |
+| SONA Adapt | <1ms |
+| RLM Query | 50-200ms |
 
-```rust
-use ruvllm::trm::{TrmEngine, TrmEngineBuilder, TrmConfig, RecursiveReasoner};
+### SIMD Optimizations
 
-// Build TRM engine with custom configuration
-let mut engine = TrmEngineBuilder::new()
-    .hidden_dim(256)
-    .embedding_dim(256)
-    .default_k(10)           // Default K iterations
-    .n_inner(4)              // Inner latent updates per K
-    .confidence_threshold(0.95)  // Early stopping threshold
-    .build()
-    .unwrap();
+- AVX2/AVX-512 on x86_64
+- NEON on ARM64
+- 4-8x speedup on vector operations
 
-// Prepare question and answer embeddings
-let question = vec![0.5; 256];  // Question embedding
-let mut answer = vec![0.1; 256]; // Initial answer (refined in-place)
+## CLI Usage
 
-// Perform recursive reasoning
-let result = engine.reason(&question, &mut answer);
+```bash
+# Query a model
+ruvllm query "What is machine learning?"
 
-println!("Confidence: {:.2}%", result.confidence * 100.0);
-println!("Iterations used: {}/{}", result.iterations_used, result.max_iterations);
-println!("Early stopped: {}", result.early_stopped);
+# Stream output
+ruvllm query --stream "Write a poem"
 
-// With SONA routing for adaptive K
-use ruvllm::trm::SonaBridge;
+# Download a model
+ruvllm download ruvector/ruvltra-small-q4km
 
-let bridge = SonaBridge::new(256, 256);
-let routing = bridge.compute_routing(&question, 0.8);  // quality hint
+# Benchmark
+ruvllm bench ./models/model.gguf
 
-let result = engine.reason_with_routing(&question, &mut answer, &routing);
-println!("Adaptive K used: {}", routing.k);
+# Run evaluation (SWE-Bench)
+ruvllm eval --model ./models/model.gguf --subset lite --max-tasks 50
 ```
 
-### Federated Learning
+## Training
 
-```rust
-use ruvector_sona::training::{EphemeralAgent, FederatedCoordinator, SonaConfig};
+### Routing Dataset Generation
 
-// Create central coordinator (persistent, large capacity)
-let mut coordinator = FederatedCoordinator::default_coordinator("main", 3072);
-coordinator.set_quality_threshold(0.4);  // Only accept high-quality trajectories
-coordinator.set_consolidation_interval(50);  // Auto-consolidate every 50 agents
+Generate training data for agent routing:
 
-// Create ephemeral agents for distributed learning
-let mut agent = EphemeralAgent::default_federated("agent-1", 3072);
+```bash
+# Generate routing dataset
+node scripts/training/routing-dataset.js
 
-// Agent processes tasks and learns locally
-agent.process_trajectory(
-    embedding,      // Query embedding
-    activations,    // Hidden state activations
-    quality,        // Quality score [0.0, 1.0]
-    Some("gpt-4".to_string()),  // Model route
-    vec!["code".to_string()],   // Context tags
-);
+# Output: 381 examples, 793 contrastive pairs
+```
 
-// Export state before agent termination
-let export = agent.export_state();
-println!("Agent exported {} trajectories", export.trajectories.len());
+### Contrastive Fine-tuning
 
-// Coordinator aggregates learning from all agents
-let result = coordinator.aggregate(export);
-println!("Accepted: {}, Rejected: {}",
-    result.trajectories_accepted,
-    result.trajectories_rejected
-);
+Fine-tune models with LoRA adapters:
 
-// Get patterns for warm-starting new agents
-let patterns = coordinator.get_initial_patterns(10);
-```
+```typescript
+import { ContrastiveTrainer, ContrastivePair } from '@ruvector/ruvllm';
 
-### WASM Usage (Browser/Edge)
+const trainer = new ContrastiveTrainer({
+  modelPath: './models/base.gguf',
+  loraRank: 8,
+  loraAlpha: 16,
+  learningRate: 1e-4,
+});
 
-Build SONA for WebAssembly:
+// Training pairs from routing dataset
+const pairs: ContrastivePair[] = [
+  {
+    anchor: 'Fix the authentication bug in login.ts',
+    positive: 'coder',
+    negative: 'researcher',
+  },
+  // ... more pairs
+];
 
-```bash
-# Build WASM package
-cd crates/sona
-wasm-pack build --target web --features wasm
+await trainer.train(pairs, { epochs: 10, batchSize: 32 });
+await trainer.save('./adapters/routing-lora');
 ```
 
-Use in JavaScript:
+### Training Scripts
 
-```javascript
-import init, { WasmSonaEngine } from './pkg/sona.js';
+Located in `scripts/training/`:
 
-async function main() {
-  await init();
+| Script | Description |
+|--------|-------------|
+| `routing-dataset.js` | Generate 381 routing examples |
+| `claude-code-synth.js` | Synthetic data generation |
+| `contrastive-finetune.js` | LoRA fine-tuning pipeline |
 
-  // Create SONA engine
-  const engine = new WasmSonaEngine(256);  // hidden_dim = 256
+## Model Links
 
-  // Or with custom configuration
-  const engineCustom = WasmSonaEngine.withConfig({
-    hidden_dim: 256,
-    embedding_dim: 256,
-    micro_lora_rank: 2,
-    base_lora_rank: 16,
-    ewc_lambda: 1000.0,
-    pattern_clusters: 128,
-  });
+### HuggingFace Repository
 
-  // Start trajectory
-  const embedding = new Float32Array(256).fill(0.1);
-  const trajectoryId = engine.startTrajectory(embedding);
+**URL**: [https://huggingface.co/ruv/ruvltra](https://huggingface.co/ruv/ruvltra)
 
-  // Record steps
-  engine.recordStep(trajectoryId, 42, 0.8, 1000);
+### Available Models
 
-  // End trajectory with quality score
-  engine.endTrajectory(trajectoryId, 0.85);
+| Model | File | Size | Purpose |
+|-------|------|------|---------|
+| RuvLTRA Claude Code 0.5B | `ruvltra-claude-code-0.5b-q4_k_m.gguf` | ~400MB | Agent routing (100% with hybrid) |
+| RuvLTRA Small 0.5B | `ruvltra-0.5b-q4_k_m.gguf` | ~400MB | General embeddings |
+| RuvLTRA Medium 3B | `ruvltra-3b-q4_k_m.gguf` | ~2GB | Full LLM inference |
 
-  // Apply LoRA transformation
-  const input = new Float32Array(256).fill(1.0);
-  const output = engine.applyLora(input);
+### Download Models
 
-  // Run learning cycles
-  engine.runInstantCycle();  // Flush micro-LoRA updates
-  if (engine.tick()) {       // Background learning
-    console.log('Background learning completed');
-  }
+```bash
+# Using CLI
+ruvllm download ruv/ruvltra
 
-  // Get statistics
-  const stats = engine.stats();
-  console.log('Patterns:', stats.patterns_stored);
-}
+# Using HuggingFace CLI
+huggingface-cli download ruv/ruvltra ruvltra-claude-code-0.5b-q4_k_m.gguf
+
+# Programmatic download
+import { downloadModel } from '@ruvector/ruvllm';
+await downloadModel('ruv/ruvltra', { quantization: 'q4_k_m' });
 ```
 
----
+### Environment Variables
 
-## HuggingFace Export
+```bash
+# HuggingFace authentication (any of these)
+HF_TOKEN=hf_xxx
+HUGGING_FACE_HUB_TOKEN=hf_xxx
+HUGGINGFACE_API_KEY=hf_xxx
+```
 
-Export learned patterns, LoRA weights, and preference pairs to HuggingFace:
+### Local Model Storage
 
 ```bash
-# Export LoRA weights in PEFT-compatible SafeTensors format
-ruvllm-export safetensors ./exports/lora
-
-# Export learned patterns as JSONL dataset
-ruvllm-export patterns ./exports/patterns
+~/.ruvllm/models/       # Downloaded GGUF models
+~/.ruvllm/training/     # Training data and configs
+```
 
-# Export DPO/RLHF preference pairs
-ruvllm-export preferences ./exports/preferences
+## API Reference
 
-# Export all artifacts
-ruvllm-export all ./exports
+### RuvLLM Class
 
-# Push to HuggingFace Hub
-HF_TOKEN=your_token ruvllm-export push username/my-sona-model
+```typescript
+class RuvLLM {
+  constructor(config?: RuvLLMConfig);
 
-# Generate pretraining pipeline configuration
-ruvllm-export pretrain ./exports
-```
+  // Generate text
+  query(prompt: string, params?: GenerateParams): Promise<Response>;
 
----
+  // Stream generation
+  stream(prompt: string, params?: GenerateParams): AsyncIterable<string>;
 
-## Architecture Deep Dive
+  // Load a model
+  loadModel(path: string): Promise<void>;
 
-### HNSW Memory Index
+  // Memory operations
+  addMemory(text: string, metadata?: Record<string, unknown>): number;
+  searchMemory(query: string, topK?: number): MemoryResult[];
 
-The memory system uses Hierarchical Navigable Small World graphs:
+  // Get SONA learning stats
+  sonaStats(): SonaStats | null;
 
+  // Adapt on feedback
+  adapt(input: Float32Array, quality: number): void;
+}
 ```
-Layer 2:  [3] ─────────────────── [7]
-           │                       │
-Layer 1:  [3] ─── [5] ─────────── [7] ─── [9]
-           │      │                │       │
-Layer 0:  [1]─[2]─[3]─[4]─[5]─[6]─[7]─[8]─[9]─[10]
-
-• M = 32 connections per node
-• ef_construction = 200 for build quality
-• ef_search = 64 for query speed
-• O(log N) search complexity
-```
-
-### FastGRNN Router
-
-Sparse + Low-rank matrices for efficient routing:
 
-```
-           Input (128-dim)
-                │
-        ┌───────┴───────┐
-        │  LayerNorm    │
-        └───────┬───────┘
-                │
-    ┌───────────┴───────────┐
-    │   FastGRNN Cell       │
-    │                       │
-    │  W_sparse (90% zero)  │
-    │  U = A @ B (rank-8)   │
-    │                       │
-    │  z = σ(Wx + Uh + b)   │
-    │  h' = z⊙h + (1-z)⊙ν   │
-    └───────────┬───────────┘
-                │
-        ┌───────┴───────┐
-        │ Output Heads  │
-        ├───────────────┤
-        │ Model Select  │ → 4 classes
-        │ Context Size  │ → 5 buckets
-        │ Temperature   │ → continuous
-        │ Top-p         │ → continuous
-        │ Confidence    │ → continuous
-        └───────────────┘
-```
+### RlmController Class
 
-### MicroLoRA Architecture
+```typescript
+class RlmController {
+  constructor(config?: RlmConfig, engine?: RuvLLM);
 
-Two-tier LoRA system for adaptive learning:
+  // Query with recursive retrieval
+  query(input: string): Promise<RlmAnswer>;
 
-```
-┌─────────────────────────────────────────────────────────────┐
-│                      MicroLoRA (Rank 1-2)                   │
-│                   Per-Request Adaptation                    │
-├─────────────────────────────────────────────────────────────┤
-│                                                             │
-│   Input ──► Down Proj ──► Up Proj ──► Scale ──► Add        │
-│   (dim)     (dim→rank)   (rank→dim)   (α/r)    to output   │
-│                                                             │
-│   Performance: <100μs latency, 2,236 ops/sec               │
-│   Rank-2 is ~5% faster than Rank-1 (better SIMD)           │
-└─────────────────────────────────────────────────────────────┘
-
-┌─────────────────────────────────────────────────────────────┐
-│                      BaseLoRA (Rank 4-16)                   │
-│                   Background Adaptation                     │
-├─────────────────────────────────────────────────────────────┤
-│                                                             │
-│   Aggregated from successful MicroLoRA patterns             │
-│   Merged hourly into base weights                           │
-│   EWC++ regularization prevents forgetting                  │
-│                                                             │
-└─────────────────────────────────────────────────────────────┘
-```
+  // Stream query
+  queryStream(input: string): AsyncGenerator<StreamToken>;
 
-### EWC++ (Enhanced Elastic Weight Consolidation)
+  // Memory management
+  addMemory(text: string, metadata?: Record<string, unknown>): Promise<string>;
+  searchMemory(query: string, topK?: number): Promise<MemorySpan[]>;
 
-Prevents catastrophic forgetting:
+  // Cache management
+  clearCache(): void;
+  getCacheStats(): { size: number; entries: number };
 
+  // Configuration
+  updateConfig(config: Partial<RlmConfig>): void;
+  getConfig(): Required<RlmConfig>;
+}
 ```
-Loss = Task_Loss + λ * Σᵢ Fᵢ(θᵢ - θ*ᵢ)²
-
-Where:
-• Fᵢ = Online Fisher information (EMA decay 0.999)
-• θ*ᵢ = Optimal weights for previous tasks
-• λ = Adaptive (2000 default, range 100-15000)
-• Multi-task memory with circular buffer (10 tasks)
-• Automatic task boundary detection
-```
-
-### SIMD Operations
-
-Native CPU acceleration:
-
-```rust
-// AVX2 dot product (8 floats at a time)
-#[target_feature(enable = "avx2")]
-unsafe fn dot_product_avx2(a: &[f32], b: &[f32]) -> f32
 
-// SSE4.1 fallback (4 floats at a time)
-#[target_feature(enable = "sse4.1")]
-unsafe fn dot_product_sse(a: &[f32], b: &[f32]) -> f32
+### Configuration
 
-// Automatic detection and dispatch
-let result = SimdOps::dot_product(&a, &b);
+```typescript
+interface RuvLLMConfig {
+  modelPath?: string;       // Path to GGUF model
+  sonaEnabled?: boolean;    // Enable SONA learning (default: true)
+  flashAttention?: boolean; // Use Flash Attention 2 (default: true)
+  maxTokens?: number;       // Max generation tokens (default: 256)
+  temperature?: number;     // Sampling temperature (default: 0.7)
+  topP?: number;            // Top-p sampling (default: 0.9)
+}
 ```
 
----
-
-## Supported Models
+### Generate Parameters
 
-### Real Inference (CPU SIMD)
+```typescript
+interface GenerateParams {
+  maxTokens?: number;
+  temperature?: number;
+  topP?: number;
+  topK?: number;
+  repetitionPenalty?: number;
+  stopSequences?: string[];
+}
+```
 
-| Model | Parameters | Context | Repo |
-|-------|------------|---------|------|
-| SmolLM 135M | 135M | 2048 | HuggingFaceTB/SmolLM-135M |
-| SmolLM 360M | 360M | 2048 | HuggingFaceTB/SmolLM-360M |
-| Qwen2 0.5B | 500M | 4096 | Qwen/Qwen2-0.5B |
-| TinyLlama 1.1B | 1.1B | 2048 | TinyLlama/TinyLlama-1.1B-Chat |
+## SIMD Module
 
-All models support Q4_K_M quantization for efficient CPU inference.
+For direct access to optimized SIMD kernels:
 
----
+```typescript
+import { simd } from '@ruvector/ruvllm/simd';
 
-## HTTP Server API
+// Dot product
+const result = simd.dotProduct(vecA, vecB);
 
-When running with the `server` feature:
+// Matrix multiplication
+const output = simd.matmul(matrix, vector);
 
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/health` | GET | Health check |
-| `/query` | POST | Submit query |
-| `/stats` | GET | Get statistics |
-| `/feedback` | POST | Submit feedback |
-| `/session` | POST | Create new session |
+// Flash Attention
+const attended = simd.flashAttention(query, key, value, scale);
 
-```bash
-# Example query
-curl -X POST http://localhost:3000/query \
-  -H "Content-Type: application/json" \
-  -d '{"query": "What is Rust?", "session_id": null}'
+// RMS Normalization
+simd.rmsNorm(hidden, weights, epsilon);
 ```
 
----
+## Evaluation Harness
 
-## Testing
-
-```bash
-# Run all tests
-cargo test -p ruvllm
-
-# Unit tests only (47 tests)
-cargo test -p ruvllm --lib
-
-# Integration tests (15 tests)
-cargo test -p ruvllm --test integration
-
-# With output
-cargo test -p ruvllm -- --nocapture
-```
+Run model evaluations with SWE-Bench integration:
 
-### Test Coverage
+```typescript
+import { RuvLLM, EvaluationHarness, AblationMode } from '@ruvector/ruvllm';
 
-| Module | Tests | Coverage |
-|--------|-------|----------|
-| Memory (HNSW) | 12 | Search, insertion, graph expansion |
-| Router (FastGRNN) | 8 | Forward pass, training, EWC |
-| Attention | 6 | Multi-head, edge features, cross-attention |
-| Embedding | 9 | Tokenization, caching, pooling |
-| SONA | 10 | LoRA, EWC++, ReasoningBank, loops |
-| Orchestrator | 2 | End-to-end pipeline |
-| Integration | 15 | Full system tests |
+const harness = new EvaluationHarness({
+  modelPath: './models/model.gguf',
+  enableHnsw: true,
+  enableSona: true,
+});
 
----
+// Run single evaluation
+const result = await harness.evaluate(
+  'Fix the null pointer exception',
+  'def process(data): return data.split()',
+  AblationMode.Full
+);
 
-## Project Structure
+console.log(`Success: ${result.success}, Quality: ${result.qualityScore}`);
 
-```
-examples/ruvLLM/
-├── Cargo.toml              # Dependencies and features
-├── README.md               # This file
-├── src/
-│   ├── lib.rs              # Library entry point
-│   ├── config.rs           # Configuration system
-│   ├── error.rs            # Error types
-│   ├── types.rs            # Core domain types
-│   ├── orchestrator.rs     # Main RuvLLM coordinator
-│   ├── memory.rs           # HNSW memory service
-│   ├── router.rs           # FastGRNN router
-│   ├── attention.rs        # Graph attention engine
-│   ├── embedding.rs        # Embedding service
-│   ├── inference.rs        # Mock inference pool
-│   ├── inference_real.rs   # Candle-based real inference
-│   ├── simd_inference.rs   # SIMD-optimized transformer
-│   ├── learning.rs         # Self-learning service
-│   ├── compression.rs      # Memory compression
-│   ├── training.rs         # Pretraining pipeline
-│   ├── trm/                # TRM (Tiny Recursive Models) module
-│   │   ├── mod.rs          # Module exports and traits
-│   │   ├── engine.rs       # Main TRM reasoning engine
-│   │   ├── config.rs       # Configuration and builder
-│   │   ├── mlp.rs          # MLP latent updater
-│   │   ├── attention.rs    # Attention latent updater
-│   │   ├── refiner.rs      # Answer refinement
-│   │   ├── confidence.rs   # Confidence scoring
-│   │   ├── sona_bridge.rs  # SONA integration
-│   │   ├── types.rs        # TRM types and results
-│   │   └── error.rs        # Error handling
-│   ├── sona/               # SONA module
-│   │   ├── mod.rs          # Module exports
-│   │   ├── types.rs        # SONA types
-│   │   ├── lora.rs         # MicroLoRA & BaseLoRA
-│   │   ├── ewc.rs          # EWC++ implementation
-│   │   ├── reasoning_bank.rs  # Pattern storage
-│   │   ├── trajectory.rs   # Trajectory recording
-│   │   ├── engine.rs       # SONA engine
-│   │   └── loops/          # Temporal learning loops
-│   │       ├── instant.rs  # Per-request loop
-│   │       ├── background.rs  # Hourly loop
-│   │       └── coordinator.rs # Loop coordinator
-│   └── bin/
-│       ├── demo.rs         # Interactive demo
-│       ├── bench.rs        # Quick benchmarks
-│       ├── benchmark_suite.rs  # Full benchmark suite
-│       ├── simd_demo.rs    # SIMD capabilities demo
-│       ├── pretrain.rs     # Pretraining pipeline
-│       ├── export.rs       # HuggingFace export
-│       └── server.rs       # HTTP server
-├── tests/
-│   └── integration.rs      # Integration tests
-├── benches/
-│   ├── pipeline.rs         # Full pipeline benchmarks
-│   ├── router.rs           # Router benchmarks
-│   ├── memory.rs           # Memory benchmarks
-│   ├── attention.rs        # Attention benchmarks
-│   ├── sona_bench.rs       # SONA benchmarks
-│   └── trm_bench.rs        # TRM benchmarks
-├── config/                 # Configuration files
-└── docs/
-    └── sparc/              # SPARC methodology docs
+// Run ablation study (Baseline, RetrievalOnly, AdaptersOnly, R+A, Full)
+const report = await harness.runAblationStudy(tasks);
+for (const [mode, metrics] of Object.entries(report.modeMetrics)) {
+  console.log(`${mode}: ${metrics.successRate * 100}% success`);
+}
 ```
 
----
+## mistral-rs Backend (Production Serving)
 
-## Feature Flags
+For production deployments with 10-100+ concurrent users, use the mistral-rs backend:
 
-### RuvLLM Features
+```typescript
+import { RuvLLM, MistralBackend, PagedAttentionConfig } from '@ruvector/ruvllm';
 
-| Feature | Default | Description |
-|---------|---------|-------------|
-| `storage` | ✓ | Persistent storage and HNSW indexing |
-| `metrics` | ✓ | Prometheus metrics export |
-| `server` | ✗ | HTTP server with Axum |
-| `real-inference` | ✗ | Candle-based real LLM inference |
-| `hf-export` | ✗ | HuggingFace export via ruvector-sona |
-| `full` | ✗ | All features enabled |
+// Configure for production serving
+const backend = new MistralBackend({
+  // PagedAttention: 5-10x more concurrent users
+  pagedAttention: {
+    blockSize: 16,
+    maxBlocks: 4096,
+    gpuMemoryFraction: 0.9,
+    prefixCaching: true,
+  },
+  // X-LoRA: Per-token adapter routing
+  xlora: {
+    adapters: ['./adapters/coder', './adapters/researcher'],
+    topK: 2,
+  },
+  // ISQ: Runtime quantization
+  isq: {
+    bits: 4,
+    method: 'awq',
+  },
+});
 
-```bash
-# Build with all features
-cargo build --release --features full
-```
+const llm = new RuvLLM({ backend });
+await llm.loadModel('mistralai/Mistral-7B-Instruct-v0.2');
 
-### ruvector-sona Features (Dependency)
+// Serve multiple concurrent requests
+const response = await llm.query('Write production code');
+```
 
-| Feature | Default | Description |
-|---------|---------|-------------|
-| `serde-support` | ✓ | Serialization for export, training, and federated learning |
-| `wasm` | ✗ | WebAssembly bindings for browser/edge deployment |
-| `napi` | ✗ | N-API bindings for Node.js integration |
+> **Note**: mistral-rs features require the Rust backend with `mistral-rs` feature enabled. Native bindings will use mistral-rs when available.
 
-```bash
-# Build SONA with WASM support
-cd crates/sona
-wasm-pack build --target web --features wasm
-```
+## Supported Models
 
----
-
-## Configuration Options
-
-| Option | Default | Description |
-|--------|---------|-------------|
-| `embedding.dimension` | 768 | Embedding vector size |
-| `embedding.max_tokens` | 512 | Max tokens per input |
-| `memory.hnsw_m` | 16 | HNSW connections per node |
-| `memory.hnsw_ef_construction` | 100 | Build quality parameter |
-| `memory.hnsw_ef_search` | 64 | Search quality parameter |
-| `router.input_dim` | 128 | Router input features |
-| `router.hidden_dim` | 64 | FastGRNN hidden size |
-| `router.sparsity` | 0.9 | Weight matrix sparsity |
-| `router.rank` | 8 | Low-rank decomposition |
-| `learning.enabled` | true | Enable self-learning |
-| `learning.quality_threshold` | 0.7 | Min quality for writeback |
-| `learning.ewc_lambda` | 2000 | EWC regularization strength |
-| `sona.pattern_clusters` | 100 | K-means++ clusters |
-| `sona.micro_lora_rank` | 2 | MicroLoRA rank |
-
-### Federated Learning Configuration
-
-| Option | Default | Description |
-|--------|---------|-------------|
-| `federated.quality_threshold` | 0.4 | Min quality for trajectory acceptance |
-| `federated.consolidation_interval` | 50 | Auto-consolidate every N agents |
-| `federated.coordinator_capacity` | 50000 | Trajectory buffer size for coordinator |
-| `federated.agent_capacity` | 500 | Trajectory buffer size per agent |
-| `federated.base_lora_rank` | 16 | Coordinator LoRA rank (deeper for aggregation) |
-
----
-
-## Self-Learning Improvement Over Time
-
-| Epoch | Queries | Quality | Routing | Cache Hit | Memory | Improvement |
-|-------|---------|---------|---------|-----------|--------|-------------|
-| 0 | 0 | 65.0% | 50.0% | 0.0% | 0 | 0.0% (baseline) |
-| 1 | 50 | 67.2% | 58.0% | 10.0% | 25 | +3.4% |
-| 2 | 100 | 69.8% | 66.0% | 20.0% | 50 | +7.4% |
-| 3 | 150 | 71.5% | 74.0% | 30.0% | 75 | +10.0% |
-| 4 | 200 | 73.2% | 82.0% | 40.0% | 100 | +12.6% |
-| 5 | 250 | 74.8% | 90.0% | 50.0% | 125 | +15.1% |
-
----
-
-## References
-
-- [TinyRecursiveModels](https://github.com/SamsungSAILMontreal/TinyRecursiveModels) - Samsung SAIL Montreal's recursive reasoning approach
-- [LFM2: Liquid Foundation Models](https://arxiv.org/abs/2511.23404v1) - Gated convolutions + grouped query attention
-- [FastGRNN](https://arxiv.org/abs/1901.02358) - Fast, Accurate, Stable and Tiny GRU
-- [HNSW](https://arxiv.org/abs/1603.09320) - Hierarchical Navigable Small World Graphs
-- [EWC](https://arxiv.org/abs/1612.00796) - Elastic Weight Consolidation
-- [LoRA](https://arxiv.org/abs/2106.09685) - Low-Rank Adaptation of Large Language Models
-
----
+- **RuvLTRA-Small** (494M) - Q4K, Q5K, Q8
+- **RuvLTRA-Medium** (3B) - Q4K, Q5K, Q8
+- **Qwen 2.5** (0.5B-72B)
+- **Llama 3.x** (8B-70B)
+- **Mistral** (7B-22B)
+- **Phi-3** (3.8B-14B)
+- **Gemma-2** (2B-27B)
 
-## License
+## Platform Support
 
-Licensed under either of:
+| Platform | Architecture | Status |
+|----------|--------------|--------|
+| macOS | arm64 (M1-M4) | Full support |
+| macOS | x64 | Supported |
+| Linux | x64 | Supported |
+| Linux | arm64 | Supported |
+| Windows | x64 | Supported |
 
-- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
-- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
+## Related Packages
 
-at your option.
+- [@ruvector/core](https://www.npmjs.com/package/@ruvector/core) - Vector operations
+- [@ruvector/sona](https://www.npmjs.com/package/@ruvector/sona) - SONA learning engine
+- [@ruvector/ruvector](https://www.npmjs.com/package/@ruvector/ruvector) - Full Ruvector SDK
 
-## Contributing
+## Links
 
-Contributions are welcome! Please feel free to submit a Pull Request.
+- [GitHub Repository](https://github.com/ruvnet/ruvector)
+- [HuggingFace Models](https://huggingface.co/ruv/ruvltra)
+- [API Documentation](https://docs.rs/ruvllm)
+- [Crate (Rust)](https://crates.io/crates/ruvllm)
 
----
+## License
 
-<p align="center">
-  <b>Built with Rust + Ruvector</b><br>
-  <i>Self-Learning AI that gets smarter with every interaction</i>
-</p>
+MIT OR Apache-2.0