@ruvector/ruvllm 2.4.0 → 2.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +262 -298
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,6 +1,42 @@
1
- # @ruvector/ruvllm v2.4
1
+ <div align="center">
2
2
 
3
- Self-learning LLM orchestration with SONA adaptive learning, HNSW memory, RLM recursive retrieval, and SIMD inference for Node.js.
3
+ # @ruvector/ruvllm
4
+
5
+ ### The First Purpose-Built LLM Runtime for Claude Code Agent Orchestration
6
+
7
+ **100% Routing Accuracy | Sub-Millisecond Inference | Self-Learning**
8
+
9
+ [![npm](https://img.shields.io/npm/v/@ruvector/ruvllm)](https://www.npmjs.com/package/@ruvector/ruvllm)
10
+ [![Downloads](https://img.shields.io/npm/dm/@ruvector/ruvllm)](https://www.npmjs.com/package/@ruvector/ruvllm)
11
+ [![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue)](LICENSE)
12
+ [![Tests](https://img.shields.io/badge/tests-145%20passing-brightgreen)](./test)
13
+
14
+ [Quick Start](#quick-start) | [RLM](#rlm-recursive-language-model) | [Training](#training) | [Models](#models) | [API](#api-reference)
15
+
16
+ </div>
17
+
18
+ ---
19
+
20
+ ## What is @ruvector/ruvllm?
21
+
22
+ **@ruvector/ruvllm** is a TypeScript/JavaScript SDK for intelligent LLM orchestration, specifically designed for **Claude Code** and multi-agent systems. It provides:
23
+
24
+ - **RLM (Recursive Language Model)** - Break complex queries into sub-queries, synthesize coherent answers
25
+ - **100% Routing Accuracy** - Hybrid keyword + embedding strategy for perfect agent selection
26
+ - **SONA Self-Learning** - Model improves with every successful interaction
27
+ - **SIMD Acceleration** - AVX2/NEON optimized inference
28
+
29
+ ### Why @ruvector/ruvllm?
30
+
31
+ | Challenge | Traditional Approach | @ruvector/ruvllm Solution |
32
+ |-----------|---------------------|---------------------------|
33
+ | Agent selection | Manual or keyword-based | Semantic + keyword hybrid = **100%** |
34
+ | Complex queries | Single-shot RAG | Recursive decomposition + synthesis |
35
+ | Response latency | 2-5 seconds | **<1ms** cache, 50-200ms full |
36
+ | Learning | Static models | **Self-improving** (SONA) |
37
+ | Cost per route | $0.01+ (API call) | **$0** (local inference) |
38
+
39
+ ---
4
40
 
5
41
  ## Installation
6
42
 
@@ -11,43 +47,99 @@ npm install @ruvector/ruvllm
11
47
  ## Quick Start
12
48
 
13
49
  ```typescript
14
- import { RuvLLM, RuvLLMConfig } from '@ruvector/ruvllm';
15
-
16
- // Initialize with default configuration
17
- const llm = new RuvLLM();
50
+ import { RuvLLM, RlmController } from '@ruvector/ruvllm';
18
51
 
19
- // Or with custom configuration
52
+ // Simple LLM inference
20
53
  const llm = new RuvLLM({
21
- modelPath: './models/ruvltra-small-q4km.gguf',
54
+ modelPath: '~/.ruvllm/models/ruvltra-claude-code-0.5b-q4_k_m.gguf',
22
55
  sonaEnabled: true,
23
- flashAttention: true,
24
- maxTokens: 256,
25
56
  });
26
57
 
27
- // Generate text
28
58
  const response = await llm.query('Explain quantum computing');
29
59
  console.log(response.text);
30
60
 
31
- // Stream generation
32
- for await (const token of llm.stream('Write a haiku about Rust')) {
33
- process.stdout.write(token);
34
- }
61
+ // Recursive Language Model for complex queries
62
+ const rlm = new RlmController({ maxDepth: 5 });
63
+ const answer = await rlm.query('What are the causes AND solutions for slow API responses?');
64
+ // Automatically decomposes into sub-queries, retrieves context, synthesizes answer
35
65
  ```
36
66
 
37
- ## What's New in v2.4
67
+ ---
68
+
69
+ ## Core Features
70
+
71
+ ### 1. Claude Code Native Routing
72
+
73
+ Built **by** Claude Code, **for** Claude Code. Routes tasks to 60+ agent types:
74
+
75
+ ```typescript
76
+ import { RuvLLM } from '@ruvector/ruvllm';
38
77
 
39
- | Feature | Description |
40
- |---------|-------------|
41
- | **RLM (Recursive Language Model)** | Query decomposition with recursive retrieval and synthesis |
42
- | **100% Routing Accuracy** | Hybrid keyword-first strategy achieves 100% on Claude Code tasks |
43
- | **145 Tests Passing** | Comprehensive test coverage across all modules |
44
- | **Contrastive Fine-tuning** | LoRA-based training with 793 contrastive pairs |
45
- | **Training Scripts** | Generate routing datasets and fine-tune models |
46
- | **HuggingFace Models** | Pre-trained RuvLTRA models available |
78
+ const llm = new RuvLLM({ model: 'ruv/ruvltra' });
79
+
80
+ // Intelligent routing
81
+ const route = await llm.route('implement OAuth2 authentication');
82
+ console.log(route.agent); // 'security-architect'
83
+ console.log(route.confidence); // 0.98
84
+ console.log(route.tier); // 2 (Haiku-level complexity)
85
+
86
+ // Multi-agent teams for complex tasks
87
+ const team = await llm.routeComplex('build full-stack app with auth');
88
+ // Returns: [system-architect, backend-dev, coder, security-architect, tester]
89
+ ```
90
+
91
+ ### 2. 3-Tier Intelligent Routing
92
+
93
+ ```
94
+ ┌─────────────────────────────────────────────────────────┐
95
+ │ User Request │
96
+ └─────────────────────┬───────────────────────────────────┘
97
+ ↓
98
+ [RuvLTRA Routing]
99
+ ↓
100
+ ┌─────────────┼─────────────┐
101
+ ↓ ↓ ↓
102
+ ┌───────────┐ ┌───────────┐ ┌───────────┐
103
+ │ Tier 1 │ │ Tier 2 │ │ Tier 3 │
104
+ │ Booster │ │ Haiku │ │ Opus │
105
+ │ <1ms │ │ ~500ms │ │ 2-5s │
106
+ │ $0 │ │ $0.0002 │ │ $0.015 │
107
+ └───────────┘ └───────────┘ └───────────┘
108
+ ```
109
+
110
+ ### 3. Self-Learning (SONA)
111
+
112
+ Every successful interaction improves the model:
113
+
114
+ ```typescript
115
+ // First routing: Full inference
116
+ llm.route('implement OAuth2') → security-architect (97%)
117
+
118
+ // Later: Pattern hit in <25μs (learned from success)
119
+ llm.route('add OAuth2 flow') → security-architect (99%, cached pattern)
120
+ ```
121
+
122
+ ---
47
123
 
48
124
  ## RLM (Recursive Language Model)
49
125
 
50
- RLM provides recursive retrieval-augmented generation that breaks down complex queries into sub-queries and synthesizes answers from retrieved context.
126
+ RLM provides **recursive query decomposition** - unlike traditional RAG that retrieves once, RLM breaks complex questions into sub-queries and synthesizes coherent answers.
127
+
128
+ ### How It Works
129
+
130
+ ```
131
+ Query: "What are the causes AND solutions for slow API responses?"
132
+ ↓
133
+ [Decomposition]
134
+ / \
135
+ "Causes of slow API?" "Solutions for slow API?"
136
+ ↓ ↓
137
+ [Sub-answers] [Sub-answers]
138
+ \ /
139
+ [Synthesis]
140
+ ↓
141
+ Coherent combined answer with sources
142
+ ```
51
143
 
52
144
  ### Basic Usage
53
145
 
@@ -66,19 +158,15 @@ await rlm.addMemory('React is a library for building user interfaces.');
66
158
 
67
159
  // Query with recursive retrieval
68
160
  const answer = await rlm.query('What are causes and solutions for type errors in React?');
69
- console.log(answer.text);
70
- console.log('Sources:', answer.sources);
71
- console.log('Quality Score:', answer.qualityScore);
72
- console.log('Confidence:', answer.confidence);
161
+ console.log(answer.text); // Comprehensive synthesized answer
162
+ console.log(answer.sources); // Source attributions
163
+ console.log(answer.qualityScore); // 0.0-1.0
164
+ console.log(answer.confidence); // Routing confidence
73
165
  ```
74
166
 
75
167
  ### Streaming
76
168
 
77
169
  ```typescript
78
- import { RlmController } from '@ruvector/ruvllm';
79
-
80
- const rlm = new RlmController();
81
-
82
170
  for await (const event of rlm.queryStream('Explain machine learning')) {
83
171
  if (event.type === 'token') {
84
172
  process.stdout.write(event.text);
@@ -88,18 +176,16 @@ for await (const event of rlm.queryStream('Explain machine learning')) {
88
176
  }
89
177
  ```
90
178
 
91
- ### With Reflection
179
+ ### With Self-Reflection
92
180
 
93
181
  ```typescript
94
- import { RlmController } from '@ruvector/ruvllm';
95
-
96
182
  const rlm = new RlmController({
97
183
  enableReflection: true,
98
184
  maxReflectionIterations: 2,
99
185
  minQualityScore: 0.8,
100
186
  });
101
187
 
102
- // Answers will be iteratively refined until quality >= 0.8
188
+ // Answers are iteratively refined until quality >= 0.8
103
189
  const answer = await rlm.query('Complex multi-part technical question...');
104
190
  ```
105
191
 
@@ -119,58 +205,68 @@ interface RlmConfig {
119
205
  }
120
206
  ```
121
207
 
122
- ## Exports
208
+ ---
209
+
210
+ ## Unique Capabilities
211
+
212
+ ### 1. Memory-Augmented Routing
213
+
214
+ Every successful routing is stored in HNSW-indexed memory for instant recall:
123
215
 
124
216
  ```typescript
125
- import {
126
- // Core
127
- RuvLLM,
128
- RuvLLMConfig,
129
-
130
- // RLM - Recursive Language Model
131
- RlmController,
132
- RlmConfig,
133
- RlmAnswer,
134
- MemorySpan,
135
- StreamToken,
136
-
137
- // RLM Training
138
- RlmTrainer,
139
- RlmTrainingConfig,
140
- RlmTrainingExample,
141
- createRlmTrainer,
142
- DEFAULT_RLM_CONFIG,
143
- FAST_RLM_CONFIG,
144
- THOROUGH_RLM_CONFIG,
145
- ROUTING_FOCUSED_CONFIG,
217
+ // First time: Full inference (~50ms)
218
+ route("implement OAuth2") → security-architect (97% confidence)
146
219
 
147
- // SONA Learning
148
- SonaCoordinator,
149
- TrajectoryBuilder,
220
+ // Later: Memory hit (<25μs)
221
+ route("add OAuth2 flow") → security-architect (99% confidence, cached)
222
+ ```
150
223
 
151
- // Federated Learning
152
- EphemeralAgent,
153
- FederatedCoordinator,
224
+ ### 2. Confidence-Aware Escalation
154
225
 
155
- // LoRA Adapters
156
- LoraAdapter,
157
- LoraManager,
226
+ ```typescript
227
+ // Low confidence automatically escalates
228
+ Confidence > 0.9 → Use recommended agent
229
+ Confidence 0.7-0.9 → Use with human confirmation
230
+ Confidence < 0.7 → Escalate to higher tier
231
+ ```
158
232
 
159
- // Sessions
160
- SessionManager,
233
+ ### 3. Batch SIMD Operations
161
234
 
162
- // Contrastive Training
163
- ContrastiveTrainer,
235
+ ```typescript
236
+ import { simd } from '@ruvector/ruvllm/simd';
164
237
 
165
- // Benchmarks
166
- ModelComparisonBenchmark,
167
- RoutingBenchmark,
168
- EmbeddingBenchmark,
169
- } from '@ruvector/ruvllm';
238
+ // 4x faster vector operations with AVX2/NEON
239
+ const similarity = simd.batchCosineSimilarity(query, targets);
240
+ const attended = simd.flashAttention(q, k, v, scale);
170
241
  ```
171
242
 
243
+ ### 4. Zero-Copy Caching
244
+
245
+ Arc-based string interning for 100-1000x faster cache hits on large responses.
246
+
247
+ ---
248
+
172
249
  ## Performance
173
250
 
251
+ ### Benchmarks (M4 Pro)
252
+
253
+ | Operation | Latency | Throughput |
254
+ |-----------|---------|------------|
255
+ | Query decomposition | 340 ns | 2.9M/s |
256
+ | Cache lookup | 23.5 ns | 42.5M/s |
257
+ | Embedding (384d) | 293 ns | 3.4M/s |
258
+ | Memory search (10k) | 0.4 ms | 2.5K/s |
259
+ | End-to-end routing | <1 ms | 1K+/s |
260
+ | Full RLM query | 50-200 ms | 5-20/s |
261
+
262
+ ### Routing Accuracy
263
+
264
+ | Strategy | RuvLTRA | Qwen Base | OpenAI |
265
+ |----------|---------|-----------|--------|
266
+ | Embedding Only | 45% | 40% | 52% |
267
+ | Keyword Only | 78% | 78% | N/A |
268
+ | **Hybrid** | **100%** | 95% | N/A |
269
+
174
270
  ### Test Results
175
271
 
176
272
  ```
@@ -184,67 +280,57 @@ import {
184
280
  - Benchmarks: 26 tests
185
281
  ```
186
282
 
187
- ### Routing Accuracy (Claude Code Tasks)
283
+ ---
188
284
 
189
- | Strategy | RuvLTRA | Qwen Base |
190
- |----------|---------|-----------|
191
- | Embedding Only | 45% | 40% |
192
- | Keyword-First (Hybrid) | **100%** | 95% |
285
+ ## Models
193
286
 
194
- ### Inference Performance (M4 Pro)
287
+ ### HuggingFace Repository
195
288
 
196
- | Operation | Performance |
197
- |-----------|-------------|
198
- | Inference | 88-135 tok/s |
199
- | Flash Attention | 320us (seq=2048) |
200
- | HNSW Search | 17-62us |
201
- | SONA Adapt | <1ms |
202
- | RLM Query | 50-200ms |
289
+ **URL**: [https://huggingface.co/ruv/ruvltra](https://huggingface.co/ruv/ruvltra)
203
290
 
204
- ### SIMD Optimizations
291
+ ### Available Models
205
292
 
206
- - AVX2/AVX-512 on x86_64
207
- - NEON on ARM64
208
- - 4-8x speedup on vector operations
293
+ | Model | Size | Purpose | Accuracy |
294
+ |-------|------|---------|----------|
295
+ | **ruvltra-claude-code-0.5b-q4_k_m** | 398 MB | Agent routing | **100%** (hybrid) |
296
+ | ruvltra-small-0.5b-q4_k_m | ~400 MB | Embeddings | - |
297
+ | ruvltra-medium-1.1b-q4_k_m | ~1 GB | Full inference | - |
209
298
 
210
- ## CLI Usage
299
+ ### Download Models
211
300
 
212
- ```bash
213
- # Query a model
214
- ruvllm query "What is machine learning?"
301
+ ```typescript
302
+ // Programmatic
303
+ import { downloadModel } from '@ruvector/ruvllm';
304
+ await downloadModel('ruv/ruvltra', { quantization: 'q4_k_m' });
215
305
 
216
- # Stream output
217
- ruvllm query --stream "Write a poem"
306
+ // CLI
307
+ ruvllm download ruv/ruvltra
308
+ ```
218
309
 
219
- # Download a model
220
- ruvllm download ruvector/ruvltra-small-q4km
310
+ ### Auto-Download
221
311
 
222
- # Benchmark
223
- ruvllm bench ./models/model.gguf
312
+ Models are automatically downloaded on first use:
224
313
 
225
- # Run evaluation (SWE-Bench)
226
- ruvllm eval --model ./models/model.gguf --subset lite --max-tasks 50
314
+ ```typescript
315
+ const llm = new RuvLLM({ model: 'ruv/ruvltra' });
316
+ // Downloads to ~/.ruvllm/models/ if not present
227
317
  ```
228
318
 
229
- ## Training
319
+ ---
230
320
 
231
- ### Routing Dataset Generation
321
+ ## Training
232
322
 
233
- Generate training data for agent routing:
323
+ ### Generate Routing Dataset
234
324
 
235
325
  ```bash
236
- # Generate routing dataset
237
326
  node scripts/training/routing-dataset.js
238
-
239
- # Output: 381 examples, 793 contrastive pairs
327
+ # Output: 381 examples, 793 contrastive pairs, 156 hard negatives
240
328
  ```
241
329
 
242
330
  ### Contrastive Fine-tuning
243
331
 
244
- Fine-tune models with LoRA adapters:
245
-
246
332
  ```typescript
247
- import { ContrastiveTrainer, ContrastivePair } from '@ruvector/ruvllm';
333
+ import { ContrastiveTrainer } from '@ruvector/ruvllm';
248
334
 
249
335
  const trainer = new ContrastiveTrainer({
250
336
  modelPath: './models/base.gguf',
@@ -253,73 +339,25 @@ const trainer = new ContrastiveTrainer({
253
339
  learningRate: 1e-4,
254
340
  });
255
341
 
256
- // Training pairs from routing dataset
257
- const pairs: ContrastivePair[] = [
258
- {
259
- anchor: 'Fix the authentication bug in login.ts',
260
- positive: 'coder',
261
- negative: 'researcher',
262
- },
342
+ const pairs = [
343
+ { anchor: 'Fix auth bug', positive: 'coder', negative: 'researcher' },
263
344
  // ... more pairs
264
345
  ];
265
346
 
266
- await trainer.train(pairs, { epochs: 10, batchSize: 32 });
347
+ await trainer.train(pairs, { epochs: 10 });
267
348
  await trainer.save('./adapters/routing-lora');
268
349
  ```
269
350
 
270
351
  ### Training Scripts
271
352
 
272
- Located in `scripts/training/`:
273
-
274
353
  | Script | Description |
275
354
  |--------|-------------|
276
355
  | `routing-dataset.js` | Generate 381 routing examples |
277
356
  | `claude-code-synth.js` | Synthetic data generation |
278
357
  | `contrastive-finetune.js` | LoRA fine-tuning pipeline |
358
+ | `rlm-dataset.js` | RLM training data (500 examples) |
279
359
 
280
- ## Model Links
281
-
282
- ### HuggingFace Repository
283
-
284
- **URL**: [https://huggingface.co/ruv/ruvltra](https://huggingface.co/ruv/ruvltra)
285
-
286
- ### Available Models
287
-
288
- | Model | File | Size | Purpose |
289
- |-------|------|------|---------|
290
- | RuvLTRA Claude Code 0.5B | `ruvltra-claude-code-0.5b-q4_k_m.gguf` | ~400MB | Agent routing (100% with hybrid) |
291
- | RuvLTRA Small 0.5B | `ruvltra-0.5b-q4_k_m.gguf` | ~400MB | General embeddings |
292
- | RuvLTRA Medium 3B | `ruvltra-3b-q4_k_m.gguf` | ~2GB | Full LLM inference |
293
-
294
- ### Download Models
295
-
296
- ```bash
297
- # Using CLI
298
- ruvllm download ruv/ruvltra
299
-
300
- # Using HuggingFace CLI
301
- huggingface-cli download ruv/ruvltra ruvltra-claude-code-0.5b-q4_k_m.gguf
302
-
303
- # Programmatic download
304
- import { downloadModel } from '@ruvector/ruvllm';
305
- await downloadModel('ruv/ruvltra', { quantization: 'q4_k_m' });
306
- ```
307
-
308
- ### Environment Variables
309
-
310
- ```bash
311
- # HuggingFace authentication (any of these)
312
- HF_TOKEN=hf_xxx
313
- HUGGING_FACE_HUB_TOKEN=hf_xxx
314
- HUGGINGFACE_API_KEY=hf_xxx
315
- ```
316
-
317
- ### Local Model Storage
318
-
319
- ```bash
320
- ~/.ruvllm/models/ # Downloaded GGUF models
321
- ~/.ruvllm/training/ # Training data and configs
322
- ```
360
+ ---
323
361
 
324
362
  ## API Reference
325
363
 
@@ -329,23 +367,16 @@ HUGGINGFACE_API_KEY=hf_xxx
329
367
  class RuvLLM {
330
368
  constructor(config?: RuvLLMConfig);
331
369
 
332
- // Generate text
333
370
  query(prompt: string, params?: GenerateParams): Promise<Response>;
334
-
335
- // Stream generation
336
371
  stream(prompt: string, params?: GenerateParams): AsyncIterable<string>;
372
+ route(task: string): Promise<RoutingResult>;
373
+ routeComplex(task: string): Promise<AgentTeam[]>;
337
374
 
338
- // Load a model
339
375
  loadModel(path: string): Promise<void>;
340
-
341
- // Memory operations
342
- addMemory(text: string, metadata?: Record<string, unknown>): number;
376
+ addMemory(text: string, metadata?: object): number;
343
377
  searchMemory(query: string, topK?: number): MemoryResult[];
344
378
 
345
- // Get SONA learning stats
346
379
  sonaStats(): SonaStats | null;
347
-
348
- // Adapt on feedback
349
380
  adapt(input: Float32Array, quality: number): void;
350
381
  }
351
382
  ```
@@ -356,147 +387,68 @@ class RuvLLM {
356
387
  class RlmController {
357
388
  constructor(config?: RlmConfig, engine?: RuvLLM);
358
389
 
359
- // Query with recursive retrieval
360
390
  query(input: string): Promise<RlmAnswer>;
361
-
362
- // Stream query
363
391
  queryStream(input: string): AsyncGenerator<StreamToken>;
364
392
 
365
- // Memory management
366
- addMemory(text: string, metadata?: Record<string, unknown>): Promise<string>;
393
+ addMemory(text: string, metadata?: object): Promise<string>;
367
394
  searchMemory(query: string, topK?: number): Promise<MemorySpan[]>;
368
395
 
369
- // Cache management
370
396
  clearCache(): void;
371
397
  getCacheStats(): { size: number; entries: number };
372
398
 
373
- // Configuration
374
399
  updateConfig(config: Partial<RlmConfig>): void;
375
400
  getConfig(): Required<RlmConfig>;
376
401
  }
377
402
  ```
378
403
 
379
- ### Configuration
404
+ ### All Exports
380
405
 
381
406
  ```typescript
382
- interface RuvLLMConfig {
383
- modelPath?: string; // Path to GGUF model
384
- sonaEnabled?: boolean; // Enable SONA learning (default: true)
385
- flashAttention?: boolean; // Use Flash Attention 2 (default: true)
386
- maxTokens?: number; // Max generation tokens (default: 256)
387
- temperature?: number; // Sampling temperature (default: 0.7)
388
- topP?: number; // Top-p sampling (default: 0.9)
389
- }
390
- ```
391
-
392
- ### Generate Parameters
393
-
394
- ```typescript
395
- interface GenerateParams {
396
- maxTokens?: number;
397
- temperature?: number;
398
- topP?: number;
399
- topK?: number;
400
- repetitionPenalty?: number;
401
- stopSequences?: string[];
402
- }
403
- ```
404
-
405
- ## SIMD Module
406
-
407
- For direct access to optimized SIMD kernels:
407
+ import {
408
+ // Core
409
+ RuvLLM, RuvLLMConfig,
408
410
 
409
- ```typescript
410
- import { simd } from '@ruvector/ruvllm/simd';
411
+ // RLM
412
+ RlmController, RlmConfig, RlmAnswer, MemorySpan, StreamToken,
411
413
 
412
- // Dot product
413
- const result = simd.dotProduct(vecA, vecB);
414
+ // Training
415
+ RlmTrainer, ContrastiveTrainer, createRlmTrainer,
416
+ DEFAULT_RLM_CONFIG, FAST_RLM_CONFIG, THOROUGH_RLM_CONFIG,
414
417
 
415
- // Matrix multiplication
416
- const output = simd.matmul(matrix, vector);
418
+ // SONA Learning
419
+ SonaCoordinator, TrajectoryBuilder,
417
420
 
418
- // Flash Attention
419
- const attended = simd.flashAttention(query, key, value, scale);
421
+ // LoRA
422
+ LoraAdapter, LoraManager,
420
423
 
421
- // RMS Normalization
422
- simd.rmsNorm(hidden, weights, epsilon);
424
+ // Benchmarks
425
+ ModelComparisonBenchmark, RoutingBenchmark, EmbeddingBenchmark,
426
+ } from '@ruvector/ruvllm';
423
427
  ```
424
428
 
425
- ## Evaluation Harness
426
-
427
- Run model evaluations with SWE-Bench integration:
428
-
429
- ```typescript
430
- import { RuvLLM, EvaluationHarness, AblationMode } from '@ruvector/ruvllm';
431
-
432
- const harness = new EvaluationHarness({
433
- modelPath: './models/model.gguf',
434
- enableHnsw: true,
435
- enableSona: true,
436
- });
437
-
438
- // Run single evaluation
439
- const result = await harness.evaluate(
440
- 'Fix the null pointer exception',
441
- 'def process(data): return data.split()',
442
- AblationMode.Full
443
- );
429
+ ---
444
430
 
445
- console.log(`Success: ${result.success}, Quality: ${result.qualityScore}`);
431
+ ## CLI
446
432
 
447
- // Run ablation study (Baseline, RetrievalOnly, AdaptersOnly, R+A, Full)
448
- const report = await harness.runAblationStudy(tasks);
449
- for (const [mode, metrics] of Object.entries(report.modeMetrics)) {
450
- console.log(`${mode}: ${metrics.successRate * 100}% success`);
451
- }
452
- ```
453
-
454
- ## mistral-rs Backend (Production Serving)
433
+ ```bash
434
+ # Route a task
435
+ ruvllm route "add unit tests for auth module"
436
+ # → Agent: tester | Confidence: 0.96 | Tier: 2
455
437
 
456
- For production deployments with 10-100+ concurrent users, use the mistral-rs backend:
438
+ # Query with streaming
439
+ ruvllm query --stream "Explain machine learning"
457
440
 
458
- ```typescript
459
- import { RuvLLM, MistralBackend, PagedAttentionConfig } from '@ruvector/ruvllm';
460
-
461
- // Configure for production serving
462
- const backend = new MistralBackend({
463
- // PagedAttention: 5-10x more concurrent users
464
- pagedAttention: {
465
- blockSize: 16,
466
- maxBlocks: 4096,
467
- gpuMemoryFraction: 0.9,
468
- prefixCaching: true,
469
- },
470
- // X-LoRA: Per-token adapter routing
471
- xlora: {
472
- adapters: ['./adapters/coder', './adapters/researcher'],
473
- topK: 2,
474
- },
475
- // ISQ: Runtime quantization
476
- isq: {
477
- bits: 4,
478
- method: 'awq',
479
- },
480
- });
441
+ # Download models
442
+ ruvllm download ruv/ruvltra
481
443
 
482
- const llm = new RuvLLM({ backend });
483
- await llm.loadModel('mistralai/Mistral-7B-Instruct-v0.2');
444
+ # Run benchmarks
445
+ ruvllm bench ./models/model.gguf
484
446
 
485
- // Serve multiple concurrent requests
486
- const response = await llm.query('Write production code');
447
+ # Evaluate (SWE-Bench)
448
+ ruvllm eval --model ./models/model.gguf --subset lite
487
449
  ```
488
450
 
489
- > **Note**: mistral-rs features require the Rust backend with `mistral-rs` feature enabled. Native bindings will use mistral-rs when available.
490
-
491
- ## Supported Models
492
-
493
- - **RuvLTRA-Small** (494M) - Q4K, Q5K, Q8
494
- - **RuvLTRA-Medium** (3B) - Q4K, Q5K, Q8
495
- - **Qwen 2.5** (0.5B-72B)
496
- - **Llama 3.x** (8B-70B)
497
- - **Mistral** (7B-22B)
498
- - **Phi-3** (3.8B-14B)
499
- - **Gemma-2** (2B-27B)
451
+ ---
500
452
 
501
453
  ## Platform Support
502
454
 
@@ -508,19 +460,31 @@ const response = await llm.query('Write production code');
508
460
  | Linux | arm64 | Supported |
509
461
  | Windows | x64 | Supported |
510
462
 
511
- ## Related Packages
512
-
513
- - [@ruvector/core](https://www.npmjs.com/package/@ruvector/core) - Vector operations
514
- - [@ruvector/sona](https://www.npmjs.com/package/@ruvector/sona) - SONA learning engine
515
- - [@ruvector/ruvector](https://www.npmjs.com/package/@ruvector/ruvector) - Full Ruvector SDK
463
+ ---
516
464
 
517
465
  ## Links
518
466
 
519
- - [GitHub Repository](https://github.com/ruvnet/ruvector)
520
- - [HuggingFace Models](https://huggingface.co/ruv/ruvltra)
521
- - [API Documentation](https://docs.rs/ruvllm)
522
- - [Crate (Rust)](https://crates.io/crates/ruvllm)
467
+ | Resource | URL |
468
+ |----------|-----|
469
+ | **npm** | [npmjs.com/package/@ruvector/ruvllm](https://www.npmjs.com/package/@ruvector/ruvllm) |
470
+ | **HuggingFace** | [huggingface.co/ruv/ruvltra](https://huggingface.co/ruv/ruvltra) |
471
+ | **Crate (Rust)** | [crates.io/crates/ruvllm](https://crates.io/crates/ruvllm) |
472
+ | **Documentation** | [docs.rs/ruvllm](https://docs.rs/ruvllm) |
473
+ | **GitHub** | [github.com/ruvnet/ruvector](https://github.com/ruvnet/ruvector) |
474
+ | **Claude Flow** | [github.com/ruvnet/claude-flow](https://github.com/ruvnet/claude-flow) |
475
+
476
+ ---
523
477
 
524
478
  ## License
525
479
 
526
480
  MIT OR Apache-2.0
481
+
482
+ ---
483
+
484
+ <div align="center">
485
+
486
+ **Built for Claude Code. Optimized for agents. Designed for speed.**
487
+
488
+ [Get Started](#quick-start) | [View on GitHub](https://github.com/ruvnet/ruvector)
489
+
490
+ </div>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ruvector/ruvllm",
3
- "version": "2.4.0",
3
+ "version": "2.4.1",
4
4
  "description": "Self-learning LLM orchestration with SONA adaptive learning, HNSW memory, RLM recursive retrieval, FastGRNN routing, and SIMD inference",
5
5
  "main": "dist/cjs/index.js",
6
6
  "module": "dist/esm/index.js",