llmjs2 1.3.9 → 1.6.1

package/{docs → core/docs}/README.md

@@ -1,47 +1,47 @@
-# llmjs2 Documentation
-
-Welcome to the llmjs2 documentation! This folder contains all the documentation for using and understanding the llmjs2 library.
-
-## Documentation Overview
-
-### 📖 Getting Started
-
-- **[GET_STARTED.md](GET_STARTED.md)** - Quick setup guide for new users (5 minutes to first completion)
-
-### 🔧 Usage Guides
-
-- **[BASIC_USAGE.md](BASIC_USAGE.md)** - Core API patterns, configuration, and common use cases
-- **[ROUTER_GUIDE.md](ROUTER_GUIDE.md)** - Model routing and load balancing
-- **[GUARDRAILS_GUIDE.md](GUARDRAILS_GUIDE.md)** - Content filtering and request processing
-- **[SERVER_MODE.md](SERVER_MODE.md)** - Run llmjs2 as an OpenAI-compatible API server with routing
-- **[CLI.md](CLI.md)** - Command-line interface for server management
-
-## Quick Navigation
-
-### New to llmjs2?
-
-Start with **[GET_STARTED.md](GET_STARTED.md)** to get up and running quickly.
-
-### Want to use the API directly?
-
-Check out **[BASIC_USAGE.md](BASIC_USAGE.md)** for different API patterns and examples.
-
-### Need routing and load balancing?
-
-See **[ROUTER_GUIDE.md](ROUTER_GUIDE.md)** for intelligent model routing.
-
-### Need content filtering or custom processing?
-
-See **[GUARDRAILS_GUIDE.md](GUARDRAILS_GUIDE.md)** for guardrails and request processing.
-
-### Need to set up a server?
-
-See **[SERVER_MODE.md](SERVER_MODE.md)** for OpenAI-compatible server setup.
-
-### Prefer command-line tools?
-
-**[CLI.md](CLI.md)** covers the command-line interface and configuration files.
-
-## Contributing
-
-Documentation improvements are welcome! Please ensure any changes maintain consistency across all documentation files.
+# llmjs2 Documentation
+
+Welcome to the llmjs2 documentation! This folder contains all the documentation for using and understanding the llmjs2 library.
+
+## Documentation Overview
+
+### 📖 Getting Started
+
+- **[GET_STARTED.md](GET_STARTED.md)** - Quick setup guide for new users (5 minutes to first completion)
+
+### 🔧 Usage Guides
+
+- **[BASIC_USAGE.md](BASIC_USAGE.md)** - Core API patterns, configuration, and common use cases
+- **[ROUTER_GUIDE.md](ROUTER_GUIDE.md)** - Model routing and load balancing
+- **[GUARDRAILS_GUIDE.md](GUARDRAILS_GUIDE.md)** - Content filtering and request processing
+- **[SERVER_MODE.md](SERVER_MODE.md)** - Run llmjs2 as an OpenAI-compatible API server with routing
+- **[CLI.md](CLI.md)** - Command-line interface for server management
+
+## Quick Navigation
+
+### New to llmjs2?
+
+Start with **[GET_STARTED.md](GET_STARTED.md)** to get up and running quickly.
+
+### Want to use the API directly?
+
+Check out **[BASIC_USAGE.md](BASIC_USAGE.md)** for different API patterns and examples.
+
+### Need routing and load balancing?
+
+See **[ROUTER_GUIDE.md](ROUTER_GUIDE.md)** for intelligent model routing.
+
+### Need content filtering or custom processing?
+
+See **[GUARDRAILS_GUIDE.md](GUARDRAILS_GUIDE.md)** for guardrails and request processing.
+
+### Need to set up a server?
+
+See **[SERVER_MODE.md](SERVER_MODE.md)** for OpenAI-compatible server setup.
+
+### Prefer command-line tools?
+
+**[CLI.md](CLI.md)** covers the command-line interface and configuration files.
+
+## Contributing
+
+Documentation improvements are welcome! Please ensure any changes maintain consistency across all documentation files.

package/core/docs/ROUTER_GUIDE.md

@@ -0,0 +1,199 @@
+# llmjs2 Router Usage Guide
+
+The llmjs2 router provides intelligent model routing and load balancing capabilities, allowing you to distribute requests across multiple model deployments with different strategies.
+
+## Overview
+
+The router system enables:
+
+- **Load balancing** across models with the same name
+- **Multiple routing strategies** (default, random, sequential)
+- **Provider-agnostic routing** with unified API
+- **Flexible model configuration** for different providers
+- 
+
+## Quick Start
+
+### Basic Setup
+
+```javascript
+import { router } from 'llmjs2';
+
+// Define your model deployments
+const modelList = [
+  {
+    "model_name": "gpt-3.5-turbo",
+    "llm_params": {
+      "model": "ollama/chatgpt-v-2",
+      "api_key": process.env.OLLAMA_API_KEY,
+      "api_base": process.env.OLLAMA_API_BASE
+    }
+  },
+  {
+    "model_name": "openai-turbo",
+    "llm_params": {
+      "model": "gpt-3.5-turbo",
+      "api_key": process.env.OPENAI_API_KEY
+    }
+  },
+  {
+    "model_name": "gpt-4",
+    "llm_params": {
+      "model": "ollama/gpt-4",
+      "api_key": process.env.OLLAMA_API_KEY,
+      "api_base": process.env.OLLAMA_API_BASE
+    }
+  }
+];
+
+// Create routers with different strategies
+const defaultRouter = router(modelList);
+const randomRouter = router(modelList, 'random');
+const sequentialRouter = router(modelList, 'sequential');
+```
+
+### Basic Usage
+
+```javascript
+// Route to specific model
+const response = await defaultRouter.completion({
+  model: "gpt-3.5-turbo",
+  messages: [{"role": "user", "content": "Hey, how's it going?"}]
+});
+
+// Auto-route with random strategy
+const randomResponse = await randomRouter.completion({
+  messages: [{"role": "user", "content": "Hey, how's it going?"}]
+});
+
+// Auto-route with sequential strategy
+const seqResponse = await sequentialRouter.completion({
+  messages: [{"role": "user", "content": "Hey, how's it going?"}]
+});
+```
+
+## Model Configuration
+
+### Model List Format
+
+Each model in the list is defined with:
+
+```javascript
+{
+  "model_name": "string",     // Alias for routing (can have multiple providers)
+  "llm_params": {             // Provider-specific parameters
+    "model": "string",        // Actual model identifier for the provider
+    "api_key": "string",      // API key (can use environment variables)
+    "api_base": "string?",    // Custom API base URL (optional)
+    // ... other provider-specific params
+  }
+}
+```
+
+### Supported Providers
+
+#### Ollama
+
+```javascript
+{
+  "model_name": "my-ollama-model",
+  "llm_params": {
+    "model": "ollama/llama2",
+    "api_key": process.env.OLLAMA_API_KEY,
+    "api_base": process.env.OLLAMA_API_BASE
+  }
+}
+```
+
+#### OpenRouter
+
+```javascript
+{
+  "model_name": "my-openrouter-model",
+  "llm_params": {
+    "model": "openrouter/free-model",
+    "api_key": process.env.OPEN_ROUTER_API_KEY
+  }
+}
+```
+
+#### OpenAI
+
+```javascript
+{
+  "model_name": "my-openai-model",
+  "llm_params": {
+    "model": "openai/gpt-4",
+    "api_key": process.env.OPENAI_API_KEY
+  }
+}
+```
+
+## Routing Strategies
+
+### Default Strategy
+
+When no strategy is specified, uses sequential selection across all available models for auto-routing (when no specific model is requested).
+
+```javascript
+const route = router(modelList); // or router(modelList, 'default')
+
+// Auto-route with sequential selection (cycles through all models)
+const response = await route.completion({
+  messages: [...]
+});
+
+// Routes to one of the models with model_name="gpt-3.5-turbo" (load balancing)
+const response = await route.completion({
+  model: "gpt-3.5-turbo",
+  messages: [...]
+});
+```
+
+### Random Strategy
+
+Randomly selects from available models when no specific model is requested.
+
+```javascript
+const route = router(modelList, 'random');
+
+// Randomly selects from ALL models in the list
+const response = await route.completion({
+  messages: [...]
+});
+```
+
+### Sequential Strategy
+
+Cycles through models in order for each request.
+
+```javascript
+const route = router(modelList, 'sequential');
+
+// Uses first model, then second, then third, etc.
+const response1 = await route.completion({ messages: [...] }); // model 1
+const response2 = await route.completion({ messages: [...] }); // model 2
+const response3 = await route.completion({ messages: [...] }); // model 3
+// ... cycles back to model 1
+```
+
+## Error Handling
+
+```javascript
+try {
+  const response = await route.completion({
+    model: "non-existent-model",
+    messages: [{"role": "user", "content": "Hello"}]
+  });
+} catch (error) {
+  if (error.message.includes('Model not found')) {
+    console.log('Model not configured in router');
+  } else if (error.message.includes('API key')) {
+    console.log('Provider API key missing');
+  } else {
+    console.log('Routing error:', error.message);
+  }
+}
+```
+
+##