llmjs2 1.3.8 → 1.6.1

package/docs/ROUTER_GUIDE.md

@@ -1,402 +0,0 @@
-# llmjs2 Router Usage Guide
-
-The llmjs2 router provides intelligent model routing and load balancing capabilities, allowing you to distribute requests across multiple model deployments with different strategies.
-
-## Overview
-
-The router system enables:
-
-- **Load balancing** across models with the same name
-- **Multiple routing strategies** (default, random, sequential)
-- **Provider-agnostic routing** with unified API
-- **Flexible model configuration** for different providers
-- 
-
-## Quick Start
-
-### Basic Setup
-
-```javascript
-import { router } from 'llmjs2';
-
-// Define your model deployments
-const modelList = [
-  {
-    "model_name": "gpt-3.5-turbo",
-    "llm_params": {
-      "model": "ollama/chatgpt-v-2",
-      "api_key": process.env.OLLAMA_API_KEY,
-      "api_base": process.env.OLLAMA_API_BASE
-    }
-  },
-  {
-    "model_name": "openai-turbo",
-    "llm_params": {
-      "model": "gpt-3.5-turbo",
-      "api_key": process.env.OPENAI_API_KEY
-    }
-  },
-  {
-    "model_name": "gpt-4",
-    "llm_params": {
-      "model": "ollama/gpt-4",
-      "api_key": process.env.OLLAMA_API_KEY,
-      "api_base": process.env.OLLAMA_API_BASE
-    }
-  }
-];
-
-// Create routers with different strategies
-const defaultRouter = router(modelList);
-const randomRouter = router(modelList, 'random');
-const sequentialRouter = router(modelList, 'sequential');
-```
-
-### Basic Usage
-
-```javascript
-// Route to specific model
-const response = await defaultRouter.completion({
-  model: "gpt-3.5-turbo",
-  messages: [{"role": "user", "content": "Hey, how's it going?"}]
-});
-
-// Auto-route with random strategy
-const randomResponse = await randomRouter.completion({
-  messages: [{"role": "user", "content": "Hey, how's it going?"}]
-});
-
-// Auto-route with sequential strategy
-const seqResponse = await sequentialRouter.completion({
-  messages: [{"role": "user", "content": "Hey, how's it going?"}]
-});
-```
-
-## Model Configuration
-
-### Model List Format
-
-Each model in the list is defined with:
-
-```javascript
-{
-  "model_name": "string",     // Alias for routing (can have multiple providers)
-  "llm_params": {             // Provider-specific parameters
-    "model": "string",        // Actual model identifier for the provider
-    "api_key": "string",      // API key (can use environment variables)
-    "api_base": "string?",    // Custom API base URL (optional)
-    // ... other provider-specific params
-  }
-}
-```
-
-### Supported Providers
-
-#### Ollama
-
-```javascript
-{
-  "model_name": "my-ollama-model",
-  "llm_params": {
-    "model": "ollama/llama2",
-    "api_key": process.env.OLLAMA_API_KEY,
-    "api_base": process.env.OLLAMA_API_BASE
-  }
-}
-```
-
-#### OpenRouter
-
-```javascript
-{
-  "model_name": "my-openrouter-model",
-  "llm_params": {
-    "model": "openrouter/free-model",
-    "api_key": process.env.OPEN_ROUTER_API_KEY
-  }
-}
-```
-
-#### OpenAI
-
-```javascript
-{
-  "model_name": "my-openai-model",
-  "llm_params": {
-    "model": "openai/gpt-4",
-    "api_key": process.env.OPENAI_API_KEY
-  }
-}
-```
-
-## Routing Strategies
-
-### Default Strategy
-
-When no strategy is specified, uses sequential selection across all available models for auto-routing (when no specific model is requested).
-
-```javascript
-const route = router(modelList); // or router(modelList, 'default')
-
-// Auto-route with sequential selection (cycles through all models)
-const response = await route.completion({
-  messages: [...]
-});
-
-// Routes to one of the models with model_name="gpt-3.5-turbo" (load balancing)
-const response = await route.completion({
-  model: "gpt-3.5-turbo",
-  messages: [...]
-});
-```
-
-### Random Strategy
-
-Randomly selects from available models when no specific model is requested.
-
-```javascript
-const route = router(modelList, 'random');
-
-// Randomly selects from ALL models in the list
-const response = await route.completion({
-  messages: [...]
-});
-```
-
-### Sequential Strategy
-
-Cycles through models in order for each request.
-
-```javascript
-const route = router(modelList, 'sequential');
-
-// Uses first model, then second, then third, etc.
-const response1 = await route.completion({ messages: [...] }); // model 1
-const response2 = await route.completion({ messages: [...] }); // model 2
-const response3 = await route.completion({ messages: [...] }); // model 3
-// ... cycles back to model 1
-```
-
-## Advanced Usage
-
-### Load Balancing
-
-Multiple models with the same `model_name` enable load balancing:
-
-```javascript
-const modelList = [
-  {
-    "model_name": "gpt-3.5-turbo",
-    "llm_params": {
-      "model": "ollama/chatgpt-v-2",
-      "api_key": process.env.OLLAMA_API_KEY
-    }
-  },
-  {
-    "model_name": "gpt-3.5-turbo",  // Same name - load balancing
-    "llm_params": {
-      "model": "openai/gpt-3.5-turbo",
-      "api_key": process.env.OPENAI_API_KEY
-    }
-  },
-  {
-    "model_name": "gpt-3.5-turbo",  // Same name - load balancing
-    "llm_params": {
-      "model": "openrouter/minimax2.5",
-      "api_key": process.env.OPEN_ROUTER_API_KEY,
-      "api_base": process.env.OPEN_ROUTER_BASE
-    }
-  }
-];
-
-const route = router(modelList);
-
-// This will load balance across all 3 "gpt-3.5-turbo" models
-const response = await route.completion({
-  model: "gpt-3.5-turbo",
-  messages: [...]
-});
-```
-
-### Environment Variables
-
-Use environment variables for configuration:
-
-```javascript
-const modelList = [
-  {
-    "model_name": "production-gpt4",
-    "llm_params": {
-      "model": "openai/gpt-4",
-      "api_key": process.env.OPENAI_API_KEY,
-      "api_base": process.env.OPENAI_API_BASE || "https://api.openai.com/v1"
-    }
-  },
-  {
-    "model_name": "staging-gpt4",
-    "llm_params": {
-      "model": "ollama/gpt-4",
-      "api_key": process.env.OLLAMA_API_KEY,
-      "api_base": process.env.OLLAMA_API_BASE
-    }
-  }
-];
-```
-
-### Complete API Reference
-
-```javascript
-import { router } from 'llmjs2';
-
-// Create router
-const myRouter = router(modelList, strategy);
-
-// Completion with specific model
-const response1 = await myRouter.completion({
-  model: "model_name",           // Optional: specific model to route to
-  messages: [...],               // Required: chat messages
-  tools: [...],                  // Optional: function calling tools
-  // ... other completion params
-});
-
-// Auto-routing completion
-const response2 = await myRouter.completion({
-  messages: [...],               // Required: chat messages
-  // Uses routing strategy when no model specified
-});
-```
-
-## Error Handling
-
-```javascript
-try {
-  const response = await route.completion({
-    model: "non-existent-model",
-    messages: [{"role": "user", "content": "Hello"}]
-  });
-} catch (error) {
-  if (error.message.includes('Model not found')) {
-    console.log('Model not configured in router');
-  } else if (error.message.includes('API key')) {
-    console.log('Provider API key missing');
-  } else {
-    console.log('Routing error:', error.message);
-  }
-}
-```
-
-## Use Cases
-
-### Multi-Provider Fallback
-
-```javascript
-const fallbackModels = [
-  {
-    "model_name": "gpt-4_1",
-    "llm_params": { "model": "openai/gpt-4", "api_key": process.env.OPENAI_API_KEY }
-  },
-  {
-    "model_name": "gpt-4_2",
-    "llm_params": { "model": "ollama/gpt-4", "api_key": process.env.OLLAMA_API_KEY }
-  },
-  {
-    "model_name": "gpt-4_3",
-    "llm_params": { "model": "openrouter/gpt-4", "api_key": process.env.OPEN_ROUTER_API_KEY }
-  }
-];
-
-const route = router(fallbackModels);
-
-// Automatically tries different providers if one fails
-const response = await route.completion({
-  messages: [...]
-});
-```
-
-### Cost Optimization
-
-```javascript
-const costOptimizedModels = [
-  {
-    "model_name": "text-davinci-001",
-    "llm_params": { "model": "ollama/text-davinci-003", "api_key": process.env.OLLAMA_API_KEY }
-  },
-  {
-    "model_name": "text-davinci-002",
-    "llm_params": { "model": "openrouter/text-davinci-003", "api_key": process.env.OPENROUTER_API_KEY }
-  },
-  {
-    "model_name": "text-davinci-003",
-    "llm_params": { "model": "openai/gpt-3.5-turbo", "api_key": process.env.OPENAI_API_KEY }
-  }
-];
-
-const route = router(costOptimizedModels, 'random');
-
-// Load balances across cheaper providers
-const response = await route.completion({
-  model: "text-davinci-003",
-  messages: [...]
-});
-```
-
-### A/B Testing
-
-```javascript
-const abTestModels = [
-  {
-    "model_name": "experiment-a",
-    "llm_params": { "model": "gpt-4", "api_key": process.env.OPENAI_API_KEY }
-  },
-  {
-    "model_name": "experiment-b",
-    "llm_params": { "model": "ollama/gpt-4", "api_key": process.env.OLLAMA_API_KEY }
-  }
-];
-
-const route = router(abTestModels, 'random');
-
-// Randomly routes between experiment variants
-const response = await route.completion({
-  model: "experiment-a",  // or "experiment-b"
-  messages: [...]
-});
-```
-
-## Configuration Examples
-
-### Production Setup
-
-```javascript
-const productionModels = [
-  // Primary OpenAI models
-  { "model_name": "gpt-4", "llm_params": { "model": "gpt-4", "api_key": process.env.OPENAI_API_KEY } },
-  { "model_name": "gpt-3.5-turbo", "llm_params": { "model": "openai/gpt-3.5-turbo", "api_key": process.env.OPENAI_API_KEY } },
-
-  // Fallback Ollama models
-  { "model_name": "gpt-4", "llm_params": { "model": "ollama/gpt-4", "api_key": process.env.OLLAMA_API_KEY } },
-  { "model_name": "gpt-3.5-turbo", "llm_params": { "model": "ollama/gpt-3.5-turbo", "api_key": process.env.OLLAMA_API_KEY } },
-
-  // Cost-effective alternatives
-  { "model_name": "gpt-3.5-turbo", "llm_params": { "model": "openrouter/openrouter/free", "api_key": process.env.OPENROUTER_API_KEY } }
-];
-
-export const productionRouter = router(productionModels);
-export const stagingRouter = router(productionModels, 'sequential');
-```
-
-### Development Setup
-
-```javascript
-const devModels = [
-  // Mock/echo models for testing
-  { "model_name": "echo", "llm_params": { "model": "echo", "api_key": "dev" } },
-
-  // Single provider for consistency
-  { "model_name": "gpt-3.5-turbo", "llm_params": { "model": "ollama/gpt-3.5-turbo", "api_key": process.env.OLLAMA_API_KEY } }
-];
-
-export const devRouter = router(devModels, 'sequential');
-```
-
-This router system provides powerful routing capabilities while maintaining a simple, unified API for LLM completion across multiple providers and deployment strategies.

package/index.js

@@ -1,265 +0,0 @@
-const OpenAIProvider = require('./providers/openai');
-const OllamaProvider = require('./providers/ollama');
-const OpenRouterProvider = require('./providers/openrouter');
-const { router } = require('./router');
-const { app } = require('./server');
-const logger = require('./logger');
-
-// Module-level variable to track sequential provider selection across completion calls
-let autoProviderIndex = 0;
-
-class LLMJS2 {
-  constructor(config = {}) {
-    this.providers = {
-      openai: new OpenAIProvider(config.openai || {}),
-      ollama: new OllamaProvider(config.ollama || {}),
-      openrouter: new OpenRouterProvider(config.openrouter || {})
-    };
-    this.defaultProvider = config.defaultProvider;
-    this.timeout = config.timeout || 60000;
-  }
-
-  /**
-   * Get available providers based on API keys
-   */
-  getAvailableProviders() {
-    const available = [];
-
-    const openaiKey = process.env.OPENAI_API_KEY || this.providers.openai.apiKey;
-    const ollamaKey = process.env.OLLAMA_API_KEY || this.providers.ollama.apiKey;
-    const openrouterKey = process.env.OPEN_ROUTER_API_KEY || this.providers.openrouter.apiKey;
-
-    // Check if keys are non-empty and not placeholder values
-    if (openaiKey && typeof openaiKey === 'string' && openaiKey.trim() && !openaiKey.startsWith(':')) {
-      available.push('openai');
-    }
-    if (ollamaKey && typeof ollamaKey === 'string' && ollamaKey.trim() && !ollamaKey.startsWith(':')) {
-      available.push('ollama');
-    }
-    if (openrouterKey && typeof openrouterKey === 'string' && openrouterKey.trim() && !openrouterKey.startsWith(':')) {
-      available.push('openrouter');
-    }
-
-    return available;
-  }
-
-  /**
-   * Parse model string like 'provider/model_name' or just 'model_name'
-   * Only splits on the first '/', since model names can contain '/' characters
-   */
-  parseModel(modelString) {
-    if (!modelString || typeof modelString !== 'string') {
-      return { provider: null, model: null };
-    }
-
-    const firstSlashIndex = modelString.indexOf('/');
-    if (firstSlashIndex !== -1) {
-      return {
-        provider: modelString.substring(0, firstSlashIndex),
-        model: modelString.substring(firstSlashIndex + 1)
-      };
-    }
-
-    return { provider: null, model: modelString };
-  }
-
-  /**
-   * Auto-select provider based on available API keys
-   * Cycles through available providers sequentially
-   */
-  getAutoProvider() {
-    // Check which API keys are available
-    const availableProviders = [];
-
-    if (process.env.OLLAMA_API_KEY) {
-      availableProviders.push('ollama');
-    }
-    if (process.env.OPEN_ROUTER_API_KEY) {
-      availableProviders.push('openrouter');
-    }
-    if (process.env.OPENAI_API_KEY) {
-      availableProviders.push('openai');
-    }
-
-    if (availableProviders.length === 0) {
-      throw new Error('No API keys found. Set OLLAMA_API_KEY, OPEN_ROUTER_API_KEY, or OPENAI_API_KEY environment variables.');
-    }
-
-    // Get next provider in sequence using module-level index
-    const providerName = availableProviders[autoProviderIndex];
-    autoProviderIndex = (autoProviderIndex + 1) % availableProviders.length;
-
-    const provider = this.providers[providerName];
-    const model = provider.defaultModel;
-
-    return { provider, model };
-  }
-
-  /**
-   * Determine which provider to use
-   */
-  getProvider(modelString, options = {}) {
-    const { provider: specifiedProvider, model } = this.parseModel(modelString);
-
-    if (specifiedProvider) {
-      if (!this.providers[specifiedProvider]) {
-        throw new Error(`Unknown provider: ${specifiedProvider}`);
-      }
-      return { provider: this.providers[specifiedProvider], model };
-    }
-
-    // Auto-detect provider
-    const availableProviders = this.getAvailableProviders();
-
-    if (availableProviders.length === 0) {
-      throw new Error('No API keys configured. Set OPENAI_API_KEY, OLLAMA_API_KEY, or OPEN_ROUTER_API_KEY environment variables.');
-    }
-
-    // Use default provider if specified, otherwise use first available
-    const providerName = this.defaultProvider || availableProviders[0];
-    const provider = this.providers[providerName];
-
-    if (!provider) {
-      throw new Error(`Provider ${providerName} is not available`);
-    }
-
-    return { provider, model: model || provider.defaultModel };
-  }
-
-  /**
-   * Validate input parameters
-   */
-  validateInput(input) {
-    if (typeof input === 'string') {
-      // Simple string prompt
-      if (!input.trim()) {
-        throw new Error('Prompt cannot be empty');
-      }
-      return {
-        model: null,
-        messages: [{ role: 'user', content: input }],
-        options: { final: true }
-      };
-    }
-
-    if (typeof input === 'object' && input !== null) {
-      // Object-based API
-      if (!input.messages || !Array.isArray(input.messages)) {
-        throw new Error('messages must be an array');
-      }
-
-      if (input.messages.length === 0) {
-        throw new Error('messages array cannot be empty');
-      }
-
-      // Validate message format
-      for (const msg of input.messages) {
-        if (!msg.role || !msg.content) {
-          throw new Error('Each message must have role and content properties');
-        }
-        if (!['system', 'user', 'assistant'].includes(msg.role)) {
-          throw new Error('Message role must be system, user, or assistant');
-        }
-      }
-
-      return {
-        model: input.model,
-        messages: input.messages,
-        options: {
-          temperature: input.temperature,
-          maxTokens: input.max_tokens || input.maxTokens,
-          topP: input.top_p || input.topP,
-          frequencyPenalty: input.frequency_penalty || input.frequencyPenalty,
-          presencePenalty: input.presence_penalty || input.presencePenalty,
-          stop: input.stop,
-          tools: input.tools,
-          toolChoice: input.tool_choice || input.toolChoice,
-          apiKey: input.apiKey,
-          timeout: input.timeout,
-          final: input.final ?? true
-        }
-      };
-    }
-
-    throw new Error('Input must be a string or object with messages');
-  }
-
-  /**
-   * Main completion function
-   */
-  async completion(input) {
-    try {
-      const { model, messages, options } = this.validateInput(input);
-      const { final, ...providerOptions } = options;
-
-      let provider, finalModel;
-
-      if (model) {
-        // Specific model requested
-        const result = this.getProvider(model, options);
-        provider = result.provider;
-        finalModel = result.model;
-      } else {
-        // No model specified - auto-select provider based on available API keys
-        const autoSelection = this.getAutoProvider();
-        provider = autoSelection.provider;
-        finalModel = autoSelection.model;
-      }
-
-      // Override provider API key if specified in options
-      if (options.apiKey) {
-        provider.apiKey = options.apiKey;
-      }
-
-      // Override timeout if specified
-      if (options.timeout) {
-        provider.timeout = options.timeout;
-      }
-
-      // Log request information
-      const apiKeyPreview = provider.apiKey ? provider.apiKey.substring(0, 10) + '...' : 'none';
-      logger.info('LLMJS2 📤 Sending to LLM provider', {
-        source: 'completion',
-        provider: provider.constructor.name.replace('Provider', '').toLowerCase(),
-        model: finalModel,
-        apiKey: apiKeyPreview,
-        messages: messages
-      });
-
-      const result = await provider.createCompletion(messages, { ...providerOptions, model: finalModel });
-
-      // Log response information
-      logger.info('LLMJS2 📥 Received from LLM provider', {
-        source: 'completion',
-        ...result
-      });
-
-      return final ? result.content : result;
-
-    } catch (error) {
-      // Sanitize error message to avoid leaking sensitive information
-      const message = error.message || 'Unknown error occurred';
-
-      // Don't include API keys or other sensitive data in error messages
-      const sanitizedMessage = message.replace(/Bearer\s+[^\s]+/gi, 'Bearer [REDACTED]');
-
-      throw new Error(`LLMJS2 completion failed: ${sanitizedMessage}`);
-    }
-  }
-}
-
-// Export the completion function directly for convenience
-function completion(input) {
-  const llm = new LLMJS2();
-  return llm.completion(input);
-}
-
-module.exports = {
-  completion,
-  LLMJS2,
-  router,
-  app,
-  OpenAIProvider,
-  OllamaProvider,
-  OpenRouterProvider
-};