llmjs2 1.7.1 → 2.0.1

package/core/docs/README.md

@@ -1,47 +0,0 @@
-# llmjs2 Documentation
-
-Welcome to the llmjs2 documentation! This folder contains all the documentation for using and understanding the llmjs2 library.
-
-## Documentation Overview
-
-### 📖 Getting Started
-
-- **[GET_STARTED.md](GET_STARTED.md)** - Quick setup guide for new users (5 minutes to first completion)
-
-### 🔧 Usage Guides
-
-- **[BASIC_USAGE.md](BASIC_USAGE.md)** - Core API patterns, configuration, and common use cases
-- **[ROUTER_GUIDE.md](ROUTER_GUIDE.md)** - Model routing and load balancing
-- **[GUARDRAILS_GUIDE.md](GUARDRAILS_GUIDE.md)** - Content filtering and request processing
-- **[SERVER_MODE.md](SERVER_MODE.md)** - Run llmjs2 as an OpenAI-compatible API server with routing
-- **[CLI.md](CLI.md)** - Command-line interface for server management
-
-## Quick Navigation
-
-### New to llmjs2?
-
-Start with **[GET_STARTED.md](GET_STARTED.md)** to get up and running quickly.
-
-### Want to use the API directly?
-
-Check out **[BASIC_USAGE.md](BASIC_USAGE.md)** for different API patterns and examples.
-
-### Need routing and load balancing?
-
-See **[ROUTER_GUIDE.md](ROUTER_GUIDE.md)** for intelligent model routing.
-
-### Need content filtering or custom processing?
-
-See **[GUARDRAILS_GUIDE.md](GUARDRAILS_GUIDE.md)** for guardrails and request processing.
-
-### Need to set up a server?
-
-See **[SERVER_MODE.md](SERVER_MODE.md)** for OpenAI-compatible server setup.
-
-### Prefer command-line tools?
-
-**[CLI.md](CLI.md)** covers the command-line interface and configuration files.
-
-## Contributing
-
-Documentation improvements are welcome! Please ensure any changes maintain consistency across all documentation files.

package/core/docs/ROUTER_GUIDE.md

@@ -1,199 +0,0 @@
-# llmjs2 Router Usage Guide
-
-The llmjs2 router provides intelligent model routing and load balancing capabilities, allowing you to distribute requests across multiple model deployments with different strategies.
-
-## Overview
-
-The router system enables:
-
-- **Load balancing** across models with the same name
-- **Multiple routing strategies** (default, random, sequential)
-- **Provider-agnostic routing** with unified API
-- **Flexible model configuration** for different providers
-- 
-
-## Quick Start
-
-### Basic Setup
-
-```javascript
-import { router } from 'llmjs2';
-
-// Define your model deployments
-const modelList = [
-  {
-    "model_name": "gpt-3.5-turbo",
-    "llm_params": {
-      "model": "ollama/chatgpt-v-2",
-      "api_key": process.env.OLLAMA_API_KEY,
-      "api_base": process.env.OLLAMA_API_BASE
-    }
-  },
-  {
-    "model_name": "openai-turbo",
-    "llm_params": {
-      "model": "gpt-3.5-turbo",
-      "api_key": process.env.OPENAI_API_KEY
-    }
-  },
-  {
-    "model_name": "gpt-4",
-    "llm_params": {
-      "model": "ollama/gpt-4",
-      "api_key": process.env.OLLAMA_API_KEY,
-      "api_base": process.env.OLLAMA_API_BASE
-    }
-  }
-];
-
-// Create routers with different strategies
-const defaultRouter = router(modelList);
-const randomRouter = router(modelList, 'random');
-const sequentialRouter = router(modelList, 'sequential');
-```
-
-### Basic Usage
-
-```javascript
-// Route to specific model
-const response = await defaultRouter.completion({
-  model: "gpt-3.5-turbo",
-  messages: [{"role": "user", "content": "Hey, how's it going?"}]
-});
-
-// Auto-route with random strategy
-const randomResponse = await randomRouter.completion({
-  messages: [{"role": "user", "content": "Hey, how's it going?"}]
-});
-
-// Auto-route with sequential strategy
-const seqResponse = await sequentialRouter.completion({
-  messages: [{"role": "user", "content": "Hey, how's it going?"}]
-});
-```
-
-## Model Configuration
-
-### Model List Format
-
-Each model in the list is defined with:
-
-```javascript
-{
-  "model_name": "string",     // Alias for routing (can have multiple providers)
-  "llm_params": {             // Provider-specific parameters
-    "model": "string",        // Actual model identifier for the provider
-    "api_key": "string",      // API key (can use environment variables)
-    "api_base": "string?",    // Custom API base URL (optional)
-    // ... other provider-specific params
-  }
-}
-```
-
-### Supported Providers
-
-#### Ollama
-
-```javascript
-{
-  "model_name": "my-ollama-model",
-  "llm_params": {
-    "model": "ollama/llama2",
-    "api_key": process.env.OLLAMA_API_KEY,
-    "api_base": process.env.OLLAMA_API_BASE
-  }
-}
-```
-
-#### OpenRouter
-
-```javascript
-{
-  "model_name": "my-openrouter-model",
-  "llm_params": {
-    "model": "openrouter/free-model",
-    "api_key": process.env.OPEN_ROUTER_API_KEY
-  }
-}
-```
-
-#### OpenAI
-
-```javascript
-{
-  "model_name": "my-openai-model",
-  "llm_params": {
-    "model": "openai/gpt-4",
-    "api_key": process.env.OPENAI_API_KEY
-  }
-}
-```
-
-## Routing Strategies
-
-### Default Strategy
-
-When no strategy is specified, uses sequential selection across all available models for auto-routing (when no specific model is requested).
-
-```javascript
-const route = router(modelList); // or router(modelList, 'default')
-
-// Auto-route with sequential selection (cycles through all models)
-const response = await route.completion({
-  messages: [...]
-});
-
-// Routes to one of the models with model_name="gpt-3.5-turbo" (load balancing)
-const response = await route.completion({
-  model: "gpt-3.5-turbo",
-  messages: [...]
-});
-```
-
-### Random Strategy
-
-Randomly selects from available models when no specific model is requested.
-
-```javascript
-const route = router(modelList, 'random');
-
-// Randomly selects from ALL models in the list
-const response = await route.completion({
-  messages: [...]
-});
-```
-
-### Sequential Strategy
-
-Cycles through models in order for each request.
-
-```javascript
-const route = router(modelList, 'sequential');
-
-// Uses first model, then second, then third, etc.
-const response1 = await route.completion({ messages: [...] }); // model 1
-const response2 = await route.completion({ messages: [...] }); // model 2
-const response3 = await route.completion({ messages: [...] }); // model 3
-// ... cycles back to model 1
-```
-
-## Error Handling
-
-```javascript
-try {
-  const response = await route.completion({
-    model: "non-existent-model",
-    messages: [{"role": "user", "content": "Hello"}]
-  });
-} catch (error) {
-  if (error.message.includes('Model not found')) {
-    console.log('Model not configured in router');
-  } else if (error.message.includes('API key')) {
-    console.log('Provider API key missing');
-  } else {
-    console.log('Routing error:', error.message);
-  }
-}
-```
-
-## 

package/core/docs/SERVER_MODE.md

@@ -1,358 +0,0 @@
-# Server Mode Guide
-
-Run llmjs2 as an OpenAI-compatible API server with intelligent routing and load balancing capabilities to integrate with existing OpenAI clients and applications.
-
-## Quick Start Server
-
-### Method 1: Simple JavaScript Server
-
-Create a server file:
-
-```javascript
-// server.js
-import { app } from 'llmjs2';
-
-// Start the server
-app.listen(3000, () => {
-  console.log('🚀 llmjs2 server running on http://localhost:3000');
-});
-```
-
-Run it:
-
-```bash
-node server.js
-```
-
-# 
-
-## API Endpoints
-
-### Chat Completions
-
-**Endpoint:** `POST /v1/chat/completions`
-
-**Content-Type:** `application/json`
-
-**Request Format:**
-
-```json
-{
-  "model": "ollama/minimax-m2.5:cloud",
-  "messages": [
-    {
-      "role": "user",
-      "content": "Hello! How are you?"
-    }
-  ],
-  "tools": [] // optional
-}
-```
-
-**Response Format:**
-
-The server returns an OpenAI-style chat completion response. A legacy `messages` array is also included for backward compatibility:
-
-```json
-{
-  "id": "chatcmpl-123456",
-  "object": "chat.completion",
-  "created": 1640995200,
-  "model": "ollama/minimax-m2.5:cloud",
-  "choices": [
-    {
-      "index": 0,
-      "message": {
-        "role": "assistant",
-        "content": "Hello! I'm doing well, thank you for asking!"
-      },
-      "finish_reason": "stop"
-    }
-  ],
-  "messages": [
-    {
-      "role": "user",
-      "content": "Hello! How are you?"
-    },
-    {
-      "role": "assistant",
-      "content": "Hello! I'm doing well, thank you for asking!"
-    }
-  ]
-}
-```
-
-## Using with OpenAI Clients
-
-### Direct HTTP Requests
-
-You can consume the response like a normal OpenAI chat completion:
-
-```python
-import requests
-
-response = requests.post(
-    "http://localhost:3000/v1/chat/completions",
-    json={
-        "messages": [{"role": "user", "content": "Hello!"}]
-    }
-)
-
-data = response.json()
-assistant_message = data["choices"][0]["message"]
-print(f"Model used: {data['model']}")
-print(f"Assistant: {assistant_message['content']}")
-```
-
-### Node.js with fetch
-
-```javascript
-const response = await fetch('http://localhost:3000/v1/chat/completions', {
-  method: 'POST',
-  headers: {
-    'Content-Type': 'application/json'
-  },
-  body: JSON.stringify({
-    messages: [{ role: 'user', content: 'Hello!' }]
-  })
-});
-
-const data = await response.json();
-const assistantMessage = data.choices[0].message;
-console.log(`Model used: ${data.model}`);
-console.log(`Assistant: ${assistantMessage.content}`);
-```
-
-### cURL
-
-```bash
-curl -X POST http://localhost:3000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer your-api-key" \
-  -d '{
-    "model": "ollama/minimax-m2.5:cloud",
-    "messages": [
-      {
-        "role": "user",
-        "content": "Hello! How are you?"
-      }
-    ]
-   }'
-```
-
-## Router Integration
-
-Add intelligent routing and load balancing to your server:
-
-### Basic Router Setup
-
-```javascript
-import { router, app } from 'llmjs2';
-
-const costOptimizedModels = [
-  {
-    "model_name": "text-davinci-001",
-    "llm_params": {
-      "model": "ollama/text-davinci-003",
-      "api_key": process.env.OLLAMA_API_KEY
-    }
-  },
-  {
-    "model_name": "text-davinci-002",
-    "llm_params": {
-      "model": "openrouter/text-davinci-003",
-      "api_key": process.env.OPEN_ROUTER_API_KEY
-    }
-  },
-  {
-    "model_name": "text-davinci-003",
-    "llm_params": {
-      "model": "openai/gpt-3.5-turbo",
-      "api_key": process.env.OPENAI_API_KEY
-    }
-  }
-];
-
-// Create router with random strategy for load balancing
-const route = router(costOptimizedModels, 'random');
-
-// Apply router to server
-app.use(route);
-
-// Start the server
-app.listen(3000, () => {
-  console.log('🚀 llmjs2 server with routing running on http://localhost:3000');
-});
-```
-
-### Router Strategies
-
-- **`'random'`**: Randomly selects from available models
-- **`'sequential'`**: Cycles through models in order
-- **`'default'`** or none: Load balances across models with same `model_name`
-
-### API Usage with Routing
-
-```bash
-# Automatic routing (uses router strategy)
-curl -X POST http://localhost:3000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "messages": [{"role": "user", "content": "Hello!"}]
-  }'
-
-# Direct model routing (bypasses router)
-curl -X POST http://localhost:3000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "openai/gpt-3.5-turbo",
-    "messages": [{"role": "user", "content": "Hello!"}]
-  }'
-```
-
-### Advanced Routing Examples
-
-**Multi-Provider Fallback:**
-
-```javascript
-const fallbackModels = [
-  { "model_name": "gpt-4", "llm_params": { "model": "openai/gpt-4", "api_key": process.env.OPENAI_API_KEY } },
-  { "model_name": "gpt-4", "llm_params": { "model": "ollama/gpt-4", "api_key": process.env.OLLAMA_API_KEY } },
-  { "model_name": "gpt-4", "llm_params": { "model": "openrouter/gpt-4", "api_key": process.env.OPEN_ROUTER_API_KEY } }
-];
-
-const route = router(fallbackModels, 'random');
-app.use(route);
-```
-
-**Cost Optimization:**
-
-```javascript
-const costModels = [
-  { "model_name": "completion", "llm_params": { "model": "ollama/llama2", "api_key": process.env.OLLAMA_API_KEY } },
-  { "model_name": "completion", "llm_params": { "model": "openrouter/free", "api_key": process.env.OPEN_ROUTER_API_KEY } },
-  { "model_name": "completion", "llm_params": { "model": "openai/gpt-3.5-turbo", "api_key": process.env.OPENAI_API_KEY } }
-];
-
-const route = router(costModels, 'sequential'); // Try cheaper models first
-app.use(route);
-```
-
-## Function Calling (Tools) Support
-
-The server supports OpenAI-compatible function calling:
-
-```bash
-curl -X POST http://localhost:3000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer your-api-key" \
-  -d '{
-    "model": "openrouter/openrouter/free",
-    "messages": [
-      {
-        "role": "user",
-        "content": "What is the weather like in Paris?"
-      }
-    ],
-    "tools": [
-      {
-        "type": "function",
-        "function": {
-          "name": "get_weather",
-          "description": "Get the current weather in a given location",
-          "parameters": {
-            "type": "object",
-            "properties": {
-              "location": {
-                "type": "string",
-                "description": "The city and state, e.g. San Francisco, CA"
-              }
-            },
-            "required": ["location"]
-          }
-        }
-      }
-    ]
-  }'
-```
-
-#### Error Handling
-
-The server returns proper HTTP status codes and JSON error responses:
-
-```json
-{
-  "error": {
-    "message": "model is required",
-    "type": "invalid_request_error"
-  }
-}
-```
-
-Common status codes:
-
-- `400` - Bad Request (missing parameters)
-- `404` - Not Found (invalid endpoint)
-- `500` - Internal Server Error (API failures)
-
-### Environment Variables for Production
-
-```bash
-# Server configuration
-PORT=3000
-HOST=0.0.0.0
-
-# API Keys
-OLLAMA_API_KEY=your_production_key
-OPEN_ROUTER_API_KEY=your_production_key
-
-# Default models
-OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
-OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
-```
-
-## Monitoring and Logging
-
-The server includes comprehensive logging:
-
-```
-[2024-01-15T10:30:45.123Z] POST /v1/chat/completions
-Headers: {"content-type":"application/json",...}
-Body parsing completed successfully
-Starting completion with model: ollama/minimax-m2.5:cloud
-```
-
-### API Request Issues
-
-**400 Bad Request:**
-
-- Check that `model` and `messages` are provided
-- Ensure messages have `role` and `content` properties
-
-**500 Internal Server Error:**
-
-- Check API keys are valid
-- Verify internet connection
-- Check provider API status
-
-### CORS Issues
-
-If you're getting CORS errors in the browser:
-
-```javascript
-// The server includes CORS headers by default
-// If you need custom CORS, modify the server code
-res.writeHead(statusCode, {
-  'Content-Type': 'application/json',
-  'Access-Control-Allow-Origin': '*', // Change this for production
-  // ... other headers
-});
-```
-
-## Next Steps
-
-- **[CLI Guide](CLI.md)** - Use the command-line interface
-- **[Basic Usage](BASIC_USAGE.md)** - Learn different API patterns
-- **[Technical Specification](TECHNICAL_SPECIFICATION.md)** - Detailed technical information
-
-The server mode makes llmjs2 compatible with any OpenAI-compatible client or application!

package/core/index.js

@@ -1,115 +0,0 @@
-const OpenAIProvider = require('./providers/openai');
-const OllamaProvider = require('./providers/ollama');
-const OpenRouterProvider = require('./providers/openrouter');
-const { completion } = require('./completion');
-const { router } = require('./router');
-const { app } = require('./server');
-
-class LLMJS2 {
-  constructor(config = {}) {
-    this.defaultProvider = config.defaultProvider;
-    this.providers = {
-      openai: new OpenAIProvider(config.openai || {}),
-      ollama: new OllamaProvider(config.ollama || {}),
-      openrouter: new OpenRouterProvider(config.openrouter || {})
-    };
-  }
-
-  parseModel(modelString) {
-    if (!modelString || typeof modelString !== 'string') {
-      return { provider: null, model: null };
-    }
-
-    const firstSlashIndex = modelString.indexOf('/');
-    if (firstSlashIndex !== -1) {
-      return {
-        provider: modelString.substring(0, firstSlashIndex),
-        model: modelString.substring(firstSlashIndex + 1)
-      };
-    }
-
-    return { provider: null, model: modelString };
-  }
-
-  getAvailableProviders() {
-    const available = [];
-
-    const openaiKey = process.env.OPENAI_API_KEY || this.providers.openai.apiKey;
-    const ollamaKey = process.env.OLLAMA_API_KEY || this.providers.ollama.apiKey;
-    const openrouterKey = process.env.OPEN_ROUTER_API_KEY || this.providers.openrouter.apiKey;
-
-    if (openaiKey && typeof openaiKey === 'string' && openaiKey.trim() && !openaiKey.startsWith(':')) {
-      available.push('openai');
-    }
-    if (ollamaKey && typeof ollamaKey === 'string' && ollamaKey.trim() && !ollamaKey.startsWith(':')) {
-      available.push('ollama');
-    }
-    if (openrouterKey && typeof openrouterKey === 'string' && openrouterKey.trim() && !openrouterKey.startsWith(':')) {
-      available.push('openrouter');
-    }
-
-    return available;
-  }
-
-  getAutoProvider() {
-    const availableProviders = this.getAvailableProviders();
-    if (availableProviders.length === 0) {
-      throw new Error('No API keys found. Set OLLAMA_API_KEY, OPEN_ROUTER_API_KEY, or OPENAI_API_KEY environment variables.');
-    }
-
-    const randomIndex = Math.floor(Math.random() * availableProviders.length);
-    const providerName = availableProviders[randomIndex];
-    const provider = this.providers[providerName];
-
-    return {
-      provider,
-      model: provider.defaultModel
-    };
-  }
-
-  getProvider(modelString) {
-    const { provider: specifiedProvider, model } = this.parseModel(modelString);
-
-    if (specifiedProvider) {
-      const provider = this.providers[specifiedProvider];
-      if (!provider) {
-        throw new Error(`Unknown provider: ${specifiedProvider}`);
-      }
-      return { provider, model };
-    }
-
-    const availableProviders = this.getAvailableProviders();
-    if (availableProviders.length === 0) {
-      throw new Error('No API keys configured. Set OPENAI_API_KEY, OLLAMA_API_KEY, or OPEN_ROUTER_API_KEY environment variables.');
-    }
-
-    const providerName = this.defaultProvider && availableProviders.includes(this.defaultProvider)
-      ? this.defaultProvider
-      : availableProviders[0];
-    const provider = this.providers[providerName];
-
-    return {
-      provider,
-      model: model || provider.defaultModel
-    };
-  }
-
-  completion(input) {
-    return completion(input, {
-      defaultProvider: this.defaultProvider,
-      openai: { apiKey: this.providers.openai.apiKey, baseURL: this.providers.openai.baseURL, timeout: this.providers.openai.timeout },
-      ollama: { apiKey: this.providers.ollama.apiKey, baseURL: this.providers.ollama.baseURL, timeout: this.providers.ollama.timeout },
-      openrouter: { apiKey: this.providers.openrouter.apiKey, baseURL: this.providers.openrouter.baseURL, timeout: this.providers.openrouter.timeout }
-    });
-  }
-}
-
-module.exports = {
-  completion,
-  LLMJS2,
-  router,
-  app,
-  OpenAIProvider,
-  OllamaProvider,
-  OpenRouterProvider
-};