llmjs2 1.1.1 → 1.3.0

package/README.md

@@ -1,357 +1,382 @@
-# llmjs2
-
-A lightweight llm Node.js library for building simple / personal AI applications
-
-## Supported Providers
-
-- **Ollama** - Connect to Ollama's cloud API
-- **OpenRouter** - Access multiple LLM models through OpenRouter
-
-## Installation
-
-```bash
-npm install llmjs2
-```
-
-## Usage
-
-llmjs2 supports three calling conventions:
-
-### Simple API (Auto-Detection)
-
-```javascript
-import { completion } from 'llmjs2';
-
-// Just provide a prompt - the library handles the rest
-const result = await completion('Explain the use of llmjs2');
-
-// Or provide a model and prompt
-const result = await completion('ollama/minimax-m2.5:cloud', 'Explain the use of llmjs2');
-```
-
-**How it works:**
-- Looks for `OLLAMA_API_KEY` and `OPEN_ROUTER_API_KEY` environment variables
-- If only one is set, uses that provider
-- If both are set, randomly chooses one
-- Uses `OLLAMA_DEFAULT_MODEL` or defaults to `minimax-m2.5:cloud` for Ollama
-- Uses `OPEN_ROUTER_DEFAULT_MODEL` or defaults to `openrouter/free` for OpenRouter
-- If a model is provided, uses that model instead of the default
-
-### Function-Based API
-
-```javascript
-import { completion } from 'llmjs2';
-
-// Using Ollama
-const resultOllama = await completion('ollama/minimax-m2.5:cloud', 'Explain the use of llmjs2', 'your-api-key');
-
-// Using OpenRouter
-const resultOR = await completion('openrouter/openrouter/free', 'Explain the use of llmjs2', 'your-api-key');
-```
-
-### Object-Based API
-
-```javascript
-import { completion } from 'llmjs2';
-
-// Using Ollama with system message
-const resultOllama = await completion({
-  model: 'ollama/minimax-m2.5:cloud',
-  messages: [
-    { role: 'system', content: 'You are a helpful AI assistant.' },
-    { role: 'user', content: 'Explain the use of llmjs2.' }
-  ],
-  apiKey: 'your-api-key' // optional
-});
-
-// Using OpenRouter with system message
-const resultOR = await completion({
-  model: 'openrouter/openrouter/free',
-  messages: [
-    { role: 'system', content: 'You are a helpful AI assistant.' },
-    { role: 'user', content: 'Explain the use of llmjs2.' }
-  ],
-  apiKey: 'your-api-key' // optional
-});
-```
-
-## Tools Support
-
-llmjs2 supports function calling (tools) through the object-based API:
-
-```javascript
-import { completion } from 'llmjs2';
-
-const result = await completion({
-  model: 'openrouter/openrouter/free',
-  messages: [
-    { role: 'user', content: 'What is the weather like in Paris?' }
-  ],
-  tools: [
-    {
-      type: 'function',
-      function: {
-        name: 'get_weather',
-        description: 'Get the current weather in a given location',
-        parameters: {
-          type: 'object',
-          properties: {
-            location: {
-              type: 'string',
-              description: 'The city and state, e.g. San Francisco, CA'
-            },
-            unit: {
-              type: 'string',
-              enum: ['celsius', 'fahrenheit'],
-              description: 'The temperature unit to use'
-            }
-          },
-          required: ['location']
-        }
-      }
-    }
-  ]
-});
-
-// Result when tools are used:
-// {
-//   content: '',
-//   tool_calls: [
-//     {
-//       id: 'call_123',
-//       type: 'function',
-//       function: {
-//         name: 'get_weather',
-//         arguments: '{"location": "Paris, France"}'
-//       }
-//     }
-//   ]
-// }
-```
-
-## API Key Configuration
-
-You can provide API keys in four ways:
-
-### 1. Simple API (Environment Variables)
-
-```bash
-export OLLAMA_API_KEY=your-ollama-api-key
-export OPEN_ROUTER_API_KEY=your-openrouter-api-key
-
-# Optional: Set default models
-export OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
-export OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
-```
-
-```javascript
-const result = await completion('Your prompt');
-```
-
-### 2. Direct Parameter (Function API)
-
-```javascript
-const result = await completion('ollama/minimax-m2.5:cloud', 'Your prompt', 'your-api-key');
-```
-
-### 3. Object Property (Object API)
-
-```javascript
-const result = await completion({
-  model: 'ollama/minimax-m2.5:cloud',
-  messages: [{ role: 'user', content: 'Your prompt' }],
-  apiKey: 'your-api-key'
-});
-```
-
-### 4. Environment Variables (Function/Object API)
-
-```bash
-export OLLAMA_API_KEY=your-ollama-api-key
-export OPEN_ROUTER_API_KEY=your-openrouter-api-key
-```
-
-```javascript
-// Function API
-const result = await completion('ollama/minimax-m2.5:cloud', 'Your prompt');
-
-// Object API
-const result = await completion({
-  model: 'ollama/minimax-m2.5:cloud',
-  messages: [{ role: 'user', content: 'Your prompt' }]
-});
-```
-
-## Model Format
-
-Models must be specified in the format: `provider/model_name`
-
-The provider is the text before the first `/`, and the model name is everything after it.
-
-Examples:
-- `ollama/minimax-m2.5:cloud`
-- `ollama/llama2`
-- `openrouter/openrouter/free`
-- `openrouter/meta-llama/llama-2-70b-chat`
-
-## Messages Format (Object API)
-
-The `messages` parameter is an array of message objects with the following structure:
-
-```javascript
-[
-  { role: 'system', content: 'You are a helpful AI assistant.' },
-  { role: 'user', content: 'What is the capital of France?' },
-  { role: 'assistant', content: 'The capital of France is Paris.' },
-  { role: 'user', content: 'What is its population?' }
-]
-```
-
-**Supported roles:**
-- `system` - System instructions
-- `user` - User messages
-- `assistant` - Assistant responses
-
-## Tools Format (Object API)
-
-The `tools` parameter is an array of tool definitions:
-
-```javascript
-[
-  {
-    type: 'function',
-    function: {
-      name: 'function_name',
-      description: 'Description of what the function does',
-      parameters: {
-        type: 'object',
-        properties: {
-          param1: {
-            type: 'string',
-            description: 'Description of parameter'
-          }
-        },
-        required: ['param1']
-      }
-    }
-  }
-]
-```
-
-## Error Handling
-
-The library throws descriptive errors for:
-- Missing or invalid parameters
-- Missing API keys
-- API request failures
-- Invalid response formats
-- Request timeouts (60 seconds)
-- Invalid tools format
-
-```javascript
-try {
-  const result = await completion('Your prompt');
-} catch (error) {
-  console.error('Completion failed:', error.message);
-}
-```
-
-## Example Programs
-
-### Main Example
-
-A real usage test program is included in `example.js`. To run it:
-
-```bash
-# Set your API keys
-export OLLAMA_API_KEY=your-ollama-api-key
-export OPEN_ROUTER_API_KEY=your-openrouter-api-key
-
-# Run the example
-node example.js
-```
-
-The example program will:
-- Test simple API (auto-detection)
-- Test simple API with model
-- Test Ollama with function-based API
-- Test Ollama with object-based API
-- Test Ollama with tools
-- Test OpenRouter with function-based API
-- Test OpenRouter with object-based API
-- Test OpenRouter with tools
-- Display results and test summary
-
-
-
-## API Reference
-
-### completion(prompt)
-
-**Simple API (Prompt Only)**
-
-**Parameters:**
-- `prompt` (string): The prompt to send to the LLM
-
-**Returns:**
-- `Promise<string>`: The completion result
-
-**Behavior:**
-- Auto-detects provider based on available API keys
-- Uses `OLLAMA_DEFAULT_MODEL` or defaults to `minimax-m2.5:cloud` for Ollama
-- Uses `OPEN_ROUTER_DEFAULT_MODEL` or defaults to `openrouter/free` for OpenRouter
-- Randomly chooses provider if both API keys are set
-
-### completion(model, prompt)
-
-**Simple API (Model and Prompt)**
-
-**Parameters:**
-- `model` (string): Model identifier in format "provider/model_name"
-- `prompt` (string): The prompt to send to the LLM
-
-**Returns:**
-- `Promise<string>`: The completion result
-
-**Behavior:**
-- Auto-detects provider based on available API keys
-- Uses the provided model instead of the default
-- Randomly chooses provider if both API keys are set
-
-### completion(model, prompt, apiKey)
-
-**Function-Based API**
-
-**Parameters:**
-- `model` (string): Model identifier in format "provider/model_name"
-- `prompt` (string): The prompt to send to the LLM
-- `apiKey` (string, optional): API key (falls back to environment variables)
-
-**Returns:**
-- `Promise<string>`: The completion result
-
-### completion(options)
-
-**Object-Based API**
-
-**Parameters:**
-- `options` (object): Configuration object
-  - `model` (string): Model identifier in format "provider/model_name"
-  - `messages` (array): Array of message objects with role and content
-  - `apiKey` (string, optional): API key (falls back to environment variables)
-  - `tools` (array, optional): Array of tool definitions
-
-**Returns:**
-- `Promise<string|object>`: The completion result (string or object with tool calls)
-
-**Throws:**
-- Error if model format is invalid
-- Error if prompt/messages is missing
-- Error if API key is not provided
-- Error if API request fails
-- Error if request times out (60 seconds)
-- Error if tools format is invalid
-
-## License
-
-MIT
+# llmjs2
+
+A unified, enterprise-grade Node.js library for connecting to multiple Large Language Model (LLM) providers: OpenAI, Ollama, and OpenRouter.
+
+**Features:**
+- **Unified API**: Single interface for OpenAI, Ollama, and OpenRouter
+- **Intelligent Router**: Load balancing and multiple routing strategies
+- **Guardrails System**: Content filtering, logging, rate limiting, and custom processing
+- **OpenAI-Compatible Server**: Drop-in replacement for OpenAI API clients
+- **CLI Interface**: Command-line server management with configuration files
+- **Enterprise Security**: Input validation, error sanitization, and safe defaults
+- **Zero External Dependencies**: Pure Node.js implementation
+
+## Features
+
+- **Unified API**: Single interface for OpenAI, Ollama, and OpenRouter
+- **Auto-detection**: Automatically chooses available providers based on API keys
+- **Enterprise-grade**: Robust error handling, input validation, and security measures
+- **Zero dependencies**: Uses only Node.js built-in modules
+- **TypeScript-free**: Pure JavaScript, no compilation required
+- **Production-ready**: Comprehensive testing and security auditing
+
+## Installation
+
+```bash
+npm install llmjs2
+```
+
+Or for global CLI usage:
+
+```bash
+npm install -g llmjs2
+```
+
+## Quick Test
+
+Try the sample configuration:
+
+```bash
+# Validate configuration
+node validate-config.js
+
+# Start server with sample config
+node cli.js --config config.yaml --port 3001
+
+# Test the API
+curl -X POST http://localhost:3001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"messages":[{"role":"user","content":"Hello!"}]}'
+
+# Response format:
+# {
+#   "id": "chatcmpl-123456",
+#   "object": "chat.completion",
+#   "created": 1640995200,
+#   "model": "ollama/minimax-m2.5:cloud",
+#   "messages": [
+#     {"role": "user", "content": "Hello!"},
+#     {"role": "assistant", "content": "Hi there!"}
+#   ]
+# }
+```
+
+See `CONFIG_README.md` for detailed configuration examples.
+
+## Quick Start
+
+```javascript
+import { completion } from 'llmjs2';
+
+// Set API keys
+process.env.OPENAI_API_KEY = 'your-openai-key';
+process.env.OLLAMA_API_KEY = 'your-ollama-key';
+process.env.OPEN_ROUTER_API_KEY = 'your-openrouter-key';
+
+// Simple completion
+const response = await completion('Hello, how are you?');
+console.log(response);
+```
+
+## API Keys Setup
+
+Set your API keys as environment variables:
+
+```bash
+export OPENAI_API_KEY=your_openai_api_key
+export OLLAMA_API_KEY=your_ollama_api_key
+export OPEN_ROUTER_API_KEY=your_openrouter_api_key
+```
+
+## Usage Patterns
+
+### 1. Simple API (Auto-detection)
+
+```javascript
+import { completion } from 'llmjs2';
+
+const response = await completion('Explain quantum physics simply');
+```
+
+### 2. Provider-Specific Model
+
+```javascript
+// OpenAI
+const openaiResponse = await completion('openai/gpt-4', 'Write a haiku about coding');
+
+// Ollama
+const ollamaResponse = await completion('ollama/minimax-m2.5:cloud', 'What is AI?');
+
+// OpenRouter
+const openrouterResponse = await completion('openrouter/openrouter/free', 'Tell me a joke');
+```
+
+### 3. Advanced Object API
+
+```javascript
+const response = await completion({
+  model: 'openai/gpt-3.5-turbo',
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: 'What is the capital of France?' }
+  ],
+  temperature: 0.7,
+  maxTokens: 100
+});
+```
+
+## Configuration
+
+### Default Models
+
+Set default models for each provider:
+
+```bash
+export OPENAI_DEFAULT_MODEL=gpt-4
+export OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
+export OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
+```
+
+### Base URLs
+
+Customize API endpoints:
+
+```bash
+export OPENAI_BASE_URL=https://api.openai.com/v1
+export OLLAMA_BASE_URL=https://ollama.com/api/chat
+export OPEN_ROUTER_BASE_URL=https://openrouter.ai/api/v1/chat/completions
+```
+
+## Error Handling
+
+```javascript
+import { completion } from 'llmjs2';
+
+try {
+  const response = await completion('Tell me a joke');
+  console.log(response);
+} catch (error) {
+  console.error('Error:', error.message);
+}
+```
+
+## Conversations
+
+```javascript
+import { completion } from 'llmjs2';
+
+const messages = [
+  { role: 'system', content: 'You are a helpful coding assistant.' },
+  { role: 'user', content: 'How do I reverse a string in JavaScript?' }
+];
+
+let response = await completion({ model: 'openai/gpt-4', messages });
+console.log('Assistant:', response);
+
+// Continue conversation
+messages.push({ role: 'assistant', content: response });
+messages.push({ role: 'user', content: 'Can you show me with an example?' });
+
+response = await completion({ model: 'openai/gpt-4', messages });
+console.log('Assistant:', response);
+```
+
+## Function Calling (Tools)
+
+```javascript
+import { completion } from 'llmjs2';
+
+const weatherTool = {
+  type: 'function',
+  function: {
+    name: 'get_weather',
+    description: 'Get current weather for a location',
+    parameters: {
+      type: 'object',
+      properties: {
+        location: { type: 'string', description: 'City name' }
+      },
+      required: ['location']
+    }
+  }
+};
+
+const response = await completion({
+  model: 'openai/gpt-4',
+  messages: [{ role: 'user', content: 'What is the weather in Paris?' }],
+  tools: [weatherTool]
+});
+
+if (response.tool_calls) {
+  console.log('Tool calls:', response.tool_calls);
+}
+```
+
+## Router System
+
+Intelligent model routing with load balancing and multiple strategies:
+
+```javascript
+import { router } from 'llmjs2';
+
+const modelList = [
+  {
+    model_name: 'gpt-3.5-turbo',
+    llm_params: {
+      model: 'ollama/chatgpt-v-2',
+      api_key: process.env.OLLAMA_API_KEY
+    }
+  },
+  {
+    model_name: 'gpt-3.5-turbo',
+    llm_params: {
+      model: 'openai/gpt-3.5-turbo',
+      api_key: process.env.OPENAI_API_KEY
+    }
+  }
+];
+
+// Load balancing across models with same name
+const route = router(modelList, 'random');
+const response = await route.completion({
+  model: 'gpt-3.5-turbo',
+  messages: [{ role: 'user', content: 'Hello!' }]
+});
+
+// Auto-routing with different strategies
+const randomRouter = router(modelList, 'random');
+const sequentialRouter = router(modelList, 'sequential');
+```
+
+**Routing Strategies:**
+- `default`: Load balance across models with same name
+- `random`: Randomly select from all models
+- `sequential`: Cycle through models in order
+
+## Guardrails System
+
+Add custom logic before and after LLM calls for content filtering, logging, and processing:
+
+```javascript
+import { router } from 'llmjs2';
+
+const route = router(modelList);
+
+route.setGuardrails([
+  {
+    name: 'content_filter',
+    mode: 'pre_call',
+    code: (processId, input) => {
+      // Filter inappropriate content
+      const filteredMessages = input.messages.map(msg => ({
+        ...msg,
+        content: msg.content.replace(/badword/gi, '****')
+      }));
+      return { ...input, messages: filteredMessages };
+    }
+  },
+  {
+    name: 'response_logger',
+    mode: 'post_call',
+    code: (processId, result) => {
+      console.log(`[${processId}] Response:`, result);
+      return result;
+    }
+  }
+]);
+```
+
+## Server Mode
+
+Run an API server that returns responses with metadata and message arrays:
+
+```javascript
+import { router, app } from 'llmjs2';
+
+const route = router(modelList);
+app.use(route);
+app.listen(3000);
+```
+
+Or use the CLI:
+
+```bash
+llmjs2 --config config.yaml --port 3000
+```
+
+## CLI Interface
+
+Manage servers from the command line:
+
+```bash
+# Start server with defaults
+llmjs2
+
+# Use configuration file
+llmjs2 --config config.yaml
+
+# Custom port and host
+llmjs2 --port 8080 --host 0.0.0.0
+
+# Get help
+llmjs2 --help
+```
+
+## Configuration Files
+
+Use YAML for advanced configuration:
+
+```yaml
+model_list:
+  - model_name: premium
+    llm_params:
+      model: openrouter/openai/gpt-4
+      api_key: os.environ/OPEN_ROUTER_API_KEY
+
+  - model_name: standard
+    llm_params:
+      model: ollama/minimax-m2.5:cloud
+      api_key: os.environ/OLLAMA_API_KEY
+
+guardrails:
+  - name: content_filter
+    mode: pre_call
+    code: |
+      (processId, input) => {
+        // Content filtering logic
+        return input;
+      }
+
+router_settings:
+  routing_strategy: random
+```
+
+**Note**: Model names in the configuration use the format `[provider]/[actual-model-name]` (e.g., `openai/gpt-4`, `ollama/minimax-m2.5:cloud`). The `[provider]/` prefix is used for routing and is automatically stripped when sending requests to LLM providers.
+
+## Security Features
+
+- **No API key logging**: Sensitive information is never logged
+- **Input validation**: All inputs are validated and sanitized
+- **Error sanitization**: Error messages don't leak sensitive data
+- **Timeout protection**: Requests timeout to prevent hanging
+- **HTTPS only**: All communications use HTTPS
+
+## Testing
+
+Run the test suite:
+
+```bash
+npm test
+```
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions welcome! Please ensure all tests pass and add tests for new features.
+
+## Support
+
+For issues and questions, please create an issue on GitHub.