llmjs2 1.1.1 → 1.3.1

package/docs/CLI.md

@@ -0,0 +1,455 @@
+# CLI Guide
+
+Use the llmjs2 command-line interface to manage servers and configurations without writing code.
+
+## Installation
+
+If you installed llmjs2 globally, the `llmjs2` command is available everywhere:
+
+```bash
+npm install -g llmjs2
+```
+
+## Basic Usage
+
+### Start Server with Defaults
+
+```bash
+llmjs2
+```
+
+This starts a server on `http://localhost:3000` using:
+
+- Environment variables for API keys
+- Default models
+- No configuration file
+
+### Custom Server Options
+
+```bash
+# Custom port
+llmjs2 --port 8080
+
+# Custom host
+llmjs2 --host 0.0.0.0
+
+# Custom port and host
+llmjs2 --port 8080 --host 0.0.0.0
+
+# Use configuration file
+llmjs2 --config my-config.yaml
+
+# Combine options
+llmjs2 --config my-config.yaml --port 8080 --host 0.0.0.0
+```
+
+### Get Help
+
+```bash
+llmjs2 --help
+```
+
+Output:
+
+```
+🤖 llmjs2 - OpenAI-Compatible API Server
+
+USAGE:
+  llmjs2 [options]
+
+DESCRIPTION:
+  Starts an OpenAI-compatible API server with intelligent routing and guardrails.
+  Supports model load balancing, content filtering, and custom request processing.
+  The server listens for POST requests to /v1/chat/completions.
+
+OPTIONS:
+  -c, --config <file>    YAML config file with models, guardrails, and routing
+  -p, --port <port>      Port to listen on (default: 3000)
+  -H, --host <host>      Host to bind to (default: localhost)
+  -h, --help            Show this help message
+
+EXAMPLES:
+  llmjs2
+  llmjs2 --config config.yaml
+  llmjs2 --config config.yaml --port 8080 --host 0.0.0.0
+```
+
+## Configuration Files
+
+Use YAML configuration files to define models, guardrails, and router settings for advanced LLM routing and processing.
+
+### Configuration with Guardrails
+
+Create `config.yaml` with models, guardrails, and routing settings:
+
+```yaml
+model_list:
+  - model_name: premium
+    llm_params:
+      model: openrouter/openai/gpt-4-turbo-preview
+      api_key: os.environ/PROD_OPEN_ROUTER_API_KEY
+
+  - model_name: standard
+    llm_params:
+      model: ollama/minimax-m2.5:cloud
+      api_key: os.environ/PROD_OLLAMA_API_KEY
+
+guardrails:
+  - name: "content_filter"
+    mode: "pre_call"
+    code: |
+      (processId, input) => {
+        // Filter inappropriate content before LLM call
+        const { model, messages } = input;
+        const filteredMessages = messages.map(msg => ({
+          ...msg,
+          content: msg.content.replace(/badword/gi, '****')
+        }));
+        return { model, messages: filteredMessages };
+      }
+
+  - name: "response_logger"
+    mode: "post_call"
+    code: |
+      (processId, result) => {
+        console.log(`[${processId}] Response logged:`, typeof result);
+        return result;
+      }
+
+router_settings:
+  routing_strategy: "random"
+```
+
+Run with configuration:
+
+```bash
+llmjs2 --config config.yaml
+```
+
+```yaml
+
+```
+
+## Environment Variables
+
+The CLI uses the same environment variables as the library:
+
+```bash
+# Required: API Keys
+export OLLAMA_API_KEY=your_ollama_key
+export OPEN_ROUTER_API_KEY=your_openrouter_key
+
+# Optional: Default Models
+export OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
+export OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
+
+# Optional: Server Settings
+export PORT=3000
+export HOST=localhost
+```
+
+### Using .env Files
+
+Create a `.env` file in your project directory:
+
+```bash
+# .env
+OLLAMA_API_KEY=your_ollama_key_here
+OPEN_ROUTER_API_KEY=your_openrouter_key_here
+OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
+OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
+PORT=3000
+HOST=localhost
+```
+
+The CLI will automatically load `.env` files.
+
+## Example Configurations
+
+### Development Setup
+
+```yaml
+# dev-config.yaml
+model_list:
+  - model_name: fast-dev
+    llm_params:
+      model: openrouter/openrouter/free
+      api_key: os.environ/OPEN_ROUTER_API_KEY
+
+guardrails:
+  - name: "dev_logger"
+    mode: "post_call"
+    code: |
+      (processId, result) => {
+        console.log(`[DEV] ${processId}:`, typeof result);
+        return result;
+      }
+
+router_settings:
+  routing_strategy: "sequential"
+```
+
+```bash
+llmjs2 --config dev-config.yaml --port 3001
+```
+
+### Production Setup
+
+```yaml
+# prod-config.yaml
+model_list:
+  - model_name: premium
+    llm_params:
+      model: openrouter/openai/gpt-4-turbo-preview
+      api_key: os.environ/PROD_OPEN_ROUTER_API_KEY
+
+  - model_name: standard
+    llm_params:
+      model: ollama/minimax-m2.5:cloud
+      api_key: os.environ/PROD_OLLAMA_API_KEY
+
+guardrails:
+  - name: "content_filter"
+    mode: "pre_call"
+    code: |
+      (processId, input) => {
+        const { model, messages } = input;
+        const filteredMessages = messages.map(msg => ({
+          ...msg,
+          content: msg.content.replace(/badword/gi, '****')
+        }));
+        return { model, messages: filteredMessages };
+      }
+
+  - name: "rate_limiter"
+    mode: "pre_call"
+    code: |
+      (processId, input) => {
+        // Implement rate limiting logic
+        return input;
+      }
+
+router_settings:
+  routing_strategy: "random"
+```
+
+```bash
+llmjs2 --config dev-config.yaml --port 3001
+```
+
+### Production Setup
+
+```yaml
+# prod-config.yaml
+model_list:
+  - model_name: premium
+    llm_params:
+      model: openrouter/openai/gpt-4-turbo-preview
+      api_key: os.environ/PROD_OPEN_ROUTER_API_KEY
+
+   - model_name: standard
+    llm_params:
+      model: ollama/minimax-m2.5:cloud
+      api_key: os.environ/PROD_OLLAMA_API_KEY
+
+guardrails:
+  - name: "my-custom-processor"
+    mode: "pre_call" # Use 'post_call' to process the LLM's response
+    code: |
+      (processId, input) => {
+        // Filter inappropriate content before LLM call
+        const { model, messages } = input;
+        const filteredMessages = messages.map(msg => ({
+          ...msg,
+          content: msg.content.replace(/badword/gi, '****')
+        }));
+        return { model, messages: filteredMessages };
+      }
+router_settings:
+  routing_strategy: "random"         
+```
+
+### Multi-Environment Setup
+
+```yaml
+# config.yaml
+model_list:
+  - model_name: default
+    llm_params:
+      model: ollama/minimax-m2.5:cloud
+      api_key: os.environ/OLLAMA_API_KEY
+
+  - model_name: premium
+    llm_params:
+      model: openrouter/openai/gpt-4
+      api_key: os.environ/OPEN_ROUTER_API_KEY
+
+guardrails:
+  - name: "environment_logger"
+    mode: "post_call"
+    code: |
+      (processId, result) => {
+        console.log(`[${process.env.NODE_ENV || 'development'}] ${processId}:`, typeof result);
+        return result;
+      }
+
+router_settings:
+  routing_strategy: "random"
+```
+
+## Testing Your Configuration
+
+### Test with cURL
+
+```bash
+# Start server
+llmjs2 --config config.yaml --port 3001
+
+# Test direct model routing
+curl -X POST http://localhost:3001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "default",
+    "messages": [{"role": "user", "content": "Hello!"}]
+  }'
+
+# Test automatic routing (uses router strategy from config)
+curl -X POST http://localhost:3001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "messages": [{"role": "user", "content": "Hello!"}]
+  }'
+```
+
+### Test Specific Models
+
+```bash
+# Test premium model
+curl -X POST http://localhost:3001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer test-key" \
+  -d '{
+    "model": "premium",
+    "messages": [{"role": "user", "content": "Hello premium!"}]
+  }'
+```
+
+## Advanced CLI Usage
+
+### Running in Background
+
+```bash
+# Linux/Mac
+llmjs2 --config config.yaml > server.log 2>&1 &
+
+# Windows (PowerShell)
+Start-Job -ScriptBlock { llmjs2 --config config.yaml } -Name llmjs2-server
+```
+
+### Process Management
+
+```bash
+# Find process
+ps aux | grep llmjs2
+
+# Kill process (replace PID)
+kill PID
+
+# Or use pkill
+pkill -f llmjs2
+```
+
+### Systemd Service (Linux)
+
+Create `/etc/systemd/system/llmjs2.service`:
+
+```ini
+[Unit]
+Description=llmjs2 API Server
+After=network.target
+
+[Service]
+Type=simple
+User=your-user
+WorkingDirectory=/path/to/your/app
+ExecStart=/usr/bin/llmjs2 --config config.yaml
+Restart=always
+RestartSec=10
+Environment=OLLAMA_API_KEY=your_key
+Environment=OPEN_ROUTER_API_KEY=your_key
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Enable and start:
+
+```bash
+sudo systemctl enable llmjs2
+sudo systemctl start llmjs2
+sudo systemctl status llmjs2
+```
+
+## Troubleshooting CLI Issues
+
+### Configuration File Errors
+
+**YAML syntax error:**
+
+```yaml
+# Check your YAML syntax
+yamllint config.yaml
+```
+
+**File not found:**
+
+```bash
+# Check file exists and path is correct
+ls -la config.yaml
+llmjs2 --config ./config.yaml
+```
+
+### Environment Variable Issues
+
+**API key not found:**
+
+```bash
+# Check environment variables are set
+echo $OLLAMA_API_KEY
+echo $OPEN_ROUTER_API_KEY
+
+# Or use a .env file
+echo "OLLAMA_API_KEY=your_key" > .env
+```
+
+### Port Issues
+
+**Port already in use:**
+
+```bash
+# Check what's using the port
+lsof -i :3000
+
+# Use a different port
+llmjs2 --port 8080
+```
+
+### Permission Issues
+
+**Cannot bind to port:**
+
+```bash
+# Try ports above 1024
+llmjs2 --port 8080
+
+# Or run with sudo (not recommended for production)
+sudo llmjs2 --port 80
+```
+
+## Next Steps
+
+- **[Get Started](GET_STARTED.md)** - Basic setup and first steps
+- **[Basic Usage](BASIC_USAGE.md)** - Learn different API patterns
+- **[Server Mode](SERVER_MODE.md)** - Advanced server configuration
+
+The CLI makes it easy to deploy and manage llmjs2 servers in any environment!

package/docs/GET_STARTED.md

@@ -0,0 +1,129 @@
+# Getting Started with llmjs2
+
+Welcome to llmjs2! This guide will get you up and running in just 5 minutes.
+
+## Prerequisites
+
+Before you begin, make sure you have:
+
+- Node.js version 14.0.0 or higher
+- API keys for at least one LLM provider (Ollama or OpenRouter)
+
+## Installation
+
+### Option 1: Global Installation (Recommended)
+
+Install llmjs2 globally to use it from anywhere:
+
+```bash
+npm install -g llmjs2
+```
+
+### Option 2: Local Installation
+
+Install in your project directory:
+
+```bash
+npm install llmjs2
+```
+
+## Quick Setup
+
+### Step 1: Set API Keys
+
+Set your API keys as environment variables:
+
+```bash
+# For Ollama (get your key from https://ollama.com)
+export OLLAMA_API_KEY=your_ollama_api_key_here
+
+# For OpenRouter (get your key from https://openrouter.ai)
+export OPEN_ROUTER_API_KEY=your_openrouter_api_key_here
+```
+
+### Step 2: Your First Completion
+
+Create a simple test script:
+
+```javascript
+// hello-world.js
+import { completion } from 'llmjs2';
+
+async function helloWorld() {
+  try {
+    const response = await completion('Hello! Introduce yourself in one sentence.');
+    console.log('Response:', response);
+  } catch (error) {
+    console.error('Error:', error.message);
+  }
+}
+
+helloWorld();
+```
+
+Run it:
+
+```bash
+node hello-world.js
+```
+
+**Expected Output:**
+
+```
+Response: Hello! I'm an AI assistant powered by llmjs2, designed to help with various tasks and provide intelligent responses.
+```
+
+### Step 3: Test Different Models
+
+Try using specific models:
+
+```javascript
+// test-models.js
+import { completion } from 'llmjs2';
+
+async function testModels() {
+  try {
+    // Test Ollama
+    if (process.env.OLLAMA_API_KEY) {
+      const ollamaResponse = await completion('ollama/minimax-m2.5:cloud', 'What is AI?');
+      console.log('Ollama:', ollamaResponse);
+    }
+
+    // Test OpenRouter
+    if (process.env.OPEN_ROUTER_API_KEY) {
+      const openrouterResponse = await completion('openrouter/openrouter/free', 'What is machine learning?');
+      console.log('OpenRouter:', openrouterResponse);
+    }
+  } catch (error) {
+    console.error('Error:', error.message);
+  }
+}
+
+testModels();
+```
+
+## What's Next?
+
+Now that you have llmjs2 working, you can:
+
+1. **[Learn Basic Usage](BASIC_USAGE.md)** - Explore different API patterns and configuration options
+2. **[Run as a Server](SERVER_MODE.md)** - Set up an OpenAI-compatible API server
+3. **[Use the CLI](CLI.md)** - Manage servers and configurations from the command line
+
+## Supported Providers
+
+llmjs2 supports multiple LLM providers:
+
+- **Ollama** - Connect to Ollama's cloud API
+- **OpenRouter** - Access various models through OpenRouter
+
+## Need Help?
+
+If you encounter any issues:
+
+1. Check that your API keys are valid and have credits
+2. Ensure you're using the correct model format: `provider/model_name`
+3. Try the examples in this guide first
+4. Check your internet connection
+
+For more detailed usage, see the other guides in this documentation.