llmjs2 1.3.8 → 1.6.1

package/docs/CLI.md

@@ -1,455 +0,0 @@
-# CLI Guide
-
-Use the llmjs2 command-line interface to manage servers and configurations without writing code.
-
-## Installation
-
-If you installed llmjs2 globally, the `llmjs2` command is available everywhere:
-
-```bash
-npm install -g llmjs2
-```
-
-## Basic Usage
-
-### Start Server with Defaults
-
-```bash
-llmjs2
-```
-
-This starts a server on `http://localhost:3000` using:
-
-- Environment variables for API keys
-- Default models
-- No configuration file
-
-### Custom Server Options
-
-```bash
-# Custom port
-llmjs2 --port 8080
-
-# Custom host
-llmjs2 --host 0.0.0.0
-
-# Custom port and host
-llmjs2 --port 8080 --host 0.0.0.0
-
-# Use configuration file
-llmjs2 --config my-config.yaml
-
-# Combine options
-llmjs2 --config my-config.yaml --port 8080 --host 0.0.0.0
-```
-
-### Get Help
-
-```bash
-llmjs2 --help
-```
-
-Output:
-
-```
-🤖 llmjs2 - OpenAI-Compatible API Server
-
-USAGE:
-  llmjs2 [options]
-
-DESCRIPTION:
-  Starts an OpenAI-compatible API server with intelligent routing and guardrails.
-  Supports model load balancing, content filtering, and custom request processing.
-  The server listens for POST requests to /v1/chat/completions.
-
-OPTIONS:
-  -c, --config <file>    YAML config file with models, guardrails, and routing
-  -p, --port <port>      Port to listen on (default: 3000)
-  -H, --host <host>      Host to bind to (default: localhost)
-  -h, --help            Show this help message
-
-EXAMPLES:
-  llmjs2
-  llmjs2 --config config.yaml
-  llmjs2 --config config.yaml --port 8080 --host 0.0.0.0
-```
-
-## Configuration Files
-
-Use YAML configuration files to define models, guardrails, and router settings for advanced LLM routing and processing.
-
-### Configuration with Guardrails
-
-Create `config.yaml` with models, guardrails, and routing settings:
-
-```yaml
-model_list:
-  - model_name: premium
-    llm_params:
-      model: openrouter/openai/gpt-4-turbo-preview
-      api_key: os.environ/PROD_OPEN_ROUTER_API_KEY
-
-  - model_name: standard
-    llm_params:
-      model: ollama/minimax-m2.5:cloud
-      api_key: os.environ/PROD_OLLAMA_API_KEY
-
-guardrails:
-  - name: "content_filter"
-    mode: "pre_call"
-    code: |
-      (processId, input) => {
-        // Filter inappropriate content before LLM call
-        const { model, messages } = input;
-        const filteredMessages = messages.map(msg => ({
-          ...msg,
-          content: msg.content.replace(/badword/gi, '****')
-        }));
-        return { model, messages: filteredMessages };
-      }
-
-  - name: "response_logger"
-    mode: "post_call"
-    code: |
-      (processId, result) => {
-        console.log(`[${processId}] Response logged:`, typeof result);
-        return result;
-      }
-
-router_settings:
-  routing_strategy: "random"
-```
-
-Run with configuration:
-
-```bash
-llmjs2 --config config.yaml
-```
-
-```yaml
-
-```
-
-## Environment Variables
-
-The CLI uses the same environment variables as the library:
-
-```bash
-# Required: API Keys
-export OLLAMA_API_KEY=your_ollama_key
-export OPEN_ROUTER_API_KEY=your_openrouter_key
-
-# Optional: Default Models
-export OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
-export OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
-
-# Optional: Server Settings
-export PORT=3000
-export HOST=localhost
-```
-
-### Using .env Files
-
-Create a `.env` file in your project directory:
-
-```bash
-# .env
-OLLAMA_API_KEY=your_ollama_key_here
-OPEN_ROUTER_API_KEY=your_openrouter_key_here
-OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
-OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
-PORT=3000
-HOST=localhost
-```
-
-The CLI will automatically load `.env` files.
-
-## Example Configurations
-
-### Development Setup
-
-```yaml
-# dev-config.yaml
-model_list:
-  - model_name: fast-dev
-    llm_params:
-      model: openrouter/openrouter/free
-      api_key: os.environ/OPEN_ROUTER_API_KEY
-
-guardrails:
-  - name: "dev_logger"
-    mode: "post_call"
-    code: |
-      (processId, result) => {
-        console.log(`[DEV] ${processId}:`, typeof result);
-        return result;
-      }
-
-router_settings:
-  routing_strategy: "sequential"
-```
-
-```bash
-llmjs2 --config dev-config.yaml --port 3001
-```
-
-### Production Setup
-
-```yaml
-# prod-config.yaml
-model_list:
-  - model_name: premium
-    llm_params:
-      model: openrouter/openai/gpt-4-turbo-preview
-      api_key: os.environ/PROD_OPEN_ROUTER_API_KEY
-
-  - model_name: standard
-    llm_params:
-      model: ollama/minimax-m2.5:cloud
-      api_key: os.environ/PROD_OLLAMA_API_KEY
-
-guardrails:
-  - name: "content_filter"
-    mode: "pre_call"
-    code: |
-      (processId, input) => {
-        const { model, messages } = input;
-        const filteredMessages = messages.map(msg => ({
-          ...msg,
-          content: msg.content.replace(/badword/gi, '****')
-        }));
-        return { model, messages: filteredMessages };
-      }
-
-  - name: "rate_limiter"
-    mode: "pre_call"
-    code: |
-      (processId, input) => {
-        // Implement rate limiting logic
-        return input;
-      }
-
-router_settings:
-  routing_strategy: "random"
-```
-
-```bash
-llmjs2 --config dev-config.yaml --port 3001
-```
-
-### Production Setup
-
-```yaml
-# prod-config.yaml
-model_list:
-  - model_name: premium
-    llm_params:
-      model: openrouter/openai/gpt-4-turbo-preview
-      api_key: os.environ/PROD_OPEN_ROUTER_API_KEY
-
-   - model_name: standard
-    llm_params:
-      model: ollama/minimax-m2.5:cloud
-      api_key: os.environ/PROD_OLLAMA_API_KEY
-
-guardrails:
-  - name: "my-custom-processor"
-    mode: "pre_call" # Use 'post_call' to process the LLM's response
-    code: |
-      (processId, input) => {
-        // Filter inappropriate content before LLM call
-        const { model, messages } = input;
-        const filteredMessages = messages.map(msg => ({
-          ...msg,
-          content: msg.content.replace(/badword/gi, '****')
-        }));
-        return { model, messages: filteredMessages };
-      }
-router_settings:
-  routing_strategy: "random"         
-```
-
-### Multi-Environment Setup
-
-```yaml
-# config.yaml
-model_list:
-  - model_name: default
-    llm_params:
-      model: ollama/minimax-m2.5:cloud
-      api_key: os.environ/OLLAMA_API_KEY
-
-  - model_name: premium
-    llm_params:
-      model: openrouter/openai/gpt-4
-      api_key: os.environ/OPEN_ROUTER_API_KEY
-
-guardrails:
-  - name: "environment_logger"
-    mode: "post_call"
-    code: |
-      (processId, result) => {
-        console.log(`[${process.env.NODE_ENV || 'development'}] ${processId}:`, typeof result);
-        return result;
-      }
-
-router_settings:
-  routing_strategy: "random"
-```
-
-## Testing Your Configuration
-
-### Test with cURL
-
-```bash
-# Start server
-llmjs2 --config config.yaml --port 3001
-
-# Test direct model routing
-curl -X POST http://localhost:3001/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "default",
-    "messages": [{"role": "user", "content": "Hello!"}]
-  }'
-
-# Test automatic routing (uses router strategy from config)
-curl -X POST http://localhost:3001/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "messages": [{"role": "user", "content": "Hello!"}]
-  }'
-```
-
-### Test Specific Models
-
-```bash
-# Test premium model
-curl -X POST http://localhost:3001/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer test-key" \
-  -d '{
-    "model": "premium",
-    "messages": [{"role": "user", "content": "Hello premium!"}]
-  }'
-```
-
-## Advanced CLI Usage
-
-### Running in Background
-
-```bash
-# Linux/Mac
-llmjs2 --config config.yaml > server.log 2>&1 &
-
-# Windows (PowerShell)
-Start-Job -ScriptBlock { llmjs2 --config config.yaml } -Name llmjs2-server
-```
-
-### Process Management
-
-```bash
-# Find process
-ps aux | grep llmjs2
-
-# Kill process (replace PID)
-kill PID
-
-# Or use pkill
-pkill -f llmjs2
-```
-
-### Systemd Service (Linux)
-
-Create `/etc/systemd/system/llmjs2.service`:
-
-```ini
-[Unit]
-Description=llmjs2 API Server
-After=network.target
-
-[Service]
-Type=simple
-User=your-user
-WorkingDirectory=/path/to/your/app
-ExecStart=/usr/bin/llmjs2 --config config.yaml
-Restart=always
-RestartSec=10
-Environment=OLLAMA_API_KEY=your_key
-Environment=OPEN_ROUTER_API_KEY=your_key
-
-[Install]
-WantedBy=multi-user.target
-```
-
-Enable and start:
-
-```bash
-sudo systemctl enable llmjs2
-sudo systemctl start llmjs2
-sudo systemctl status llmjs2
-```
-
-## Troubleshooting CLI Issues
-
-### Configuration File Errors
-
-**YAML syntax error:**
-
-```yaml
-# Check your YAML syntax
-yamllint config.yaml
-```
-
-**File not found:**
-
-```bash
-# Check file exists and path is correct
-ls -la config.yaml
-llmjs2 --config ./config.yaml
-```
-
-### Environment Variable Issues
-
-**API key not found:**
-
-```bash
-# Check environment variables are set
-echo $OLLAMA_API_KEY
-echo $OPEN_ROUTER_API_KEY
-
-# Or use a .env file
-echo "OLLAMA_API_KEY=your_key" > .env
-```
-
-### Port Issues
-
-**Port already in use:**
-
-```bash
-# Check what's using the port
-lsof -i :3000
-
-# Use a different port
-llmjs2 --port 8080
-```
-
-### Permission Issues
-
-**Cannot bind to port:**
-
-```bash
-# Try ports above 1024
-llmjs2 --port 8080
-
-# Or run with sudo (not recommended for production)
-sudo llmjs2 --port 80
-```
-
-## Next Steps
-
-- **[Get Started](GET_STARTED.md)** - Basic setup and first steps
-- **[Basic Usage](BASIC_USAGE.md)** - Learn different API patterns
-- **[Server Mode](SERVER_MODE.md)** - Advanced server configuration
-
-The CLI makes it easy to deploy and manage llmjs2 servers in any environment!