llmjs2 1.1.0 → 1.3.0

package/CONFIG_README.md

@@ -0,0 +1,98 @@
+# llmjs2 Configuration Guide
+
+This directory contains sample configuration files for testing the llmjs2 library.
+
+## Files
+
+- `config.yaml` - Comprehensive sample configuration with models, guardrails, and routing
+- `.env` - Sample environment variables (add your API keys here)
+- `validate-config.js` - Script to validate your configuration
+
+## Quick Start
+
+1. **Add your API keys** to `.env`:
+   ```bash
+   # Uncomment and add your keys
+   OPENAI_API_KEY=your_actual_openai_key
+   # OLLAMA_API_KEY and OPEN_ROUTER_API_KEY are already set for testing
+   ```
+
+2. **Validate the configuration**:
+   ```bash
+   node validate-config.js
+   ```
+
+3. **Start the server**:
+   ```bash
+   node cli.js --config config.yaml --port 3001
+   ```
+
+4. **Test the API**:
+   ```bash
+   curl -X POST http://localhost:3001/v1/chat/completions \
+     -H "Content-Type: application/json" \
+     -d '{"messages":[{"role":"user","content":"Hello!"}]}'
+   ```
+
+## Configuration Structure
+
+### Model List
+Defines available models and their providers:
+- `model_name`: Alias for routing (can have multiple providers)
+- `llm_params`: Provider-specific configuration
+  - `model`: Full model identifier in format `[provider]/[actual-model-name]` (e.g., `openai/gpt-4`, `ollama/minimax-m2.5:cloud`)
+  - `api_key`: API key (supports `os.environ/VAR_NAME` syntax)
+  - `api_base`: Optional custom API endpoint
+
+**Important**: The `[provider]/` prefix is used internally for routing. When requests are sent to LLM providers, only the `[actual-model-name]` part is used.
+
+### Guardrails
+Custom processing logic for requests/responses:
+- `name`: Unique identifier
+- `mode`: `pre_call` (before LLM) or `post_call` (after LLM)
+- `code`: JavaScript function as string
+
+### Router Settings
+- `routing_strategy`: `default`, `random`, or `sequential`
+
+## Testing Different Features
+
+### Load Balancing
+The config includes multiple models with the same `model_name` (like `free-model`) to demonstrate load balancing.
+
+### Guardrails
+Try sending requests with inappropriate content to see filtering in action:
+```bash
+curl -X POST http://localhost:3001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"messages":[{"role":"user","content":"This contains badword"}]}'
+```
+
+### Different Routing Strategies
+Edit `router_settings.routing_strategy` to test:
+- `random`: Random model selection
+- `sequential`: Cycle through models
+- `default`: Load balance same model names
+
+## Environment Variables
+
+The configuration supports environment variables:
+- `os.environ/VAR_NAME` syntax in YAML
+- `.env` file loaded automatically
+- Standard environment variables override defaults
+
+## Troubleshooting
+
+- **"Model not found"**: Check that API keys are set in `.env`
+- **"Rate limit exceeded"**: The config includes rate limiting (10 requests/minute)
+- **Server won't start**: Run `node validate-config.js` to check configuration
+- **API errors**: Check server logs for detailed error messages
+
+## Production Usage
+
+For production:
+1. Remove test API keys from `.env`
+2. Add your actual API keys
+3. Configure proper rate limiting and content filtering
+4. Set up proper logging and monitoring
+5. Use environment-specific config files (dev/prod)