llmjs2 1.3.9 → 1.7.1

package/core/README.md

@@ -0,0 +1,485 @@
+# llmjs2
+
+A unified Node.js library for connecting to multiple Large Language Model (LLM) providers: OpenAI, Ollama, and OpenRouter.
+
+**Features:**
+- **Unified API**: Single interface for OpenAI, Ollama, and OpenRouter
+- **Intelligent Router**: Load balancing and multiple routing strategies
+- **Guardrails System**: Content filtering, logging, rate limiting, and custom processing
+- **OpenAI-Compatible Server**: OpenAI-style chat completions schema
+- **CLI Interface**: Command-line server management with configuration files
+- **Enterprise Security**: Input validation, error sanitization, and safe defaults
+- **Lightweight Dependencies**: Uses Node.js core modules plus YAML parsing
+
+## Features
+
+- **Unified API**: Single interface for OpenAI, Ollama, and OpenRouter
+- **Auto-detection**: Automatically chooses available providers based on API keys
+- **Enterprise-grade**: Robust error handling, input validation, and security measures
+- **Minimal dependencies**: Uses only one external dependency (`yaml`)
+- **TypeScript-free**: Pure JavaScript, no compilation required
+- **Production-ready**: Comprehensive testing and security auditing
+
+## Installation
+
+```bash
+npm install llmjs2
+```
+
+Or for global CLI usage:
+
+```bash
+npm install -g llmjs2
+```
+
+## Quick Test
+
+Try the sample configuration:
+
+```bash
+# Start server with sample config
+llmjs2 --config config.yaml --port 3001
+
+# Test the API
+curl -X POST http://localhost:3001/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"messages":[{"role":"user","content":"Hello!"}]}'
+
+# Response format:
+# {
+#   "id": "chatcmpl-123456",
+#   "object": "chat.completion",
+#   "created": 1640995200,
+#   "model": "ollama/minimax-m2.5:cloud",
+#   "messages": [
+#     {"role": "user", "content": "Hello!"},
+#     {"role": "assistant", "content": "Hi there!"}
+#   ]
+# }
+
+## Programmatic Configuration
+
+For advanced users, you can configure llmjs2 router programmatically instead of using YAML:
+
+```bash
+npm run router:example
+```
+
+See `core/docs/ROUTER_GUIDE.md` for complete routing examples and guardrail configuration.
+
+## AI Chat App
+
+Experience llmjs2 with a simple terminal-based chat interface:
+
+```bash
+npm run chat
+```
+
+Features:
+- Conversational chat with message history
+- Automatic model routing (random selection)
+- Shows which model was used for each response
+- Simple guardrails (logging)
+- Graceful exit with "exit", "quit", or "bye"
+
+The chat app uses the same router configuration as the programmatic examples but provides an interactive chat experience.
+
+See `core/docs/CLI.md` and `core/docs/SERVER_MODE.md` for configuration and server examples.
+
+## Quick Start
+
+```javascript
+import { completion } from 'llmjs2';
+
+// Set API keys
+process.env.OPENAI_API_KEY = 'your-openai-key';
+process.env.OLLAMA_API_KEY = 'your-ollama-key';
+process.env.OPEN_ROUTER_API_KEY = 'your-openrouter-key';
+
+// Simple completion
+const response = await completion('Hello, how are you?');
+console.log(response);
+```
+
+## API Keys Setup
+
+Set your API keys as environment variables:
+
+```bash
+export OPENAI_API_KEY=your_openai_api_key
+export OLLAMA_API_KEY=your_ollama_api_key
+export OPEN_ROUTER_API_KEY=your_openrouter_api_key
+```
+
+## Usage Patterns
+
+### 1. Simple API (Auto-detection)
+
+```javascript
+import { completion } from 'llmjs2';
+
+// Auto-selects provider based on available API keys
+// Cycles through: Ollama → OpenRouter → OpenAI → Ollama...
+const response = await completion('Explain quantum physics simply');
+```
+
+**Auto-Selection Logic:**
+- Checks for `OLLAMA_API_KEY`, `OPEN_ROUTER_API_KEY`, `OPENAI_API_KEY`
+- Cycles sequentially through available providers
+- Uses default model for each provider
+- Falls back gracefully if keys are missing
+
+### 2. Provider-Specific Model
+
+```javascript
+// OpenAI
+const openaiResponse = await completion({
+  model: 'openai/gpt-4',
+  messages: [{ role: 'user', content: 'Write a haiku about coding' }]
+});
+
+// Ollama
+const ollamaResponse = await completion({
+  model: 'ollama/minimax-m2.5:cloud',
+  messages: [{ role: 'user', content: 'What is AI?' }]
+});
+
+// OpenRouter
+const openrouterResponse = await completion({
+  model: 'openrouter/openrouter/free',
+  messages: [{ role: 'user', content: 'Tell me a joke' }]
+});
+```
+
+### 3. Advanced Object API
+
+```javascript
+const response = await completion({
+  model: 'openai/gpt-3.5-turbo',
+  messages: [
+    { role: 'system', content: 'You are a helpful assistant.' },
+    { role: 'user', content: 'What is the capital of France?' }
+  ],
+  temperature: 0.7,
+  maxTokens: 100
+});
+```
+
+## Configuration
+
+### Default Models
+
+Set default models for each provider:
+
+```bash
+export OPENAI_DEFAULT_MODEL=gpt-4
+export OLLAMA_DEFAULT_MODEL=minimax-m2.5:cloud
+export OPEN_ROUTER_DEFAULT_MODEL=openrouter/free
+```
+
+### Base URLs
+
+Customize API endpoints:
+
+```bash
+export OPENAI_BASE_URL=https://api.openai.com/v1
+export OLLAMA_BASE_URL=https://ollama.com/api/chat
+export OPEN_ROUTER_BASE_URL=https://openrouter.ai/api/v1/chat/completions
+```
+
+## Error Handling
+
+```javascript
+import { completion } from 'llmjs2';
+
+try {
+  const response = await completion('Tell me a joke');
+  console.log(response);
+} catch (error) {
+  console.error('Error:', error.message);
+}
+```
+
+## Conversations
+
+```javascript
+import { completion } from 'llmjs2';
+
+const messages = [
+  { role: 'system', content: 'You are a helpful coding assistant.' },
+  { role: 'user', content: 'How do I reverse a string in JavaScript?' }
+];
+
+let response = await completion({ model: 'openai/gpt-4', messages });
+console.log('Assistant:', response);
+
+// Continue conversation
+messages.push({ role: 'assistant', content: response });
+messages.push({ role: 'user', content: 'Can you show me with an example?' });
+
+response = await completion({ model: 'openai/gpt-4', messages });
+console.log('Assistant:', response);
+```
+
+## Function Calling (Tools)
+
+```javascript
+import { completion } from 'llmjs2';
+
+const weatherTool = {
+  type: 'function',
+  function: {
+    name: 'get_weather',
+    description: 'Get current weather for a location',
+    parameters: {
+      type: 'object',
+      properties: {
+        location: { type: 'string', description: 'City name' }
+      },
+      required: ['location']
+    }
+  }
+};
+
+const response = await completion({
+  model: 'openai/gpt-4',
+  messages: [{ role: 'user', content: 'What is the weather in Paris?' }],
+  tools: [weatherTool]
+});
+
+if (response.tool_calls) {
+  console.log('Tool calls:', response.tool_calls);
+}
+```
+
+## Router System
+
+Intelligent model routing with load balancing and multiple strategies:
+
+```javascript
+import { router } from 'llmjs2';
+
+const modelList = [
+  {
+    model_name: 'gpt-3.5-turbo',
+    llm_params: {
+      model: 'ollama/chatgpt-v-2',
+      api_key: process.env.OLLAMA_API_KEY
+    }
+  },
+  {
+    model_name: 'gpt-3.5-turbo',
+    llm_params: {
+      model: 'openai/gpt-3.5-turbo',
+      api_key: process.env.OPENAI_API_KEY
+    }
+  }
+];
+
+// Load balancing across models with same name
+const route = router(modelList, 'random');
+const response = await route.completion({
+  model: 'gpt-3.5-turbo',
+  messages: [{ role: 'user', content: 'Hello!' }]
+});
+
+// Auto-routing with different strategies
+const randomRouter = router(modelList, 'random');
+const sequentialRouter = router(modelList, 'sequential');
+```
+
+**Routing Strategies:**
+- `default`: Load balance across models with same name
+- `random`: Randomly select from all models
+- `sequential`: Cycle through models in order
+
+## Guardrails System
+
+Add custom logic before and after LLM calls for content filtering, logging, and processing:
+
+```javascript
+import { router } from 'llmjs2';
+
+const route = router(modelList);
+
+route.setGuardrails([
+  {
+    name: 'content_filter',
+    mode: 'pre_call',
+    code: (processId, input) => {
+      // Filter inappropriate content
+      const filteredMessages = input.messages.map(msg => ({
+        ...msg,
+        content: msg.content.replace(/badword/gi, '****')
+      }));
+      return { ...input, messages: filteredMessages };
+    }
+  },
+  {
+    name: 'response_logger',
+    mode: 'post_call',
+    code: (processId, result) => {
+      console.log(`[${processId}] Response:`, result);
+      return result;
+    }
+  }
+]);
+```
+
+## Server Mode
+
+Run an API server that returns responses with metadata and message arrays:
+
+```javascript
+import { router, app } from 'llmjs2';
+
+const route = router(modelList);
+app.use(route);
+app.listen(3000);
+```
+
+Or use the CLI:
+
+```bash
+llmjs2 --config config.yaml --port 3000
+```
+
+## CLI Interface
+
+Manage servers from the command line:
+
+```bash
+# Start server with defaults
+llmjs2
+
+# Use configuration file
+llmjs2 --config config.yaml
+
+# Custom port and host
+llmjs2 --port 8080 --host 0.0.0.0
+
+# Get help
+llmjs2 --help
+```
+
+## Configuration Files
+
+Use YAML for advanced configuration:
+
+```yaml
+model_list:
+  - model_name: premium
+    llm_params:
+      model: openrouter/openai/gpt-4
+      api_key: os.environ/OPEN_ROUTER_API_KEY
+
+  - model_name: standard
+    llm_params:
+      model: ollama/minimax-m2.5:cloud
+      api_key: os.environ/OLLAMA_API_KEY
+
+guardrails:
+  - name: content_filter
+    mode: pre_call
+    code: |
+      (processId, input) => {
+        // Content filtering logic
+        return input;
+      }
+
+router_settings:
+  routing_strategy: random
+```
+
+**Note**: Model names in the configuration use the format `[provider]/[actual-model-name]` (e.g., `openai/gpt-4`, `ollama/minimax-m2.5:cloud`). The `[provider]/` prefix is used for routing and is automatically stripped when sending requests to LLM providers.
+
+## Security Features
+
+- **No API key logging**: Sensitive information is never logged
+- **Input validation**: All inputs are validated and sanitized
+- **Error sanitization**: Error messages don't leak sensitive data
+- **Timeout protection**: Requests timeout to prevent hanging
+- **HTTPS only**: All communications use HTTPS
+
+## Testing
+
+Run the test suite:
+
+```bash
+npm test
+```
+
+Test basic completion functionality:
+
+```bash
+npm run test:completion
+```
+
+## Logging
+
+Configure logging levels for LLM provider requests and responses:
+
+```bash
+# Show all logs (DEBUG, WARN, INFO, ERROR)
+LLMJS2_LOG=debug node your-script.js
+
+# Show INFO and ERROR logs only
+LLMJS2_LOG=info node your-script.js
+
+# Show WARN, INFO, and ERROR logs
+LLMJS2_LOG=warn node your-script.js
+
+# Show ERROR logs only
+LLMJS2_LOG=error node your-script.js
+
+# Examples
+LLMJS2_LOG=debug npm run chat
+LLMJS2_LOG=info npm run test:completion
+```
+
+### Log Levels
+
+- **DEBUG**: All logs including detailed request/response data
+- **INFO**: LLM provider communication and important events
+- **WARN**: Warnings and important notifications
+- **ERROR**: Errors and failures
+
+### Log Format
+
+```
+[TIMESTAMP] [LEVEL] MESSAGE
+DATA_OBJECT (JSON formatted)
+```
+
+### Example Output
+
+```
+[2026-04-13T10:14:58.123Z] [INFO] LLMJS2 📤 Sending to LLM provider
+{
+  "source": "completion",
+  "provider": "ollama",
+  "model": "minimax-m2.5:cloud",
+  "apiKey": "2620e31dea...",
+  "messages": [{"role": "user", "content": "Hello!"}]
+}
+
+[2026-04-13T10:15:01.456Z] [INFO] LLMJS2 📥 Received from LLM provider
+{
+  "source": "completion",
+  "content": "Hello! How can I help you?",
+  "role": "assistant",
+  "usage": {"prompt_eval_count": 10, "eval_count": 15}
+}
+```
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions welcome! Please ensure all tests pass and add tests for new features.
+
+## Support
+
+For issues and questions, please create an issue on GitHub.