lynkr 7.2.5 → 8.0.1

package/PLAN-per-client-model-routing.md

@@ -1,252 +0,0 @@
-# Per-Client Model Routing Implementation Plan
-
-## Current State
-
-### How Model Routing Works Now
-
-1. **Global Provider**: Set via `MODEL_PROVIDER` in `.env`
-2. **No Per-Client Routing**: All clients use the same provider
-3. **Smart Routing**: Only when `PREFER_OLLAMA=true` (routes based on tool count/complexity)
-4. **No Client-Side Indication**: Clients don't know which model was used
-
-### Current Configuration Options
-
-| Config | Purpose | Example Values |
-|--------|---------|-----------------|
-| `MODEL_PROVIDER` | Primary provider | databricks, ollama, openrouter, azure-openai, bedrock |
-| `PREFER_OLLAMA` | Enable smart routing | true/false |
-| `FALLBACK_ENABLED` | Allow fallback to cloud | true/false (default: true) |
-| `FALLBACK_PROVIDER` | Cloud fallback provider | databricks, azure-anthropic, bedrock |
-| `OLLAMA_MAX_TOOLS_FOR_ROUTING` | Tool threshold for Ollama | 3 (default) |
-
----
-
-## Implementation Plan
-
-### Phase 1: Add Client-to-Provider Mapping Config
-
-**File**: `.env`
-
-```bash
-# Per-client provider mapping (new)
-CLIENT_PROVIDER_MAP=claude-code:ollama,codex:azure-openai,cline:openrouter,kilo:zai
-CLIENT_PROVIDER_DEFAULT=azure-openai
-```
-
-**File**: `src/config/index.js`
-
-```javascript
-// Add helper function
-function parseClientProviderMap(envValue) {
-  if (!envValue) return {};
-  const map = {};
-  envValue.split(',').forEach(pair => {
-    const [client, provider] = pair.split(':').map(s => s.trim().toLowerCase());
-    if (client && provider) {
-      map[client] = provider;
-    }
-  });
-  return map;
-}
-
-// Add to config object
-clientProviderMap: parseClientProviderMap(process.env.CLIENT_PROVIDER_MAP),
-clientProviderDefault: process.env.CLIENT_PROVIDER_DEFAULT || process.env.MODEL_PROVIDER || 'azure-openai',
-```
-
----
-
-### Phase 2: Detect Client & Override Provider
-
-**File**: `src/orchestrator/index.js` (around line 769)
-
-```javascript
-// Add at top of file
-const VALID_PROVIDERS = new Set([
-  'databricks', 'azure-anthropic', 'azure-openai', 'openrouter',
-  'openai', 'ollama', 'llamacpp', 'lmstudio', 'bedrock', 'zai', 'vertex'
-]);
-
-// Add helper function
-function detectClientType(headers) {
-  const userAgent = (headers?.["user-agent"] || "").toLowerCase();
-  const clientHeader = (headers?.["x-client"] || headers?.["x-client-name"] || "").toLowerCase();
-
-  if (userAgent.includes("claude") || clientHeader.includes("claude")) return "claude-code";
-  if (userAgent.includes("codex") || clientHeader.includes("codex")) return "codex";
-  if (userAgent.includes("kilo") || clientHeader.includes("kilo")) return "kilo";
-  if (userAgent.includes("cline") || clientHeader.includes("cline")) return "cline";
-  if (userAgent.includes("continue") || clientHeader.includes("continue")) return "continue";
-  if (userAgent.includes("cursor") || clientHeader.includes("cursor")) return "cursor";
-  if (userAgent.includes("windsurf") || clientHeader.includes("windsurf")) return "windsurf";
-
-  return "unknown";
-}
-
-function getProviderForClient(headers, defaultProvider) {
-  const clientType = detectClientType(headers);
-  const clientMap = config.clientProviderMap || {};
-
-  // 1. Check for explicit header override
-  const headerOverride = headers?.["x-model-provider"]?.toLowerCase();
-  if (headerOverride && VALID_PROVIDERS.has(headerOverride)) {
-    logger.info({ clientType, headerOverride }, "Using header-specified provider");
-    return headerOverride;
-  }
-
-  // 2. Check client-specific mapping from config
-  if (clientMap[clientType]) {
-    logger.info({ clientType, provider: clientMap[clientType] }, "Using client-mapped provider");
-    return clientMap[clientType];
-  }
-
-  // 3. Fall back to default
-  logger.debug({ clientType, defaultProvider }, "Using default provider");
-  return config.clientProviderDefault || defaultProvider;
-}
-
-// Replace current static provider selection:
-// OLD: const providerType = config.modelProvider?.type ?? "databricks";
-// NEW:
-const providerType = getProviderForClient(headers, config.modelProvider?.type ?? "databricks");
-```
-
----
-
-### Phase 3: Add Response Header for Model Indication
-
-**File**: `src/api/openai-router.js`
-
-In the streaming response section, add headers:
-
-```javascript
-// After res.setHeader("Content-Type", "text/event-stream");
-res.setHeader("X-Lynkr-Provider", providerType);
-res.setHeader("X-Lynkr-Client-Detected", clientType);
-```
-
-**File**: `src/api/openai-router.js` (in /v1/responses endpoint)
-
-In the response.completed event, add metadata:
-
-```javascript
-const completedEvent = {
-  type: "response.completed",
-  response: {
-    id: responseId,
-    object: "response",
-    status: "completed",
-    // ... existing fields
-
-    // NEW: Add provider metadata
-    _lynkr: {
-      provider: providerType,
-      client_detected: clientType,
-      model_actual: actualModelUsed
-    }
-  },
-  sequence_number: sequenceNumber++
-};
-```
-
----
-
-### Phase 4: Hot Reload Support
-
-**File**: `src/config/index.js`
-
-```javascript
-reloadConfig() {
-  // ... existing reload logic
-
-  // Add client provider map reload
-  this.clientProviderMap = parseClientProviderMap(process.env.CLIENT_PROVIDER_MAP);
-  this.clientProviderDefault = process.env.CLIENT_PROVIDER_DEFAULT || process.env.MODEL_PROVIDER;
-
-  logger.info({
-    clientProviderMap: this.clientProviderMap,
-    clientProviderDefault: this.clientProviderDefault
-  }, "Client provider mapping reloaded");
-}
-```
-
----
-
-### Phase 5: Logging & Observability
-
-**File**: `src/orchestrator/index.js`
-
-Add logging for routing decisions:
-
-```javascript
-logger.info({
-  sessionId: session?.id,
-  clientType,
-  requestedModel: payload.model,
-  resolvedProvider: providerType,
-  headerOverride: headers?.["x-model-provider"],
-  configuredMapping: config.clientProviderMap[clientType]
-}, "Model routing decision");
-```
-
----
-
-## Files to Modify
-
-| File | Changes |
-|------|---------|
-| `.env` | Add `CLIENT_PROVIDER_MAP`, `CLIENT_PROVIDER_DEFAULT` |
-| `src/config/index.js` | Parse new config, add helper function, add to exports |
-| `src/orchestrator/index.js` | Add `getProviderForClient()`, `detectClientType()`, use headers |
-| `src/api/openai-router.js` | Add `X-Lynkr-Provider` response header, add `_lynkr` metadata |
-
----
-
-## Example Configuration
-
-```bash
-# .env
-
-# Default provider (used if no client mapping matches)
-MODEL_PROVIDER=azure-openai
-CLIENT_PROVIDER_DEFAULT=azure-openai
-
-# Per-client provider mapping
-# Format: client1:provider1,client2:provider2,...
-CLIENT_PROVIDER_MAP=claude-code:ollama,codex:azure-openai,cline:openrouter,kilo:zai,cursor:databricks
-
-# Individual provider configs
-OLLAMA_MODEL=qwen2.5-coder:7b
-AZURE_OPENAI_DEPLOYMENT=gpt-4o
-OPENROUTER_MODEL=anthropic/claude-3.5-sonnet
-ZAI_MODEL=GLM-4.7
-```
-
-**Result**:
-- Claude Code → Ollama (local, fast, free)
-- Codex CLI → Azure OpenAI (GPT-4o/GPT-5)
-- Cline → OpenRouter (Claude 3.5 Sonnet)
-- Kilo Code → Z.AI (GLM-4.7, cheap)
-- Cursor → Databricks (Claude via Databricks)
-- Unknown → Azure OpenAI (default)
-
----
-
-## Testing
-
-1. Start server with new config
-2. Make request from Claude Code → should route to Ollama
-3. Check logs for "Model routing decision"
-4. Check response header `X-Lynkr-Provider`
-5. Make request from Codex → should route to Azure OpenAI
-6. Test header override: `curl -H "X-Model-Provider: ollama" ...`
-
----
-
-## Future Enhancements
-
-1. **Per-Model Routing**: Route based on requested model name, not just client
-2. **Cost-Based Routing**: Route to cheapest provider that supports the request
-3. **Latency-Based Routing**: Route to fastest available provider
-4. **Load Balancing**: Distribute across multiple providers
-5. **UI Dashboard**: Visual config for routing rules

package/ROUTER_COMPARISON.md

@@ -1,173 +0,0 @@
-# Comparison: claude-code-router vs Lynkr Proxy
-
-## Architecture Differences
-
-**claude-code-router:**
-- **CLI-first design** - `ccr` commands for interactive model switching
-- **Request interceptor** - Sits between Claude Code CLI and LLM providers
-- **Transformer pipeline** - Middleware system for request/response modification
-- **Built with Fastify** (web framework)
-- **TypeScript + esbuild** compilation
-- **Web UI** for configuration
-
-**Lynkr:**
-- **HTTP proxy server** - Express-based API endpoint
-- **Provider abstraction** - Unified interface for 7+ providers
-- **Long-term memory system** (Titans-inspired)
-- **Built with Express** (web framework)
-- **Pure JavaScript** (no compilation)
-- **Token optimization focus** (6 optimization phases)
-
----
-
-## Key Feature Comparison
-
-| Feature | claude-code-router | Lynkr | Winner |
-|---------|-------------------|-------|--------|
-| **Dynamic Model Switching** | ✅ Runtime `/model` command | ❌ Static .env config | 🏆 Router |
-| **Routing Logic** | ✅ Context-aware (think/background/long-context) | ❌ Simple provider fallback only | 🏆 Router |
-| **Custom Router Scripts** | ✅ JavaScript-based routing rules | ❌ No custom routing | 🏆 Router |
-| **Web UI** | ✅ `ccr ui` browser interface | ❌ No UI | 🏆 Router |
-| **Long-Term Memory** | ❌ None | ✅ Vector search + surprise scoring | 🏆 Lynkr |
-| **Token Optimization** | ⚠️ Basic (long-context detection) | ✅ 6 phases (smart tools, compression, etc.) | 🏆 Lynkr |
-| **Smart Tool Selection** | ❌ None | ✅ Heuristic-based (just implemented) | 🏆 Lynkr |
-| **History Compression** | ❌ None | ✅ Automatic + token budget enforcement | 🏆 Lynkr |
-| **Prompt Caching** | ✅ Via transformer | ✅ Built-in | 🟰 Tie |
-| **Provider Count** | 6 (OpenRouter, DeepSeek, Ollama, Gemini, etc.) | 7 (Databricks, Azure, OpenAI, OpenRouter, Ollama, llama.cpp) | 🟰 Tie |
-| **Tool Enhancement** | ✅ `enhancetool` transformer | ❌ Basic passthrough | 🏆 Router |
-| **GitHub Actions** | ✅ CI/CD integration | ❌ None | 🏆 Router |
-| **Logging** | ✅ Rotating file logs | ✅ Pino logger | 🟰 Tie |
-| **TypeScript** | ✅ Full TypeScript | ❌ JavaScript only | 🏆 Router |
-
----
-
-## Improvements for Lynkr (Ranked by Impact)
-
-### 🔴 **Critical - High Impact, High Value**
-
-#### 1. Dynamic Model Switching via `/model` Command
-- **What**: Allow users to switch models mid-conversation without restarting server
-- **Why**: Router's killer feature - flexibility without configuration edits
-- **Implementation**: Add chat command parser, session-level model overrides
-- **Effort**: Medium (2-3 days)
-
-#### 2. Context-Aware Routing (Background/Think/Long-Context)
-- **What**: Automatically route requests based on context type
-- **Why**: Cost optimization + performance (cheap models for background, reasoning models for planning)
-- **Example**:
-  - Background tasks → `gpt-4o-mini` ($0.15/1M)
-  - Planning/thinking → `o1-preview` (reasoning model)
-  - Long context (>60k tokens) → `claude-sonnet-4` (200k context)
-- **Effort**: Medium (3-4 days)
-
-#### 3. Custom Router Scripts (JavaScript-based)
-- **What**: Let users define routing logic in JavaScript
-- **Why**: Ultimate flexibility - enterprise users need custom rules
-- **Example**:
-  ```javascript
-  // router.js
-  module.exports = function(request) {
-    if (request.tools.length > 5) return 'gpt-4o'; // Complex task
-    if (request.content.includes('urgent')) return 'databricks'; // Fast provider
-    return 'openrouter/nova-lite'; // Default cheap
-  }
-  ```
-- **Effort**: High (5-7 days)
-
-#### 4. Web UI for Configuration
-- **What**: Browser-based interface at `http://localhost:8081/ui`
-- **Why**: Non-technical users can't edit .env files
-- **Features**: Model selection, provider config, logs viewer, cost tracking
-- **Effort**: High (7-10 days)
-
-### 🟡 **High Impact, Medium Complexity**
-
-#### 5. Tool Enhancement Transformer
-- **What**: Add error tolerance and response buffering to tool calls
-- **Why**: Prevents cascade failures when tools return malformed JSON
-- **Example**: Retry tool calls with exponential backoff, validate tool outputs
-- **Effort**: Low (1-2 days)
-
-#### 6. Request/Response Transformer Pipeline
-- **What**: Middleware system to modify requests/responses per provider
-- **Why**: Provider-specific quirks (Azure needs different format, Ollama strips thinking blocks)
-- **Current**: Hardcoded in client adapters
-- **Improved**: Pluggable transformer chain
-- **Effort**: Medium (3-4 days)
-
-#### 7. Token-Based Auto-Routing
-- **What**: Switch to high-context models when input exceeds threshold
-- **Why**: Prevent truncation errors, automatic upgrade
-- **Example**: Request >100k tokens → auto-switch from `gpt-4o` (128k) to `claude-sonnet-4` (200k)
-- **Effort**: Low (1-2 days) - you already have token counting
-
-### 🟢 **Nice to Have - Lower Priority**
-
-#### 8. GitHub Actions Integration
-- **What**: Trigger Claude Code workflows in CI/CD
-- **Why**: Automated code reviews, documentation generation
-- **Use Case**: PR opens → Claude reviews code → posts comments
-- **Effort**: Medium (3-4 days)
-
-#### 9. CLI Commands (`lynkr model`, `lynkr ui`)
-- **What**: Interactive terminal commands for management
-- **Why**: Better DX than editing .env and restarting
-- **Effort**: Medium (2-3 days)
-
-#### 10. Rotating File Logs
-- **What**: Auto-rotate logs by size/date (keep last 7 days)
-- **Why**: Prevent disk bloat in production
-- **Current**: Pino logs to stdout only
-- **Effort**: Low (1 day) - use `pino-rotating-file-stream`
-
-#### 11. LRU Caching for Responses
-- **What**: Cache identical requests for X minutes
-- **Why**: Save money on repeated queries
-- **Example**: User asks "what is 2+2?" 3 times → only 1 LLM call
-- **Effort**: Low (1-2 days)
-
-#### 12. TypeScript Migration
-- **What**: Convert codebase to TypeScript
-- **Why**: Type safety, better IDE support, fewer runtime errors
-- **Effort**: Very High (15-20 days) - 87 files to convert
-
----
-
-## Unique Strengths of Lynkr (Don't Lose These!)
-
-1. **Long-term memory system** - Router doesn't have this
-2. **Smart tool selection** - Just implemented, very valuable
-3. **6-phase token optimization** - Industry-leading
-4. **History compression** - Automatic context management
-5. **7 providers** - Broader support than Router
-6. **Hybrid routing** - Ollama + cloud fallback
-
----
-
-## Recommended Implementation Order
-
-### Phase 1: Quick Wins (1-2 weeks)
-1. Token-based auto-routing
-2. Tool enhancement transformer
-3. Rotating file logs
-4. LRU caching
-
-### Phase 2: Game Changers (3-4 weeks)
-5. Dynamic model switching via `/model` command
-6. Context-aware routing (background/think/long-context)
-7. Request/Response transformer pipeline
-
-### Phase 3: Enterprise Features (4-6 weeks)
-8. Custom router scripts
-9. Web UI
-10. CLI commands
-11. GitHub Actions integration
-
-### Phase 4: Long-Term (Optional)
-12. TypeScript migration
-
----
-
-## Bottom Line
-
-Router excels at **flexibility and user experience** (dynamic switching, routing logic, Web UI). Lynkr excels at **optimization and intelligence** (memory, token optimization, smart tools). Merging the best of both would create the ultimate Claude Code proxy.