adaptive-memory-multi-model-router 2.1.0 → 2.2.0

package/docs/API.md

@@ -1,562 +1,855 @@
-# A3M Router API Documentation
+# A3M Router API Reference
+
+Complete reference for the A3M Router — TypeScript SDK, Python SDK, CLI, REST API, and integrations.
+
+---
 
 ## Table of Contents
 
-- [Core Functions](#core-functions)
-- [Memory Module](#memory-module)
-- [Compression Module](#compression-module)
-- [Auto-Fetch Module](#auto-fetch-module)
-- [OAuth Module](#oauth-module)
-- [Provider Registry](#provider-registry)
-- [Cost Tracker](#cost-tracker)
-- [Circuit Breaker](#circuit-breaker)
+- [TypeScript SDK](#typescript-sdk)
+- [Python SDK](#python-sdk)
+- [CLI](#cli)
+- [REST API](#rest-api)
+- [OpenAI SDK Compatibility](#openai-sdk-compatibility)
+- [LangChain Integration](#langchain-integration)
+- [Configuration](#configuration)
+- [Error Handling](#error-handling)
 
 ---
 
-## Core Functions
+## TypeScript SDK
 
-### createA3MRouter
+### Installation
 
-Creates an A3M Router instance with specified configuration.
-
-```javascript
-import { createA3MRouter } from 'adaptive-memory-multi-model-router';
-
-const router = createA3MRouter({
-  memory: true,           // Enable memory tree
-  costBudget: 0.05,      // Max cost per request in USD
-  providers: ['openai', 'groq', 'anthropic'],  // Enabled providers
-  maxLatency: 2000,     // Max latency in ms
-  cacheEnabled: true     // Enable caching
-});
+```bash
+npm install adaptive-memory-multi-model-router
 ```
 
-**Parameters:**
+### Import
 
-| Name | Type | Default | Description |
-|------|------|---------|-------------|
-| `memory` | boolean | `false` | Enable memory tree |
-| `costBudget` | number | `0.10` | Max cost per request |
-| `providers` | string[] | `['openai']` | Enabled providers |
-| `maxLatency` | number | `5000` | Max latency in ms |
-| `cacheEnabled` | boolean | `true` | Enable response cache |
+```typescript
+import { A3MRouter } from 'adaptive-memory-multi-model-router/sdk';
+```
 
-**Returns:** `A3MRouter` instance
+### Constructor
 
----
+```typescript
+const router = new A3MRouter(config?: A3MRouterConfig);
+```
 
-### router.route()
+**A3MRouterConfig:**
 
-Routes a request to the optimal provider.
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `defaultModel` | `string` | — | Fallback model when routing is ambiguous |
+| `maxCostPerQuery` | `number` | — | Max cost per query in USD |
+| `preferSpeedOverQuality` | `boolean` | `false` | Prefer fast models over higher quality |
+| `providers` | `string[]` | all | Restrict routing to these provider IDs |
 
-```javascript
-const result = await router.route({
-  prompt: 'Your prompt here',
-  context: { 
-    type: 'coding',    // 'coding', 'analysis', 'creative'
-    language: 'python',  // Optional language hint
-    domain: 'tech'       // Optional domain hint
-  },
-  options: {
-    maxCost: 0.02,
-    maxLatency: 2000
-  }
-});
-```
+### Methods
 
-**Parameters:**
+#### `route(query: string): RoutingResult`
 
-| Name | Type | Description |
-|------|------|-------------|
-| `prompt` | string | The input prompt |
-| `context` | object | Routing context |
-| `options.maxCost` | number | Max cost for this request |
-| `options.maxLatency` | number | Max latency in ms |
+Route a query to the best available model. Returns the decision without executing the query.
 
-**Returns:**
+```typescript
+const decision = router.route("Write a Python function to sort a list");
 
-```javascript
-{
-  output: 'Response text',
-  provider: 'openai',
-  model: 'gpt-4o',
-  cost: 0.0015,
-  latency: 850,
-  tokens: { prompt: 150, completion: 200 }
-}
+console.log(decision.model);          // "groq/llama-3.3-70b-versatile"
+console.log(decision.tier);           // "cheap"
+console.log(decision.cost);           // 0.00035
+console.log(decision.complexity);     // 0.35
+console.log(decision.reasoning);      // "Selected Groq/llama-3.3-70b for code detected"
+console.log(decision.fallbackModels); // ["cerebras/llama-3.3-70b", "mistral/mistral-small"]
+console.log(decision.isFree);         // false
+console.log(decision.isExpert);       // false
 ```
 
----
+**RoutingResult:**
 
-## Memory Module
+| Field | Type | Description |
+|-------|------|-------------|
+| `model` | `string` | Selected model identifier |
+| `tier` | `'free' \| 'cheap' \| 'mid' \| 'premium'` | Cost tier |
+| `cost` | `number` | Estimated cost in USD |
+| `complexity` | `number` | Query complexity score (0.0–1.0) |
+| `reasoning` | `string` | Human-readable routing reason |
+| `fallbackModels` | `string[]` | Alternative models in priority order |
+| `isFree` | `boolean` | Whether the selected model is free |
+| `isExpert` | `boolean` | Whether this is an expert-level query |
 
-### MemoryTree
+#### `routeBatch(queries: string[]): RoutingResult[]`
 
-Hierarchical memory with LRU cache and fast index.
+Route multiple queries at once.
 
-```javascript
-import { MemoryTree } from 'adaptive-memory-multi-model-router/memory';
-
-const tree = new MemoryTree(maxChunkSize = 3000);
+```typescript
+const decisions = router.routeBatch([
+  "What is 2+2?",
+  "Design a distributed database",
+  "Translate to French"
+]);
+
+decisions.forEach((d, i) => {
+  console.log(`Query ${i}: ${d.model} ($${d.cost.toFixed(6)})`);
+});
 ```
 
-#### tree.add(data)
+#### `recommend(task: string): RoutingResult`
 
-Adds data to memory tree.
+Get model recommendation for a task category.
 
-```javascript
-await tree.add('Your context content here');
-await tree.add('More content for the tree');
+```typescript
+const rec = router.recommend("code generation");
+console.log(rec.model); // "deepseek/deepseek-chat"
 ```
 
-#### tree.search(query)
+#### `analyze(query: string): QueryFeatures`
 
-Fast indexed search.
+Extract detailed features from a query for debugging.
 
-```javascript
-const results = tree.search('keyword');
-// Returns: [{ id, content, score, accessCount, ... }]
+```typescript
+const features = router.analyze("Design a secure authentication system for a healthcare app");
+
+console.log(features.complexity);       // 0.72
+console.log(features.has_code);         // false
+console.log(features.requires_reasoning); // true
+console.log(features.is_security);      // true
+console.log(features.detected_domain);  // "security"
+console.log(features.domain_score);     // 0.30
+```
+
+**QueryFeatures:**
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `complexity` | `number` | Overall complexity score (0.0–1.0) |
+| `length` | `number` | Word count |
+| `has_code` | `boolean` | Code-related keywords detected |
+| `has_math` | `boolean` | Math-related keywords detected |
+| `is_multilingual` | `boolean` | Non-ASCII characters detected |
+| `is_translation` | `boolean` | Translation request detected |
+| `is_creative` | `boolean` | Creative writing request |
+| `requires_reasoning` | `boolean` | Analytical/reasoning verbs detected |
+| `is_security` | `boolean` | Security domain keywords |
+| `is_devops` | `boolean` | DevOps/infrastructure keywords |
+| `is_data` | `boolean` | Data/ML keywords |
+| `detected_domain` | `string` | Best matching domain (legal, medical, finance, security, architecture, ml_research) |
+| `domain_score` | `number` | Domain match confidence |
+
+#### `serve(port?: number): Promise<string>`
+
+Start the OpenAI-compatible proxy server.
+
+```typescript
+const proxyURL = await router.serve(8787);
+console.log(proxyURL); // "http://localhost:8787/v1"
+
+// Now use with any OpenAI SDK
+import OpenAI from 'openai';
+const client = new OpenAI({ baseURL: proxyURL, apiKey: 'not-needed' });
 ```
 
-#### tree.getContext(maxTokens)
+#### `proxyURL` (getter)
 
-Gets context for routing.
+Returns the proxy URL. Available after `serve()` is called, otherwise returns the default.
 
-```javascript
-const context = tree.getContext(3000);
-// Returns: concatenated context string
+```typescript
+router.serve(3000);
+console.log(router.proxyURL); // "http://localhost:3000/v1"
 ```
 
-#### tree.toMarkdown()
+### Tier Classification
 
-Exports memory as Obsidian-compatible markdown.
+Complexity scores map to tiers:
 
-```javascript
-const md = tree.toMarkdown();
-fs.writeFileSync('./memory.md', md);
-```
+| Tier | Complexity Range | Typical Models | Cost/1M tokens |
+|------|:----------------:|----------------|:--------------:|
+| `free` | 0.00 – 0.19 | CommandCode, Ollama, LM Studio | $0.00 |
+| `cheap` | 0.20 – 0.44 | Groq Llama, Cerebras, DeepSeek | ~$0.60 |
+| `mid` | 0.45 – 0.64 | Mistral Small, GPT-4o-mini | ~$1.50 |
+| `premium` | 0.65 – 1.00 | GPT-4o, Claude Sonnet, Gemini Pro | $2.50+ |
 
-#### tree.getStats()
+### Low-Level API
 
-Returns tree statistics.
+The SDK wraps these lower-level exports, also available directly:
 
-```javascript
-const stats = tree.getStats();
-// { totalChunks, maxDepth, indexSize, lruSize }
+```typescript
+import {
+  routeQuery,
+  routeBatch,
+  recommendForTask,
+  extractQueryFeatures,
+  createA3MRouter,
+  getAvailableProviders,
+  registerProvider,
+  createProxyServer,
+} from 'adaptive-memory-multi-model-router';
+
+// Direct routing
+const result = routeQuery("What is 2+2?");
+// => { primary_model, fallback_models, confidence, reasoning, estimated_cost, ... }
+
+// Router instance (v1 style)
+const router = createA3MRouter();
+const decision = router.route("Hello");
 ```
 
 ---
 
-### EpisodicMemoryStore
+## Python SDK
 
-Episodic memory for routing decisions.
+### Installation
 
-```javascript
-import { EpisodicMemoryStore } from 'adaptive-memory-multi-model-router';
-
-const memory = new EpisodicMemoryStore();
+```bash
+pip install a3m-router
 ```
 
-#### memory.record(request, response, routing)
+### Usage
 
-Records a routing decision.
+```python
+from a3m import A3MRouter
 
-```javascript
-await memory.record(request, response, {
-  provider: 'openai',
-  model: 'gpt-4o',
-  reasoning: 'Simple query, fast model sufficient'
-});
-```
+# Create router instance
+router = A3MRouter()
 
-#### memory.getSimilar(request)
+# Route a query
+decision = router.route("Write a Python function")
+print(decision.model)     # "groq/llama-3.3-70b"
+print(decision.tier)      # "cheap"
+print(decision.cost)      # 0.0004
+print(decision.complexity) # 0.35
 
-Gets similar past requests.
+# Chat through the router (auto-selects model)
+response = router.chat("What is 2+2?")
+print(response.content)   # "4"
+print(response.model)     # "groq/llama-3.3-70b"
 
-```javascript
-const similar = await memory.getSimilar({ prompt: 'Debug Python' });
-// Returns routing decisions for similar prompts
+# Analyze query features
+features = router.analyze("Design a microservice architecture")
+print(features.complexity)  # 0.58
+print(features.has_code)    # False
+print(features.detected_domain) # "architecture"
 ```
 
----
-
-### ObsidianVault
+### Async Usage
 
-Exports routing decisions as markdown files.
+```python
+import asyncio
+from a3m import A3MRouter
 
-```javascript
-import { ObsidianVault } from 'adaptive-memory-multi-model-router/vault';
+async def main():
+    router = A3MRouter()
+    
+    # Async chat
+    response = await router.achat("Explain quantum computing")
+    print(response.content)
+    
+    # Batch routing
+    decisions = router.route_batch([
+        "What is 2+2?",
+        "Design a distributed system",
+        "Write a poem"
+    ])
+    for d in decisions:
+        print(f"{d.model} ({d.tier}) — ${d.cost:.6f}")
 
-const vault = new ObsidianVault({ path: './vault' });
+asyncio.run(main())
 ```
 
-#### vault.saveDecision(decision)
+### With OpenAI Python SDK
 
-Saves a routing decision as markdown.
+```python
+from openai import OpenAI
 
-```javascript
-await vault.saveDecision({
-  id: 'decision-001',
-  timestamp: Date.now(),
-  prompt: 'Explain quantum',
-  selectedProvider: 'openai',
-  selectedModel: 'gpt-4o',
-  reasoning: 'Complex topic, use best model',
-  cost: 0.003,
-  latency: 1200
-});
-```
-
-#### vault.getRecentDecisions(count)
-
-Gets recent decisions.
+# Point OpenAI SDK at the A3M proxy
+client = OpenAI(
+    base_url="http://localhost:8787/v1",
+    api_key="not-needed"
+)
 
-```javascript
-const recent = vault.getRecentDecisions(10);
+response = client.chat.completions.create(
+    model="auto",
+    messages=[{"role": "user", "content": "Hello!"}]
+)
+print(response.choices[0].message.content)
 ```
 
 ---
 
-## Compression Module
-
-### EnhancedCompression
+## CLI
 
-TokenJuice-style compression with caching.
+### Installation
 
-```javascript
-import { EnhancedCompression } from 'adaptive-memory-multi-model-router/compression';
-
-const compressor = new EnhancedCompression();
+```bash
+npm install -g adaptive-memory-multi-model-router
+# or use without installing:
+npx a3m-router <command>
 ```
 
-#### compressor.compress(text)
+### Commands
 
-Compresses text (HTML→Markdown, URL shortening, etc).
+#### `route` — Route a query
 
-```javascript
-const compressed = compressor.compress('<h1>Hello</h1><p>URL: https://very-long-url.com</p>');
-// Returns: '# Hello\n\nURL: very-long-url.com/...'
+```bash
+npx a3m-router route "Your query here"
 ```
 
-#### compressor.getStats(original, compressed)
+Output:
+```
+  Primary:   groq/llama-3.3-70b-versatile
+  Fallbacks: cerebras/llama-3.3-70b, mistral/mistral-small
+  Est. Cost: $0.000350
+  Type:      api
+  Reason:    Selected Groq for code detected, fast tier
+```
 
-Gets compression statistics.
+#### `serve` — Start proxy server
 
-```javascript
-const stats = compressor.getStats(original, compressed);
-// { original: 200, compressed: 80, reduction: '60.0%', ratio: '0.40' }
+```bash
+npx a3m-router serve --port 8787
 ```
 
----
+Options:
+- `--port` / `-p` — Port to listen on (default: 8787)
 
-### isonEncode / isonDecode
+#### `benchmark` — Run routing accuracy benchmark
 
-ISON format compression.
+```bash
+npx a3m-router benchmark
+```
 
-```javascript
-import { isonEncode, isonDecode } from 'adaptive-memory-multi-model-router/utils/compression';
+#### `providers` — List configured providers
 
-const encoded = isonEncode(messages);
-const decoded = isonDecode(encoded);
+```bash
+npx a3m-router providers
 ```
 
----
+#### `models` — List known models
 
-## Auto-Fetch Module
+```bash
+npx a3m-router models
+```
 
-### AutoFetch
+#### `recommend` — Get model recommendation for a task
 
-Periodically syncs data from connected tools.
+```bash
+npx a3m-router recommend "code generation"
+```
 
-```javascript
-import { AutoFetch } from 'adaptive-memory-multi-model-router/autofetch';
+#### `cost` — Estimate token cost
 
-const fetcher = new AutoFetch({
-  intervalMs: 20 * 60 * 1000, // 20 minutes
-  targets: ['github', 'notion', 'slack', 'gmail', 'calendar']
-});
+```bash
+npx a3m-router cost "Your text here"
 ```
 
-#### fetcher.start()
-
-Starts the sync loop.
+#### `token` — Count tokens
 
-```javascript
-fetcher.start();
+```bash
+npx a3m-router token "Your text here"
 ```
 
-#### fetcher.syncAll()
+#### `batch` — Route multiple queries
 
-Triggers immediate sync.
-
-```javascript
-const results = await fetcher.syncAll();
-// Returns: Map<target, SyncResult>
+```bash
+npx a3m-router batch "What is 2+2?" "Design a system" "Write a poem"
 ```
 
-#### fetcher.stop()
+#### `compare` — Compare providers side by side
 
-Stops the sync loop.
+```bash
+npx a3m-router compare "Your query"
+```
 
-```javascript
-fetcher.stop();
+#### `test` — Test all configured providers
+
+```bash
+npx a3m-router test
 ```
 
-#### fetcher.getStats()
+#### `memory` — Memory operations
 
-Gets sync statistics.
+```bash
+npx a3m-router memory add "key" "value"
+npx a3m-router memory search "query"
+npx a3m-router memory stats
+```
 
-```javascript
-const stats = fetcher.getStats();
-// { totalTargets: 5, failedTargets: 0 }
+#### `status` — Show router status
+
+```bash
+npx a3m-router status
 ```
 
 ---
 
-## OAuth Module
+## REST API
 
-### OAuthManager
+Start the server:
 
-One-click OAuth for integrations.
+```bash
+npx a3m-router serve --port 8787
+```
 
-```javascript
-import { OAuthManager } from 'adaptive-memory-multi-model-router/oauth';
+Base URL: `http://localhost:8787`
 
-const oauth = new OAuthManager();
-```
+### POST /v1/chat/completions
 
-#### oauth.configure(provider, config)
+OpenAI-compatible chat endpoint. Supports both streaming and non-streaming.
 
-Configures an OAuth provider.
+**Request:**
 
-```javascript
-oauth.configure('github', {
-  clientId: 'your-client-id',
-  clientSecret: 'your-secret',
-  redirectUri: 'http://localhost:3000/callback'
-});
+```bash
+curl http://localhost:8787/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "messages": [
+      {"role": "system", "content": "You are a helpful assistant."},
+      {"role": "user", "content": "What is 2+2?"}
+    ],
+    "temperature": 0.7,
+    "max_tokens": 1024,
+    "stream": false
+  }'
 ```
 
-#### oauth.getAuthUrl(provider)
+**Request Body:**
 
-Gets authorization URL.
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `model` | `string` | `"auto"` | Model to use. `"auto"` triggers routing. Accepts `provider/model` format. |
+| `messages` | `array` | required | OpenAI-format message array |
+| `temperature` | `number` | provider default | Sampling temperature (0–2) |
+| `max_tokens` | `number` | `1024` | Maximum tokens to generate |
+| `stream` | `boolean` | `false` | Enable SSE streaming |
+| `stop` | `string \| string[]` | — | Stop sequences |
 
-```javascript
-const url = oauth.getAuthUrl('github');
-// Opens OAuth flow
-```
+**Non-streaming response:**
 
-#### oauth.handleCallback(provider, code, state)
+```json
+{
+  "id": "chatcmpl-a1b2c3d4",
+  "object": "chat.completion",
+  "created": 1716000000,
+  "model": "llama-3.3-70b-versatile",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "2 + 2 = 4"
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 24,
+    "completion_tokens": 5,
+    "total_tokens": 29
+  }
+}
+```
 
-Handles OAuth callback.
+**Streaming response:**
 
-```javascript
-const tokens = await oauth.handleCallback('github', code, state);
 ```
+data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"...","choices":[{"index":0,"delta":{"content":"2"},"finish_reason":null}]}
 
-#### oauth.isConnected(provider)
+data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"...","choices":[{"index":0,"delta":{"content":" +"},"finish_reason":null}]}
 
-Checks if provider is connected.
+data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"...","choices":[{"index":0,"delta":{"content":" 2 = 4"},"finish_reason":null}]}
 
-```javascript
-const connected = oauth.isConnected('github');
+data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"...","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
+
+data: [DONE]
 ```
 
-#### oauth.getAccessToken(provider)
+**Model routing:**
 
-Gets valid access token (auto-refreshes).
+| Model value | Behavior |
+|-------------|----------|
+| `"auto"` | Routes based on query complexity (recommended) |
+| `"groq/llama-3.3-70b-versatile"` | Uses specific provider/model |
+| `"gpt-4o"` | Maps to configured OpenAI provider |
+| `"claude-sonnet"` | Maps to configured Anthropic provider |
 
-```javascript
-const token = await oauth.getAccessToken('github');
+### POST /v1/completions
+
+OpenAI-compatible text completions endpoint.
+
+```bash
+curl http://localhost:8787/v1/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "auto",
+    "prompt": "The capital of France is",
+    "max_tokens": 50
+  }'
 ```
 
----
+**Response:**
 
-## Provider Registry
+```json
+{
+  "id": "chatcmpl-...",
+  "object": "text_completion",
+  "created": 1716000000,
+  "model": "llama-3.3-70b-versatile",
+  "choices": [
+    {
+      "text": " Paris.",
+      "index": 0,
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 5,
+    "completion_tokens": 2,
+    "total_tokens": 7
+  }
+}
+```
 
-### ProviderRegistry
+### POST /v1/route
 
-Manages 14 LLM providers.
+Route a query without executing it. Returns the routing decision.
 
-```javascript
-import { ProviderRegistry } from 'adaptive-memory-multi-model-router/providers';
+```bash
+curl -X POST http://localhost:8787/v1/route \
+  -H "Content-Type: application/json" \
+  -d '{"query": "Write a Python function to sort a list"}'
+```
 
-const registry = new ProviderRegistry();
+**Response:**
+
+```json
+{
+  "primary_model": "groq/llama-3.3-70b-versatile",
+  "fallback_models": ["cerebras/llama-3.3-70b", "mistral/mistral-small"],
+  "confidence": 0.82,
+  "reasoning": "Selected Groq for code detected, fast tier",
+  "estimated_cost": 0.00035,
+  "estimated_latency_ms": 800,
+  "features": {
+    "complexity": 0.35,
+    "has_code": true,
+    "has_math": false,
+    "requires_reasoning": false,
+    "detected_domain": ""
+  }
+}
 ```
 
-#### registry.getReadyProviders()
+### GET /v1/models
 
-Gets currently available providers.
+List all available models.
 
-```javascript
-const ready = registry.getReadyProviders();
-// ['openai', 'groq', 'anthropic']
+```bash
+curl http://localhost:8787/v1/models
 ```
 
-#### registry.selectModel()
+**Response:**
 
-Selects optimal model based on cost-latency tradeoff.
-
-```javascript
-const model = registry.selectModel();
-// 'openai/gpt-4o'
+```json
+{
+  "object": "list",
+  "data": [
+    {
+      "id": "groq/llama-3.3-70b-versatile",
+      "object": "model",
+      "owned_by": "groq"
+    }
+  ]
+}
 ```
 
-#### registry.recordSuccess(provider)
+### GET /health
 
-Records successful request.
+Health check with provider status and cost summary.
 
-```javascript
-registry.recordSuccess('openai');
+```bash
+curl http://localhost:8787/health
 ```
 
-#### registry.recordFailure(provider)
+**Response:**
 
-Records failed request.
+```json
+{
+  "status": "ok",
+  "version": "2.1.0",
+  "providers": {
+    "total": 12,
+    "healthy": 8,
+    "details": {
+      "groq": {
+        "name": "Groq",
+        "type": "api",
+        "models": 3,
+        "available": true
+      }
+    }
+  },
+  "cost": {
+    "total": 0.0042,
+    "requests": 127
+  },
+  "uptime": 86400.5,
+  "recentRequests": []
+}
+```
 
-```javascript
-registry.recordFailure('openai');
+### GET /dashboard
+
+Interactive web dashboard (if `public/` directory is present).
+
+```
+http://localhost:8787/
 ```
 
 ---
 
-## Cost Tracker
+## OpenAI SDK Compatibility
 
-### CostTracker
+The A3M Router proxy is fully compatible with the OpenAI SDK. Point the `baseURL` at the proxy and use `model: "auto"` for routing.
 
-Tracks request costs and budgets.
+### JavaScript/TypeScript
 
 ```javascript
-import { CostTracker } from 'adaptive-memory-multi-model-router/cost';
+import OpenAI from 'openai';
 
-const tracker = new CostTracker();
-```
+const client = new OpenAI({
+  baseURL: 'http://localhost:8787/v1',
+  apiKey: 'not-needed'
+});
 
-#### tracker.record(requestInfo)
+// Auto-routing
+const response = await client.chat.completions.create({
+  model: 'auto',
+  messages: [{ role: 'user', content: 'Hello!' }]
+});
 
-Records a request.
+// Specific model
+const response2 = await client.chat.completions.create({
+  model: 'groq/llama-3.3-70b-versatile',
+  messages: [{ role: 'user', content: 'Hello!' }]
+});
 
-```javascript
-tracker.record({
-  provider: 'openai',
-  model: 'gpt-4o',
-  promptTokens: 150,
-  completionTokens: 200,
-  cost: 0.003
+// Streaming
+const stream = await client.chat.completions.create({
+  model: 'auto',
+  messages: [{ role: 'user', content: 'Tell me a story' }],
+  stream: true
 });
+for await (const chunk of stream) {
+  process.stdout.write(chunk.choices[0]?.delta?.content || '');
+}
 ```
 
-#### tracker.getSummary()
+### Python
 
-Gets cost summary.
+```python
+from openai import OpenAI
 
-```javascript
-const summary = tracker.getSummary();
-// { totalRequests, totalCost, byProvider, byModel }
-```
+client = OpenAI(
+    base_url="http://localhost:8787/v1",
+    api_key="not-needed"
+)
 
-#### tracker.getRemainingBudget(provider)
+response = client.chat.completions.create(
+    model="auto",
+    messages=[{"role": "user", "content": "Hello!"}]
+)
+print(response.choices[0].message.content)
+```
 
-Gets remaining budget.
+### cURL
 
-```javascript
-const remaining = tracker.getRemainingBudget('openai');
+```bash
+curl http://localhost:8787/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'
 ```
 
 ---
 
-## Circuit Breaker
+## LangChain Integration
 
-### CircuitBreaker
+```typescript
+import { A3MChatModel } from 'adaptive-memory-multi-model-router/langchain';
+import { HumanMessage } from '@langchain/core/messages';
 
-Prevents cascading failures.
+const model = new A3MChatModel({
+  modelName: 'auto', // or specific model
+  temperature: 0.7,
+});
 
-```javascript
-import { CircuitBreaker } from 'adaptive-memory-multi-model-router/utils/reliability';
+// Invoke
+const response = await model.invoke([
+  new HumanMessage("What is 2+2?")
+]);
+console.log(response.content);
+
+// Streaming
+const stream = await model.stream([
+  new HumanMessage("Tell me a story")
+]);
+for await (const chunk of stream) {
+  process.stdout.write(chunk.content);
+}
+```
 
-const breaker = new CircuitBreaker({
-  name: 'openai',
-  failureThreshold: 3,
-  resetTimeout: 60000
-});
+**Prerequisites:**
+
+```bash
+npm install @langchain/core @langchain/openai
 ```
 
-#### breaker.canExecute()
+LangChain is a peer dependency — install it separately.
 
-Checks if request can proceed.
+---
 
-```javascript
-if (breaker.canExecute()) {
-  // Proceed with request
-}
-```
+## Configuration
 
-#### breaker.recordSuccess()
+### Environment Variables
 
-Records success.
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `PORT` | Proxy server port | `8787` |
+| `GROQ_API_KEY` | Groq provider API key | — |
+| `CEREBRAS_API_KEY` | Cerebras provider API key | — |
+| `OPENAI_API_KEY` | OpenAI provider API key | — |
+| `ANTHROPIC_API_KEY` | Anthropic provider API key | — |
+| `GOOGLE_API_KEY` | Google/Gemini provider API key | — |
+| `MISTRAL_API_KEY` | Mistral provider API key | — |
+| `DEEPSEEK_API_KEY` | DeepSeek provider API key | — |
+| `OPENROUTER_API_KEY` | OpenRouter provider API key | — |
 
-```javascript
-breaker.recordSuccess();
-```
+### Config File
 
-#### breaker.recordFailure()
+Provider configuration is stored at `~/.config/a3m-router/providers.json`:
 
-Records failure.
+```json
+{
+  "providers": {
+    "groq": {
+      "apiKey": "gsk_...",
+      "models": ["llama-3.3-70b-versatile", "mixtral-8x7b"]
+    }
+  }
+}
+```
 
-```javascript
-breaker.recordFailure();
+### Programmatic Provider Registration
+
+```typescript
+import { registerProvider } from 'adaptive-memory-multi-model-router';
+
+registerProvider('my-provider', {
+  name: 'My Provider',
+  type: 'api',
+  apiKey: 'sk-...',
+  baseUrl: 'https://api.my-provider.com/v1/chat/completions',
+  models: ['my-model-v1'],
+  costPerK: { input: 0.50, output: 1.50 },
+  maxTokens: 8192,
+  priority: 5,
+});
 ```
 
 ---
 
-## CLI Commands
+## Error Handling
 
-| Command | Description |
-|---------|-------------|
-| `a3m-router route "prompt"` | Route to optimal model |
-| `a3m-router parallel "t1" "t2" "t3"` | Parallel execution |
-| `a3m-router compare "prompt"` | Compare models |
-| `a3m-router cost` | Show cost summary |
-| `a3m-router count "text"` | Count tokens |
-| `a3m-router compress "text"` | Compress text |
-| `a3m-router local "prompt"` | Local Ollama |
-| `a3m-router providers` | List providers |
+### HTTP Error Responses
 
----
+All errors follow the OpenAI error format:
 
-## Error Handling
+```json
+{
+  "error": {
+    "message": "No provider available for model \"gpt-4o\". Configure API keys.",
+    "type": "server_error",
+    "code": 503
+  }
+}
+```
 
-```javascript
-import { CircuitBreaker, withRetry } from 'adaptive-memory-multi-model-router';
+| Status | Type | Description |
+|--------|------|-------------|
+| 400 | `invalid_request_error` | Malformed request body or missing fields |
+| 404 | `not_found` | Unknown endpoint |
+| 502 | `upstream_error` | Provider returned an error |
+| 503 | `server_error` | No providers available |
 
-const breaker = new CircuitBreaker({ name: 'test' });
+### Fallback Behavior
+
+When the primary provider fails, the proxy automatically tries alternative providers in order:
+
+1. Primary provider (routed or specified)
+2. Other configured API providers
+3. Returns 502 if all providers fail
+
+### SDK Error Handling
+
+```typescript
+const router = new A3MRouter();
 
 try {
-  const result = await withRetry(
-    () => router.route({ prompt: 'test' }),
-    { maxRetries: 3, retryDelay: 1000 }
-  );
-} catch (error) {
-  if (breaker.getState().status === 'open') {
-    console.log('Circuit open - using fallback');
+  const decision = router.route("Your query");
+  if (decision.model === 'unknown') {
+    // No providers available
   }
+} catch (err) {
+  console.error('Routing failed:', err);
 }
 ```
 
 ---
 
-## TypeScript Support
+## Provider Support
 
-A3M Router is written in TypeScript with full type definitions.
+### Supported Provider Types
+
+| Type | Providers | Protocol |
+|------|-----------|----------|
+| **API** | Groq, Cerebras, OpenAI, Anthropic, Google, Mistral, DeepSeek, OpenRouter | REST API |
+| **Local** | Ollama, vLLM, LM Studio | OpenAI-compatible local API |
+| **CLI** | CommandCode, OpenCode | Local CLI tools |
+
+### Adding Custom Providers
 
 ```typescript
-import { createA3MRouter, type A3MRouterConfig } from 'adaptive-memory-multi-model-router';
+import { registerProvider } from 'adaptive-memory-multi-model-router';
+
+registerProvider('custom', {
+  name: 'Custom LLM',
+  type: 'api',
+  apiKey: process.env.CUSTOM_API_KEY,
+  baseUrl: 'https://api.custom.com/v1/chat/completions',
+  models: ['custom-v1'],
+  costPerK: { input: 1.0, output: 2.0 },
+  maxTokens: 4096,
+  priority: 5,
+});
+```
 
-const config: A3MRouterConfig = {
-  memory: true,
-  costBudget: 0.05,
-  providers: ['openai', 'groq']
-};
+---
 
-const router = createA3MRouter(config);
+## Architecture
+
+```
+┌──────────────┐     ┌──────────────────────────────────────────┐
+│  Your Code   │────>│  A3M Router                              │
+│              │     │                                          │
+│  OpenAI SDK  │     │  ┌─────────┐    ┌──────────────────┐   │
+│  LangChain   │     │  │ Routing │───>│ Model Selection  │   │
+│  Python SDK  │     │  │ Engine  │    │ (cost/quality)   │   │
+│  CLI         │     │  └─────────┘    └──────┬───────────┘   │
+│  cURL        │     │                        │               │
+└──────────────┘     │  ┌─────────────────────▼────────────┐  │
+                     │  │  Provider Layer                    │  │
+                     │  │  Groq | Cerebras | OpenAI | ...   │  │
+                     │  │  Ollama | vLLM | LM Studio        │  │
+                     │  └───────────────────────────────────┘  │
+                     └──────────────────────────────────────────┘
 ```