adaptive-memory-multi-model-router 2.16.3 → 2.16.4

package/README.md

@@ -1,329 +1,224 @@
-# A3M Router
+# LLM Routing That Cuts Your AI Bill by 90%
 
-> *"Intelligence now has a universal medium of exchange: tokens. Tokens are the new dollars."* — Sarah Wang, a16z GP
+**GPT-4o costs $0.03/run. A3M routes the same request to Groq/Mistral for $0.0001.**
 
-**Building AI's nervous system, the biologically inspired way.**
-
-When Stripe acquired OpenRouter, it validated that AI routing is critical infrastructure. But it also raised a question: **who controls the mint?**
-
-A3M Router is the open-source answer. Built on 3 billion years of biological intelligence.
-
----
+```python
+# Before: Expensive and slow
+response = openai.ChatCompletion.create(model="gpt-4o", messages=[...])
+# $0.03 per request. Every time.
 
-**Intelligent LLM routing across 80+ providers — saves 70-95% on AI costs.**
+# After: Same API, 99.7% cheaper
+from openai import OpenAI
+client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
+response = client.chat.completions.create(model="auto", messages=[...])
+# Routes to cheapest capable provider. $0.0001 per request.
+```
 
-[![npm version](https://img.shields.io/npm/v/adaptive-memory-multi-model-router)](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
-[![npm downloads](https://img.shields.io/npm/dm/adaptive-memory-multi-model-router)](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
+[![npm version](https://img.shields.io/npm/v/adaptive-memory-multi-model-router)](https://www.npmjs.com/npm/package/adaptive-memory-multi-model-router)
+[![npm downloads](https://img.shields.io/npm/dm/adaptive-memory-multi-model-router)](https://www.npmjs.com/npm/package/adaptive-memory-multi-model-router)
 [![PyPI version](https://img.shields.io/pypi/v/a3m-router)](https://pypi.org/project/a3m-router/)
 [![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
-[![Tests](https://img.shields.io/badge/Tests-28%2F28%20passing-brightgreen)](https://github.com/Das-rebel/a3m-router/actions)
+[![CI](https://github.com/Das-rebel/a3m-router/actions/workflows/ci.yml/badge.svg)](https://github.com/Das-rebel/a3m-router/actions)
 [![GitHub stars](https://img.shields.io/github/stars/Das-rebel/a3m-router)](https://github.com/Das-rebel/a3m-router/stargazers)
 
-**📦 Available on:**
-[![npm](https://img.shields.io/badge/npm-6%2C000%2B%2Fmonth-CB3837?style=flat-square&logo=npm)](https://www.npmjs.com/package/adaptive-memory-multi-model-router) 
-[![PyPI](https://img.shields.io/badge/PyPI-700%2Fmonth-3776AB?style=flat-square&logo=pypi)](https://pypi.org/project/a3m-router/) 
-[![GitHub](https://img.shields.io/badge/GitHub-14%20%E2%98%85-100000?style=flat-square&logo=github)](https://github.com/Das-rebel/a3m-router/stargazers)
-
----
-
-## The Problem with Centralization
-
-When one company controls the "dollar of intelligence," what happens to innovation?
-
-History offers cautionary tales. When GitHub was acquired by Microsoft, forks proliferated. GitLab gained market share. The acquirer's brand became a liability for some users.
-
-The same dynamic plays out here. A segment of OpenRouter's user base will start asking: **"Is there an open-source alternative?"**
-
-**We're that alternative.** Not "better" — a different philosophy.
-
----
-
-## Biology-Inspired Intelligence
-
-Nature has been solving the routing problem for 3 billion years. Here's what we borrowed:
-
-### 🐜 Swarm Intelligence → 99.99% Uptime
-
-Ants never ask for directions. Yet colonies reliably find the shortest paths to food.
-
-How? **Pheromone trails.** Each request leaves a trail. If a model fails, its trail weakens and requests avoid it. New paths emerge automatically.
-
-This is how A3M Router achieves 99.99% uptime. Not one giant brain managing everything — millions of tiny smart decisions adding up to a resilient whole.
-
-### 🧠 Neural Plasticity → Adaptive Learning
-
-Your brain isn't static. It constantly rewires, strengthening used pathways and pruning unused ones.
-
-A3M Router does the same: **time-decayed weights** prevent overfitting to outdated provider behavior. Recent performance matters more than old data.
-
-Always learning. Always adapting. Never stuck in the past.
-
-### 📊 Competitive Exclusion → Diversity
-
-In nature, no species can dominate indefinitely. Success creates conditions for others to challenge it.
-
-A3M Router implements **diversity penalty** (EXP3 algorithm). Higher market share = bigger penalty = natural equilibrium.
-
-No monoculture. The plankton paradox solved.
-
-### 🦚 Handicap Principle → Cost as Signal
-
-Why does a peacock have an extravagant tail? Expensive signals are more credible. A peacock that survives despite its handicap must be truly exceptional.
-
-A3M Router sees cost as a **credibility signal**. High cost = high computational investment = better quality for high-stakes queries.
-
-Intelligent resource allocation based on task criticality.
-
 ---
 
-## TL;DR — What Is This?
-
-**Before:**
-```python
-# Pay GPT-4o prices for EVERY query
-client = OpenAI(api_key="sk-...")
-response = client.chat.completions.create(
-    model="gpt-4o",
-    messages=[{"role": "user", "content": "What is 2+2?"}]
-)  # Costs: $0.03
-```
-
-**After:**
-```python
-# A3M Router picks the right model automatically
-client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
-response = client.chat.completions.create(
-    model="auto",  # ← Just change this
-    messages=[{"role": "user", "content": "What is 2+2?"}]
-)  # Routes to Groq/Mistral — costs: $0.0001
 ```
+$ npx a3m-router serve
+   ___  ___  ____ ____  _  _ __  ___
+  / __)/  \/ ___)(  __)( \/ )(  )(  _)
+ ( (__(  O ))__)  ) _)  )  (  )(/(
+  \___)\__/(____)(____)(_)\_)(____/
 
-**Result:** Simple questions cost 300x less. Complex queries still go to premium models when needed.
-
-**New — System One routing (`model="jev-auto"`):** a single-pass, calibrated
-decision head distilled from the heuristic router — ~2ms, 97.8% agreement,
-per-choice probabilities, and it scores **unseen providers** through their
-text (dynamic option sets). Falls back to the full router below p<0.22.
-
----
-
-## 🚀 Performance Benchmarks
+  A3M Router v2.16.3
+  Serving at http://localhost:8787/v1
 
-> *"We don't just claim our router is fast — we *prove* it through 8,400+ real-world queries."*
+  Providers: 80+ | Mode: auto | Memory: enabled
 
-| Metric | A3M Router | OpenRouter | Advantage |
-|--------|-------------|------------|-----------|
-| **Latency (P99)** | **162ms** | 189ms | **14% faster** |
-| **Cost per 1K tokens** | **$0.00012** | $0.0015 | **92% cheaper** |
-| **Quality Score** | **94%** | 92% | **2% better** |
-| **Provider Coverage** | **80+** | 45 | **78% more** |
-| **Uptime** | **99.99%** | Provider-dependent | **Always on** |
-
-### Cost by Query Type
-
-| Query Type | GPT-4o Cost | A3M Router Cost | Savings |
-|------------|-------------|-----------------|---------|
-| "What is 2+2?" | $0.03 | $0.0001 (Groq) | **99.7%** |
-| "Write a Python function" | $0.05 | $0.002 (DeepSeek) | **96%** |
-| "Design a database schema" | $0.15 | $0.008 (Mixed) | **95%** |
-| "Complex reasoning" | $0.15 | $0.15 (GPT-4o) | **0%** (correctly routed) |
+  → POST /v1/chat/completions
+  → GET  /v1/models
+  → GET  /health
+```
 
 ---
 
-## Quick Start
+## Get Started in 30 Seconds
 
 ```bash
-# npm
 npm install adaptive-memory-multi-model-router
 npx a3m-router serve
 
-# Python
-pip install a3m-router
-python -m a3m_router.serve
-
-# Docker
-docker run -p 8787:8787 ghcr.io/das-rebel/a3m-router
+# Then use it like OpenAI:
 ```
 
-Then use it like any OpenAI-compatible API:
-
 ```python
 from openai import OpenAI
-
 client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
 
+# model="auto" → routes to cheapest capable provider
 response = client.chat.completions.create(
-    model="auto",  # ← Magic: A3M picks the best provider
-    messages=[{"role": "user", "content": "What is Python?"}]
+    model="auto",
+    messages=[{"role": "user", "content": "Write a Python fibonacci function"}]
 )
+
+print(response.choices[0].message.content)
+# Output: GPT-4o quality, DeepSeek/Groq price
 ```
 
 ---
 
-## How Routing Works
-
-A3M analyzes every request:
-
-| Signal | Detects |
-|--------|---------|
-| **Domain** | Legal, medical, code, finance, ML keywords |
-| **Task type** | Code, translation, analysis, creative |
-| **Complexity** | Clause count, multi-step markers |
-| **Verb intensity** | "design/architect" → complex, "what/who" → simple |
+## Why Your AI Costs Too Much
 
-Then routes to the right tier:
+Most requests don't need GPT-4o. A simple question costs the same as a complex one.
 
-| Tier | Providers | Use When |
-|------|-----------|----------|
-| **Free** | Ollama, Llama.cpp | Experimentation |
-| **Cheap** | Groq, DeepSeek, Mistral | Simple Q&A, short code |
-| **Mid** | GPT-4o-mini, Claude-haiku | Standard tasks |
-| **Premium** | GPT-4o, Claude-sonnet, Gemini | Complex reasoning |
+| Query | GPT-4o | A3M Routes To | You Save |
+|-------|--------|---------------|----------|
+| "What is 2+2?" | $0.03 | Groq ($0.0001) | **99.7%** |
+| "Explain quantum" | $0.03 | Mistral ($0.0002) | **99.3%** |
+| "Write a Python function" | $0.05 | DeepSeek ($0.002) | **96%** |
+| Complex reasoning | $0.15 | GPT-4o ($0.15) | **0%** (correctly routed) |
 
-### Two routing engines
-
-| Mode | Engine | Latency | Notes |
-|------|--------|---------|-------|
-| `model="auto"` | Heuristic System 2 (features + EXP3 diversity) | ~0.4ms | Default |
-| `model="jev-auto"` | **System One** option-attention head | ~2ms warm | Calibrated probs, dynamic option sets, confidence-guarded fallback to `auto` |
-
-The Jev head ([open System One interface pattern](https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD))
-is distilled from the heuristic router via `npm run jev:distill && npm run jev:train`.
-Optionally point it at a Jev-compatible server ([openjev-sglang](https://github.com/ekzhang/openjev-sglang)
-or api.typesafe.ai) with `A3M_JEV_URL`.
-
----
-
-## Provider Coverage
-
-**80+ providers** including OpenAI, Anthropic, Google, Groq, DeepSeek, Mistral, NVIDIA, Ollama, vLLM, and more.
-
-Availability checked at runtime.
+A3M analyzes your prompt and routes to the cheapest provider that can answer it correctly.
 
 ---
 
-## Architecture
+## How Routing Works
 
 ```
-Request → Guardrails → Semantic Cache → Router → Provider → Response
-                          ↓
-                    Memory Layer
-                    (optional)
+Your Request
+     │
+     ▼
+┌────────────┐
+│  Semantic  │  ← "Is this a duplicate?" (free cache hit?)
+└─────┬──────┘
+     ▼
+┌────────────┐
+│  Router    │  ← "Simple question or complex reasoning?"
+└─────┬──────┘
+     ▼
+┌────────────┐
+│ Provider   │  ← Groq / Mistral / DeepSeek / GPT-4o / Claude...
+└────────────┘
 ```
 
-- **Guardrails** — Prompt injection detection, PII filtering
-- **Semantic Cache** — Instant hits for repeated queries (zero cost)
-- **Router** — Scores query, selects tier, picks cheapest healthy provider
-- **Ensemble** — Optional parallel calls for best-answer mode
+**Two modes:**
+- `model="auto"` — Heuristic router, ~0.4ms overhead, zero extra cost
+- `model="jev-auto"` — ML decision head, calibrated probabilities, ~2ms warm
 
 ---
 
-## CLI Commands
+## 80+ Providers, Zero Config
 
 ```bash
-npx a3m-router serve              # Start server (port 8787)
-npx a3m-router route "query"    # See routing decision
-npx a3m-router health           # Provider status
-npx a3m-router benchmark        # Local accuracy test
+npx a3m-router providers list
 ```
 
+| Tier | Examples |
+|------|----------|
+| Free | Ollama, Llama.cpp, HuggingFace Inference |
+| Budget | Groq, DeepSeek, Mistral, Cloudflare Workers AI |
+| Mid | GPT-4o-mini, Claude-haiku, Gemini-flash |
+| Premium | GPT-4o, Claude-sonnet, Gemini-pro |
+
+Provider availability checked at runtime — no hardcoded uptimes.
+
 ---
 
-## Parallel Ensemble — Best Answer Mode
+## Ship in Minutes, Not Days
 
-Need the best answer regardless of cost? Call multiple providers in parallel:
+**Drop-in OpenAI replacement:**
+```python
+# Just change the base URL — your existing code works
+client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
+```
 
+**Or use the full API:**
 ```python
 from a3m.router import A3MRouter
 
 router = A3MRouter(
     model="auto",
-    parallel_ensemble=3,  # ← Call 3 providers simultaneously
-)
-
-result = router.route(
-    messages=[{"role": "user", "content": "Explain quantum entanglement"}],
-    ensemble_timeout_ms=10000,
+    parallel_ensemble=3,  # Call 3 providers, take the best
+    memory={"type": "semantic", "window": 10},  # Remember context
 )
 
-print(f"Best from: {result.provider}")
-print(f"Response: {result.content}")
+result = router.route(messages=[{"role": "user", "content": "..."}])
+print(f"Provider: {result.provider}")
+print(f"Cost: ${result.cost}")
 ```
 
 ---
 
-## Memory & Context
+## Self-Hosted, No Lock-In
 
-A3M Router includes semantic memory capabilities:
+OpenRouter takes a cut. A3M runs on your machine.
 
-```python
-router = A3MRouter(
-    model="auto",
-    memory={
-        "type": "semantic",
-        "window": 10,  # Last 10 exchanges
-        "similarity_threshold": 0.85,
-    }
-)
+```bash
+# Docker (one command)
+docker run -p 8787:8787 ghcr.io/das-rebel/a3m-router:latest
 
-# Second call uses cached context automatically
-result = router.route(
-    messages=[{"role": "user", "content": "What framework should I use?"}]
-)
-# A3M knows "Python web app" from previous context
+# Or Node.js / Python directly
+npm install adaptive-memory-multi-model-router
+python -m a3m_router.serve
 ```
 
+No API key to share. No vendor lock-in. Your prompts stay on your infrastructure.
+
 ---
 
-## Why Not Just Use OpenRouter?
+## The Fine Print
 
-| Feature | OpenRouter | A3M Router |
-|---------|------------|-------------|
-| **Open-source** | Partial | 100% |
-| **Self-hostable** | No | Yes |
-| **Biology-inspired** | No | Yes |
-| **Provider diversity** | Centralized | Decentralized |
-| **Cost** | $0.0015/1K | $0.00012/1K |
+**Works great when:**
+- You're building AI features and need cost control
+- You want fallback providers (if Groq is down, we route elsewhere)
+- You need semantic caching across conversation turns
+- You want to compare provider quality on the same prompts
 
-We're not competing — we're offering a different philosophy.
+**Not the right tool when:**
+- You need exactly GPT-4o for every request (then just use GPT-4o)
+- Your infrastructure can't run a local service
 
 ---
 
-## Contributing
+## CLI Reference
 
-- 📖 [Contributing Guide](CONTRIBUTING.md)
-- 🐛 [Issue Tracker](https://github.com/Das-rebel/a3m-router/issues)
-- 💬 [Discussions](https://github.com/Das-rebel/a3m-router/discussions)
-- 📜 [Changelog](CHANGELOG.md)
+```bash
+npx a3m-router serve              # Start server (port 8787)
+npx a3m-router route "prompt"    # Preview routing decision
+npx a3m-router health            # Live provider availability
+npx a3m-router benchmark         # Local quality benchmark
+npx a3m-router providers list    # Show all providers
+```
+
+**Environment variables:**
+```bash
+A3M_LOG_LEVEL=debug    # Debug logging
+PORT=8787              # Server port
+A3M_JEV_URL=...        # Optional: remote Jev ML backend
+```
 
 ---
 
-## License
+## Contributing
 
-[MIT License](LICENSE)
+See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, project structure, and code conventions.
+
+- [Issue Tracker](https://github.com/Das-rebel/a3m-router/issues)
+- [Discussions](https://github.com/Das-rebel/a3m-router/discussions)
+- [Changelog](CHANGELOG.md)
 
 ---
 
-## Project Stats
+## The Philosophy (For the Curious)
 
-- **npm downloads:** ~5,989/month
-- **PyPI downloads:** ~620/month
-- **Providers:** 80+
-- **Tests:** 28/28 passing
-- **License:** MIT
+A3M is built on biological intelligence: evolution solved the routing problem 3 billion years ago. The immune system doesn't use the same response for every pathogen — it routes resources based on threat level.
 
----
+Same idea here: simple questions get cheap answers. Complex reasoning gets premium models. The router learns from traffic and improves over time.
 
-<p align="center">
-  <strong>Built on 3 billion years of biological intelligence.</strong><br>
-  <a href="https://github.com/Das-rebel/a3m-router">GitHub</a> •
-  <a href="https://twitter.com/a3m_router">Twitter</a> •
-  <a href="https://www.npmjs.com/package/adaptive-memory-multi-model-router">npm</a> •
-  <a href="https://pypi.org/project/a3m-router/">PyPI</a>
-</p>
+A3M is 100% open-source, self-hostable, and community-driven. We're not competing with OpenRouter — we're offering a different philosophy: open, decentralized, and yours.
 
 ---
 
-## 🔧 More from Das-rebel
+## ⭐ Star History
 
-- **[FinWipe](https://github.com/Das-rebel/finwipe)** — DIY financial data deletion CLI for India. Exercise your right to erasure under the DPDP Act 2023 across 230+ NBFCs, fintechs and banks.
+[![Star History Chart](https://api.star-history.com/svg?repos=Das-rebel/a3m-router&type=Timeline)](https://star-history.com/#Das-rebel/a3m-router&Timeline)