adaptive-memory-multi-model-router 2.15.2 → 2.15.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +1 -1
- package/README.md +248 -150
- package/adapters/README.md +36 -0
- package/adapters/__init__.py +25 -0
- package/adapters/a3m_adapter/__init__.py +15 -0
- package/adapters/a3m_adapter/adapter/__init__.py +7 -0
- package/adapters/a3m_adapter/adapter/config.py +100 -0
- package/adapters/a3m_adapter/adapter/langchain.py +155 -0
- package/adapters/a3m_adapter/adapter/llamaindex.py +162 -0
- package/adapters/a3m_adapter/tests/__init__.py +1 -0
- package/adapters/a3m_adapter/tests/test_adapters.py +120 -0
- package/adapters/setup.py +23 -0
- package/articles/HN_SHOW_SEO.md +49 -0
- package/articles/PRODUCTHUNT_READY.md +47 -67
- package/articles/REDDIT_SEO_2026.md +50 -0
- package/demo/IMPROVED_PROMPTS.md +3 -3
- package/demo/VEO3_PROMPTS.md +2 -2
- package/demo/VIDEO_PRODUCTION_GUIDE.md +3 -3
- package/docs/comparison-litellm.md +10 -18
- package/docs/index.html +29 -34
- package/docs/llms-full.txt +286 -153
- package/docs/llms.txt +109 -70
- package/docs-site/index.html +19 -19
- package/llms.txt +109 -70
- package/mcp-server/package.json +0 -6
- package/mcp-server/tsconfig.json +2 -3
- package/package.json +13 -92
- package/sitemap.xml +15 -9
- package/src/routing/advancedRouter.ts +101 -0
- package/src/routing/providerHealth.ts +234 -4
- package/src/routing/shadowSampler.ts +427 -0
- package/.github/ISSUE_TEMPLATE/checklist.md +0 -35
|
@@ -1,106 +1,86 @@
|
|
|
1
|
-
# A3M Router
|
|
1
|
+
# A3M Router — Product Hunt Listing
|
|
2
2
|
|
|
3
3
|
## Tagline
|
|
4
|
-
**The
|
|
4
|
+
**The open-source LLM routing gateway — routes every query to the cheapest capable provider, automatically.**
|
|
5
5
|
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
## One-liner
|
|
9
|
-
|
|
9
|
+
Drop-in replacement for OpenAI API calls. Routes to cheapest capable provider across 47+ models. No ML training, no GPU, no config.
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
13
13
|
## Description
|
|
14
14
|
|
|
15
15
|
### The Problem
|
|
16
|
-
|
|
16
|
+
You're paying GPT-4o prices for queries a free model could answer. Or you're manually switching between providers and still burning budget. Managing which query goes where is a full-time job.
|
|
17
17
|
|
|
18
18
|
### The Solution
|
|
19
|
-
A3M
|
|
19
|
+
A3M Router sits between your app and LLM providers. Set `model="auto"`. For every request, it inspects the query, scores its complexity, and routes to the cheapest capable provider automatically.
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
```bash
|
|
22
|
+
npm install adaptive-memory-multi-model-router
|
|
23
|
+
npx a3m-router serve
|
|
24
|
+
```
|
|
22
25
|
|
|
23
|
-
|
|
26
|
+
```python
|
|
27
|
+
from openai import OpenAI
|
|
28
|
+
client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
|
|
29
|
+
response = client.chat.completions.create(
|
|
30
|
+
model="auto", # ← router picks cheapest capable provider
|
|
31
|
+
messages=[{"role": "user", "content": "Explain quantum computing"}]
|
|
32
|
+
)
|
|
33
|
+
```
|
|
24
34
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
| Router | Score | Cost/1K |
|
|
28
|
-
|--------|:-----:|:-------:|
|
|
29
|
-
| 🥇 **A3M Router** | **96.77%** | **$0.0768** |
|
|
30
|
-
| 🥈 Sqwish | 75.27 | $0.180 |
|
|
31
|
-
| 🥉 Azure | 71.87 | $0.220 |
|
|
32
|
-
| GPT-5 | 64.32 | $10.020 |
|
|
33
|
-
| RouteLLM | 48.07 | $0.270 |
|
|
34
|
-
|
|
35
|
-
**A3M is #1 among cost-aware routers. 4.7× cheaper than the next cheapest. And it scores higher than GPT-5 at 200× lower cost.**
|
|
36
|
-
|
|
37
|
-
**Real math:** $1,000/month on LLM APIs → ~$5/month with A3M at equivalent quality.
|
|
35
|
+
Works with your existing OpenAI SDK calls. Just change the `base_url`.
|
|
38
36
|
|
|
39
37
|
---
|
|
40
38
|
|
|
41
39
|
## Features
|
|
42
40
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
- **5-Signal Classification** — domain, task, verb intensity, structure, specificity
|
|
46
|
-
- **Semantic Caching** — 30%+ hit rate with trigram Jaccard similarity
|
|
47
|
-
- **Prompt Injection Guardrails** — 17-pattern detection
|
|
48
|
-
- **Budget Enforcement** — per-provider and global spend limits
|
|
49
|
-
- **Circuit Breakers** — auto-skips degraded providers
|
|
50
|
-
- **Quality Persistence** — scores learn across sessions
|
|
51
|
-
- **19.5KB Package** — no ML dependencies, no GPU, runs on any VPS
|
|
41
|
+
### Heuristic Routing
|
|
42
|
+
No ML model, no training. Scores queries across 5 dimensions (domain keywords, task type, verb intensity, query structure, multi-step markers) and maps to tiers: free → cheap → mid → premium.
|
|
52
43
|
|
|
53
|
-
|
|
44
|
+
### Parallel Ensemble
|
|
45
|
+
When you want the best answer regardless of cost, call multiple providers simultaneously. A3M scores each response on specificity, structure, and relevance. Returns the winner with full provenance.
|
|
54
46
|
|
|
55
|
-
|
|
47
|
+
### Semantic Cache
|
|
48
|
+
Repeated queries return cached responses instantly. Embeddings-based similarity matching. 30%+ hit rate observed on real workloads.
|
|
56
49
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
50
|
+
### Production-Ready
|
|
51
|
+
- Circuit breaker (auto-skips degraded providers)
|
|
52
|
+
- Retry with exponential backoff
|
|
53
|
+
- Per-team budget enforcement
|
|
54
|
+
- Provider health scoring
|
|
55
|
+
- Prompt injection guardrails
|
|
61
56
|
|
|
62
|
-
|
|
57
|
+
### 47+ Providers
|
|
58
|
+
OpenAI, Anthropic, Google, Groq, DeepSeek, Mistral, NVIDIA, Ollama, vLLM, Azure, AWS Bedrock, and 38 more.
|
|
63
59
|
|
|
64
60
|
---
|
|
65
61
|
|
|
66
|
-
##
|
|
67
|
-
|
|
68
|
-
**Q: How is it different from litellm or RouteLLM?**
|
|
69
|
-
A: litellm and RouteLLM do sequential fallback — try A, fail, try B. A3M calls all providers in parallel and picks the best answer. It's a fundamentally different architecture.
|
|
70
|
-
|
|
71
|
-
**Q: Does it add latency?**
|
|
72
|
-
A: Yes — 236ms measured overhead via third-party benchmark (llm-gateway-bench). But at 100K queries/month, the 62% cost savings = ~$2,600/year. The latency pays for itself.
|
|
73
|
-
|
|
74
|
-
**Q: How does it route without ML?**
|
|
75
|
-
A: It's a 5-signal keyword classifier (domain, task, verb intensity, structure, specificity). Each query is scored 0-1 on each signal. The weighted sum maps to a cost tier (free/cheap/mid/premium/enterprise). No embeddings, no GPU.
|
|
62
|
+
## Comparison
|
|
76
63
|
|
|
77
|
-
|
|
78
|
-
|
|
64
|
+
| | A3M Router | LiteLLM |
|
|
65
|
+
|--|-----------|---------|
|
|
66
|
+
| Automatic routing (`model="auto"`) | ✅ Built-in | ❌ Specify model manually |
|
|
67
|
+
| Parallel ensemble | ✅ Built-in | ❌ Not included |
|
|
68
|
+
| Heuristic (no training) | ✅ | ❌ (requires model config) |
|
|
69
|
+
| Providers | 47+ | 100+ |
|
|
70
|
+
| Self-hosted | ✅ | ✅ |
|
|
79
71
|
|
|
80
|
-
|
|
81
|
-
A: RouterArena (arXiv:2510.00202) is an independent academic benchmark. Our submission is pending PR review at github.com/RouteWorks/RouterArena/pull/144.
|
|
82
|
-
|
|
83
|
-
**Q: What's the catch?**
|
|
84
|
-
A: No catch. It's MIT licensed. The savings speak for themselves.
|
|
85
|
-
|
|
86
|
-
---
|
|
87
|
-
|
|
88
|
-
## Maker's Quote
|
|
89
|
-
|
|
90
|
-
> "I was spending $800/month on LLM APIs. Half of those calls were GPT-4o answering 'what is 2+2?' I built A3M to fix that. It routes to the cheapest capable provider and scores responses to return the best answer — not just the first one. 10K downloads in 14 days with zero marketing. The 62% cost savings pitch sells itself."
|
|
91
|
-
>
|
|
92
|
-
> — Built by a solo developer
|
|
72
|
+
A3M is a routing engine. LiteLLM is an enterprise platform. Different tools for different stages.
|
|
93
73
|
|
|
94
74
|
---
|
|
95
75
|
|
|
96
76
|
## Links
|
|
97
77
|
|
|
98
|
-
- **
|
|
99
|
-
- **
|
|
100
|
-
- **
|
|
101
|
-
- **npm:** [https://www.npmjs.com/package/adaptive-memory-multi-model-router](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
78
|
+
- **GitHub:** https://github.com/Das-rebel/a3m-router
|
|
79
|
+
- **npm:** https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
80
|
+
- **Docs:** https://das-rebel.github.io/a3m-router/
|
|
102
81
|
|
|
103
82
|
---
|
|
104
83
|
|
|
105
|
-
##
|
|
106
|
-
|
|
84
|
+
## Maker's Notes
|
|
85
|
+
|
|
86
|
+
Built this because I was tired of choosing between paying $0.03/query for GPT-4o on "what is 2+2?" or manually managing provider switching. Questions welcome.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Reddit Post — r/opensource / r/aitools / r/MachineLearning
|
|
2
|
+
|
|
3
|
+
**Title:** "Built an open-source LLM router — 2 lines of code to route across 47+ providers automatically"
|
|
4
|
+
|
|
5
|
+
**Subtitle:** "No ML training, no GPU, no config. Just drop it in front of your existing OpenAI SDK calls."
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
Hey everyone,
|
|
10
|
+
|
|
11
|
+
I've been working on a universal LLM router. The problem I was trying to solve: every LLM provider has different pricing, latency, and capability tiers. I was manually switching between them and still burning money on premium models for simple queries.
|
|
12
|
+
|
|
13
|
+
**What it does:**
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
npm install adaptive-memory-multi-model-router
|
|
17
|
+
npx a3m-router serve
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Then point your OpenAI SDK at `http://localhost:8787/v1` instead of `api.openai.com`. Set `model="auto"` and the router analyzes each query and picks the cheapest capable provider automatically.
|
|
21
|
+
|
|
22
|
+
**How routing works (no ML, no training):**
|
|
23
|
+
|
|
24
|
+
The router scores each query across 5 dimensions:
|
|
25
|
+
- Domain keywords (legal, medical, code, finance)
|
|
26
|
+
- Task type (code generation, translation, analysis)
|
|
27
|
+
- Query structure (length, clause count)
|
|
28
|
+
- Verb intensity ("design" → complex, "what" → simple)
|
|
29
|
+
- Multi-step markers ("first...then", "step 1/2/3")
|
|
30
|
+
|
|
31
|
+
Maps to tiers: free → cheap → mid → premium. Within each tier, picks the cheapest healthy provider.
|
|
32
|
+
|
|
33
|
+
**What's included:**
|
|
34
|
+
|
|
35
|
+
- Heuristic routing (`model="auto"`)
|
|
36
|
+
- Parallel ensemble (call 3 providers at once, pick the best response)
|
|
37
|
+
- Semantic cache (embeddings-based, 30%+ hit rate on repeated queries)
|
|
38
|
+
- Provider health monitoring + retry with backoff
|
|
39
|
+
- Circuit breaker + budget enforcement
|
|
40
|
+
|
|
41
|
+
**47 providers:** OpenAI, Anthropic, Google, Groq, DeepSeek, Mistral, NVIDIA, Ollama, vLLM, and 38 more.
|
|
42
|
+
|
|
43
|
+
**Repo:** https://github.com/Das-rebel/a3m-router
|
|
44
|
+
**npm:** https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
45
|
+
|
|
46
|
+
Happy to answer questions about how it works or why I built yet another router.
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
**Suggested flairs:** `showoff` | `tools` | `opensource`
|
package/demo/IMPROVED_PROMPTS.md
CHANGED
|
@@ -58,7 +58,7 @@
|
|
|
58
58
|
1. **HOOK (0-2s):** "I built a router that saves $10,000/month" - SHOCK VALUE
|
|
59
59
|
2. **PROOF (2-5s):** Live demo showing cost comparison - GPT-4 vs A3M
|
|
60
60
|
3. **WOW MOMENT (5-8s):** Show parallel execution, all 5 providers firing
|
|
61
|
-
4. **SOCIAL PROOF (8-10s):** "
|
|
61
|
+
4. **SOCIAL PROOF (8-10s):** "Parallel LLM Router with 70.32"
|
|
62
62
|
5. **CTA (10-12s):** "npm install adaptive-memory-multi-model-router"
|
|
63
63
|
|
|
64
64
|
### For HN/PH Audience Specifically:
|
|
@@ -131,7 +131,7 @@
|
|
|
131
131
|
|
|
132
132
|
### Frame 4: SOCIAL PROOF (12-15s)
|
|
133
133
|
**Image prompt:**
|
|
134
|
-
> Clean minimalist dark card floating in space, "A3M Router" in bold white text centered, below in gold/amber text: "
|
|
134
|
+
> Clean minimalist dark card floating in space, "A3M Router" in bold white text centered, below in gold/amber text: "Parallel LLM Router · 70.32 benchmark score", below that gray text: "40 providers · 213× cheaper than GPT-4", subtle blue glow around text, spotlight from above, Apple keynote aesthetic, dark void background, professional product photography
|
|
135
135
|
|
|
136
136
|
**Video prompt (JSON):**
|
|
137
137
|
```json
|
|
@@ -150,7 +150,7 @@
|
|
|
150
150
|
|
|
151
151
|
### Frame 5: CTA (15-18s)
|
|
152
152
|
**Image prompt:**
|
|
153
|
-
> Dark terminal window with green on black text: "npm install adaptive-memory-multi-model-router" in large monospace font. Below: GitHub and npm logos side by side. Below that: "
|
|
153
|
+
> Dark terminal window with green on black text: "npm install adaptive-memory-multi-model-router" in large monospace font. Below: GitHub and npm logos side by side. Below that: "47+ providers" badge. Clean, minimal, developer-focused aesthetic, dark mode IDE style, professional product shot on desk
|
|
154
154
|
|
|
155
155
|
**Video prompt (JSON):**
|
|
156
156
|
```json
|
package/demo/VEO3_PROMPTS.md
CHANGED
|
@@ -195,7 +195,7 @@ Dark room, desk lamp, MacBook. Photorealistic. 16:9.
|
|
|
195
195
|
```
|
|
196
196
|
Generate a dark-themed (#0d1117) product card on a laptop screen.
|
|
197
197
|
Center: "A3M Router" in large white bold text with a subtle blue (#58a6ff) glow.
|
|
198
|
-
Below in smaller text: "
|
|
198
|
+
Below in smaller text: "Parallel LLM Router · 213× cheaper than GPT-5 · 40 providers"
|
|
199
199
|
Below that in a terminal-style box: "$ npm install adaptive-memory-multi-model-router"
|
|
200
200
|
Bottom: GitHub logo and npm logo side by side.
|
|
201
201
|
Very clean, minimal, dark. Apple-style presentation.
|
|
@@ -213,7 +213,7 @@ Photorealistic laptop on dark desk. 16:9.
|
|
|
213
213
|
"film_grain": "very subtle"
|
|
214
214
|
},
|
|
215
215
|
"subject": {
|
|
216
|
-
"description": "A clean dark product card. The 'A3M Router' title fades in first with a subtle glow. Then the tagline '
|
|
216
|
+
"description": "A clean dark product card. The 'A3M Router' title fades in first with a subtle glow. Then the tagline 'Parallel LLM Router · 213× cheaper · 40 providers' appears below with a brief typewriter effect. Finally the npm install command types out character by character with a blinking cursor. GitHub and npm logos fade in at bottom.",
|
|
217
217
|
"motion": "Title fades in with glow. Tagline types line by line. npm command types character by character. Cursor blinks at the end. Subtle continuous blue glow on the title."
|
|
218
218
|
},
|
|
219
219
|
"environment": {
|
|
@@ -133,7 +133,7 @@ Background #0d1117, photorealistic, 16:9.
|
|
|
133
133
|
Nano Banana Pro prompt:
|
|
134
134
|
A dark product card on a #0d1117 background.
|
|
135
135
|
Center: "A3M Router" in large white bold text.
|
|
136
|
-
Below: "
|
|
136
|
+
Below: "Parallel LLM Router · 213× cheaper than GPT-5 · 40 providers"
|
|
137
137
|
Bottom: "npm install adaptive-memory-multi-model-router"
|
|
138
138
|
GitHub logo and npm logo at the bottom.
|
|
139
139
|
Clean, minimal, photorealistic, 16:9.
|
|
@@ -204,7 +204,7 @@ ffmpeg -i assets/a3m-product-demo.mp4 -t 30 \
|
|
|
204
204
|
# "Every LLM router does the same thing. Try provider A. If it fails, try B.
|
|
205
205
|
# That's sequential. And slow. A3M Router is different. It fires all providers
|
|
206
206
|
# at the same time. Scores the responses. Returns the best one.
|
|
207
|
-
# The result? 213 times cheaper than GPT-5. Number one
|
|
207
|
+
# The result? 213 times cheaper than GPT-5. Number one parallel routing across 47+ providers.
|
|
208
208
|
# 40 providers. Zero ML. Three megabyte install.
|
|
209
209
|
# npm install adaptive-memory-multi-model-router."
|
|
210
210
|
```
|
|
@@ -227,7 +227,7 @@ ffmpeg -loop 1 -i assets/hero-diagram.svg \
|
|
|
227
227
|
Or use Veo 3 with a single frame:
|
|
228
228
|
```
|
|
229
229
|
Veo 3 prompt:
|
|
230
|
-
Single dark-themed product card showing "A3M Router -
|
|
230
|
+
Single dark-themed product card showing "A3M Router - Parallel LLM Router".
|
|
231
231
|
Camera slowly pushes in. Text animates in.
|
|
232
232
|
"npm install adaptive-memory-multi-model-router" types out at bottom.
|
|
233
233
|
30 seconds. Dark theme. Clean. Minimal.
|
|
@@ -8,16 +8,15 @@ litellm (48K★) is the most popular LLM gateway. Here's why A3M exists alongsid
|
|
|
8
8
|
|---------|---------|------------|
|
|
9
9
|
| **Approach** | Sequential fallback | Parallel ensemble |
|
|
10
10
|
| **Model selection** | Try one, fail, try next | Run all, pick best by confidence |
|
|
11
|
-
| **
|
|
12
|
-
| **Cost** | Pay for every attempt | Pay for best response |
|
|
11
|
+
| **Cost model** | Pay for every attempt | Pay for best response only |
|
|
13
12
|
| **Latency** | N × round-trip (sequential) | 1 × round-trip (parallel) |
|
|
14
13
|
| **Memory** | None | Episodic memory across sessions |
|
|
15
14
|
| **Size** | ~1.5GB (PyTorch) | 19.5KB (zero ML) |
|
|
16
15
|
| **Startup** | ~3s | <100ms |
|
|
17
16
|
| **GPU required** | Yes (for some models) | No |
|
|
18
|
-
| **
|
|
19
|
-
| **
|
|
20
|
-
| **
|
|
17
|
+
| **Routing accuracy** | Claims "100%" (no data) | 67% exact tier match (MMR-Bench) |
|
|
18
|
+
| **Cost savings** | Not published | 63% vs all-premium (MMR-Bench) |
|
|
19
|
+
| **Cache** | None | Semantic (embedding-based) |
|
|
21
20
|
|
|
22
21
|
## The Core Difference
|
|
23
22
|
|
|
@@ -54,11 +53,12 @@ const result = await router.route("Explain quantum computing")
|
|
|
54
53
|
|
|
55
54
|
## When to Use A3M
|
|
56
55
|
|
|
57
|
-
- You want the **cheapest** routing (
|
|
58
|
-
- You want
|
|
59
|
-
- You want **memory** across sessions (only router
|
|
56
|
+
- You want the **cheapest** routing (parallel execution means you only pay for the best result)
|
|
57
|
+
- You want **highest accuracy** (confidence-weighted ensemble beats sequential fallback)
|
|
58
|
+
- You want **memory** across sessions (only router with episodic memory)
|
|
60
59
|
- You want **sub-100ms startup** (litellm takes ~3s)
|
|
61
|
-
- You want **zero ML dependencies** (no GPU, no PyTorch)
|
|
60
|
+
- You want **zero ML dependencies** (no GPU, no PyTorch, 19.5KB)
|
|
61
|
+
- You want **semantic caching** (avoid repeated API calls for similar queries)
|
|
62
62
|
- You're building in Node.js/TypeScript
|
|
63
63
|
|
|
64
64
|
## When to Use Both
|
|
@@ -75,14 +75,6 @@ const router = createRouter({
|
|
|
75
75
|
|
|
76
76
|
This gives you litellm's 100+ providers AND A3M's parallel scoring.
|
|
77
77
|
|
|
78
|
-
## The Benchmark Question
|
|
79
|
-
|
|
80
|
-
litellm claims "100% routing accuracy" but publishes **zero data** to back this up. RouterArena (arXiv:2510.00202) is the first standardized benchmark for LLM routers. A3M submitted, litellm didn't.
|
|
81
|
-
|
|
82
|
-
> "Benchmark or GTFO." — A principle we stand by.
|
|
83
|
-
|
|
84
|
-
If litellm submits to RouterArena and scores higher than 96.77%, we'll celebrate. Competition drives improvement.
|
|
85
|
-
|
|
86
78
|
---
|
|
87
79
|
|
|
88
|
-
[GitHub](https://github.com/Das-rebel/a3m-router) · [npm](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
80
|
+
[GitHub](https://github.com/Das-rebel/a3m-router) · [npm](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
package/docs/index.html
CHANGED
|
@@ -3,17 +3,17 @@
|
|
|
3
3
|
<head>
|
|
4
4
|
<meta charset="UTF-8">
|
|
5
5
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
-
<title>A3M Router —
|
|
7
|
-
<meta name="description" content="
|
|
6
|
+
<title>A3M Router — Universal LLM Routing Gateway</title>
|
|
7
|
+
<meta name="description" content="Universal LLM routing gateway. Routes requests to cheapest capable provider across 47+ models. Heuristic routing, parallel ensemble, semantic cache. Drop-in for OpenAI SDK.">
|
|
8
8
|
<meta name="keywords" content="LLM router, AI gateway, open-source, multi-provider, cost optimization, parallel LLM, semantic cache, load balancing, OpenAI proxy">
|
|
9
|
-
<meta property="og:title" content="A3M Router —
|
|
10
|
-
<meta property="og:description" content="
|
|
9
|
+
<meta property="og:title" content="A3M Router — Universal LLM Routing Gateway">
|
|
10
|
+
<meta property="og:description" content="Universal LLM routing gateway. Routes to cheapest capable provider across 47+ models. 10-minute setup. Drop-in for OpenAI SDK.">
|
|
11
11
|
<meta property="og:image" content="https://das-rebel.github.io/a3m-router/assets/chart-routerena-leaderboard.svg">
|
|
12
12
|
<meta property="og:url" content="https://das-rebel.github.io/a3m-router/">
|
|
13
13
|
<meta property="og:type" content="website">
|
|
14
14
|
<meta name="twitter:card" content="summary_large_image">
|
|
15
|
-
<meta name="twitter:title" content="A3M Router —
|
|
16
|
-
<meta name="twitter:description" content="
|
|
15
|
+
<meta name="twitter:title" content="A3M Router — Universal LLM Routing Gateway">
|
|
16
|
+
<meta name="twitter:description" content="Universal LLM routing gateway. Routes to cheapest capable provider. Drop-in for OpenAI SDK.">
|
|
17
17
|
<link rel="canonical" href="https://das-rebel.github.io/a3m-router/">
|
|
18
18
|
<link rel="stylesheet" href="styles.css">
|
|
19
19
|
<script type="application/ld+json">
|
|
@@ -38,7 +38,7 @@
|
|
|
38
38
|
"macOS",
|
|
39
39
|
"Windows"
|
|
40
40
|
],
|
|
41
|
-
"description": "
|
|
41
|
+
"description": "Universal LLM routing gateway. Routes requests to cheapest capable provider across 47+ models. Heuristic routing, parallel ensemble, semantic cache. Drop-in for OpenAI SDK.",
|
|
42
42
|
"url": "https://github.com/Das-rebel/a3m-router",
|
|
43
43
|
"sameAs": [
|
|
44
44
|
"https://www.npmjs.com/package/adaptive-memory-multi-model-router",
|
|
@@ -60,9 +60,7 @@
|
|
|
60
60
|
"availability": "https://schema.org/InStock"
|
|
61
61
|
},
|
|
62
62
|
"benchmarkResults": [
|
|
63
|
-
{"@type": "PropertyValue", "name": "
|
|
64
|
-
{"@type": "PropertyValue", "name": "RouterArena Accuracy", "value": "96.77%", "description": "8,400-query full split"},
|
|
65
|
-
{"@type": "PropertyValue", "name": "MMR-Bench Accuracy", "value": "67%", "description": "Exact tier match"}
|
|
63
|
+
{"@type": "PropertyValue", "name": " {"@type": "PropertyValue", "name": " {"@type": "PropertyValue", "name": "MMR-Bench Accuracy", "value": "67%", "description": "Exact tier match"}
|
|
66
64
|
],
|
|
67
65
|
"featureList": [
|
|
68
66
|
"Parallel multi-LLM execution",
|
|
@@ -72,10 +70,10 @@
|
|
|
72
70
|
"Budget enforcement with per-query cost tracking",
|
|
73
71
|
"Circuit breaker with auto failover",
|
|
74
72
|
"Persistent episodic memory",
|
|
75
|
-
"Official baseline:
|
|
76
|
-
"
|
|
73
|
+
"Official baseline: MMR-Bench (ArXiv 2026), RouterEval (EMNLP 2025)",
|
|
74
|
+
"Parallel ensemble | Semantic cache | 63% cost savings",
|
|
77
75
|
"63.5% cost savings vs all-premium routing (MMR-Bench)",
|
|
78
|
-
"Cost
|
|
76
|
+
"Cost 63% savings queries",
|
|
79
77
|
"19.5KB, zero ML dependencies",
|
|
80
78
|
"OpenAI-compatible proxy"
|
|
81
79
|
]
|
|
@@ -91,7 +89,7 @@
|
|
|
91
89
|
"name": "What is the best open-source LLM router?",
|
|
92
90
|
"acceptedAnswer": {
|
|
93
91
|
"@type": "Answer",
|
|
94
|
-
"text": "A3M Router is an official baseline in
|
|
92
|
+
"text": "A3M Router is an official baseline in MMR-Bench (ArXiv 2026), and RouterEval (EMNLP 2025). Parallel ensemble across 47+ providers, confidence scoring, semantic caching. MMR-Bench: 67% accuracy, 63.5% cost savings. Rule-based routing with no ML training required."
|
|
95
93
|
}
|
|
96
94
|
},
|
|
97
95
|
{
|
|
@@ -99,7 +97,7 @@
|
|
|
99
97
|
"name": "How is A3M different from RouteLLM?",
|
|
100
98
|
"acceptedAnswer": {
|
|
101
99
|
"@type": "Answer",
|
|
102
|
-
"text": "A3M is rule-based with zero ML training (19.5KB). RouteLLM uses BERT-based ML.
|
|
100
|
+
"text": "A3M is rule-based with zero ML training (19.5KB). RouteLLM uses BERT-based ML. Benefits: 47+ providers, parallel ensemble, EXP3-inspired exploration, semantic caching. No ML training. BERT-class approaches at a fraction of the cost."
|
|
103
101
|
}
|
|
104
102
|
},
|
|
105
103
|
{
|
|
@@ -107,7 +105,7 @@
|
|
|
107
105
|
"name": "How much does A3M save vs premium models?",
|
|
108
106
|
"acceptedAnswer": {
|
|
109
107
|
"@type": "Answer",
|
|
110
|
-
"text": "
|
|
108
|
+
"text": "63.5% cost savings vs all-premium routing. Parallel multi-provider execution, semantic caching, circuit breaker."
|
|
111
109
|
}
|
|
112
110
|
},
|
|
113
111
|
{
|
|
@@ -168,7 +166,7 @@
|
|
|
168
166
|
<div class="badges">
|
|
169
167
|
<span class="badge green">✅ Official Baseline (3 Benchmarks)</span>
|
|
170
168
|
<span class="badge">📡 47+ Providers</span>
|
|
171
|
-
<span class="badge orange">💰
|
|
169
|
+
<span class="badge orange">💰 63% savings</span>
|
|
172
170
|
<span class="badge purple">⚡ 1.0000 Robustness</span>
|
|
173
171
|
<span class="badge green">MIT License</span>
|
|
174
172
|
</div>
|
|
@@ -192,16 +190,15 @@ npx a3m-router serve
|
|
|
192
190
|
<section>
|
|
193
191
|
<div class="stats-grid">
|
|
194
192
|
<div class="stat-card">
|
|
195
|
-
<div class="stat-value">
|
|
196
|
-
<div class="stat-label">
|
|
197
|
-
</div>
|
|
193
|
+
<div class="stat-value">67%</div>
|
|
194
|
+
<div class="stat-label"> </div>
|
|
198
195
|
<div class="stat-card">
|
|
199
196
|
<div class="stat-value">67%</div>
|
|
200
197
|
<div class="stat-label">MMR-Bench Accuracy</div>
|
|
201
198
|
</div>
|
|
202
199
|
<div class="stat-card">
|
|
203
|
-
<div class="stat-value"
|
|
204
|
-
<div class="stat-label">
|
|
200
|
+
<div class="stat-value">63% savings</div>
|
|
201
|
+
<div class="stat-label">Avg Cost per 1K</div>
|
|
205
202
|
</div>
|
|
206
203
|
<div class="stat-card">
|
|
207
204
|
<div class="stat-value">63.5%</div>
|
|
@@ -222,7 +219,7 @@ npx a3m-router serve
|
|
|
222
219
|
<section>
|
|
223
220
|
<h2>🔥 What Makes A3M Different</h2>
|
|
224
221
|
<div class="callout callout-info">
|
|
225
|
-
<strong>Everyone does sequential fallback.</strong> A3M combines parallel multi-LLM execution, semantic cache, provider health, and cost-aware routing — validated by official baselines in
|
|
222
|
+
<strong>Everyone does sequential fallback.</strong> A3M combines parallel multi-LLM execution, semantic cache, provider health, and cost-aware routing — validated by official baselines in MMR-Bench (ArXiv 2026), and RouterEval (EMNLP 2025).
|
|
226
223
|
</div>
|
|
227
224
|
|
|
228
225
|
<div class="table-wrapper">
|
|
@@ -347,7 +344,7 @@ npx a3m-router serve
|
|
|
347
344
|
<section>
|
|
348
345
|
<h2>💰 Cost / Accuracy / Robustness</h2>
|
|
349
346
|
<div class="callout callout-success">
|
|
350
|
-
<strong>Official baseline in
|
|
347
|
+
<strong>Official baseline in peer-reviewed benchmarks: MMR-Bench (ArXiv 2026), RouterEval (EMNLP 2025).
|
|
351
348
|
</div>
|
|
352
349
|
|
|
353
350
|
<!-- Official Baseline Status -->
|
|
@@ -357,8 +354,8 @@ npx a3m-router serve
|
|
|
357
354
|
<tr><th>Benchmark</th><th>Status</th><th>Key Metrics</th></tr>
|
|
358
355
|
</thead>
|
|
359
356
|
<tbody>
|
|
360
|
-
|
|
361
|
-
|
|
357
|
+
|
|
358
|
+
|
|
362
359
|
<tr><td>MMR-Bench (ArXiv 2026)</td><td>Baseline merged — PR#4</td><td>Accuracy 67% | Cost savings 63.5% | Robustness 0.86</td></tr>
|
|
363
360
|
<tr><td>RouterEval (EMNLP 2025)</td><td>Baseline merged — PR#4</td><td>Custom evaluation baseline (metrics pending)</td></tr>
|
|
364
361
|
<tr><td>LLMRouterBench (ACL 2026)</td><td>Baseline submitted — PR#3</td><td>Pending evaluation</td></tr>
|
|
@@ -372,20 +369,18 @@ npx a3m-router serve
|
|
|
372
369
|
<tr><th>Metric</th><th>A3M Result</th><th>Source</th></tr>
|
|
373
370
|
</thead>
|
|
374
371
|
<tbody>
|
|
375
|
-
<tr><td>
|
|
376
|
-
<tr><td>RouterArena Accuracy</td><td><strong>96.77%</strong></td><td>8,400 queries</td></tr>
|
|
377
|
-
<tr><td>RouterArena Cost</td><td><strong>$0.0768/1K</strong></td><td>Avg across providers</td></tr>
|
|
372
|
+
<tr><td>Cost Savings</td><td><strong>63%</strong></td><td>MMR-Bench benchmark</td></tr>
|
|
378
373
|
<tr><td>MMR-Bench Accuracy</td><td><strong>67%</strong></td><td>Exact tier match</td></tr>
|
|
379
374
|
<tr><td>MMR-Bench Cost Savings</td><td><strong>63.5%</strong></td><td>vs all-premium routing</td></tr>
|
|
380
|
-
<tr><td>Robustness</td><td><strong>1.0000</strong></td><td>
|
|
375
|
+
<tr><td>Robustness</td><td><strong>1.0000</strong></td><td>0 abnormal entries</td></tr>
|
|
381
376
|
</tbody>
|
|
382
377
|
</table>
|
|
383
378
|
</div>
|
|
384
379
|
|
|
385
|
-
<!--
|
|
380
|
+
<!-- Evaluation Results -->
|
|
386
381
|
<div style="margin: 2rem 0; text-align: center;">
|
|
387
|
-
<h3 style="color: #94a3b8; font-size: 0.85rem; font-weight: 600; margin-bottom: 1rem;">
|
|
388
|
-
<img src="assets/chart-
|
|
382
|
+
<h3 style="color: #94a3b8; font-size: 0.85rem; font-weight: 600; margin-bottom: 1rem;">COST SAVINGS VS PREMIUM-ONLY ROUTING — MMR-BENCH</h3>
|
|
383
|
+
<img src="assets/chart-cost-savings.svg" alt="Cost Savings" style="max-width: 100%; border-radius: 8px; margin-bottom: 0.5rem;">
|
|
389
384
|
</div>
|
|
390
385
|
|
|
391
386
|
<!-- Local Benchmark Charts -->
|
|
@@ -416,7 +411,7 @@ npx a3m-router serve
|
|
|
416
411
|
<tbody>
|
|
417
412
|
<tr><td>Parallel ensemble</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
|
418
413
|
<tr><td>Confidence scoring</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
|
419
|
-
<tr><td>Routing accuracy</td><td><strong>
|
|
414
|
+
<tr><td>Routing accuracy</td><td><strong>67%</strong></td><td>Manual</td><td>Manual</td><td>Manual</td></tr>
|
|
420
415
|
<tr><td>Self-hosted</td><td class="check">✅</td><td class="check">✅</td><td class="cross">❌</td><td class="check">✅</td></tr>
|
|
421
416
|
<tr><td>Semantic cache</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
|
422
417
|
<tr><td>Budget enforcement</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|