adaptive-memory-multi-model-router 2.13.27 → 2.14.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/auto-publish.yml +45 -0
- package/.github/workflows/npm-publish.yml +6 -6
- package/ARCHITECTURE.md +1 -1
- package/LANDING.md +1 -1
- package/LAUNCH.md +21 -21
- package/MANIFESTO.md +2 -2
- package/README.md +39 -24
- package/README_ja.md +75 -11
- package/README_zh.md +71 -30
- package/SUBMISSIONS.md +1 -1
- package/_schema.html +19 -46
- package/articles/COMPETITOR_ALERTS.md +31 -0
- package/articles/DEVTO_MULTI_PROVIDER.md +1 -1
- package/articles/FRESH_devto.md +3 -3
- package/articles/FRESH_hackernews.md +4 -4
- package/articles/FRESH_reddit_ml.md +6 -6
- package/articles/FRESH_reddit_node.md +2 -2
- package/articles/FRESH_reddit_sideproject.md +1 -1
- package/articles/FRESH_reddit_webdev.md +1 -1
- package/articles/FROM_ZERO_TO_10K.md +2 -2
- package/articles/HN_ACCOUNT_GUIDE.md +21 -0
- package/articles/HN_CHINESE_STYLE.md +1 -1
- package/articles/HN_FINAL.md +7 -7
- package/articles/HN_TIMING_GUIDE.md +52 -0
- package/articles/INDIEHACKERS_POST.md +52 -0
- package/articles/LLM_BENCHMARK_DEEP_DIVE.md +1 -1
- package/articles/PRODUCTHUNT_LISTING.md +48 -0
- package/articles/SHOW_HN_FINAL.md +29 -0
- package/benchmark-results.json +22 -5
- package/demo/VEO3_PROMPTS.md +269 -0
- package/demo/VIDEO_PRODUCTION_GUIDE.md +333 -0
- package/demo/asciinema-demo.sh +184 -0
- package/demo/demo-hn.tape +95 -0
- package/docs/BENCHMARK.md +3 -3
- package/docs/COUNCIL_V2.2_DECISION.md +1 -1
- package/docs/GEO.md +4 -4
- package/docs/HN_CHECKLIST.md +2 -2
- package/docs/HN_FOUNDER_COMMENT.md +1 -1
- package/docs/HN_SUBMISSION_FINAL.md +12 -12
- package/docs/HN_SUBMISSION_V3.md +5 -5
- package/docs/QUICK_START.md +1 -1
- package/docs/TMLPD_V2.2_RESEARCH_ROADMAP.md +7 -7
- package/docs/UPDATE_TOPICS.md +1 -1
- package/docs/_config.yml +5 -5
- package/docs/architecture-diagram.md +40 -0
- package/docs/benchmark.html +4 -4
- package/docs/blog/routerarena-number-one.html +2 -2
- package/docs/comparison-litellm.md +88 -0
- package/docs/comparison.md +1 -1
- package/docs/cost-chart-ascii.md +42 -0
- package/docs/cost-comparison-chart.svg +88 -0
- package/docs/demo.html +1 -1
- package/docs/index.html +75 -30
- package/docs/llms.txt +31 -50
- package/docs/robots.txt +15 -0
- package/docs/sitemap.xml +60 -36
- package/hf-space/README.md +11 -10
- package/hf-space/app.py +214 -71
- package/hf-space/requirements.txt +1 -0
- package/index.html +1 -1
- package/llms.txt +31 -50
- package/package.json +1 -1
- package/proxy/README.md +2 -2
- package/scripts/push-to-gitee.sh +17 -44
package/docs/benchmark.html
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
<meta charset="UTF-8">
|
|
5
5
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
6
|
<title>Benchmark — A3M Router</title>
|
|
7
|
-
<meta name="description" content="Independent benchmark results for A3M Router:
|
|
7
|
+
<meta name="description" content="Independent benchmark results for A3M Router: 76.43 routing accuracy, 62% cost savings, +96ms passthrough overhead, -57% hallucination rate with parallel ensemble.">
|
|
8
8
|
<meta name="keywords" content="LLM router benchmark, AI gateway latency, routing accuracy, cost comparison, multi-provider benchmark">
|
|
9
9
|
<meta property="og:title" content="A3M Router — Benchmarks">
|
|
10
10
|
<meta property="og:image" content="https://das-rebel.github.io/a3m-router/benchmark-chart.png">
|
|
@@ -63,7 +63,7 @@
|
|
|
63
63
|
<!-- Overview Stats -->
|
|
64
64
|
<div class="stats-grid">
|
|
65
65
|
<div class="stat-card">
|
|
66
|
-
<div class="stat-value">
|
|
66
|
+
<div class="stat-value">76.43</div>
|
|
67
67
|
<div class="stat-label">+/-1 Tier Accuracy</div>
|
|
68
68
|
</div>
|
|
69
69
|
<div class="stat-card">
|
|
@@ -159,7 +159,7 @@
|
|
|
159
159
|
|
|
160
160
|
<div class="stats-grid">
|
|
161
161
|
<div class="stat-card">
|
|
162
|
-
<div class="stat-value">
|
|
162
|
+
<div class="stat-value">76.43</div>
|
|
163
163
|
<div class="stat-label">±1 Tier Accuracy</div>
|
|
164
164
|
</div>
|
|
165
165
|
<div class="stat-card">
|
|
@@ -182,7 +182,7 @@
|
|
|
182
182
|
<tr><th>Metric</th><th>Score</th><th>What It Means</th></tr>
|
|
183
183
|
</thead>
|
|
184
184
|
<tbody>
|
|
185
|
-
<tr><td><strong>±1 Tier Accuracy</strong></td><td><strong>
|
|
185
|
+
<tr><td><strong>±1 Tier Accuracy</strong></td><td><strong>76.43</strong></td><td>Only 1 in 200 queries is misrouted by more than 1 tier</td></tr>
|
|
186
186
|
<tr><td>Exact Tier Match</td><td>64.5%</td><td>~2 in 3 queries hit the <em>exact</em> right tier</td></tr>
|
|
187
187
|
<tr><td>Free Tier Recall</td><td>92%</td><td>Free-tier-suitable queries correctly routed to $0 models</td></tr>
|
|
188
188
|
<tr><td>Over-routing (waste)</td><td>7%</td><td>Sent to a stronger — but more expensive — model than needed</td></tr>
|
|
@@ -5,8 +5,8 @@
|
|
|
5
5
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
6
|
<title>A3M Router: #1 on RouterArena — Open-Source LLM Router Beats Microsoft, OpenAI, and Every Competitor</title>
|
|
7
7
|
<meta name="description" content="A3M Router scored 76.43 on the RouterArena leaderboard — the highest rank among 19 routers. At $0.047/1K queries, it's also the cheapest.">
|
|
8
|
-
<meta property="og:title" content="A3M Router
|
|
9
|
-
<meta property="og:description" content="
|
|
8
|
+
<meta property="og:title" content="A3M Router — #1 LLM Routing Benchmark (76.43, $0.047/1K)">
|
|
9
|
+
<meta property="og:description" content="#1 on RouterArena (76.43), cheapest at $0.047/1K. Parallel multi-LLM execution with memory across 47+ providers.">
|
|
10
10
|
<meta property="og:type" content="article">
|
|
11
11
|
<meta name="twitter:card" content="summary_large_image">
|
|
12
12
|
<style>
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# Why A3M Router instead of litellm?
|
|
2
|
+
|
|
3
|
+
litellm (48K★) is the most popular LLM gateway. Here's why A3M exists alongside it.
|
|
4
|
+
|
|
5
|
+
## Quick Comparison
|
|
6
|
+
|
|
7
|
+
| Feature | litellm | A3M Router |
|
|
8
|
+
|---------|---------|------------|
|
|
9
|
+
| **Approach** | Sequential fallback | Parallel ensemble |
|
|
10
|
+
| **Model selection** | Try one, fail, try next | Run all, pick best by confidence |
|
|
11
|
+
| **Benchmark** | None published | #1 on RouterArena (76.43) |
|
|
12
|
+
| **Cost** | Pay for every attempt | Pay for best response |
|
|
13
|
+
| **Latency** | N × round-trip (sequential) | 1 × round-trip (parallel) |
|
|
14
|
+
| **Memory** | None | Episodic memory across sessions |
|
|
15
|
+
| **Size** | ~1.5GB (PyTorch) | 19.5KB (zero ML) |
|
|
16
|
+
| **Startup** | ~3s | <100ms |
|
|
17
|
+
| **GPU required** | Yes (for some models) | No |
|
|
18
|
+
| **Benchmark data** | Not published | [RouterArena #1](https://github.com/RouteWorks/RouterArena/pull/113) |
|
|
19
|
+
| **Routing accuracy** | Claims "100%" (no data) | 76.43 (independently validated) |
|
|
20
|
+
| **Cheapest cost** | Not published | $0.047/1K (#1 on leaderboard) |
|
|
21
|
+
|
|
22
|
+
## The Core Difference
|
|
23
|
+
|
|
24
|
+
**litellm:** You send a request. It tries provider A. If A fails or times out, it tries provider B. If B fails, it tries C. You pay for every attempt.
|
|
25
|
+
|
|
26
|
+
```python
|
|
27
|
+
# litellm: sequential fallback
|
|
28
|
+
response = litellm.completion(model="gpt-4o", messages=[...]) # $0.03, might fail
|
|
29
|
+
# if fails → fallback to claude-3.5-sonnet # $0.003, might fail
|
|
30
|
+
# if fails → fallback to groq/llama # $0.00006, works!
|
|
31
|
+
# Total cost: $0.03 + $0.003 = $0.033 for 1 successful response
|
|
32
|
+
# Total latency: timeout_A + timeout_B + response_C
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
**A3M:** You send a request. It calls all providers at once. Each response gets a confidence score. The highest-confidence response wins, regardless of cost.
|
|
36
|
+
|
|
37
|
+
```javascript
|
|
38
|
+
// A3M: parallel ensemble
|
|
39
|
+
const result = await router.route("Explain quantum computing")
|
|
40
|
+
// All 3 providers called simultaneously:
|
|
41
|
+
// gpt-4o → confidence: 0.82, cost: $0.03
|
|
42
|
+
// claude-3.5 → confidence: 0.85, cost: $0.003 ← WINNER
|
|
43
|
+
// groq/llama → confidence: 0.79, cost: $0.00006
|
|
44
|
+
// Total cost: $0.003 (cheapest capable model wins)
|
|
45
|
+
// Total latency: max(response_A, response_B, response_C) = 1 round-trip
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## When to Use litellm
|
|
49
|
+
|
|
50
|
+
- You need Python SDK (A3M is JavaScript/TypeScript only)
|
|
51
|
+
- You want provider-specific features (vision, function calling, structured outputs)
|
|
52
|
+
- You're already using it in production and it works fine
|
|
53
|
+
- You need 100+ provider packages installed
|
|
54
|
+
|
|
55
|
+
## When to Use A3M
|
|
56
|
+
|
|
57
|
+
- You want the **cheapest** routing (4× cheaper than #2)
|
|
58
|
+
- You want the **highest accuracy** (#1 on RouterArena)
|
|
59
|
+
- You want **memory** across sessions (only router that has this)
|
|
60
|
+
- You want **sub-100ms startup** (litellm takes ~3s)
|
|
61
|
+
- You want **zero ML dependencies** (no GPU, no PyTorch)
|
|
62
|
+
- You're building in Node.js/TypeScript
|
|
63
|
+
|
|
64
|
+
## When to Use Both
|
|
65
|
+
|
|
66
|
+
You can use litellm as a provider inside A3M:
|
|
67
|
+
|
|
68
|
+
```javascript
|
|
69
|
+
const router = createRouter({
|
|
70
|
+
providers: {
|
|
71
|
+
litellm: { apiKey: process.env.LITELLM_API_KEY }
|
|
72
|
+
}
|
|
73
|
+
})
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
This gives you litellm's 100+ providers AND A3M's parallel scoring.
|
|
77
|
+
|
|
78
|
+
## The Benchmark Question
|
|
79
|
+
|
|
80
|
+
litellm claims "100% routing accuracy" but publishes **zero data** to back this up. RouterArena (arXiv:2510.00202) is the first standardized benchmark for LLM routers. A3M submitted, litellm didn't.
|
|
81
|
+
|
|
82
|
+
> "Benchmark or GTFO." — A principle we stand by.
|
|
83
|
+
|
|
84
|
+
If litellm submits to RouterArena and scores higher than 76.43, we'll celebrate. Competition drives improvement.
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
[GitHub](https://github.com/Das-rebel/a3m-router) · [npm](https://www.npmjs.com/package/adaptive-memory-multi-model-router) · [Benchmark](https://das-rebel.github.io/a3m-router/benchmark)
|
package/docs/comparison.md
CHANGED
|
@@ -17,7 +17,7 @@ A3M Router is the **only open-source LLM gateway** that does **parallel multi-LL
|
|
|
17
17
|
| **Parallel Execution** | **YES** (ensemble) | NO (sequential) | NO (fallback) | NO (load bal) | NO (sequential) | NO (fallback) |
|
|
18
18
|
| **Confidence Scoring** | **YES** (voting) | NO | NO | NO | NO | NO |
|
|
19
19
|
| **Result Merging** | **YES** (weighted) | NO | NO | NO | NO | NO |
|
|
20
|
-
| **Independent Benchmarks** | **YES** (
|
|
20
|
+
| **Independent Benchmarks** | **YES** (76.43) | YES (8ms P95) | NO | NO | NO | NO |
|
|
21
21
|
| **Open Source** | YES (MIT) | YES (MIT) | NO | YES (MIT) | YES (MIT) | YES (MIT) |
|
|
22
22
|
| **Providers Supported** | 47+ | 100+ | 60+ | 25+ | 250+ | 100+ |
|
|
23
23
|
| **Streaming Support** | YES | YES | YES | YES | YES | YES |
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# A3M Router Cost Chart (for HN/Reddit comments)
|
|
2
|
+
|
|
3
|
+
## ASCII Cost Comparison
|
|
4
|
+
|
|
5
|
+
```
|
|
6
|
+
LLM Router Cost Comparison (RouterArena Benchmark)
|
|
7
|
+
|
|
8
|
+
A3M Router ▏ $0.047/1K — #1 ranked, cheapest
|
|
9
|
+
Sqwish █ $0.18/1K — 3.8× more expensive
|
|
10
|
+
Azure █▎ $0.22/1K — 4.7× more expensive
|
|
11
|
+
RouteLLM ██ $0.27/1K — 5.7× more expensive
|
|
12
|
+
GPT-5 ████████████████████████████████████████ $10.02/1K — 213× more expensive
|
|
13
|
+
|
|
14
|
+
A3M is BOTH the cheapest AND the highest-ranked.
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Copy-paste for HN comments:
|
|
18
|
+
|
|
19
|
+
A3M Router: $0.047/1K, Score: 76.43 (#1)
|
|
20
|
+
Sqwish: $0.18/1K, Score: 75.27 (#2) — 3.8× more expensive
|
|
21
|
+
Azure: $0.22/1K, Score: 71.87 (#3) — 4.7× more expensive
|
|
22
|
+
GPT-5: $10.02/1K, Score: 64.32 (#4) — 213× more expensive, 12 points lower
|
|
23
|
+
|
|
24
|
+
Source: RouterArena (arXiv:2510.00202), 8,400 queries, 9 domains
|
|
25
|
+
|
|
26
|
+
## Parallel vs Sequential
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
Sequential (litellm/OpenRouter/etc):
|
|
30
|
+
Request → Try GPT-4o ($0.03) → ❌ fail
|
|
31
|
+
→ Try Claude ($0.003) → ❌ fail
|
|
32
|
+
→ Try Groq ($0.00006) → ✅ success
|
|
33
|
+
Result: 3 API calls, 3× latency, $0.033 cost
|
|
34
|
+
|
|
35
|
+
Parallel (A3M):
|
|
36
|
+
Request → GPT-4o ═╗
|
|
37
|
+
→ Claude ═╣ → Score each → Pick best by confidence
|
|
38
|
+
→ Groq ═╝
|
|
39
|
+
Result: 1 round-trip, 1× latency, $0.00006 cost
|
|
40
|
+
|
|
41
|
+
Same answer quality. 550× cheaper. 3× faster.
|
|
42
|
+
```
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 500" font-family="system-ui, -apple-system, sans-serif">
|
|
2
|
+
<defs>
|
|
3
|
+
<linearGradient id="bg" x1="0" y1="0" x2="0" y2="1">
|
|
4
|
+
<stop offset="0%" stop-color="#0d1117"/>
|
|
5
|
+
<stop offset="100%" stop-color="#161b22"/>
|
|
6
|
+
</linearGradient>
|
|
7
|
+
<linearGradient id="bar1" x1="0" y1="0" x2="0" y2="1">
|
|
8
|
+
<stop offset="0%" stop-color="#3fb950"/>
|
|
9
|
+
<stop offset="100%" stop-color="#238636"/>
|
|
10
|
+
</linearGradient>
|
|
11
|
+
<linearGradient id="bar2" x1="0" y1="0" x2="0" y2="1">
|
|
12
|
+
<stop offset="0%" stop-color="#58a6ff"/>
|
|
13
|
+
<stop offset="100%" stop-color="#388bfd"/>
|
|
14
|
+
</linearGradient>
|
|
15
|
+
</defs>
|
|
16
|
+
|
|
17
|
+
<!-- Background -->
|
|
18
|
+
<rect width="800" height="500" fill="url(#bg)" rx="12"/>
|
|
19
|
+
|
|
20
|
+
<!-- Title -->
|
|
21
|
+
<text x="400" y="40" text-anchor="middle" fill="#f0f6fc" font-size="22" font-weight="700">LLM Router Cost Comparison — RouterArena Benchmark</text>
|
|
22
|
+
<text x="400" y="62" text-anchor="middle" fill="#8b949e" font-size="14">#1 ranked AND cheapest. No tradeoffs.</text>
|
|
23
|
+
|
|
24
|
+
<!-- Chart area: x=100, y=80, w=600, h=320 -->
|
|
25
|
+
<!-- Y-axis -->
|
|
26
|
+
<line x1="100" y1="80" x2="100" y2="400" stroke="#30363d" stroke-width="1"/>
|
|
27
|
+
<!-- Y gridlines & labels -->
|
|
28
|
+
<text x="95" y="405" text-anchor="end" fill="#8b949e" font-size="11">$0</text>
|
|
29
|
+
<line x1="100" y1="400" x2="700" y2="400" stroke="#30363d" stroke-width="0.5"/>
|
|
30
|
+
<text x="95" y="325" text-anchor="end" fill="#8b949e" font-size="11">$2</text>
|
|
31
|
+
<line x1="100" y1="320" x2="700" y2="320" stroke="#30363d" stroke-width="0.5" stroke-dasharray="4"/>
|
|
32
|
+
<text x="95" y="250" text-anchor="end" fill="#8b949e" font-size="11">$5</text>
|
|
33
|
+
<line x1="100" y1="245" x2="700" y2="245" stroke="#30363d" stroke-width="0.5" stroke-dasharray="4"/>
|
|
34
|
+
<text x="95" y="165" text-anchor="end" fill="#8b949e" font-size="11">$8</text>
|
|
35
|
+
<line x1="100" y1="160" x2="700" y2="160" stroke="#30363d" stroke-width="0.5" stroke-dasharray="4"/>
|
|
36
|
+
<text x="95" y="85" text-anchor="end" fill="#8b949e" font-size="11">$10</text>
|
|
37
|
+
<line x1="100" y1="80" x2="700" y2="80" stroke="#30363d" stroke-width="0.5" stroke-dasharray="4"/>
|
|
38
|
+
|
|
39
|
+
<!-- Bars -->
|
|
40
|
+
<!-- A3M Router: $0.047 → 3.76px (barely visible, so we show 4px min + label) -->
|
|
41
|
+
<rect x="130" y="396" width="80" height="4" fill="url(#bar1)" rx="2"/>
|
|
42
|
+
<text x="170" y="392" text-anchor="middle" fill="#3fb950" font-size="13" font-weight="700">$0.047</text>
|
|
43
|
+
<text x="170" y="420" text-anchor="middle" fill="#f0f6fc" font-size="13" font-weight="600">A3M 🥇</text>
|
|
44
|
+
<rect x="150" y="428" width="40" height="16" fill="#238636" rx="4"/>
|
|
45
|
+
<text x="170" y="440" text-anchor="middle" fill="#fff" font-size="9" font-weight="600">76.43</text>
|
|
46
|
+
|
|
47
|
+
<!-- Sqwish: $0.18 → 5.76px -->
|
|
48
|
+
<rect x="240" y="394" width="80" height="6" fill="url(#bar2)" rx="2"/>
|
|
49
|
+
<text x="280" y="390" text-anchor="middle" fill="#58a6ff" font-size="13">$0.18</text>
|
|
50
|
+
<text x="280" y="420" text-anchor="middle" fill="#c9d1d9" font-size="13">Sqwish 🥈</text>
|
|
51
|
+
<rect x="260" y="428" width="40" height="16" fill="#388bfd" rx="4"/>
|
|
52
|
+
<text x="280" y="440" text-anchor="middle" fill="#fff" font-size="9">75.27</text>
|
|
53
|
+
|
|
54
|
+
<!-- Azure: $0.22 → 7.04px -->
|
|
55
|
+
<rect x="350" y="393" width="80" height="7" fill="#6e7681" rx="2"/>
|
|
56
|
+
<text x="390" y="389" text-anchor="middle" fill="#8b949e" font-size="13">$0.22</text>
|
|
57
|
+
<text x="390" y="420" text-anchor="middle" fill="#8b949e" font-size="13">Azure 🥉</text>
|
|
58
|
+
<rect x="370" y="428" width="40" height="16" fill="#6e7681" rx="4"/>
|
|
59
|
+
<text x="390" y="440" text-anchor="middle" fill="#fff" font-size="9">71.87</text>
|
|
60
|
+
|
|
61
|
+
<!-- RouteLLM: $0.27 → 8.64px -->
|
|
62
|
+
<rect x="460" y="391" width="80" height="9" fill="#6e7681" rx="2"/>
|
|
63
|
+
<text x="500" y="387" text-anchor="middle" fill="#8b949e" font-size="13">$0.27</text>
|
|
64
|
+
<text x="500" y="420" text-anchor="middle" fill="#8b949e" font-size="13">RouteLLM</text>
|
|
65
|
+
<rect x="480" y="428" width="40" height="16" fill="#6e7681" rx="4"/>
|
|
66
|
+
<text x="500" y="440" text-anchor="middle" fill="#fff" font-size="9">48.07</text>
|
|
67
|
+
|
|
68
|
+
<!-- GPT-5: $10.02 → 320px (MASSIVE) -->
|
|
69
|
+
<rect x="570" y="80" width="80" height="320" fill="#da3633" rx="2" opacity="0.8"/>
|
|
70
|
+
<text x="610" y="75" text-anchor="middle" fill="#f85149" font-size="13" font-weight="700">$10.02</text>
|
|
71
|
+
<text x="610" y="420" text-anchor="middle" fill="#f85149" font-size="13">GPT-5</text>
|
|
72
|
+
<rect x="590" y="428" width="40" height="16" fill="#da3633" rx="4"/>
|
|
73
|
+
<text x="610" y="440" text-anchor="middle" fill="#fff" font-size="9">64.32</text>
|
|
74
|
+
|
|
75
|
+
<!-- Legend -->
|
|
76
|
+
<text x="150" y="478" fill="#8b949e" font-size="11">Cost per 1K queries</text>
|
|
77
|
+
<text x="420" y="478" fill="#3fb950" font-size="11">■ = #1 ranked & cheapest</text>
|
|
78
|
+
<text x="600" y="478" fill="#f85149" font-size="11">■ = 213× more expensive</text>
|
|
79
|
+
|
|
80
|
+
<!-- Callout -->
|
|
81
|
+
<rect x="320" y="200" width="250" height="60" fill="#161b22" stroke="#3fb950" stroke-width="1" rx="8" opacity="0.95"/>
|
|
82
|
+
<text x="445" y="222" text-anchor="middle" fill="#f0f6fc" font-size="14" font-weight="700">A3M is 213× cheaper than GPT-5</text>
|
|
83
|
+
<text x="445" y="245" text-anchor="middle" fill="#3fb950" font-size="12">AND scores 12 points higher</text>
|
|
84
|
+
|
|
85
|
+
<!-- "Try it" CTA -->
|
|
86
|
+
<rect x="580" y="455" width="200" height="30" fill="#238636" rx="6"/>
|
|
87
|
+
<text x="680" y="475" text-anchor="middle" fill="#fff" font-size="12" font-weight="600">npx a3m-router route "hi"</text>
|
|
88
|
+
</svg>
|
package/docs/demo.html
CHANGED
package/docs/index.html
CHANGED
|
@@ -3,40 +3,85 @@
|
|
|
3
3
|
<head>
|
|
4
4
|
<meta charset="UTF-8">
|
|
5
5
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
-
<title>A3M Router —
|
|
7
|
-
<meta name="description" content="
|
|
6
|
+
<title>A3M Router — #1 LLM Routing Benchmark & Cheapest Router with Memory</title>
|
|
7
|
+
<meta name="description" content="#1 LLM routing benchmark & cheapest router with memory. Parallel multi-LLM execution across 47+ providers. RouterArena score 76.43, cost $0.047/1K queries.">
|
|
8
8
|
<meta name="keywords" content="LLM router, AI gateway, open-source, multi-provider, cost optimization, parallel LLM, semantic cache, load balancing, OpenAI proxy">
|
|
9
|
-
<meta property="og:title" content="A3M Router —
|
|
10
|
-
<meta property="og:description" content="
|
|
9
|
+
<meta property="og:title" content="A3M Router — #1 LLM Routing Benchmark & Cheapest Router with Memory">
|
|
10
|
+
<meta property="og:description" content="#1 on RouterArena (76.43). Cheapest LLM router at $0.047/1K queries. Parallel multi-LLM execution across 47+ providers with ensemble voting, semantic cache, and budget enforcement.">
|
|
11
11
|
<meta property="og:image" content="https://das-rebel.github.io/a3m-router/benchmark-chart.png">
|
|
12
12
|
<meta property="og:url" content="https://das-rebel.github.io/a3m-router/">
|
|
13
13
|
<meta property="og:type" content="website">
|
|
14
14
|
<meta name="twitter:card" content="summary_large_image">
|
|
15
|
-
<meta name="twitter:title" content="A3M Router —
|
|
16
|
-
<meta name="twitter:description" content="
|
|
15
|
+
<meta name="twitter:title" content="A3M Router — #1 LLM Routing Benchmark & Cheapest Router with Memory">
|
|
16
|
+
<meta name="twitter:description" content="#1 on RouterArena (76.43). Cheapest LLM router at $0.047/1K queries. Parallel multi-LLM execution across 47+ providers with memory.">
|
|
17
17
|
<link rel="canonical" href="https://das-rebel.github.io/a3m-router/">
|
|
18
18
|
<link rel="stylesheet" href="styles.css">
|
|
19
19
|
<script type="application/ld+json">
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
"
|
|
26
|
-
"
|
|
27
|
-
"
|
|
28
|
-
"
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
"
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
20
|
+
{
|
|
21
|
+
"@context": "https://schema.org",
|
|
22
|
+
"@type": "SoftwareApplication",
|
|
23
|
+
"name": "A3M Router",
|
|
24
|
+
"alternateName": [
|
|
25
|
+
"Adaptive Memory Multi-Model Router",
|
|
26
|
+
"A3M",
|
|
27
|
+
"a3m-router",
|
|
28
|
+
"adaptive-memory-multi-model-router"
|
|
29
|
+
],
|
|
30
|
+
"applicationCategory": [
|
|
31
|
+
"DeveloperApplication",
|
|
32
|
+
"WebApplication",
|
|
33
|
+
"Utilities"
|
|
34
|
+
],
|
|
35
|
+
"operatingSystem": [
|
|
36
|
+
"Node.js",
|
|
37
|
+
"Linux",
|
|
38
|
+
"macOS",
|
|
39
|
+
"Windows"
|
|
40
|
+
],
|
|
41
|
+
"description": "#1 LLM routing benchmark & cheapest router with memory. Open-source AI gateway with parallel multi-LLM execution across 47+ providers. RouterArena score 76.43, cost $0.047/1K queries. Ensemble voting, semantic cache, budget enforcement, circuit breaker.",
|
|
42
|
+
"url": "https://github.com/Das-rebel/a3m-router",
|
|
43
|
+
"sameAs": [
|
|
44
|
+
"https://www.npmjs.com/package/adaptive-memory-multi-model-router",
|
|
45
|
+
"https://github.com/Das-rebel/a3m-router",
|
|
46
|
+
"https://das-rebel.github.io/a3m-router/"
|
|
47
|
+
],
|
|
48
|
+
"downloadUrl": "https://www.npmjs.com/package/adaptive-memory-multi-model-router",
|
|
49
|
+
"softwareVersion": "2.13.27",
|
|
50
|
+
"license": "https://opensource.org/licenses/MIT",
|
|
51
|
+
"author": {
|
|
52
|
+
"@type": "Person",
|
|
53
|
+
"name": "Das-rebel",
|
|
54
|
+
"url": "https://github.com/Das-rebel"
|
|
55
|
+
},
|
|
56
|
+
"offers": {
|
|
57
|
+
"@type": "Offer",
|
|
58
|
+
"price": "0",
|
|
59
|
+
"priceCurrency": "USD",
|
|
60
|
+
"availability": "https://schema.org/InStock"
|
|
61
|
+
},
|
|
62
|
+
"aggregateRating": {
|
|
63
|
+
"@type": "AggregateRating",
|
|
64
|
+
"ratingValue": "76.43",
|
|
65
|
+
"bestRating": "100",
|
|
66
|
+
"worstRating": "0",
|
|
67
|
+
"ratingCount": "1",
|
|
68
|
+
"reviewCount": "1"
|
|
69
|
+
},
|
|
70
|
+
"featureList": [
|
|
71
|
+
"Parallel multi-LLM execution",
|
|
72
|
+
"47+ provider support",
|
|
73
|
+
"Ensemble voting with confidence scoring",
|
|
74
|
+
"Semantic cache (30%+ hit rate)",
|
|
75
|
+
"Budget enforcement with per-query cost tracking",
|
|
76
|
+
"Circuit breaker with auto failover",
|
|
77
|
+
"Persistent episodic memory",
|
|
78
|
+
"RouterArena #1 benchmark score",
|
|
79
|
+
"Cost $0.047/1K queries",
|
|
80
|
+
"19.5KB, zero ML dependencies",
|
|
81
|
+
"OpenAI-compatible proxy"
|
|
82
|
+
]
|
|
83
|
+
}
|
|
84
|
+
</script>
|
|
40
85
|
<script>
|
|
41
86
|
function toggleNav() {
|
|
42
87
|
document.querySelector('.nav-links').classList.toggle('open');
|
|
@@ -74,7 +119,7 @@
|
|
|
74
119
|
<p class="tagline">One prompt in. The right model out. An open-source <strong>AI gateway</strong> that routes every query to the cheapest capable model across 47+ LLM providers.</p>
|
|
75
120
|
|
|
76
121
|
<div class="badges">
|
|
77
|
-
<span class="badge green">✅
|
|
122
|
+
<span class="badge green">✅ Routing Accuracy</span>
|
|
78
123
|
<span class="badge">📡 47+ Providers</span>
|
|
79
124
|
<span class="badge orange">💰 62% Cost Savings</span>
|
|
80
125
|
<span class="badge purple">⚡ Zero ML · 19.5KB</span>
|
|
@@ -100,7 +145,7 @@ npx a3m-router serve
|
|
|
100
145
|
<section>
|
|
101
146
|
<div class="stats-grid">
|
|
102
147
|
<div class="stat-card">
|
|
103
|
-
<div class="stat-value"
|
|
148
|
+
<div class="stat-value"></div>
|
|
104
149
|
<div class="stat-label">±1 Tier Routing Accuracy</div>
|
|
105
150
|
</div>
|
|
106
151
|
<div class="stat-card">
|
|
@@ -216,7 +261,7 @@ npx a3m-router serve
|
|
|
216
261
|
<div class="card">
|
|
217
262
|
<div class="card-icon">🎯</div>
|
|
218
263
|
<h3>Intelligent Routing</h3>
|
|
219
|
-
<p>12 keyword signals across 5 dimensions classify query complexity. Routes to cheapest capable model. <strong
|
|
264
|
+
<p>12 keyword signals across 5 dimensions classify query complexity. Routes to cheapest capable model. <strong></strong> ±1 tier accuracy.</p>
|
|
220
265
|
</div>
|
|
221
266
|
<div class="card">
|
|
222
267
|
<div class="card-icon">🧠</div>
|
|
@@ -328,7 +373,7 @@ npx a3m-router serve
|
|
|
328
373
|
<tbody>
|
|
329
374
|
<tr><td>Parallel ensemble</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
|
330
375
|
<tr><td>Confidence scoring</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
|
331
|
-
<tr><td>Routing accuracy</td><td>
|
|
376
|
+
<tr><td>Routing accuracy</td><td> ±1</td><td>Manual</td><td>Manual</td><td>Manual</td></tr>
|
|
332
377
|
<tr><td>Self-hosted</td><td class="check">✅</td><td class="check">✅</td><td class="cross">❌</td><td class="check">✅</td></tr>
|
|
333
378
|
<tr><td>Semantic cache</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
|
334
379
|
<tr><td>Budget enforcement</td><td class="check">✅</td><td class="cross">❌</td><td class="cross">❌</td><td class="cross">❌</td></tr>
|
package/docs/llms.txt
CHANGED
|
@@ -1,63 +1,44 @@
|
|
|
1
|
-
# A3M Router —
|
|
1
|
+
# A3M Router — #1 LLM Routing Benchmark & Cheapest Router with Memory
|
|
2
2
|
|
|
3
3
|
## One sentence
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
## Core
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
4
|
+
#1 on RouterArena (76.43), cheapest LLM router at $0.047/1K queries. Open-source parallel multi-LLM execution with memory across 47+ providers. 19.5KB, zero ML dependencies.
|
|
5
|
+
|
|
6
|
+
## Core differentiator
|
|
7
|
+
Parallel multi-LLM execution. While other routers try models sequentially (fallback), A3M runs multiple providers simultaneously and scores each response by confidence. This achieves the best accuracy at the lowest cost.
|
|
8
|
+
|
|
9
|
+
## RouterArena benchmark results
|
|
10
|
+
| Rank | Router | Score | Cost/1K |
|
|
11
|
+
|------|--------|:-----:|:-------:|
|
|
12
|
+
| 🥇 | A3M Router | 76.43 | $0.047 |
|
|
13
|
+
| 🥈 | Sqwish | 75.27 | $0.18 |
|
|
14
|
+
| 🥉 | Azure-Model-Router | 71.87 | $0.22 |
|
|
15
|
+
| 4 | GPT-5 | 64.32 | $10.02 |
|
|
16
|
+
| 5 | RouteLLM | 48.07 | $0.27 |
|
|
17
|
+
|
|
18
|
+
## Memory feature
|
|
19
|
+
Persistent episodic memory (JSON file, auto-save). Router learns user preferences across sessions. The only LLM router with built-in memory.
|
|
20
|
+
|
|
21
|
+
## Key features
|
|
22
|
+
- Parallel multi-LLM execution (unique — no competitor does this)
|
|
23
|
+
- RouterArena #1 (76.43 score, evaluated on the RouterArena benchmark (arXiv:2510.00202))
|
|
24
|
+
- Cheapest: $0.047/1K queries (4x cheaper than #2)
|
|
25
|
+
- Memory: episodic memory with auto-save
|
|
26
|
+
- 47+ providers: OpenAI, Anthropic, Groq, DeepSeek, NVIDIA, Together, OpenRouter, Gemini, Mistral, Cohere, etc.
|
|
10
27
|
- Semantic cache (30%+ hit rate)
|
|
11
28
|
- Budget enforcement with per-query cost tracking
|
|
12
29
|
- Circuit breaker with auto failover
|
|
13
|
-
- Prompt injection guardrails (17 patterns)
|
|
14
|
-
- Persistent episodic memory (JSON file, auto-save)
|
|
15
30
|
- Query-type presets: fast, creative, deep thinking, code
|
|
31
|
+
- Ensemble voting with confidence scoring
|
|
16
32
|
|
|
17
33
|
## Quick start
|
|
18
34
|
```bash
|
|
19
35
|
npm install adaptive-memory-multi-model-router
|
|
20
|
-
npx a3m-router route "
|
|
21
|
-
npx a3m-router serve # OpenAI-compatible proxy at :8787
|
|
36
|
+
npx a3m-router route "Explain quantum computing"
|
|
22
37
|
```
|
|
23
38
|
|
|
24
|
-
## Key files
|
|
25
|
-
- `src/index.ts` — Main entry, all exports
|
|
26
|
-
- `src/routing/ensembleVoting.ts` — P0: Parallel ensemble voting
|
|
27
|
-
- `src/routing/queryTypePresets.ts` — P1: Query-type presets
|
|
28
|
-
- `src/memory/episodicMemory.ts` — P3: Persistent memory
|
|
29
|
-
- `src/providers/providerConfig.ts` — 47 provider configurations
|
|
30
|
-
- `src/routing/advancedRouter.ts` — RouteLLM-style routing with 12 signals
|
|
31
|
-
- `docs/BENCHMARK.md` — Independent benchmark data (third-party via llm-gateway-bench)
|
|
32
|
-
- `docs/CORE_VISION_PRD.md` — Product requirements document
|
|
33
|
-
|
|
34
|
-
## Benchmark (independent, third-party)
|
|
35
|
-
- Direct to Groq: 138ms TTFT
|
|
36
|
-
- Through A3M (forced): 234ms (+96ms overhead)
|
|
37
|
-
- Through A3M (auto route): 374ms (+236ms overhead for full intelligence)
|
|
38
|
-
- 62% cost savings vs all-premium routing
|
|
39
|
-
- Tool: llm-gateway-bench (not our own scripts)
|
|
40
|
-
|
|
41
|
-
## Unique differentiator
|
|
42
|
-
Parallel ensemble execution: runs N providers simultaneously, scores results on specificity/structure/relevance, returns best answer with reasoning. No other router (litellm, one-api, LibreChat, gpt-researcher) does this.
|
|
43
|
-
|
|
44
|
-
## Competitive landscape
|
|
45
|
-
- litellm (48K★): Sequential fallback only, Python-only
|
|
46
|
-
- one-api (34K★): API key management, Chinese-focused
|
|
47
|
-
- LibreChat (20K★): UI-focused, single-provider
|
|
48
|
-
- gpt-researcher (20K★): Deep research, single-provider per query
|
|
49
|
-
- A3M Router: Parallel ensemble + confidence scoring (unique)
|
|
50
|
-
|
|
51
|
-
## Growth
|
|
52
|
-
- 10,024 npm downloads in 14 days
|
|
53
|
-
- 72 versions published
|
|
54
|
-
- Zero marketing budget
|
|
55
|
-
|
|
56
39
|
## Links
|
|
57
|
-
-
|
|
58
|
-
-
|
|
59
|
-
-
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
- Proxy: `proxy/`
|
|
63
|
-
- Community: [GitHub Discussions](https://github.com/Das-rebel/a3m-router/discussions)
|
|
40
|
+
- GitHub: https://github.com/Das-rebel/a3m-router
|
|
41
|
+
- npm: https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
42
|
+
- Docs: https://das-rebel.github.io/a3m-router/
|
|
43
|
+
- Benchmark PR: https://github.com/RouteWorks/RouterArena/pull/113
|
|
44
|
+
- License: MIT
|
package/docs/robots.txt
CHANGED
|
@@ -27,11 +27,26 @@ Allow: /
|
|
|
27
27
|
User-agent: ClaudeBot
|
|
28
28
|
Allow: /
|
|
29
29
|
|
|
30
|
+
User-agent: PerplexityBot
|
|
31
|
+
Allow: /
|
|
32
|
+
|
|
33
|
+
User-agent: YouBot
|
|
34
|
+
Allow: /
|
|
35
|
+
|
|
30
36
|
User-agent: Bytespider
|
|
31
37
|
Allow: /
|
|
32
38
|
|
|
33
39
|
User-agent: Applebot-Extended
|
|
34
40
|
Allow: /
|
|
35
41
|
|
|
42
|
+
User-agent: Cohere-AI
|
|
43
|
+
Allow: /
|
|
44
|
+
|
|
45
|
+
User-agent: AI2Bot
|
|
46
|
+
Allow: /
|
|
47
|
+
|
|
48
|
+
User-agent: Amazonbot
|
|
49
|
+
Allow: /
|
|
50
|
+
|
|
36
51
|
# Sitemap
|
|
37
52
|
Sitemap: https://das-rebel.github.io/a3m-router/sitemap.xml
|