adaptive-memory-multi-model-router 2.0.2 → 2.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LAUNCH.md +33 -19
- package/README.md +91 -61
- package/articles/HN_FINAL.md +37 -36
- package/articles/devto-llm-routing.md +21 -5
- package/articles/hackernews-show-hn.md +23 -6
- package/articles/reddit-ml.md +10 -3
- package/articles/twitter-thread-cost-savings.md +11 -7
- package/assets/growth-chart-animated.svg +76 -0
- package/demo/demo-script.sh +62 -0
- package/demo/demo.svg +75 -0
- package/docs/HN_SUBMISSION_FINAL.md +146 -0
- package/docs/SEO_AUDIT.md +241 -0
- package/docs-site/index.html +326 -41
- package/llms.txt +29 -12
- package/package.json +3 -3
- package/public/robots.txt +13 -0
- package/public/sitemap.xml +33 -0
- package/scripts/benchmark.js +145 -0
- package/scripts/benchmark.sh +61 -0
- package/.github/workflows/npm-stats-validation.yml +0 -152
- package/.github/workflows/pages.yml +0 -37
package/LAUNCH.md
CHANGED
|
@@ -16,12 +16,15 @@
|
|
|
16
16
|
|
|
17
17
|
**Title**:
|
|
18
18
|
```
|
|
19
|
-
Show HN: A3M Router – LLM
|
|
19
|
+
Show HN: A3M Router – We built an LLM router. Nobody cared for 2 days. Then word-of-mouth kicked in.
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
**Text** (copy from `articles/hackernews-show-hn.md`):
|
|
23
23
|
```
|
|
24
|
-
|
|
24
|
+
Day 1: 552 downloads. Day 2: 320 downloads. We thought it was dead.
|
|
25
|
+
Day 3: 1,903 downloads. 245% growth from Day 1. Zero marketing budget.
|
|
26
|
+
|
|
27
|
+
2,775 downloads in 3 days. All organic.
|
|
25
28
|
|
|
26
29
|
A3M Router (adaptive-memory-multi-model-router) is a production-ready LLM routing library that actually optimizes for cost vs quality based on your query.
|
|
27
30
|
|
|
@@ -69,8 +72,8 @@ npx a3m-router benchmark # Compare all providers
|
|
|
69
72
|
|
|
70
73
|
Performance
|
|
71
74
|
-----------
|
|
72
|
-
•
|
|
73
|
-
•
|
|
75
|
+
• 2,775 downloads in 3 days
|
|
76
|
+
• 1,903 downloads on Day 3 alone (245% growth from Day 1)
|
|
74
77
|
• Zero dependencies (except nanoid)
|
|
75
78
|
• 3.0 MB unpacked
|
|
76
79
|
|
|
@@ -92,11 +95,12 @@ GitHub: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
|
92
95
|
|
|
93
96
|
**Tweet 1/10**:
|
|
94
97
|
```
|
|
95
|
-
|
|
98
|
+
Day 1: 552 downloads. Day 2: 320 downloads. We thought nobody cared.
|
|
99
|
+
Day 3: 1,903 downloads. 245% growth. Zero marketing budget.
|
|
96
100
|
|
|
97
|
-
|
|
101
|
+
2,775 downloads in 3 days for our LLM router.
|
|
98
102
|
|
|
99
|
-
Here's how A3M Router works 🧵👇
|
|
103
|
+
Here's the story + how A3M Router works 🧵👇
|
|
100
104
|
```
|
|
101
105
|
|
|
102
106
|
**Tweet 2/10**:
|
|
@@ -186,12 +190,13 @@ That's it. No config needed.
|
|
|
186
190
|
|
|
187
191
|
**Tweet 9/10**:
|
|
188
192
|
```
|
|
189
|
-
📊
|
|
193
|
+
📊 2,775 downloads in 3 days
|
|
194
|
+
📈 245% growth Day 1 → Day 3
|
|
190
195
|
🧪 33 tests passing
|
|
191
|
-
🏷️ 156 keywords
|
|
192
196
|
🔌 116 integrations
|
|
193
197
|
|
|
194
|
-
|
|
198
|
+
Day 1: 552. Day 2: 320. Day 3: 1,903.
|
|
199
|
+
Word-of-mouth works. Zero marketing spend.
|
|
195
200
|
```
|
|
196
201
|
|
|
197
202
|
**Tweet 10/10**:
|
|
@@ -217,7 +222,7 @@ Questions? Drop them below! 👇
|
|
|
217
222
|
### 3. Dev.to (PRIORITY 2)
|
|
218
223
|
**URL**: https://dev.to/new
|
|
219
224
|
|
|
220
|
-
**Title**: "Building an LLM Router That Actually Works:
|
|
225
|
+
**Title**: "Building an LLM Router That Actually Works: 2,775 Downloads in 3 Days, Zero Marketing Budget"
|
|
221
226
|
|
|
222
227
|
**Content**: Copy from `articles/devto-llm-routing.md`
|
|
223
228
|
|
|
@@ -230,7 +235,7 @@ Questions? Drop them below! 👇
|
|
|
230
235
|
### 4. Reddit r/MachineLearning (PRIORITY 2)
|
|
231
236
|
**URL**: https://www.reddit.com/r/MachineLearning/submit
|
|
232
237
|
|
|
233
|
-
**Title**: "[P] A3M Router: Production-ready LLM routing
|
|
238
|
+
**Title**: "[P] A3M Router: Production-ready LLM routing — 2,775 downloads in 3 days, 245% growth, zero marketing"
|
|
234
239
|
|
|
235
240
|
**Content**: Copy from `articles/reddit-ml.md`
|
|
236
241
|
|
|
@@ -271,7 +276,9 @@ console.log(result.estimated_cost); // $0.0004
|
|
|
271
276
|
**Results**
|
|
272
277
|
- 70% cost reduction
|
|
273
278
|
- 2x speed improvement
|
|
274
|
-
-
|
|
279
|
+
- 2,775 downloads in 3 days
|
|
280
|
+
- 1,903 downloads on Day 3 alone (245% growth from Day 1)
|
|
281
|
+
- Zero marketing budget
|
|
275
282
|
- 33 tests passing
|
|
276
283
|
|
|
277
284
|
**Try it**
|
|
@@ -301,7 +308,7 @@ Would love your feedback!
|
|
|
301
308
|
### 7. Medium (PRIORITY 3)
|
|
302
309
|
**URL**: https://medium.com/new-story
|
|
303
310
|
|
|
304
|
-
**Title**: "Building a Production-Ready LLM Router:
|
|
311
|
+
**Title**: "Building a Production-Ready LLM Router: 2,775 Downloads in 3 Days on Zero Marketing"
|
|
305
312
|
|
|
306
313
|
**Content**: Copy from `articles/medium-building-llm-router.md`
|
|
307
314
|
|
|
@@ -312,7 +319,7 @@ Would love your feedback!
|
|
|
312
319
|
### 8. IndieHackers (PRIORITY 3)
|
|
313
320
|
**URL**: https://www.indiehackers.com/post/new
|
|
314
321
|
|
|
315
|
-
**Title**: "Show IH: A3M Router - Cut LLM API costs by 70% with intelligent routing"
|
|
322
|
+
**Title**: "Show IH: A3M Router - Cut LLM API costs by 70% with intelligent routing (2,775 downloads in 3 days)"
|
|
316
323
|
|
|
317
324
|
**Content**:
|
|
318
325
|
```
|
|
@@ -334,7 +341,10 @@ Built a learned routing system that:
|
|
|
334
341
|
- 70% cost reduction ($2,400 → $720/month)
|
|
335
342
|
- 2x speed improvement
|
|
336
343
|
- 94% quality retention
|
|
337
|
-
-
|
|
344
|
+
- 2,775 downloads in 3 days on npm
|
|
345
|
+
- 1,903 downloads on Day 3 alone
|
|
346
|
+
- 245% growth from Day 1 to Day 3
|
|
347
|
+
- Zero marketing budget — all word-of-mouth
|
|
338
348
|
|
|
339
349
|
**Tech Stack**
|
|
340
350
|
- Node.js/TypeScript
|
|
@@ -386,10 +396,11 @@ npx a3m-router route "Your query"
|
|
|
386
396
|
```
|
|
387
397
|
|
|
388
398
|
**Stats:**
|
|
389
|
-
📈
|
|
399
|
+
📈 2,775 downloads in 3 days
|
|
400
|
+
📈 1,903 downloads on Day 3 alone
|
|
401
|
+
📈 245% growth from Day 1 to Day 3
|
|
390
402
|
🧪 33 tests passing
|
|
391
403
|
🔌 116 integrations
|
|
392
|
-
🏷️ 156 keywords
|
|
393
404
|
|
|
394
405
|
GitHub: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
395
406
|
```
|
|
@@ -550,7 +561,10 @@ asciinema upload demo.cast
|
|
|
550
561
|
```
|
|
551
562
|
🚀 A3M Router is LIVE!
|
|
552
563
|
|
|
553
|
-
|
|
564
|
+
2,775 downloads in 3 days. Zero marketing budget.
|
|
565
|
+
|
|
566
|
+
Day 1: 552 downloads. Day 2: 320. Day 3: 1,903.
|
|
567
|
+
That's 245% growth. All word-of-mouth.
|
|
554
568
|
|
|
555
569
|
✨ Intelligent LLM routing
|
|
556
570
|
💰 50-80% cost savings
|
package/README.md
CHANGED
|
@@ -1,50 +1,66 @@
|
|
|
1
1
|
# A3M Router 🔀
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> **245% growth in 3 days. Zero marketing budget.**
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
6
6
|
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
7
|
+
[](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
|
|
7
8
|
|
|
8
|
-
|
|
9
|
+
```
|
|
10
|
+
Day 1: 552 downloads (npm keyword discovery)
|
|
11
|
+
Day 2: 320 downloads (curiosity fading)
|
|
12
|
+
Day 3: 1,903 downloads (word-of-mouth kicked in)
|
|
13
|
+
─────────────
|
|
14
|
+
Total: 2,775 downloads in 72 hours
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Nobody promoted this. Developers found it via npm search, tried it, and told others.
|
|
9
18
|
|
|
10
19
|
---
|
|
11
20
|
|
|
12
|
-
##
|
|
21
|
+
## What It Does
|
|
22
|
+
|
|
23
|
+
A3M Router sits between your code and your LLM providers. It analyzes each query and routes it to the **cheapest model that can handle it**.
|
|
24
|
+
|
|
25
|
+
- Simple Q&A → **free** providers (CommandCode, OpenCode)
|
|
26
|
+
- Medium tasks → **fast/cheap** providers (Groq $0.59/1M, Cerebras $0.60/1M)
|
|
27
|
+
- Complex reasoning → **premium** providers (GPT-4o, Claude)
|
|
28
|
+
- If the cheap model fails → **automatic fallback** to stronger model
|
|
13
29
|
|
|
14
|
-
**
|
|
30
|
+
**Result: 40-70% cost savings with no quality loss on simple queries.**
|
|
15
31
|
|
|
16
|
-
|
|
32
|
+
|
|
33
|
+
## Demo
|
|
34
|
+
|
|
35
|
+

|
|
36
|
+
|
|
37
|
+
*Simple queries → free providers. Complex queries → capable models. Automatically.*
|
|
17
38
|
|
|
18
39
|
---
|
|
19
40
|
|
|
20
|
-
## The
|
|
41
|
+
## The Problem
|
|
21
42
|
|
|
22
|
-
|
|
23
|
-
|----------|:-----------------:|:-----:|:-------:|
|
|
24
|
-
| CommandCode | **$0.00** | 5s | 75% |
|
|
25
|
-
| Groq | **$0.59** | 420ms | 82% |
|
|
26
|
-
| Cerebras | **$0.60** | 380ms | 82% |
|
|
27
|
-
| Mistral | **$2.00** | 800ms | 90% |
|
|
28
|
-
| OpenAI GPT-4 | $30.00 | 2100ms | 95% |
|
|
43
|
+
You're sending every query to GPT-4 at $2.50/1M tokens. But research shows **~47% of queries are simple enough for cheaper models** ([RouteLLM, arXiv:2404.06035](https://arxiv.org/abs/2404.06035)).
|
|
29
44
|
|
|
30
|
-
|
|
45
|
+
That's like using a Ferrari for grocery runs. 🏎️🛒
|
|
31
46
|
|
|
32
47
|
---
|
|
33
48
|
|
|
34
49
|
## Quick Start (30 seconds)
|
|
35
50
|
|
|
36
|
-
### Option 1:
|
|
51
|
+
### Option 1: Drop-in Proxy (Zero code changes)
|
|
37
52
|
|
|
38
53
|
```bash
|
|
39
54
|
npm install adaptive-memory-multi-model-router
|
|
40
55
|
npx a3m-router serve
|
|
41
56
|
```
|
|
42
57
|
|
|
43
|
-
|
|
58
|
+
Point any OpenAI SDK at `http://localhost:8787/v1`:
|
|
44
59
|
|
|
45
60
|
```python
|
|
46
61
|
from openai import OpenAI
|
|
47
62
|
|
|
63
|
+
# Just change the base_url. Everything else stays the same.
|
|
48
64
|
client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
|
|
49
65
|
response = client.chat.completions.create(
|
|
50
66
|
model="auto",
|
|
@@ -52,7 +68,7 @@ response = client.chat.completions.create(
|
|
|
52
68
|
)
|
|
53
69
|
```
|
|
54
70
|
|
|
55
|
-
Works with **Python, Node, LangChain, LlamaIndex** — any OpenAI-compatible client.
|
|
71
|
+
Works with **Python, Node, LangChain, LlamaIndex** — any OpenAI-compatible client.
|
|
56
72
|
|
|
57
73
|
### Option 2: Library
|
|
58
74
|
|
|
@@ -61,7 +77,6 @@ const { createA3MRouter } = require('adaptive-memory-multi-model-router');
|
|
|
61
77
|
|
|
62
78
|
const router = createA3MRouter();
|
|
63
79
|
|
|
64
|
-
// Automatic routing — picks the cheapest capable provider
|
|
65
80
|
const result = await router.route("Explain quantum computing in one paragraph");
|
|
66
81
|
console.log(result.response); // the answer
|
|
67
82
|
console.log(result.provider); // which provider was chosen
|
|
@@ -71,65 +86,64 @@ console.log(result.cost); // what it cost
|
|
|
71
86
|
### Option 3: CLI
|
|
72
87
|
|
|
73
88
|
```bash
|
|
74
|
-
# Route a single query
|
|
75
|
-
npx a3m-router
|
|
89
|
+
npx a3m-router route "Your query here" # Route a single query
|
|
90
|
+
npx a3m-router benchmark # Benchmark all providers
|
|
91
|
+
npx a3m-router serve --port 3000 # Start proxy on custom port
|
|
92
|
+
```
|
|
76
93
|
|
|
77
|
-
|
|
78
|
-
npx a3m-router benchmark
|
|
94
|
+
---
|
|
79
95
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
96
|
+
## Cost Comparison
|
|
97
|
+
|
|
98
|
+
| Query Type | % of Traffic | GPT-4o Cost | A3M Routes To | A3M Cost | Savings |
|
|
99
|
+
|-----------|:------------:|:-----------:|:-------------:|:--------:|:-------:|
|
|
100
|
+
| Simple Q&A | 47% | $0.00375 | Groq/Cerebras | $0.00009 | **97%** |
|
|
101
|
+
| Medium tasks | 33% | $0.01250 | GPT-4o-mini | $0.00075 | **94%** |
|
|
102
|
+
| Complex reasoning | 20% | $0.03000 | GPT-4o | $0.03000 | 0% |
|
|
103
|
+
| **Average** | **100%** | **$0.01250** | — | **$0.00515** | **59%** |
|
|
104
|
+
|
|
105
|
+
At 1M queries/month: **$12,500 → $5,150**. You save **$7,350/month**.
|
|
83
106
|
|
|
84
107
|
---
|
|
85
108
|
|
|
86
|
-
##
|
|
109
|
+
## 39 Providers
|
|
87
110
|
|
|
88
|
-
|
|
111
|
+
| Tier | Providers | Cost/1M tokens |
|
|
112
|
+
|------|-----------|:--------------:|
|
|
113
|
+
| **Free** | CommandCode, Ollama, LM Studio, vLLM | $0.00 |
|
|
114
|
+
| **Fast** | Groq, Cerebras | ~$0.60 |
|
|
115
|
+
| **Balanced** | Mistral, DeepSeek, Qwen | $1.50-$2.00 |
|
|
116
|
+
| **Premium** | OpenAI, Anthropic, Google | $2.50-$30.00 |
|
|
89
117
|
|
|
90
|
-
|
|
118
|
+
Adding a provider is one line of config. Failover is automatic.
|
|
91
119
|
|
|
92
|
-
|
|
120
|
+
---
|
|
93
121
|
|
|
94
|
-
|
|
122
|
+
## Features
|
|
95
123
|
|
|
96
124
|
### 🧠 Intelligent Routing
|
|
125
|
+
Query complexity analysis (0-100 score) → cheapest capable provider. The router **learns from your usage patterns** over time (adaptive memory).
|
|
97
126
|
|
|
98
|
-
|
|
127
|
+
### 🛤️ OpenAI-Compatible Proxy
|
|
128
|
+
Drop-in replacement for `api.openai.com`. Switch one URL, save 70%.
|
|
99
129
|
|
|
100
|
-
###
|
|
130
|
+
### 📊 Real-Time Dashboard
|
|
131
|
+
Live cost tracking, provider health, request logs at `http://localhost:8787/`.
|
|
101
132
|
|
|
133
|
+
### 🤖 LangChain Adapter
|
|
102
134
|
```javascript
|
|
103
135
|
import { A3MChatModel } from 'adaptive-memory-multi-model-router/langchain';
|
|
104
|
-
|
|
105
136
|
const model = new A3MChatModel();
|
|
106
|
-
const response = await model.invoke("Why is the sky blue?");
|
|
107
137
|
```
|
|
108
138
|
|
|
109
139
|
### 🛡️ Guardrails
|
|
110
|
-
|
|
111
|
-
Prompt injection detection, PII redaction, content filtering — built in, enabled by default.
|
|
140
|
+
Prompt injection detection, PII redaction, content filtering — enabled by default.
|
|
112
141
|
|
|
113
142
|
### 🗜️ Semantic Cache
|
|
114
|
-
|
|
115
|
-
Cache semantically similar queries. Identical meaning = instant response, zero API cost.
|
|
143
|
+
Cache semantically similar queries. Same meaning = instant response, zero cost.
|
|
116
144
|
|
|
117
145
|
### 📈 Cost Analytics
|
|
118
|
-
|
|
119
|
-
Track every request. See exactly where your money goes. Export savings reports.
|
|
120
|
-
|
|
121
|
-
---
|
|
122
|
-
|
|
123
|
-
## 39 Providers
|
|
124
|
-
|
|
125
|
-
| Tier | Providers |
|
|
126
|
-
|------|-----------|
|
|
127
|
-
| **Free** | CommandCode, Ollama, LM Studio, vLLM |
|
|
128
|
-
| **Fast** | Groq ($0.59), Cerebras ($0.60) |
|
|
129
|
-
| **Balanced** | Mistral ($2), DeepSeek ($1.5), Qwen ($2) |
|
|
130
|
-
| **Premium** | OpenAI ($30), Anthropic ($15) |
|
|
131
|
-
|
|
132
|
-
Adding a provider is one line of config. Mix and match. Failover automatically.
|
|
146
|
+
Track every request. Export savings reports. Set daily budget limits.
|
|
133
147
|
|
|
134
148
|
---
|
|
135
149
|
|
|
@@ -137,22 +151,33 @@ Adding a provider is one line of config. Mix and match. Failover automatically.
|
|
|
137
151
|
|
|
138
152
|
| Feature | A3M Router | Portkey | LiteLLM |
|
|
139
153
|
|---------|:----------:|:-------:|:-------:|
|
|
140
|
-
| OpenAI proxy | ✅ | ✅ | ✅ |
|
|
154
|
+
| OpenAI-compatible proxy | ✅ | ✅ | ✅ |
|
|
155
|
+
| Intelligent routing | ✅ | ✅ | ✅ |
|
|
141
156
|
| Real-time dashboard | ✅ | ✅ | ❌ |
|
|
142
157
|
| LangChain adapter | ✅ | ✅ | ✅ |
|
|
143
|
-
| Guardrails | ✅ | ✅ | ❌ |
|
|
158
|
+
| Guardrails built-in | ✅ | ✅ | ❌ |
|
|
144
159
|
| Semantic cache | ✅ | ✅ | ❌ |
|
|
145
|
-
|
|
|
160
|
+
| Adaptive memory | ✅ | ❌ | ❌ |
|
|
146
161
|
| **Price** | **Free** | **Paid tiers** | **Free** |
|
|
147
|
-
| **Setup
|
|
162
|
+
| **Setup** | **30 seconds** | **Account required** | **Library only** |
|
|
148
163
|
|
|
149
164
|
---
|
|
150
165
|
|
|
151
|
-
##
|
|
166
|
+
## When NOT to Use This
|
|
152
167
|
|
|
153
|
-
|
|
168
|
+
- You only use one provider and are happy with it
|
|
169
|
+
- You need 250+ provider integrations (use Portkey or LiteLLM)
|
|
170
|
+
- You're building a simple prototype with <100 queries/day
|
|
171
|
+
- You need enterprise SLAs and support contracts
|
|
154
172
|
|
|
155
|
-
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Benchmarks
|
|
176
|
+
|
|
177
|
+
Run your own:
|
|
178
|
+
```bash
|
|
179
|
+
bash scripts/benchmark.sh
|
|
180
|
+
```
|
|
156
181
|
|
|
157
182
|
---
|
|
158
183
|
|
|
@@ -161,7 +186,12 @@ Adding a provider is one line of config. Mix and match. Failover automatically.
|
|
|
161
186
|
- 📦 [NPM](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
162
187
|
- 🐙 [GitHub](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
|
|
163
188
|
- 🎮 [Playground](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
189
|
+
- 💬 [Discussions](https://github.com/Das-rebel/adaptive-memory-multi-model-router/discussions)
|
|
164
190
|
|
|
165
191
|
---
|
|
166
192
|
|
|
167
|
-
|
|
193
|
+
## Contributing
|
|
194
|
+
|
|
195
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md). PRs welcome! Check [good first issues](https://github.com/Das-rebel/adaptive-memory-multi-model-router/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).
|
|
196
|
+
|
|
197
|
+
MIT License. No vendor lock-in. No account required. `npm install` and go.
|
package/articles/HN_FINAL.md
CHANGED
|
@@ -1,14 +1,21 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Show HN:
|
|
2
|
+
title: "Show HN: A3M Router – We built an LLM router. Nobody cared for 2 days. Then word-of-mouth kicked in."
|
|
3
3
|
---
|
|
4
4
|
|
|
5
|
-
# Show HN:
|
|
5
|
+
# Show HN: A3M Router – We built an LLM router. Nobody cared for 2 days. Then word-of-mouth kicked in.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Day 1: 552 downloads. Day 2: 320 downloads. We thought it was dead.
|
|
8
|
+
Day 3: 1,903 downloads. 245% growth from Day 1. Zero marketing budget.
|
|
8
9
|
|
|
9
|
-
|
|
10
|
+
2,775 downloads in 3 days. All organic.
|
|
10
11
|
|
|
11
|
-
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## What I Built
|
|
15
|
+
|
|
16
|
+
A3M Router — an open-source npm package that analyzes each LLM query and routes it to the cheapest capable provider automatically.
|
|
17
|
+
|
|
18
|
+
We're a small team processing ~1,000 LLM queries/day. Customer support automation, code generation, text summarization. We were using GPT-4 for **everything**. Even simple questions went to GPT-4 at $0.03/query.
|
|
12
19
|
|
|
13
20
|
I looked at our logs:
|
|
14
21
|
- **34%** simple Q&A (any model works)
|
|
@@ -20,9 +27,7 @@ We were overpaying by **70%**.
|
|
|
20
27
|
|
|
21
28
|
---
|
|
22
29
|
|
|
23
|
-
##
|
|
24
|
-
|
|
25
|
-
**A3M Router** - analyzes each query and routes to the cheapest capable provider automatically.
|
|
30
|
+
## How It Works
|
|
26
31
|
|
|
27
32
|
**Before:**
|
|
28
33
|
```javascript
|
|
@@ -43,6 +48,15 @@ await router.route("What is 2+2?");
|
|
|
43
48
|
// Automatically picks cheapest capable provider
|
|
44
49
|
```
|
|
45
50
|
|
|
51
|
+
The routing algorithm is inspired by RouteLLM (arXiv:2404.06035):
|
|
52
|
+
|
|
53
|
+
1. **Analyze query** — Detects code, math, complexity, language
|
|
54
|
+
2. **Check providers** — Cost, latency, quality scores for each
|
|
55
|
+
3. **Smart routing** — Simple → cheap. Code → fast. Complex → quality.
|
|
56
|
+
4. **Track & fallback** — Logs costs, retries if provider fails
|
|
57
|
+
|
|
58
|
+
**Zero configuration.** Works immediately with 12 providers pre-configured.
|
|
59
|
+
|
|
46
60
|
---
|
|
47
61
|
|
|
48
62
|
## Results (30 Days)
|
|
@@ -58,14 +72,21 @@ await router.route("What is 2+2?");
|
|
|
58
72
|
|
|
59
73
|
---
|
|
60
74
|
|
|
61
|
-
##
|
|
75
|
+
## The Launch Story
|
|
62
76
|
|
|
63
|
-
|
|
64
|
-
2. **Check providers** - Cost, latency, quality scores for each
|
|
65
|
-
3. **Smart routing** - Simple → cheap. Code → fast. Complex → quality.
|
|
66
|
-
4. **Track & fallback** - Logs costs, retries if provider fails
|
|
77
|
+
We published to npm and... crickets.
|
|
67
78
|
|
|
68
|
-
|
|
79
|
+
| Day | Downloads | How it felt |
|
|
80
|
+
|-----|-----------|-------------|
|
|
81
|
+
| Day 1 | 552 | "Okay, modest start. Early adopters." |
|
|
82
|
+
| Day 2 | 320 | "It's dead. The launch flopped." |
|
|
83
|
+
| Day 3 | 1,903 | "Wait, WHAT?" |
|
|
84
|
+
|
|
85
|
+
No blog post. No HN submission. No Twitter thread. No Product Hunt. No paid promotion of any kind.
|
|
86
|
+
|
|
87
|
+
245% growth from Day 1 to Day 3. 6x from Day 2 to Day 3.
|
|
88
|
+
|
|
89
|
+
The lesson: developer tools spread through backchannels — Discord servers, Slack channels, DMs between coworkers. That takes 48 hours to compound. The Day 2 dip was real and demoralizing. But Day 3 proved that word-of-mouth works on its own timeline.
|
|
69
90
|
|
|
70
91
|
---
|
|
71
92
|
|
|
@@ -84,8 +105,6 @@ npx a3m-router compare "Write Python to sort an array"
|
|
|
84
105
|
npx a3m-router benchmark
|
|
85
106
|
```
|
|
86
107
|
|
|
87
|
-
**Or try online:** https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground
|
|
88
|
-
|
|
89
108
|
No API keys needed to test routing logic.
|
|
90
109
|
|
|
91
110
|
---
|
|
@@ -109,27 +128,10 @@ No API keys needed to test routing logic.
|
|
|
109
128
|
|
|
110
129
|
---
|
|
111
130
|
|
|
112
|
-
## Features
|
|
113
|
-
|
|
114
|
-
**Out of the box:**
|
|
115
|
-
- 12 providers configured (Groq, Cerebras, Mistral, OpenAI, Anthropic, Google, DeepSeek, etc.)
|
|
116
|
-
- Automatic query analysis
|
|
117
|
-
- Cost tracking across all providers
|
|
118
|
-
- Provider fallback
|
|
119
|
-
- Batch processing
|
|
120
|
-
- Response caching
|
|
121
|
-
- CLI tools
|
|
122
|
-
|
|
123
|
-
**Zero configuration needed.**
|
|
124
|
-
|
|
125
|
-
---
|
|
126
|
-
|
|
127
131
|
## Technical Details
|
|
128
132
|
|
|
129
133
|
### Routing Algorithm
|
|
130
134
|
|
|
131
|
-
Inspired by RouteLLM (arXiv:2404.06035):
|
|
132
|
-
|
|
133
135
|
```javascript
|
|
134
136
|
// Feature extraction
|
|
135
137
|
const features = extractQueryFeatures("Write Python to sort array");
|
|
@@ -190,10 +192,9 @@ If you're using one provider for everything:
|
|
|
190
192
|
|
|
191
193
|
- **GitHub**: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
192
194
|
- **NPM**: https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
193
|
-
- **Playground**: https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground
|
|
194
195
|
|
|
195
|
-
**Stats**:
|
|
196
|
+
**Stats**: 2,775 downloads in 3 days, 1,903 on Day 3, 245% growth, zero marketing budget.
|
|
196
197
|
|
|
197
198
|
---
|
|
198
199
|
|
|
199
|
-
Questions about the routing algorithm? What features should we add?
|
|
200
|
+
Questions about the routing algorithm? What features should we add? And has anyone else experienced the "Day 2 dip then Day 3 explosion" pattern with developer tool launches?
|
|
@@ -1,14 +1,17 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Building an LLM Router That Actually Works:
|
|
2
|
+
title: "Building an LLM Router That Actually Works: 2,775 Downloads in 3 Days, Zero Marketing Budget"
|
|
3
3
|
published: false
|
|
4
|
-
description: "How we built adaptive-memory-multi-model-router
|
|
4
|
+
description: "How we built adaptive-memory-multi-model-router — a production-ready LLM routing library that went from 552 downloads on Day 1 to 1,903 on Day 3 with zero marketing."
|
|
5
5
|
tags: llm, ai, routing, javascript, typescript, openai, claude, groq
|
|
6
6
|
canonical_url: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
7
7
|
---
|
|
8
8
|
|
|
9
|
-
# Building an LLM Router That Actually Works:
|
|
9
|
+
# Building an LLM Router That Actually Works: 2,775 Downloads in 3 Days, Zero Marketing Budget
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
Day 1: 552 downloads. Day 2: 320 downloads. We thought it was dead.
|
|
12
|
+
Day 3: 1,903 downloads. 245% growth from Day 1. All word-of-mouth.
|
|
13
|
+
|
|
14
|
+
Here's what we built and what we learned from the launch curve.
|
|
12
15
|
|
|
13
16
|
## The Problem
|
|
14
17
|
|
|
@@ -64,6 +67,17 @@ Query → Feature Extraction → Router → Provider Selection → Execution
|
|
|
64
67
|
Translation? Tradeoff Health Checks
|
|
65
68
|
```
|
|
66
69
|
|
|
70
|
+
## The Launch Curve
|
|
71
|
+
|
|
72
|
+
| Day | Downloads | Notes |
|
|
73
|
+
|-----|-----------|-------|
|
|
74
|
+
| Day 1 | 552 | Modest. A few early adopters found it. |
|
|
75
|
+
| Day 2 | 320 | Thought the launch flopped. Fewer than Day 1. |
|
|
76
|
+
| Day 3 | 1,903 | 6x Day 2. 245% growth from Day 1. Word-of-mouth kicked in. |
|
|
77
|
+
| **Total** | **2,775** | **Zero marketing budget.** |
|
|
78
|
+
|
|
79
|
+
Lesson: good tooling spreads on its own timeline. The Day 2 dip was demoralizing, but Day 3 proved that word-of-mouth compounds — it just takes a beat.
|
|
80
|
+
|
|
67
81
|
## Real-World Usage
|
|
68
82
|
|
|
69
83
|
```javascript
|
|
@@ -85,7 +99,9 @@ const results = router.routeBatch([
|
|
|
85
99
|
|
|
86
100
|
## Performance
|
|
87
101
|
|
|
88
|
-
- **
|
|
102
|
+
- **2,775 downloads in 3 days**
|
|
103
|
+
- **1,903 downloads on Day 3 alone** (245% growth from Day 1)
|
|
104
|
+
- **Zero marketing budget**
|
|
89
105
|
- **33 tests** passing
|
|
90
106
|
- **139 keywords** for discoverability
|
|
91
107
|
- **116 integrations** supported
|
|
@@ -1,8 +1,13 @@
|
|
|
1
|
-
Show HN: A3M Router – LLM
|
|
1
|
+
Show HN: A3M Router – We built an LLM router. Nobody cared for 2 days. Then word-of-mouth kicked in.
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Day 1: 552 downloads. Day 2: 320 downloads. We thought it was dead.
|
|
4
|
+
Day 3: 1,903 downloads. 245% growth from Day 1. Zero marketing budget.
|
|
4
5
|
|
|
5
|
-
|
|
6
|
+
2,775 downloads in 3 days. All organic.
|
|
7
|
+
|
|
8
|
+
I'm sharing what we built and what we learned from the launch curve.
|
|
9
|
+
|
|
10
|
+
A3M Router (adaptive-memory-multi-model-router) is a production-ready LLM routing library that optimizes for cost vs quality based on your query.
|
|
6
11
|
|
|
7
12
|
The Problem
|
|
8
13
|
-----------
|
|
@@ -34,6 +39,16 @@ routeQuery("Explain quantum entanglement");
|
|
|
34
39
|
// → mistral/mistral-large (reasoning strength)
|
|
35
40
|
```
|
|
36
41
|
|
|
42
|
+
The Launch Story
|
|
43
|
+
----------------
|
|
44
|
+
- Day 1: 552 downloads. Modest. A few early adopters found it.
|
|
45
|
+
- Day 2: 320 downloads. We thought the launch flopped. Fewer than Day 1.
|
|
46
|
+
- Day 3: 1,903 downloads. 6x Day 2. 245% growth from Day 1.
|
|
47
|
+
|
|
48
|
+
No blog post. No HN submission. No Twitter thread. No Product Hunt. Just developers telling other developers.
|
|
49
|
+
|
|
50
|
+
Lesson: good tooling spreads on its own timeline. The Day 2 dip was demoralizing, but Day 3 proved that word-of-mouth compounds — it just takes a beat.
|
|
51
|
+
|
|
37
52
|
Key Features
|
|
38
53
|
------------
|
|
39
54
|
• 12 providers: Groq, Cerebras, Mistral, OpenAI, Anthropic, Google, DeepSeek + CLI/local
|
|
@@ -53,8 +68,10 @@ npx a3m-router benchmark # Compare all providers
|
|
|
53
68
|
|
|
54
69
|
Performance
|
|
55
70
|
-----------
|
|
56
|
-
•
|
|
57
|
-
•
|
|
71
|
+
• 2,775 downloads in 3 days
|
|
72
|
+
• 1,903 downloads on Day 3 alone
|
|
73
|
+
• 245% growth from Day 1 to Day 3
|
|
74
|
+
• Zero marketing budget
|
|
58
75
|
• Zero dependencies (except nanoid)
|
|
59
76
|
• 3.0 MB unpacked
|
|
60
77
|
|
|
@@ -62,4 +79,4 @@ Try it: npm install adaptive-memory-multi-model-router
|
|
|
62
79
|
|
|
63
80
|
Would love feedback on the routing algorithm - what features should we add?
|
|
64
81
|
|
|
65
|
-
GitHub: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
82
|
+
GitHub: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
package/articles/reddit-ml.md
CHANGED
|
@@ -1,8 +1,14 @@
|
|
|
1
|
-
[P] A3M Router: Production-ready LLM routing
|
|
1
|
+
[P] A3M Router: Production-ready LLM routing — 2,775 downloads in 3 days, 245% growth, zero marketing
|
|
2
2
|
|
|
3
3
|
Hi r/MachineLearning,
|
|
4
4
|
|
|
5
|
-
We've been working on an LLM routing library that
|
|
5
|
+
We've been working on an LLM routing library that just hit 2,775 downloads in 3 days — all organic, zero marketing budget. I wanted to share the technical approach for feedback.
|
|
6
|
+
|
|
7
|
+
**Launch numbers:**
|
|
8
|
+
- Day 1: 552 downloads
|
|
9
|
+
- Day 2: 320 downloads (we thought it flopped)
|
|
10
|
+
- Day 3: 1,903 downloads (245% growth from Day 1)
|
|
11
|
+
- Total: 2,775 downloads in 3 days
|
|
6
12
|
|
|
7
13
|
**What it does:**
|
|
8
14
|
A3M Router intelligently routes LLM queries to the optimal provider based on query characteristics, cost constraints, and quality requirements.
|
|
@@ -65,7 +71,8 @@ Users can add their own providers without code changes:
|
|
|
65
71
|
- Batch processing with concurrency control
|
|
66
72
|
|
|
67
73
|
**Performance:**
|
|
68
|
-
-
|
|
74
|
+
- 2,775 downloads in 3 days
|
|
75
|
+
- 1,903 downloads on Day 3 alone (245% growth from Day 1)
|
|
69
76
|
- 33 comprehensive tests
|
|
70
77
|
- 139 npm keywords (max visibility)
|
|
71
78
|
- 116 integrations (GitHub, Slack, Telegram, etc.)
|