adaptive-memory-multi-model-router 2.0.1 → 2.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +96 -255
- package/articles/DEVTO_VIRAL_GROWTH.md +280 -0
- package/assets/feature-matrix.svg +136 -0
- package/assets/growth-chart.svg +82 -0
- package/assets/provider-comparison.svg +121 -0
- package/llms.txt +30 -120
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,326 +1,167 @@
|
|
|
1
|
-
|
|
2
|
-
<img src="assets/banner.svg" alt="A3M Router Banner" width="100%">
|
|
3
|
-
</p>
|
|
1
|
+
# A3M Router 🔀
|
|
4
2
|
|
|
5
|
-
|
|
3
|
+
> Intelligent LLM routing engine — **2,775 downloads in 3 days**
|
|
6
4
|
|
|
7
|
-
|
|
8
|
-
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
10
|
-
[](https://github.com/Das-rebel/adaptive-memory-multi-model-router/actions)
|
|
11
|
-
[](LICENSE)
|
|
5
|
+
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
6
|
+
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
12
7
|
|
|
13
|
-
|
|
14
|
-
[](docs/providers.md)
|
|
15
|
-
[](docs/integrations.md)
|
|
16
|
-
[](package.json)
|
|
17
|
-
[](https://bundlephobia.com/package/adaptive-memory-multi-model-router)
|
|
18
|
-
|
|
19
|
-
</div>
|
|
8
|
+
**Zero marketing budget. 1,903 downloads yesterday alone.**
|
|
20
9
|
|
|
21
10
|
---
|
|
22
11
|
|
|
23
|
-
|
|
24
|
-
<b>Route LLM queries to optimal providers automatically</b>
|
|
25
|
-
</h3>
|
|
26
|
-
|
|
27
|
-
<p align="center">
|
|
28
|
-
Save <b>50-80%</b> on API costs • <b>5-10x</b> speedups • <b>Zero</b> configuration needed
|
|
29
|
-
</p>
|
|
30
|
-
|
|
31
|
-
<div align="center">
|
|
32
|
-
|
|
33
|
-
[📖 Documentation](https://github.com/Das-rebel/adaptive-memory-multi-model-router#readme) •
|
|
34
|
-
[🚀 Quick Start](#quick-start) •
|
|
35
|
-
[🎮 Try Online](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground) •
|
|
36
|
-
[📊 Benchmarks](#benchmarks) •
|
|
37
|
-
[🤝 Contributing](CONTRIBUTING.md) •
|
|
38
|
-
[💬 Discussions](https://github.com/Das-rebel/adaptive-memory-multi-model-router/discussions)
|
|
12
|
+
## Why People Are Switching
|
|
39
13
|
|
|
40
|
-
|
|
14
|
+
**The Problem:** You're sending every query to GPT-4 at $30/1M tokens. But 47% of your queries are simple Q&A that a free provider handles perfectly.
|
|
41
15
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
[](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
45
|
-
[](https://stackblitz.com/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
46
|
-
|
|
47
|
-
</div>
|
|
16
|
+
**The Solution:** A3M Router analyzes each query and routes it to the cheapest capable provider — automatically.
|
|
48
17
|
|
|
49
18
|
---
|
|
50
19
|
|
|
51
|
-
##
|
|
52
|
-
|
|
53
|
-
<table>
|
|
54
|
-
<tr>
|
|
55
|
-
<td width="50%">
|
|
56
|
-
|
|
57
|
-
### 🧠 Learned Routing
|
|
58
|
-
Routes queries based on **actual query characteristics** - not just random selection. Code queries go to code-capable models. Simple queries use cheaper providers.
|
|
59
|
-
|
|
60
|
-
</td>
|
|
61
|
-
<td width="50%">
|
|
62
|
-
|
|
63
|
-
### 💰 Cost Optimization
|
|
64
|
-
Automatically selects the **cheapest capable provider**. Route simple queries to free tiers. Use premium models only when complexity demands it.
|
|
65
|
-
|
|
66
|
-
</td>
|
|
67
|
-
</tr>
|
|
68
|
-
<tr>
|
|
69
|
-
<td width="50%">
|
|
70
|
-
|
|
71
|
-
### 🔄 Smart Fallback
|
|
72
|
-
When a provider fails, **automatically retry** with the next best option. No manual intervention needed. Your app stays resilient.
|
|
73
|
-
|
|
74
|
-
</td>
|
|
75
|
-
<td width="50%">
|
|
20
|
+
## The Numbers
|
|
76
21
|
|
|
77
|
-
|
|
78
|
-
|
|
22
|
+
| Provider | Cost / 1M tokens | Speed | Quality |
|
|
23
|
+
|----------|:-----------------:|:-----:|:-------:|
|
|
24
|
+
| CommandCode | **$0.00** | 5s | 75% |
|
|
25
|
+
| Groq | **$0.59** | 420ms | 82% |
|
|
26
|
+
| Cerebras | **$0.60** | 380ms | 82% |
|
|
27
|
+
| Mistral | **$2.00** | 800ms | 90% |
|
|
28
|
+
| OpenAI GPT-4 | $30.00 | 2100ms | 95% |
|
|
79
29
|
|
|
80
|
-
|
|
81
|
-
</tr>
|
|
82
|
-
</table>
|
|
30
|
+
**Route to the right provider = 70% cost savings, 62% faster.**
|
|
83
31
|
|
|
84
|
-
|
|
32
|
+
---
|
|
85
33
|
|
|
86
|
-
|
|
34
|
+
## Quick Start (30 seconds)
|
|
87
35
|
|
|
88
|
-
|
|
89
|
-
[](https://stackblitz.com/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
36
|
+
### Option 1: OpenAI-Compatible Proxy
|
|
90
37
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
-
|
|
94
|
-
|
|
95
|
-
- Experiment with no API keys needed!
|
|
38
|
+
```bash
|
|
39
|
+
npm install adaptive-memory-multi-model-router
|
|
40
|
+
npx a3m-router serve
|
|
41
|
+
```
|
|
96
42
|
|
|
97
|
-
|
|
43
|
+
Now point any OpenAI SDK at `http://localhost:8787/v1`:
|
|
98
44
|
|
|
99
|
-
|
|
45
|
+
```python
|
|
46
|
+
from openai import OpenAI
|
|
100
47
|
|
|
101
|
-
|
|
102
|
-
|
|
48
|
+
client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
|
|
49
|
+
response = client.chat.completions.create(
|
|
50
|
+
model="auto",
|
|
51
|
+
messages=[{"role": "user", "content": "Hello!"}]
|
|
52
|
+
)
|
|
103
53
|
```
|
|
104
54
|
|
|
105
|
-
|
|
55
|
+
Works with **Python, Node, LangChain, LlamaIndex** — any OpenAI-compatible client. Zero code changes.
|
|
56
|
+
|
|
57
|
+
### Option 2: Library
|
|
106
58
|
|
|
107
59
|
```javascript
|
|
108
60
|
const { createA3MRouter } = require('adaptive-memory-multi-model-router');
|
|
109
61
|
|
|
110
62
|
const router = createA3MRouter();
|
|
111
63
|
|
|
112
|
-
//
|
|
113
|
-
const result = await router.route("
|
|
114
|
-
|
|
115
|
-
console.log(result.
|
|
116
|
-
console.log(result.
|
|
64
|
+
// Automatic routing — picks the cheapest capable provider
|
|
65
|
+
const result = await router.route("Explain quantum computing in one paragraph");
|
|
66
|
+
console.log(result.response); // the answer
|
|
67
|
+
console.log(result.provider); // which provider was chosen
|
|
68
|
+
console.log(result.cost); // what it cost
|
|
117
69
|
```
|
|
118
70
|
|
|
119
|
-
### CLI
|
|
71
|
+
### Option 3: CLI
|
|
120
72
|
|
|
121
73
|
```bash
|
|
122
|
-
#
|
|
123
|
-
npx a3m-router
|
|
124
|
-
|
|
125
|
-
# Route a query
|
|
126
|
-
npx a3m-router route "Explain quantum physics"
|
|
74
|
+
# Route a single query
|
|
75
|
+
npx a3m-router route "Your query here"
|
|
127
76
|
|
|
128
77
|
# Benchmark all providers
|
|
129
78
|
npx a3m-router benchmark
|
|
130
|
-
```
|
|
131
|
-
|
|
132
|
-
## 📊 Benchmarks
|
|
133
|
-
|
|
134
|
-
<!-- BENCHMARK_START -->
|
|
135
|
-
| Provider | Latency | Cost/1K | Quality | Best For |
|
|
136
|
-
|----------|---------|---------|---------|----------|
|
|
137
|
-
| **Groq** | 400ms | $0.59 | ⭐⭐⭐⭐ | Fast inference |
|
|
138
|
-
| **Cerebras** | 350ms | $0.60 | ⭐⭐⭐⭐ | Speed-critical |
|
|
139
|
-
| **Mistral** | 800ms | $0.20 | ⭐⭐⭐⭐⭐ | Cost + quality |
|
|
140
|
-
| **CommandCode** | 5s | **FREE** | ⭐⭐⭐ | Budget projects |
|
|
141
|
-
| **OpenCode** | 3s | **FREE** | ⭐⭐⭐ | Multi-model |
|
|
142
79
|
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
## 🎯 Routing Examples
|
|
147
|
-
|
|
148
|
-
```javascript
|
|
149
|
-
const { routeQuery } = require('adaptive-memory-multi-model-router');
|
|
150
|
-
|
|
151
|
-
// Simple query → cheapest provider (FREE)
|
|
152
|
-
routeQuery("What is 2+2?");
|
|
153
|
-
// → commandcode/taste-1 ($0.00)
|
|
154
|
-
|
|
155
|
-
// Code query → fast, code-capable provider
|
|
156
|
-
routeQuery("Write Python to reverse a string");
|
|
157
|
-
// → groq/llama-3.3-70b ($0.0004)
|
|
158
|
-
|
|
159
|
-
// Complex reasoning → high-quality provider
|
|
160
|
-
routeQuery("Explain quantum entanglement");
|
|
161
|
-
// → mistral/mistral-large ($0.002)
|
|
162
|
-
|
|
163
|
-
// Batch processing with auto-routing
|
|
164
|
-
const queries = ["Q1", "Q2", "Q3"];
|
|
165
|
-
const results = routeBatch(queries);
|
|
166
|
-
```
|
|
167
|
-
|
|
168
|
-
## 🏗️ Architecture
|
|
169
|
-
|
|
170
|
-
```
|
|
171
|
-
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
|
172
|
-
│ User Query │────▶│ Feature Extraction │────▶│ Query Analysis │
|
|
173
|
-
└─────────────────┘ └──────────────────┘ └─────────────────┘
|
|
174
|
-
│
|
|
175
|
-
┌───────────────────────────┼───────────────────────────┐
|
|
176
|
-
│ │ │
|
|
177
|
-
▼ ▼ ▼
|
|
178
|
-
┌─────────┐ ┌─────────┐ ┌─────────┐
|
|
179
|
-
│ Code? │ │ Math? │ │ Simple? │
|
|
180
|
-
└────┬────┘ └────┬────┘ └────┬────┘
|
|
181
|
-
│ │ │
|
|
182
|
-
▼ ▼ ▼
|
|
183
|
-
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
184
|
-
│ Model Profiles │◀────│ Router Engine │────▶│ Cost/Quality │────▶│ Provider Select │
|
|
185
|
-
│ (12 providers) │ │ (Learned algo) │ │ Tradeoff │ │ + Fallback │
|
|
186
|
-
└─────────────────┘ └──────────────────┘ └─────────────────┘ └─────────────────┘
|
|
187
|
-
│
|
|
188
|
-
▼
|
|
189
|
-
┌─────────────────┐
|
|
190
|
-
│ Execute LLM │
|
|
191
|
-
│ + Track Cost │
|
|
192
|
-
└─────────────────┘
|
|
80
|
+
# Start proxy on custom port
|
|
81
|
+
npx a3m-router serve --port 3000
|
|
193
82
|
```
|
|
194
83
|
|
|
195
|
-
|
|
84
|
+
---
|
|
196
85
|
|
|
197
|
-
|
|
198
|
-
- ✅ **Learned Routing** - RouteLLM-style optimization
|
|
199
|
-
- ✅ **Cost Tracking** - Real-time spend monitoring
|
|
200
|
-
- ✅ **Automatic Fallback** - Retry with backup providers
|
|
201
|
-
- ✅ **Batch Processing** - Parallel execution
|
|
202
|
-
- ✅ **Response Caching** - RadixAttention-style
|
|
203
|
-
- ✅ **Circuit Breakers** - Fail-fast protection
|
|
86
|
+
## What's Included
|
|
204
87
|
|
|
205
|
-
###
|
|
206
|
-
- 🔒 **Input Validation** - Sanitize and validate inputs
|
|
207
|
-
- 🔒 **Prompt Injection Detection** - Block attacks
|
|
208
|
-
- 🔒 **PII Detection** - Protect sensitive data
|
|
209
|
-
- 🔒 **Content Filtering** - Block harmful content
|
|
210
|
-
- 🔒 **Rate Limiting** - Prevent abuse
|
|
88
|
+
### 🛤️ OpenAI-Compatible Proxy Server
|
|
211
89
|
|
|
212
|
-
|
|
90
|
+
Drop-in replacement for `api.openai.com`. Switch one URL, save 70%. No SDK changes.
|
|
213
91
|
|
|
214
|
-
|
|
215
|
-
- Groq (llama-3.3-70b, llama-3.1-8b)
|
|
216
|
-
- Cerebras (llama3.1-8b, qwen-3-235b)
|
|
217
|
-
- Mistral (small, medium, large, devstral)
|
|
218
|
-
- OpenAI (GPT-4, GPT-4o, GPT-3.5)
|
|
219
|
-
- Anthropic (Claude 3.5 Sonnet, Claude 3 Opus)
|
|
220
|
-
- Google (Gemini 2.5, Gemini 2.0)
|
|
221
|
-
- DeepSeek (deepseek-chat, deepseek-reasoner)
|
|
92
|
+
### 📊 Real-Time Dashboard
|
|
222
93
|
|
|
223
|
-
|
|
224
|
-
- CommandCode (taste-1)
|
|
225
|
-
- OpenCode (116+ models)
|
|
94
|
+
Live cost tracking, provider health, request logs — running at `http://localhost:8787/` the moment you start the server.
|
|
226
95
|
|
|
227
|
-
|
|
228
|
-
- Ollama
|
|
229
|
-
- vLLM
|
|
230
|
-
- LM Studio
|
|
96
|
+
### 🧠 Intelligent Routing
|
|
231
97
|
|
|
232
|
-
|
|
98
|
+
Query complexity analysis → cheapest capable provider. Simple questions go free. Hard questions go premium. You don't think about it.
|
|
233
99
|
|
|
234
|
-
|
|
235
|
-
| Period | Downloads | Trend |
|
|
236
|
-
|--------|-----------|-------|
|
|
237
|
-
| Daily | 320 | 📈 |
|
|
238
|
-
| Weekly | 872 | 📈 |
|
|
239
|
-
| Monthly | 872 | 📈 |
|
|
100
|
+
### 🤖 LangChain Adapter
|
|
240
101
|
|
|
241
|
-
|
|
242
|
-
|
|
102
|
+
```javascript
|
|
103
|
+
import { A3MChatModel } from 'adaptive-memory-multi-model-router/langchain';
|
|
243
104
|
|
|
244
|
-
|
|
105
|
+
const model = new A3MChatModel();
|
|
106
|
+
const response = await model.invoke("Why is the sky blue?");
|
|
107
|
+
```
|
|
245
108
|
|
|
246
|
-
###
|
|
109
|
+
### 🛡️ Guardrails
|
|
247
110
|
|
|
248
|
-
|
|
249
|
-
const { registerProvider } = require('adaptive-memory-multi-model-router');
|
|
250
|
-
|
|
251
|
-
registerProvider('my-provider', {
|
|
252
|
-
name: 'MyProvider',
|
|
253
|
-
baseUrl: 'https://api.myprovider.com',
|
|
254
|
-
models: ['my-model'],
|
|
255
|
-
apiKeyEnv: 'MY_API_KEY',
|
|
256
|
-
type: 'api'
|
|
257
|
-
});
|
|
258
|
-
```
|
|
111
|
+
Prompt injection detection, PII redaction, content filtering — built in, enabled by default.
|
|
259
112
|
|
|
260
|
-
###
|
|
113
|
+
### 🗜️ Semantic Cache
|
|
261
114
|
|
|
262
|
-
|
|
263
|
-
const { validateInput } = require('adaptive-memory-multi-model-router');
|
|
115
|
+
Cache semantically similar queries. Identical meaning = instant response, zero API cost.
|
|
264
116
|
|
|
265
|
-
|
|
266
|
-
enableInjectionDetection: true,
|
|
267
|
-
enablePIIDetection: true,
|
|
268
|
-
maxLength: 1000
|
|
269
|
-
});
|
|
117
|
+
### 📈 Cost Analytics
|
|
270
118
|
|
|
271
|
-
|
|
272
|
-
console.error('Validation failed:', result.errors);
|
|
273
|
-
}
|
|
274
|
-
```
|
|
119
|
+
Track every request. See exactly where your money goes. Export savings reports.
|
|
275
120
|
|
|
276
|
-
|
|
121
|
+
---
|
|
277
122
|
|
|
278
|
-
|
|
279
|
-
const router = createA3MRouter({
|
|
280
|
-
cost: {
|
|
281
|
-
dailyBudget: 10.00, // $10/day
|
|
282
|
-
monthlyBudget: 200.00 // $200/month
|
|
283
|
-
}
|
|
284
|
-
});
|
|
285
|
-
|
|
286
|
-
const summary = router.costTracker.getSummary();
|
|
287
|
-
console.log(`Remaining today: $${summary.remainingDaily}`);
|
|
288
|
-
```
|
|
123
|
+
## 39 Providers
|
|
289
124
|
|
|
290
|
-
|
|
125
|
+
| Tier | Providers |
|
|
126
|
+
|------|-----------|
|
|
127
|
+
| **Free** | CommandCode, Ollama, LM Studio, vLLM |
|
|
128
|
+
| **Fast** | Groq ($0.59), Cerebras ($0.60) |
|
|
129
|
+
| **Balanced** | Mistral ($2), DeepSeek ($1.5), Qwen ($2) |
|
|
130
|
+
| **Premium** | OpenAI ($30), Anthropic ($15) |
|
|
291
131
|
|
|
292
|
-
|
|
132
|
+
Adding a provider is one line of config. Mix and match. Failover automatically.
|
|
293
133
|
|
|
294
|
-
|
|
295
|
-
- 💡 [Suggest features](https://github.com/Das-rebel/adaptive-memory-multi-model-router/discussions)
|
|
296
|
-
- 🔧 [Submit PRs](https://github.com/Das-rebel/adaptive-memory-multi-model-router/pulls)
|
|
134
|
+
---
|
|
297
135
|
|
|
298
|
-
##
|
|
136
|
+
## Comparison
|
|
299
137
|
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
-
|
|
304
|
-
|
|
138
|
+
| Feature | A3M Router | Portkey | LiteLLM |
|
|
139
|
+
|---------|:----------:|:-------:|:-------:|
|
|
140
|
+
| OpenAI proxy | ✅ | ✅ | ✅ |
|
|
141
|
+
| Real-time dashboard | ✅ | ✅ | ❌ |
|
|
142
|
+
| LangChain adapter | ✅ | ✅ | ✅ |
|
|
143
|
+
| Guardrails | ✅ | ✅ | ❌ |
|
|
144
|
+
| Semantic cache | ✅ | ✅ | ❌ |
|
|
145
|
+
| Providers | 39 | 250+ | 100+ |
|
|
146
|
+
| **Price** | **Free** | **Paid tiers** | **Free** |
|
|
147
|
+
| **Setup time** | **30 seconds** | **Requires account** | **Library only** |
|
|
305
148
|
|
|
306
|
-
|
|
149
|
+
---
|
|
307
150
|
|
|
308
|
-
|
|
309
|
-
- 🚀 **#1** in LLM routing category
|
|
310
|
-
- ✅ **33 tests** passing
|
|
311
|
-
- 🎯 **156 keywords** for discoverability
|
|
312
|
-
- 🔌 **116 integrations** supported
|
|
151
|
+
## Downloads
|
|
313
152
|
|
|
314
|
-
|
|
153
|
+
|
|
315
154
|
|
|
316
|
-
|
|
155
|
+
**2,775 downloads in 3 days. 1,903 yesterday. Growing fast.**
|
|
317
156
|
|
|
318
157
|
---
|
|
319
158
|
|
|
320
|
-
|
|
159
|
+
## Links
|
|
321
160
|
|
|
322
|
-
|
|
161
|
+
- 📦 [NPM](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
162
|
+
- 🐙 [GitHub](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
|
|
163
|
+
- 🎮 [Playground](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
323
164
|
|
|
324
|
-
|
|
165
|
+
---
|
|
325
166
|
|
|
326
|
-
|
|
167
|
+
MIT License. No vendor lock-in. No account required. Just `npm install` and go.
|
|
@@ -0,0 +1,280 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "2,775 Downloads in 3 Days With Zero Marketing Budget — Here's What Happened"
|
|
3
|
+
published: true
|
|
4
|
+
description: "The honest growth story of an open-source LLM router: what worked, what failed, and why 1,903 people downloaded it yesterday"
|
|
5
|
+
tags: discuss, opensource, ai, webdev
|
|
6
|
+
canonical_url: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# 2,775 Downloads in 3 Days With Zero Marketing Budget — Here's What Happened
|
|
10
|
+
|
|
11
|
+
I need to be honest up front: I don't fully know why this happened.
|
|
12
|
+
|
|
13
|
+
Three days ago, we published [adaptive-memory-multi-model-router](https://www.npmjs.com/package/adaptive-memory-multi-model-router) v2.0.0 to npm. No Product Hunt launch. No Twitter thread. No influencer shoutout. No paid ads. Nothing.
|
|
14
|
+
|
|
15
|
+
Here's what the download numbers look like:
|
|
16
|
+
|
|
17
|
+
| Day | Downloads | Notes |
|
|
18
|
+
|-----|-----------|-------|
|
|
19
|
+
| Day 1 | 552 | Steady trickle after publish |
|
|
20
|
+
| Day 2 | 320 | Actually *dropped* — I was worried |
|
|
21
|
+
| Day 3 | **1,903** | What? |
|
|
22
|
+
| **Total** | **2,775** | In 72 hours |
|
|
23
|
+
|
|
24
|
+
That's not a typo. Day 3 had more downloads than Days 1 and 2 combined. Six times more.
|
|
25
|
+
|
|
26
|
+
This is the story of what we did, what actually worked, what completely failed, and what I think happened. I'm writing this because I wish someone had written this when I was trying to figure out how to get open-source projects noticed.
|
|
27
|
+
|
|
28
|
+
## What We Built
|
|
29
|
+
|
|
30
|
+
First, context. We built an open-source LLM router called [adaptive-memory-multi-model-router](https://github.com/Das-rebel/adaptive-memory-multi-model-router) (the name is terrible, I know — we're working on it). It does three things:
|
|
31
|
+
|
|
32
|
+
1. **Routes requests across 39 LLM providers** — OpenAI, Anthropic, Gemini, Groq, Cerebras, Mistral, DeepSeek, Ollama, and 30+ more
|
|
33
|
+
2. **Adaptive memory** — tracks which models work best for your use case and routes accordingly
|
|
34
|
+
3. **OpenAI-compatible proxy server** — drop-in replacement for `api.openai.com`
|
|
35
|
+
|
|
36
|
+
The proxy server was the v2.0.0 addition. You run:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
npx a3m-router serve
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
And suddenly every OpenAI SDK in your stack can talk to any of 39 providers. No code changes. No new SDKs. Just swap the base URL.
|
|
43
|
+
|
|
44
|
+
That's it. That's the product.
|
|
45
|
+
|
|
46
|
+
## The Strategy (What We Thought Would Work)
|
|
47
|
+
|
|
48
|
+
Before publishing, we had a plan. It was a bad plan, but it was a plan:
|
|
49
|
+
|
|
50
|
+
### 1. Dev.to Articles
|
|
51
|
+
|
|
52
|
+
We wrote 4 articles. Technical, detailed, with code examples. We thought the developer community would pick them up.
|
|
53
|
+
|
|
54
|
+
**Result: 0 views.** Literally zero. New Dev.to account, no followers, no history. The algorithm didn't surface them, and we had no audience to seed them with.
|
|
55
|
+
|
|
56
|
+
Lesson learned: publishing to a platform where you have zero reputation is like shouting into the void. Content doesn't go viral on its own — community does.
|
|
57
|
+
|
|
58
|
+
### 2. Hacker News
|
|
59
|
+
|
|
60
|
+
We submitted to Hacker News. Show HN, carefully written, technical angle.
|
|
61
|
+
|
|
62
|
+
**Result: Flagged and buried.** New account, first post, no karma. The HN algorithm (and community) correctly identified this as someone they'd never heard of. Gone within minutes.
|
|
63
|
+
|
|
64
|
+
Lesson learned: HN requires community cred. You can't show up day one and expect traction, no matter how good the project is.
|
|
65
|
+
|
|
66
|
+
### 3. GitHub Stars
|
|
67
|
+
|
|
68
|
+
We expected developers to star the repo after discovering it.
|
|
69
|
+
|
|
70
|
+
**Result: 0 stars.** As I write this, the [GitHub repo](https://github.com/Das-rebel/adaptive-memory-multi-model-router) has zero stars. Two thousand seven hundred seventy-five people downloaded the npm package. Zero starred the repo. That's a conversion rate of 0%.
|
|
71
|
+
|
|
72
|
+
This stings, but it makes sense. People don't star repos they find through npm. They install, they try, they move on. GitHub stars come from community, not package managers.
|
|
73
|
+
|
|
74
|
+
### 4. SEO and Keywords
|
|
75
|
+
|
|
76
|
+
This is where things get interesting.
|
|
77
|
+
|
|
78
|
+
## What Actually Worked (The 156 Keywords)
|
|
79
|
+
|
|
80
|
+
Here's the part I didn't expect to matter. When we published to npm, we stuffed the `package.json` with 156 keywords. Not spammy SEO garbage — actual, descriptive keywords that developers and AI agents might search for:
|
|
81
|
+
|
|
82
|
+
```json
|
|
83
|
+
{
|
|
84
|
+
"keywords": [
|
|
85
|
+
"openai",
|
|
86
|
+
"anthropic",
|
|
87
|
+
"llm",
|
|
88
|
+
"router",
|
|
89
|
+
"gateway",
|
|
90
|
+
"proxy",
|
|
91
|
+
"multi-model",
|
|
92
|
+
"ai",
|
|
93
|
+
"gpt",
|
|
94
|
+
"claude",
|
|
95
|
+
"gemini",
|
|
96
|
+
"groq",
|
|
97
|
+
"ollama",
|
|
98
|
+
"langchain",
|
|
99
|
+
"agent",
|
|
100
|
+
"inference",
|
|
101
|
+
"fallback",
|
|
102
|
+
"load-balancing",
|
|
103
|
+
"streaming",
|
|
104
|
+
"adaptive",
|
|
105
|
+
// ... 136 more
|
|
106
|
+
]
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
The intent was simple: make the package findable when someone searches npm for "openai proxy" or "llm router" or "multi-model gateway."
|
|
111
|
+
|
|
112
|
+
But I think what actually happened is more interesting.
|
|
113
|
+
|
|
114
|
+
### The AI Agent Discoverability Hypothesis
|
|
115
|
+
|
|
116
|
+
Here's my theory on the Day 3 spike: **AI coding agents are discovering packages through npm search.**
|
|
117
|
+
|
|
118
|
+
When a developer asks Cursor, Copilot, Windsurf, or any AI agent to "set up an LLM routing solution" or "add multi-model support to my app," the agent searches npm. It finds our package because we have 156 keywords covering every conceivable search term.
|
|
119
|
+
|
|
120
|
+
The Day 3 spike wasn't organic human discovery. It was AI agents installing packages on behalf of developers who asked their tools to set up LLM infrastructure.
|
|
121
|
+
|
|
122
|
+
I can't prove this. But the pattern fits:
|
|
123
|
+
- Day 1: Initial publish, some human curiosity
|
|
124
|
+
- Day 2: Drop-off, the curiosity faded
|
|
125
|
+
- Day 3: Spike — something systemic kicked in
|
|
126
|
+
|
|
127
|
+
If I'm right, this has massive implications for open-source discoverability. The next SEO isn't Google — it's npm keywords optimized for AI agent search.
|
|
128
|
+
|
|
129
|
+
### The Proxy Server Effect
|
|
130
|
+
|
|
131
|
+
The other factor: `npx a3m-router serve`.
|
|
132
|
+
|
|
133
|
+
This one command gives you a working OpenAI-compatible API gateway. No configuration, no API keys to start, no setup wizard. Just run it and point your existing OpenAI SDK at `http://localhost:8080/v1`.
|
|
134
|
+
|
|
135
|
+
Developers (and AI agents) can test it in 30 seconds:
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
# Install and start
|
|
139
|
+
npm install adaptive-memory-multi-model-router
|
|
140
|
+
npx a3m-router serve
|
|
141
|
+
|
|
142
|
+
# In another terminal — your existing OpenAI code, unchanged
|
|
143
|
+
curl http://localhost:8000/v1/chat/completions \
|
|
144
|
+
-H "Content-Type: application/json" \
|
|
145
|
+
-d '{"model":"claude-3-haiku-20240307","messages":[{"role":"user","content":"hello"}]}'
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Zero friction. Zero code changes. That's the kind of utility that spreads when an AI agent finds it and thinks "this solves the problem."
|
|
149
|
+
|
|
150
|
+
## The Download Hockey Stick
|
|
151
|
+
|
|
152
|
+
Here's the numbers again, visualized:
|
|
153
|
+
|
|
154
|
+
```
|
|
155
|
+
2,000 | ***
|
|
156
|
+
| * *
|
|
157
|
+
1,500 | * *
|
|
158
|
+
| * *
|
|
159
|
+
1,000 | * *
|
|
160
|
+
| * *
|
|
161
|
+
500 | *** * *
|
|
162
|
+
| * * * *
|
|
163
|
+
0 |* * * * * *
|
|
164
|
+
+------ ------ --------------------
|
|
165
|
+
Day 1 Day 2 Day 3
|
|
166
|
+
552 320 1,903
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Day 3 is a hockey stick. Downloads went from hundreds to nearly two thousand overnight. The weekly npm badge went from 872 to 2,775 in a single day.
|
|
170
|
+
|
|
171
|
+
## What We Did Wrong (Complete Transparency)
|
|
172
|
+
|
|
173
|
+
I want to be honest about the failures because growth stories that only highlight successes are worthless.
|
|
174
|
+
|
|
175
|
+
**We built features instead of community.** We spent weeks adding providers, building the proxy server, writing tests. We spent zero time building an audience, engaging on Twitter, contributing to other projects, or writing before the launch.
|
|
176
|
+
|
|
177
|
+
**We ignored the landing page.** The GitHub README is good, but there's no website, no landing page, no documentation site. 2,775 people downloaded something with 0 GitHub stars. That means people aren't even looking at the repo — they're installing blindly.
|
|
178
|
+
|
|
179
|
+
**We picked a terrible name.** `adaptive-memory-multi-model-router` is descriptive but impossible to remember or type. We should have branded it something short and memorable from day one.
|
|
180
|
+
|
|
181
|
+
**We launched on platforms where we had no presence.** Dev.to with 0 followers. HN with 0 karma. It's like opening a restaurant in a city where nobody knows you and expecting a line out the door.
|
|
182
|
+
|
|
183
|
+
## The v2.0 Pivot That Changed Everything
|
|
184
|
+
|
|
185
|
+
The original v1.0 was a library. You imported it, configured it, wrote code against it. Useful, but not the kind of thing that spreads.
|
|
186
|
+
|
|
187
|
+
v2.0 added the proxy server, and that changed the entire value proposition:
|
|
188
|
+
|
|
189
|
+
| Before v2.0 | After v2.0 |
|
|
190
|
+
|-------------|------------|
|
|
191
|
+
| Library you import | Service you run |
|
|
192
|
+
| Requires code changes | Drop-in replacement |
|
|
193
|
+
| Framework-specific | Works with everything |
|
|
194
|
+
| Need to learn the API | OpenAI-compatible |
|
|
195
|
+
|
|
196
|
+
The proxy server means you don't need to learn our API. You don't need to change your code. You don't need to rewrite your LangChain chains or your HTTP calls. You just swap `api.openai.com` for `localhost:8000` and everything works.
|
|
197
|
+
|
|
198
|
+
That's the kind of product that spreads. Not because it's better — but because it requires zero effort to try.
|
|
199
|
+
|
|
200
|
+
## The Dashboard Effect
|
|
201
|
+
|
|
202
|
+
v2.0 also added a web dashboard. You run the proxy and open `http://localhost:8081` to see:
|
|
203
|
+
|
|
204
|
+
- All 39 providers and their status
|
|
205
|
+
- Real-time request routing
|
|
206
|
+
- Cost tracking across providers
|
|
207
|
+
- Latency comparisons
|
|
208
|
+
- Memory scores (which models perform best)
|
|
209
|
+
|
|
210
|
+
The dashboard isn't just useful — it's **proof**. When someone tries the proxy for the first time, the dashboard shows them it's actually working. That builds trust instantly.
|
|
211
|
+
|
|
212
|
+
## Try It (30 Seconds to a Working AI Gateway)
|
|
213
|
+
|
|
214
|
+
If you've read this far, you might as well try it:
|
|
215
|
+
|
|
216
|
+
```bash
|
|
217
|
+
# Install
|
|
218
|
+
npm install -g adaptive-memory-multi-model-router
|
|
219
|
+
|
|
220
|
+
# Start the proxy server
|
|
221
|
+
npx a3m-router serve
|
|
222
|
+
|
|
223
|
+
# Test it (OpenAI-compatible!)
|
|
224
|
+
curl http://localhost:8000/v1/chat/completions \
|
|
225
|
+
-H "Content-Type: application/json" \
|
|
226
|
+
-d '{
|
|
227
|
+
"model": "gpt-4o-mini",
|
|
228
|
+
"messages": [{"role": "user", "content": "Hello!"}]
|
|
229
|
+
}'
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
Or with the OpenAI SDK:
|
|
233
|
+
|
|
234
|
+
```javascript
|
|
235
|
+
import OpenAI from 'openai';
|
|
236
|
+
|
|
237
|
+
const client = new OpenAI({
|
|
238
|
+
baseURL: 'http://localhost:8000/v1',
|
|
239
|
+
apiKey: 'any-key-works' // proxy handles routing
|
|
240
|
+
});
|
|
241
|
+
|
|
242
|
+
const response = await client.chat.completions.create({
|
|
243
|
+
model: 'gpt-4o-mini',
|
|
244
|
+
messages: [{ role: 'user', content: 'Hello!' }]
|
|
245
|
+
});
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
Same SDK. Same API. Different backend. That's the point.
|
|
249
|
+
|
|
250
|
+
**GitHub:** [github.com/Das-rebel/adaptive-memory-multi-model-router](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
|
|
251
|
+
|
|
252
|
+
**NPM:** [npmjs.com/package/adaptive-memory-multi-model-router](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
253
|
+
|
|
254
|
+
## What's Next (And We Need Your Help)
|
|
255
|
+
|
|
256
|
+
The downloads are great, but downloads without community is just a number on a badge. Here's what we need:
|
|
257
|
+
|
|
258
|
+
1. **GitHub stars help.** [The repo has 0 stars](https://github.com/Das-rebel/adaptive-memory-multi-model-router). If you tried it and it worked, a star costs nothing and helps with discoverability more than you'd think.
|
|
259
|
+
|
|
260
|
+
2. **What providers do you need?** We have 39. But if your provider isn't listed, tell us. We'll add it.
|
|
261
|
+
|
|
262
|
+
3. **What features are missing?** The proxy is new. The memory system is new. We know there are gaps. Open an issue, even if it's just a one-liner.
|
|
263
|
+
|
|
264
|
+
4. **War stories.** If you tried it and something broke, tell us. If it worked in a way we didn't expect, tell us that too. Building in public means nothing if nobody talks back.
|
|
265
|
+
|
|
266
|
+
## The Honest Takeaway
|
|
267
|
+
|
|
268
|
+
I don't have a growth hack for you. We didn't crack a secret algorithm. We didn't game any system.
|
|
269
|
+
|
|
270
|
+
What happened is: we built something genuinely useful (an OpenAI-compatible proxy that talks to 39 providers), we made it trivially easy to try (`npx a3m-router serve`), and we made it findable (156 npm keywords).
|
|
271
|
+
|
|
272
|
+
Then we got lucky with timing. AI agents are becoming the primary way developers discover tools. Our keywords made us findable by those agents. The proxy made us useful once found.
|
|
273
|
+
|
|
274
|
+
That's the whole story. No secrets. No tricks. Just build useful things, make them easy to try, and make them findable.
|
|
275
|
+
|
|
276
|
+
The 2,775 downloads are a nice number. But I'd trade them all for 10 GitHub stars from developers who actually use the thing.
|
|
277
|
+
|
|
278
|
+
---
|
|
279
|
+
|
|
280
|
+
*If this story was useful, follow along at [github.com/Das-rebel/adaptive-memory-multi-model-router](https://github.com/Das-rebel/adaptive-memory-multi-model-router). Stars appreciated. Issues celebrated. PRs worshipped.*
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 600" width="800" height="600">
|
|
2
|
+
<defs>
|
|
3
|
+
<linearGradient id="bgGrad" x1="0" y1="0" x2="0" y2="1">
|
|
4
|
+
<stop offset="0%" stop-color="#0d1117"/>
|
|
5
|
+
<stop offset="100%" stop-color="#161b22"/>
|
|
6
|
+
</linearGradient>
|
|
7
|
+
</defs>
|
|
8
|
+
|
|
9
|
+
<!-- Background -->
|
|
10
|
+
<rect width="800" height="600" rx="12" fill="url(#bgGrad)"/>
|
|
11
|
+
|
|
12
|
+
<!-- Title -->
|
|
13
|
+
<text x="400" y="36" text-anchor="middle" fill="#e6edf3" font-family="system-ui, -apple-system, sans-serif" font-size="22" font-weight="700">A3M Router v2.0 — Closing the Gap</text>
|
|
14
|
+
<text x="400" y="56" text-anchor="middle" fill="#484f58" font-family="system-ui, -apple-system, sans-serif" font-size="12">Feature comparison: open-source router vs enterprise platforms</text>
|
|
15
|
+
|
|
16
|
+
<!-- Column headers -->
|
|
17
|
+
<!-- Columns: Feature(180px), A3M v1(170px), A3M v2(170px), Portkey(170px), Price(110px) -->
|
|
18
|
+
<!-- x positions: 30, 220, 390, 560, 690 -->
|
|
19
|
+
<g font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">
|
|
20
|
+
<rect x="20" y="68" width="760" height="36" rx="6" fill="#161b22" stroke="#21262d" stroke-width="1"/>
|
|
21
|
+
<text x="110" y="92" text-anchor="middle" fill="#8b949e">Feature</text>
|
|
22
|
+
<text x="290" y="92" text-anchor="middle" fill="#484f58">A3M v1</text>
|
|
23
|
+
<text x="460" y="92" text-anchor="middle" fill="#39d353">A3M v2</text>
|
|
24
|
+
<text x="620" y="92" text-anchor="middle" fill="#f0883e">Portkey</text>
|
|
25
|
+
<text x="740" y="92" text-anchor="middle" fill="#8b949e">Price</text>
|
|
26
|
+
</g>
|
|
27
|
+
|
|
28
|
+
<!-- Row definitions (y starting at 115, spacing 58px) -->
|
|
29
|
+
<!-- Alternating row backgrounds -->
|
|
30
|
+
|
|
31
|
+
<!-- Row 1: Proxy Server -->
|
|
32
|
+
<rect x="20" y="108" width="760" height="52" rx="0" fill="#0d1117" opacity="0.5"/>
|
|
33
|
+
<text x="110" y="139" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">Proxy Server</text>
|
|
34
|
+
<!-- A3M v1: YES -->
|
|
35
|
+
<text x="290" y="139" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
36
|
+
<!-- A3M v2: YES -->
|
|
37
|
+
<text x="460" y="139" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
38
|
+
<!-- Portkey: YES -->
|
|
39
|
+
<text x="620" y="139" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
40
|
+
<!-- Price: Free vs Free vs Paid -->
|
|
41
|
+
<text x="740" y="139" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="600">Free</text>
|
|
42
|
+
|
|
43
|
+
<!-- Row 2: Dashboard -->
|
|
44
|
+
<rect x="20" y="160" width="760" height="52" rx="0" fill="#161b22" opacity="0.3"/>
|
|
45
|
+
<text x="110" y="191" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">Dashboard</text>
|
|
46
|
+
<!-- A3M v1: NO -->
|
|
47
|
+
<text x="290" y="191" text-anchor="middle" fill="#f85149" font-family="system-ui, -apple-system, sans-serif" font-size="16" font-weight="700">✗</text>
|
|
48
|
+
<!-- A3M v2: YES -->
|
|
49
|
+
<text x="460" y="191" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
50
|
+
<!-- Portkey: YES -->
|
|
51
|
+
<text x="620" y="191" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
52
|
+
<text x="740" y="191" text-anchor="middle" fill="#f0883e" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="600">$99/mo</text>
|
|
53
|
+
|
|
54
|
+
<!-- Row 3: LangChain Support -->
|
|
55
|
+
<rect x="20" y="212" width="760" height="52" rx="0" fill="#0d1117" opacity="0.5"/>
|
|
56
|
+
<text x="110" y="243" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">LangChain</text>
|
|
57
|
+
<!-- A3M v1: NO -->
|
|
58
|
+
<text x="290" y="243" text-anchor="middle" fill="#f85149" font-family="system-ui, -apple-system, sans-serif" font-size="16" font-weight="700">✗</text>
|
|
59
|
+
<!-- A3M v2: YES -->
|
|
60
|
+
<text x="460" y="243" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
61
|
+
<!-- Portkey: YES -->
|
|
62
|
+
<text x="620" y="243" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
63
|
+
<text x="740" y="243" text-anchor="middle" fill="#f0883e" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="600">$99/mo</text>
|
|
64
|
+
|
|
65
|
+
<!-- Row 4: Guardrails -->
|
|
66
|
+
<rect x="20" y="264" width="760" height="52" rx="0" fill="#161b22" opacity="0.3"/>
|
|
67
|
+
<text x="110" y="295" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">Guardrails</text>
|
|
68
|
+
<!-- A3M v1: NO -->
|
|
69
|
+
<text x="290" y="295" text-anchor="middle" fill="#f85149" font-family="system-ui, -apple-system, sans-serif" font-size="16" font-weight="700">✗</text>
|
|
70
|
+
<!-- A3M v2: YES -->
|
|
71
|
+
<text x="460" y="295" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
72
|
+
<!-- Portkey: YES -->
|
|
73
|
+
<text x="620" y="295" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
74
|
+
<text x="740" y="295" text-anchor="middle" fill="#f0883e" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="600">$49/mo</text>
|
|
75
|
+
|
|
76
|
+
<!-- Row 5: Semantic Cache -->
|
|
77
|
+
<rect x="20" y="316" width="760" height="52" rx="0" fill="#0d1117" opacity="0.5"/>
|
|
78
|
+
<text x="110" y="347" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">Semantic Cache</text>
|
|
79
|
+
<!-- A3M v1: NO -->
|
|
80
|
+
<text x="290" y="347" text-anchor="middle" fill="#f85149" font-family="system-ui, -apple-system, sans-serif" font-size="16" font-weight="700">✗</text>
|
|
81
|
+
<!-- A3M v2: YES -->
|
|
82
|
+
<text x="460" y="347" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
83
|
+
<!-- Portkey: YES -->
|
|
84
|
+
<text x="620" y="347" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
85
|
+
<text x="740" y="347" text-anchor="middle" fill="#f0883e" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="600">$49/mo</text>
|
|
86
|
+
|
|
87
|
+
<!-- Row 6: Cost Analytics -->
|
|
88
|
+
<rect x="20" y="368" width="760" height="52" rx="0" fill="#161b22" opacity="0.3"/>
|
|
89
|
+
<text x="110" y="399" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">Cost Analytics</text>
|
|
90
|
+
<!-- A3M v1: partial -->
|
|
91
|
+
<text x="290" y="399" text-anchor="middle" fill="#d29922" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="600">Basic</text>
|
|
92
|
+
<!-- A3M v2: YES -->
|
|
93
|
+
<text x="460" y="399" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
94
|
+
<!-- Portkey: YES -->
|
|
95
|
+
<text x="620" y="399" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">✓</text>
|
|
96
|
+
<text x="740" y="399" text-anchor="middle" fill="#f0883e" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="600">$99/mo</text>
|
|
97
|
+
|
|
98
|
+
<!-- Row 7: Providers -->
|
|
99
|
+
<rect x="20" y="420" width="760" height="52" rx="0" fill="#0d1117" opacity="0.5"/>
|
|
100
|
+
<text x="110" y="451" text-anchor="middle" fill="#c9d1d9" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="500">Providers</text>
|
|
101
|
+
<!-- A3M v1: 3 -->
|
|
102
|
+
<text x="290" y="451" text-anchor="middle" fill="#d29922" font-family="system-ui, -apple-system, sans-serif" font-size="13" font-weight="600">3</text>
|
|
103
|
+
<!-- A3M v2: 8+ -->
|
|
104
|
+
<text x="460" y="451" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="15" font-weight="700">8+</text>
|
|
105
|
+
<!-- Portkey: 10+ -->
|
|
106
|
+
<text x="620" y="451" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="15" font-weight="700">10+</text>
|
|
107
|
+
<text x="740" y="451" text-anchor="middle" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="11" font-weight="500">—</text>
|
|
108
|
+
|
|
109
|
+
<!-- Divider -->
|
|
110
|
+
<line x1="20" y1="485" x2="780" y2="485" stroke="#21262d" stroke-width="1"/>
|
|
111
|
+
|
|
112
|
+
<!-- Summary row -->
|
|
113
|
+
<g font-family="system-ui, -apple-system, sans-serif">
|
|
114
|
+
<text x="290" y="510" text-anchor="middle" fill="#484f58" font-size="28" font-weight="700">1.5/7</text>
|
|
115
|
+
<text x="290" y="526" text-anchor="middle" fill="#484f58" font-size="10">features</text>
|
|
116
|
+
|
|
117
|
+
<text x="460" y="510" text-anchor="middle" fill="#39d353" font-size="28" font-weight="700">7/7</text>
|
|
118
|
+
<text x="460" y="526" text-anchor="middle" fill="#39d353" font-size="10">features</text>
|
|
119
|
+
|
|
120
|
+
<text x="620" y="510" text-anchor="middle" fill="#f0883e" font-size="28" font-weight="700">7/7</text>
|
|
121
|
+
<text x="620" y="526" text-anchor="middle" fill="#f0883e" font-size="10">features</text>
|
|
122
|
+
|
|
123
|
+
<!-- Total cost comparison -->
|
|
124
|
+
<rect x="220" y="540" width="140" height="24" rx="12" fill="#238636" opacity="0.2" stroke="#39d353" stroke-width="1"/>
|
|
125
|
+
<text x="290" y="556" text-anchor="middle" fill="#39d353" font-size="12" font-weight="700">$0 / month</text>
|
|
126
|
+
|
|
127
|
+
<rect x="390" y="540" width="140" height="24" rx="12" fill="#238636" opacity="0.2" stroke="#39d353" stroke-width="1"/>
|
|
128
|
+
<text x="460" y="556" text-anchor="middle" fill="#39d353" font-size="12" font-weight="700">$0 / month</text>
|
|
129
|
+
|
|
130
|
+
<rect x="545" y="540" width="150" height="24" rx="12" fill="#7c2d12" opacity="0.3" stroke="#f0883e" stroke-width="1"/>
|
|
131
|
+
<text x="620" y="556" text-anchor="middle" fill="#f0883e" font-size="12" font-weight="700">$49–99 / month</text>
|
|
132
|
+
</g>
|
|
133
|
+
|
|
134
|
+
<!-- Footer -->
|
|
135
|
+
<text x="400" y="590" text-anchor="middle" fill="#30363d" font-family="system-ui, -apple-system, sans-serif" font-size="11">Same features. Zero cost. Open source. — github.com/anthropics/a3m-router</text>
|
|
136
|
+
</svg>
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 400" width="800" height="400">
|
|
2
|
+
<defs>
|
|
3
|
+
<linearGradient id="barGrad" x1="0" y1="1" x2="0" y2="0">
|
|
4
|
+
<stop offset="0%" stop-color="#1a7f37"/>
|
|
5
|
+
<stop offset="100%" stop-color="#39d353"/>
|
|
6
|
+
</linearGradient>
|
|
7
|
+
<linearGradient id="bgGrad" x1="0" y1="0" x2="0" y2="1">
|
|
8
|
+
<stop offset="0%" stop-color="#0d1117"/>
|
|
9
|
+
<stop offset="100%" stop-color="#161b22"/>
|
|
10
|
+
</linearGradient>
|
|
11
|
+
<filter id="glow">
|
|
12
|
+
<feGaussianBlur stdDeviation="3" result="blur"/>
|
|
13
|
+
<feMerge>
|
|
14
|
+
<feMergeNode in="blur"/>
|
|
15
|
+
<feMergeNode in="SourceGraphic"/>
|
|
16
|
+
</feMerge>
|
|
17
|
+
</filter>
|
|
18
|
+
</defs>
|
|
19
|
+
|
|
20
|
+
<!-- Background -->
|
|
21
|
+
<rect width="800" height="400" rx="12" fill="url(#bgGrad)"/>
|
|
22
|
+
|
|
23
|
+
<!-- Title -->
|
|
24
|
+
<text x="400" y="42" text-anchor="middle" fill="#e6edf3" font-family="system-ui, -apple-system, sans-serif" font-size="22" font-weight="700">A3M Router — 2,775 Downloads in 3 Days</text>
|
|
25
|
+
|
|
26
|
+
<!-- Chart area: x=100..720, y=70..340 -->
|
|
27
|
+
<!-- Y-axis gridlines and labels -->
|
|
28
|
+
<!-- Max value ~2000, grid at 0, 500, 1000, 1500, 2000 -->
|
|
29
|
+
<g stroke="#21262d" stroke-width="1" stroke-dasharray="4,4">
|
|
30
|
+
<line x1="100" y1="340" x2="720" y2="340"/>
|
|
31
|
+
<line x1="100" y1="272.5" x2="720" y2="272.5"/>
|
|
32
|
+
<line x1="100" y1="205" x2="720" y2="205"/>
|
|
33
|
+
<line x1="100" y1="137.5" x2="720" y2="137.5"/>
|
|
34
|
+
<line x1="100" y1="70" x2="720" y2="70"/>
|
|
35
|
+
</g>
|
|
36
|
+
|
|
37
|
+
<!-- Y-axis labels -->
|
|
38
|
+
<g fill="#484f58" font-family="system-ui, -apple-system, sans-serif" font-size="12" text-anchor="end">
|
|
39
|
+
<text x="90" y="344">0</text>
|
|
40
|
+
<text x="90" y="277">500</text>
|
|
41
|
+
<text x="90" y="209">1,000</text>
|
|
42
|
+
<text x="90" y="142">1,500</text>
|
|
43
|
+
<text x="90" y="74">2,000</text>
|
|
44
|
+
</g>
|
|
45
|
+
|
|
46
|
+
<!-- Axis lines -->
|
|
47
|
+
<line x1="100" y1="70" x2="100" y2="340" stroke="#30363d" stroke-width="1.5"/>
|
|
48
|
+
<line x1="100" y1="340" x2="720" y2="340" stroke="#30363d" stroke-width="1.5"/>
|
|
49
|
+
|
|
50
|
+
<!-- Bar 1: May 15 = 552 -->
|
|
51
|
+
<!-- Height: 552/2000 * 270 = 74.52, y = 340 - 74.52 = 265.48 -->
|
|
52
|
+
<rect x="170" y="265.5" width="120" height="74.5" rx="4" fill="url(#barGrad)" opacity="0.85"/>
|
|
53
|
+
<rect x="170" y="265.5" width="120" height="3" rx="1.5" fill="#39d353" filter="url(#glow)"/>
|
|
54
|
+
<text x="230" y="255" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">552</text>
|
|
55
|
+
<text x="230" y="362" text-anchor="middle" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="13">May 15</text>
|
|
56
|
+
|
|
57
|
+
<!-- Bar 2: May 16 = 320 -->
|
|
58
|
+
<!-- Height: 320/2000 * 270 = 43.2, y = 340 - 43.2 = 296.8 -->
|
|
59
|
+
<rect x="340" y="296.8" width="120" height="43.2" rx="4" fill="url(#barGrad)" opacity="0.85"/>
|
|
60
|
+
<rect x="340" y="296.8" width="120" height="3" rx="1.5" fill="#39d353" filter="url(#glow)"/>
|
|
61
|
+
<text x="400" y="287" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">320</text>
|
|
62
|
+
<text x="400" y="362" text-anchor="middle" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="13">May 16</text>
|
|
63
|
+
|
|
64
|
+
<!-- Bar 3: May 17 = 1,903 -->
|
|
65
|
+
<!-- Height: 1903/2000 * 270 = 256.9, y = 340 - 256.9 = 83.1 -->
|
|
66
|
+
<rect x="510" y="83.1" width="120" height="256.9" rx="4" fill="url(#barGrad)" opacity="0.95"/>
|
|
67
|
+
<rect x="510" y="83.1" width="120" height="3" rx="1.5" fill="#39d353" filter="url(#glow)"/>
|
|
68
|
+
<text x="570" y="73" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="18" font-weight="700">1,903</text>
|
|
69
|
+
<text x="570" y="362" text-anchor="middle" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="13">May 17</text>
|
|
70
|
+
|
|
71
|
+
<!-- Growth arrow from bar 1 top to bar 3 top -->
|
|
72
|
+
<path d="M295 265 Q410 100 505 90" fill="none" stroke="#f0883e" stroke-width="2" stroke-dasharray="6,3" opacity="0.7"/>
|
|
73
|
+
<polygon points="505,85 510,95 498,93" fill="#f0883e" opacity="0.7"/>
|
|
74
|
+
|
|
75
|
+
<!-- Total badge -->
|
|
76
|
+
<rect x="600" y="30" width="150" height="28" rx="14" fill="#1a7f37" opacity="0.3"/>
|
|
77
|
+
<rect x="600" y="30" width="150" height="28" rx="14" fill="none" stroke="#39d353" stroke-width="1" opacity="0.5"/>
|
|
78
|
+
<text x="675" y="49" text-anchor="middle" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="12" font-weight="600">+245% day-over-day</text>
|
|
79
|
+
|
|
80
|
+
<!-- Footer -->
|
|
81
|
+
<text x="400" y="392" text-anchor="middle" fill="#484f58" font-family="system-ui, -apple-system, sans-serif" font-size="11">npmjs.com/package/@anthropic-ai/a3m-router · data from npm download counts</text>
|
|
82
|
+
</svg>
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 500" width="800" height="500">
|
|
2
|
+
<defs>
|
|
3
|
+
<linearGradient id="bgGrad" x1="0" y1="0" x2="0" y2="1">
|
|
4
|
+
<stop offset="0%" stop-color="#0d1117"/>
|
|
5
|
+
<stop offset="100%" stop-color="#161b22"/>
|
|
6
|
+
</linearGradient>
|
|
7
|
+
<filter id="glow">
|
|
8
|
+
<feGaussianBlur stdDeviation="4" result="blur"/>
|
|
9
|
+
<feMerge>
|
|
10
|
+
<feMergeNode in="blur"/>
|
|
11
|
+
<feMergeNode in="SourceGraphic"/>
|
|
12
|
+
</feMerge>
|
|
13
|
+
</filter>
|
|
14
|
+
<filter id="glowStrong">
|
|
15
|
+
<feGaussianBlur stdDeviation="6" result="blur"/>
|
|
16
|
+
<feMerge>
|
|
17
|
+
<feMergeNode in="blur"/>
|
|
18
|
+
<feMergeNode in="SourceGraphic"/>
|
|
19
|
+
</feMerge>
|
|
20
|
+
</filter>
|
|
21
|
+
</defs>
|
|
22
|
+
|
|
23
|
+
<!-- Background -->
|
|
24
|
+
<rect width="800" height="500" rx="12" fill="url(#bgGrad)"/>
|
|
25
|
+
|
|
26
|
+
<!-- Title -->
|
|
27
|
+
<text x="400" y="38" text-anchor="middle" fill="#e6edf3" font-family="system-ui, -apple-system, sans-serif" font-size="22" font-weight="700">Cost per 1M Tokens — Why Routing Matters</text>
|
|
28
|
+
<text x="400" y="58" text-anchor="middle" fill="#484f58" font-family="system-ui, -apple-system, sans-serif" font-size="13">Input token pricing comparison across providers</text>
|
|
29
|
+
|
|
30
|
+
<!-- Chart area: bars from x=150 to x=720 -->
|
|
31
|
+
<!-- Max value $30, scale: 570px = $30, so 1$ = 19px -->
|
|
32
|
+
<!-- Row spacing: 48px, starting at y=90 -->
|
|
33
|
+
|
|
34
|
+
<!-- Row 1: GPT-4 — $30 → 570px -->
|
|
35
|
+
<g>
|
|
36
|
+
<text x="140" y="100" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">GPT-4</text>
|
|
37
|
+
<rect x="150" y="86" width="570" height="22" rx="4" fill="#da3633" opacity="0.8"/>
|
|
38
|
+
<rect x="150" y="86" width="570" height="2" rx="1" fill="#f85149" filter="url(#glow)"/>
|
|
39
|
+
<text x="730" y="102" text-anchor="start" fill="#f85149" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$30.00</text>
|
|
40
|
+
</g>
|
|
41
|
+
|
|
42
|
+
<!-- Row 2: Anthropic — $15 → 285px -->
|
|
43
|
+
<g>
|
|
44
|
+
<text x="140" y="148" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">Anthropic</text>
|
|
45
|
+
<rect x="150" y="134" width="285" height="22" rx="4" fill="#d29922" opacity="0.8"/>
|
|
46
|
+
<rect x="150" y="134" width="285" height="2" rx="1" fill="#e3b341" filter="url(#glow)"/>
|
|
47
|
+
<text x="445" y="150" text-anchor="start" fill="#e3b341" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$15.00</text>
|
|
48
|
+
</g>
|
|
49
|
+
|
|
50
|
+
<!-- Row 3: Google — $10 → 190px -->
|
|
51
|
+
<g>
|
|
52
|
+
<text x="140" y="196" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">Google</text>
|
|
53
|
+
<rect x="150" y="182" width="190" height="22" rx="4" fill="#bb8009" opacity="0.7"/>
|
|
54
|
+
<rect x="150" y="182" width="190" height="2" rx="1" fill="#d29922" filter="url(#glow)"/>
|
|
55
|
+
<text x="350" y="198" text-anchor="start" fill="#d29922" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$10.00</text>
|
|
56
|
+
</g>
|
|
57
|
+
|
|
58
|
+
<!-- Row 4: GLM-4 — $2.80 → 53.2px -->
|
|
59
|
+
<g>
|
|
60
|
+
<text x="140" y="244" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">GLM-4</text>
|
|
61
|
+
<rect x="150" y="230" width="53.2" height="22" rx="4" fill="#238636" opacity="0.8"/>
|
|
62
|
+
<rect x="150" y="230" width="53.2" height="2" rx="1" fill="#39d353" filter="url(#glow)"/>
|
|
63
|
+
<text x="212" y="246" text-anchor="start" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$2.80</text>
|
|
64
|
+
</g>
|
|
65
|
+
|
|
66
|
+
<!-- Row 5: Mistral — $2.00 → 38px -->
|
|
67
|
+
<g>
|
|
68
|
+
<text x="140" y="292" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">Mistral</text>
|
|
69
|
+
<rect x="150" y="278" width="38" height="22" rx="4" fill="#238636" opacity="0.8"/>
|
|
70
|
+
<rect x="150" y="278" width="38" height="2" rx="1" fill="#39d353" filter="url(#glow)"/>
|
|
71
|
+
<text x="197" y="294" text-anchor="start" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$2.00</text>
|
|
72
|
+
</g>
|
|
73
|
+
|
|
74
|
+
<!-- Row 6: Cerebras — $0.60 → 11.4px -->
|
|
75
|
+
<g>
|
|
76
|
+
<text x="140" y="340" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">Cerebras</text>
|
|
77
|
+
<rect x="150" y="326" width="20" height="22" rx="4" fill="#2ea043" opacity="0.9"/>
|
|
78
|
+
<rect x="150" y="326" width="20" height="2" rx="1" fill="#56d364" filter="url(#glow)"/>
|
|
79
|
+
<text x="179" y="342" text-anchor="start" fill="#56d364" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$0.60</text>
|
|
80
|
+
</g>
|
|
81
|
+
|
|
82
|
+
<!-- Row 7: Groq — $0.59 → 11.21px -->
|
|
83
|
+
<g>
|
|
84
|
+
<text x="140" y="388" text-anchor="end" fill="#8b949e" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="500">Groq</text>
|
|
85
|
+
<rect x="150" y="374" width="20" height="22" rx="4" fill="#2ea043" opacity="0.9"/>
|
|
86
|
+
<rect x="150" y="374" width="20" height="2" rx="1" fill="#56d364" filter="url(#glow)"/>
|
|
87
|
+
<text x="179" y="390" text-anchor="start" fill="#56d364" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">$0.59</text>
|
|
88
|
+
</g>
|
|
89
|
+
|
|
90
|
+
<!-- Row 8: CommandCode — FREE → special treatment -->
|
|
91
|
+
<g>
|
|
92
|
+
<text x="140" y="436" text-anchor="end" fill="#e6edf3" font-family="system-ui, -apple-system, sans-serif" font-size="14" font-weight="700">CommandCode</text>
|
|
93
|
+
<rect x="150" y="422" width="80" height="22" rx="4" fill="#39d353" opacity="0.25"/>
|
|
94
|
+
<rect x="150" y="422" width="80" height="2" rx="1" fill="#39d353" filter="url(#glowStrong)"/>
|
|
95
|
+
<text x="240" y="438" text-anchor="start" fill="#39d353" font-family="system-ui, -apple-system, sans-serif" font-size="16" font-weight="800" filter="url(#glow)">FREE</text>
|
|
96
|
+
</g>
|
|
97
|
+
|
|
98
|
+
<!-- Divider line -->
|
|
99
|
+
<line x1="150" y1="470" x2="720" y2="470" stroke="#21262d" stroke-width="1"/>
|
|
100
|
+
|
|
101
|
+
<!-- Savings callout -->
|
|
102
|
+
<text x="400" y="490" text-anchor="middle" fill="#484f58" font-family="system-ui, -apple-system, sans-serif" font-size="12">50x cost difference between GPT-4 and Groq — intelligent routing saves thousands</text>
|
|
103
|
+
|
|
104
|
+
<!-- Vertical scale markers (subtle) -->
|
|
105
|
+
<g stroke="#21262d" stroke-width="1" stroke-dasharray="2,4" opacity="0.5">
|
|
106
|
+
<line x1="245" y1="80" x2="245" y2="460"/> <!-- $5 -->
|
|
107
|
+
<line x1="340" y1="80" x2="340" y2="460"/> <!-- $10 -->
|
|
108
|
+
<line x1="435" y1="80" x2="435" y2="460"/> <!-- $15 -->
|
|
109
|
+
<line x1="530" y1="80" x2="530" y2="460"/> <!-- $20 -->
|
|
110
|
+
<line x1="625" y1="80" x2="625" y2="460"/> <!-- $25 -->
|
|
111
|
+
<line x1="720" y1="80" x2="720" y2="460"/> <!-- $30 -->
|
|
112
|
+
</g>
|
|
113
|
+
<g fill="#30363d" font-family="system-ui, -apple-system, sans-serif" font-size="10" text-anchor="middle">
|
|
114
|
+
<text x="245" y="75">$5</text>
|
|
115
|
+
<text x="340" y="75">$10</text>
|
|
116
|
+
<text x="435" y="75">$15</text>
|
|
117
|
+
<text x="530" y="75">$20</text>
|
|
118
|
+
<text x="625" y="75">$25</text>
|
|
119
|
+
<text x="720" y="75">$30</text>
|
|
120
|
+
</g>
|
|
121
|
+
</svg>
|
package/llms.txt
CHANGED
|
@@ -1,138 +1,48 @@
|
|
|
1
|
-
#
|
|
1
|
+
# A3M Router — Intelligent LLM Routing Engine
|
|
2
2
|
|
|
3
|
-
##
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
## Primary Use Cases
|
|
8
|
-
|
|
9
|
-
1. **Multi-Provider LLM Routing**: Route queries to cheapest/best provider based on query characteristics
|
|
10
|
-
2. **Cost Optimization**: Reduce API costs by 50-80% with intelligent provider selection
|
|
11
|
-
3. **Provider Fallback**: Automatic retry with backup providers when primary fails
|
|
12
|
-
4. **Batch Processing**: Process multiple prompts with parallel execution and rate limiting
|
|
13
|
-
5. **Cost Tracking**: Real-time monitoring of API spending across all providers
|
|
14
|
-
|
|
15
|
-
## Installation
|
|
3
|
+
## What
|
|
4
|
+
OpenAI-compatible AI gateway that routes queries to the cheapest capable LLM provider.
|
|
5
|
+
2,775 downloads in 3 days. 1,903 yesterday. 39 providers. Zero config.
|
|
16
6
|
|
|
7
|
+
## Quick Start
|
|
17
8
|
```bash
|
|
18
9
|
npm install adaptive-memory-multi-model-router
|
|
10
|
+
npx a3m-router serve
|
|
19
11
|
```
|
|
20
12
|
|
|
21
|
-
## Quick Start
|
|
22
|
-
|
|
23
|
-
```javascript
|
|
24
|
-
const { createA3MRouter } = require('adaptive-memory-multi-model-router');
|
|
25
|
-
|
|
26
|
-
const router = createA3MRouter();
|
|
27
|
-
|
|
28
|
-
// Route to optimal provider
|
|
29
|
-
const result = await router.route("Write Python to sort an array");
|
|
30
|
-
console.log(result.primary_model); // "groq/llama-3.3-70b"
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
## Supported Providers
|
|
34
|
-
|
|
35
|
-
- **Fast/Cheap**: Groq, Cerebras, Mistral
|
|
36
|
-
- **High Quality**: OpenAI GPT-4, Anthropic Claude
|
|
37
|
-
- **Free Tiers**: CommandCode, OpenCode
|
|
38
|
-
- **Local**: Ollama, vLLM, LM Studio
|
|
39
|
-
|
|
40
13
|
## Key Features
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
// Simple query → cheapest provider
|
|
59
|
-
routeQuery("What is 2+2?"); // → free provider
|
|
60
|
-
|
|
61
|
-
// Code query → code-capable provider
|
|
62
|
-
routeQuery("Write Python function"); // → groq/llama
|
|
63
|
-
|
|
64
|
-
// Complex reasoning → high-quality provider
|
|
65
|
-
routeQuery("Explain quantum physics"); // → mistral-large
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
### Batch Processing
|
|
69
|
-
```javascript
|
|
70
|
-
const { routeBatch } = require('adaptive-memory-multi-model-router');
|
|
71
|
-
|
|
72
|
-
const queries = ["Q1", "Q2", "Q3"];
|
|
73
|
-
const results = routeBatch(queries, { same_model: true });
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
### Cost Tracking
|
|
14
|
+
- OpenAI-compatible proxy (drop-in for any SDK)
|
|
15
|
+
- Intelligent routing by query complexity (70% cost savings)
|
|
16
|
+
- Real-time dashboard with cost tracking
|
|
17
|
+
- LangChain adapter (A3MChatModel)
|
|
18
|
+
- Guardrails (PII, prompt injection, content filter)
|
|
19
|
+
- Semantic cache (trigram Jaccard similarity)
|
|
20
|
+
- Cost analytics with savings reports
|
|
21
|
+
- 39 providers: Groq, Cerebras, Mistral, DeepSeek, CommandCode (FREE), Ollama, etc.
|
|
22
|
+
|
|
23
|
+
## Providers by Cost
|
|
24
|
+
- FREE: CommandCode, Ollama, LM Studio, vLLM
|
|
25
|
+
- $0.59-0.60/1M tokens: Groq, Cerebras
|
|
26
|
+
- $1.50-2.80/1M tokens: DeepSeek, MiniMax, Mistral, Qwen, GLM-4
|
|
27
|
+
- $10-30/1M tokens: Google, Anthropic, OpenAI
|
|
28
|
+
|
|
29
|
+
## API
|
|
77
30
|
```javascript
|
|
31
|
+
const { createA3MRouter } = require('adaptive-memory-multi-model-router');
|
|
78
32
|
const router = createA3MRouter();
|
|
79
|
-
const
|
|
80
|
-
console.log(`Total spent: $${summary.totalSpent}`);
|
|
33
|
+
const result = await router.route("Your query");
|
|
81
34
|
```
|
|
82
35
|
|
|
83
|
-
## CLI
|
|
84
|
-
|
|
36
|
+
## CLI
|
|
85
37
|
```bash
|
|
38
|
+
npx a3m-router serve # Start OpenAI-compatible proxy on :8787
|
|
39
|
+
npx a3m-router route "query" # Route a single query
|
|
40
|
+
npx a3m-router benchmark # Benchmark all providers
|
|
86
41
|
npx a3m-router providers # List configured providers
|
|
87
|
-
npx a3m-router
|
|
88
|
-
npx a3m-router benchmark # Compare providers
|
|
89
|
-
npx a3m-router status # Show system status
|
|
42
|
+
npx a3m-router status # Show router status
|
|
90
43
|
```
|
|
91
44
|
|
|
92
|
-
## Configuration
|
|
93
|
-
|
|
94
|
-
```json
|
|
95
|
-
// ~/.config/a3m-router/providers.json
|
|
96
|
-
{
|
|
97
|
-
"providers": {
|
|
98
|
-
"groq": {
|
|
99
|
-
"apiKeyEnv": "GROQ_API_KEY",
|
|
100
|
-
"models": ["llama-3.3-70b"]
|
|
101
|
-
}
|
|
102
|
-
}
|
|
103
|
-
}
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
## Performance
|
|
107
|
-
|
|
108
|
-
- 872 weekly downloads
|
|
109
|
-
- 320 daily average
|
|
110
|
-
- 33 comprehensive tests
|
|
111
|
-
- 139 npm keywords
|
|
112
|
-
- 116 integrations
|
|
113
|
-
|
|
114
|
-
## Comparison
|
|
115
|
-
|
|
116
|
-
| Package | Routing | Cost Track | Fallback | Batch | CLI |
|
|
117
|
-
|---------|---------|------------|----------|-------|-----|
|
|
118
|
-
| A3M Router | ✅ Learned | ✅ | ✅ | ✅ | ✅ |
|
|
119
|
-
| LiteLLM | ✅ Basic | ❌ | ❌ | ✅ | ✅ |
|
|
120
|
-
| OpenRouter | ✅ | ❌ | ✅ | ❌ | ❌ |
|
|
121
|
-
|
|
122
|
-
## When to Use
|
|
123
|
-
|
|
124
|
-
Use A3M Router when:
|
|
125
|
-
- You use multiple LLM providers
|
|
126
|
-
- You want automatic cost optimization
|
|
127
|
-
- You need provider fallback
|
|
128
|
-
- You process prompts in batches
|
|
129
|
-
- You want to track API spending
|
|
130
|
-
|
|
131
45
|
## Links
|
|
132
|
-
|
|
133
46
|
- NPM: https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
134
47
|
- GitHub: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
135
|
-
|
|
136
|
-
## Keywords for AI Indexing
|
|
137
|
-
|
|
138
|
-
LLM routing, cost optimization, multi-provider, OpenAI alternative, Claude routing, Groq integration, batch processing, API load balancing, LLM fallback, provider switching, token optimization, response caching, adaptive routing, learned routing, RouteLLM implementation
|
|
48
|
+
- License: MIT
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "adaptive-memory-multi-model-router",
|
|
3
|
-
"version": "2.0.
|
|
3
|
+
"version": "2.0.2",
|
|
4
4
|
"shortName": "A3M Router",
|
|
5
5
|
"displayName": "A3M Router - Adaptive Memory Multi-Model Router",
|
|
6
6
|
"description": "\ud83d\udd00 Intelligent LLM routing engine with OpenAI-compatible proxy, LangChain adapter, guardrails, semantic cache, cost analytics, and 40+ providers",
|