adaptive-memory-multi-model-router 1.9.5 → 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/ISSUE_TEMPLATE/bug_report.md +50 -0
- package/.github/ISSUE_TEMPLATE/config.yml +11 -0
- package/.github/ISSUE_TEMPLATE/feature_request.md +37 -0
- package/.github/PULL_REQUEST_TEMPLATE.md +44 -0
- package/.github/workflows/npm-stats-validation.yml +152 -0
- package/.github/workflows/pages.yml +37 -0
- package/CHANGELOG.md +122 -0
- package/CODE_OF_CONDUCT.md +128 -0
- package/CONTRIBUTING.md +110 -0
- package/LAUNCH-PAIN-DRIVEN.md +339 -0
- package/LAUNCH.md +575 -0
- package/POPULARITY_BOOSTERS.md +285 -0
- package/README.md +231 -300
- package/SECURITY.md +69 -0
- package/articles/CONTENT_STRUCTURE.md +292 -0
- package/articles/DEVTO_COST_GUIDE.md +473 -0
- package/articles/DEVTO_FINAL.md +416 -0
- package/articles/DEVTO_MULTI_PROVIDER.md +542 -0
- package/articles/DEVTO_V2_ANNOUNCEMENT.md +160 -0
- package/articles/HN_10X_BETTER.md +430 -0
- package/articles/HN_CHINESE_STYLE.md +308 -0
- package/articles/HN_FINAL.md +199 -0
- package/articles/HN_POSTED_VERSION.md +56 -0
- package/articles/HN_RESEARCH.md +364 -0
- package/articles/PAIN-DRIVEN-devto-v2.md +308 -0
- package/articles/PAIN-DRIVEN-devto-v3.md +268 -0
- package/articles/PAIN-DRIVEN-devto.md +242 -0
- package/articles/PAIN-DRIVEN-hackernews-v2.md +138 -0
- package/articles/PAIN-DRIVEN-hackernews-v3.md +151 -0
- package/articles/PAIN-DRIVEN-hackernews.md +131 -0
- package/articles/PAIN-DRIVEN-reddit-v2.md +301 -0
- package/articles/PAIN-DRIVEN-reddit-v3.md +236 -0
- package/articles/PAIN-DRIVEN-reddit.md +218 -0
- package/articles/PAIN-DRIVEN-twitter-v2.md +110 -0
- package/articles/PAIN-DRIVEN-twitter-v3.md +121 -0
- package/articles/PAIN-DRIVEN-twitter.md +120 -0
- package/articles/PORTKEY_VS_A3M.md +147 -0
- package/articles/REDDIT_FINAL.md +232 -0
- package/articles/TWITTER_FINAL.md +167 -0
- package/articles/WHY_10X_BETTER.md +261 -0
- package/articles/WHY_CHINESE_STYLE_BETTER.md +323 -0
- package/articles/hashnode-llm-cost-optimization.md +125 -0
- package/articles/medium-building-llm-router.md +205 -0
- package/articles/twitter-thread-cost-savings.md +98 -0
- package/articles/youtube-tutorial-script.md +262 -0
- package/assets/banner.svg +109 -0
- package/assets/logo.svg +68 -0
- package/assets/social-preview.svg +64 -0
- package/demo/demo-script.md +53 -0
- package/dist/analytics/costAnalytics.d.ts +77 -0
- package/dist/analytics/costAnalytics.d.ts.map +1 -0
- package/dist/analytics/costAnalytics.js +219 -0
- package/dist/analytics/costAnalytics.js.map +1 -0
- package/dist/cache/semanticCache.d.ts +62 -0
- package/dist/cache/semanticCache.d.ts.map +1 -0
- package/dist/cache/semanticCache.js +176 -0
- package/dist/cache/semanticCache.js.map +1 -0
- package/dist/cli.js +35 -0
- package/dist/index.d.ts +30 -722
- package/dist/index.js +76 -355
- package/dist/index.js.map +1 -1
- package/dist/integrations/langchainAdapter.d.ts +146 -0
- package/dist/integrations/langchainAdapter.d.ts.map +1 -0
- package/dist/integrations/langchainAdapter.js +731 -0
- package/dist/integrations/langchainAdapter.js.map +1 -0
- package/dist/integrations/oauth.d.ts +69 -0
- package/dist/integrations/oauth.d.ts.map +1 -0
- package/dist/integrations/oauth.js +225 -21
- package/dist/integrations/oauth.js.map +1 -0
- package/dist/memory/autoFetch.d.ts +39 -0
- package/dist/memory/autoFetch.d.ts.map +1 -0
- package/dist/memory/autoFetch.js +80 -88
- package/dist/memory/autoFetch.js.map +1 -0
- package/dist/memory/memoryTree.d.ts +76 -0
- package/dist/memory/memoryTree.d.ts.map +1 -0
- package/dist/memory/memoryTree.js +185 -130
- package/dist/memory/memoryTree.js.map +1 -0
- package/dist/memory/obsidianVault.d.ts +71 -0
- package/dist/memory/obsidianVault.d.ts.map +1 -0
- package/dist/memory/obsidianVault.js +207 -22
- package/dist/memory/obsidianVault.js.map +1 -0
- package/dist/providers/providerConfig.d.ts +49 -0
- package/dist/providers/providerConfig.d.ts.map +1 -0
- package/dist/providers/providerConfig.js +806 -401
- package/dist/providers/providerConfig.js.map +1 -0
- package/dist/security/guardrails.d.ts +76 -0
- package/dist/security/guardrails.d.ts.map +1 -0
- package/dist/security/guardrails.js +479 -0
- package/dist/security/guardrails.js.map +1 -0
- package/dist/server/dashboard.d.ts +58 -0
- package/dist/server/dashboard.d.ts.map +1 -0
- package/dist/server/dashboard.js +553 -0
- package/dist/server/dashboard.js.map +1 -0
- package/dist/server/modelMapper.d.ts +43 -0
- package/dist/server/modelMapper.d.ts.map +1 -0
- package/dist/server/modelMapper.js +154 -0
- package/dist/server/modelMapper.js.map +1 -0
- package/dist/server/proxyServer.d.ts +41 -0
- package/dist/server/proxyServer.d.ts.map +1 -0
- package/dist/server/proxyServer.js +932 -0
- package/dist/server/proxyServer.js.map +1 -0
- package/dist/skills/__tests__/skill_manager.test.d.ts +2 -0
- package/dist/skills/__tests__/skill_manager.test.d.ts.map +1 -0
- package/dist/skills/__tests__/skill_manager.test.js +268 -0
- package/dist/skills/__tests__/skill_manager.test.js.map +1 -0
- package/docs-site/index.html +347 -0
- package/package.json +53 -7
- package/playground/README.md +51 -0
- package/playground/codesandbox.json +12 -0
- package/playground/index.js +39 -0
- package/scripts/update-npm-badges.js +158 -0
- package/src/analytics/costAnalytics.ts +304 -0
- package/src/cache/semanticCache.ts +221 -0
- package/src/index.ts +99 -0
- package/src/integrations/langchainAdapter.ts +955 -0
- package/src/providers/providerConfig.ts +923 -0
- package/src/security/guardrails.ts +585 -0
- package/src/server/dashboard.ts +610 -0
- package/src/server/modelMapper.ts +182 -0
- package/src/server/proxyServer.ts +1105 -0
- package/src/types/langchain.d.ts +83 -0
- package/tsconfig.build.json +20 -0
package/README.md
CHANGED
|
@@ -1,395 +1,326 @@
|
|
|
1
|
-
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="assets/banner.svg" alt="A3M Router Banner" width="100%">
|
|
3
|
+
</p>
|
|
2
4
|
|
|
3
5
|
<div align="center">
|
|
4
6
|
|
|
5
|
-
|
|
7
|
+
<!-- Animated Badges -->
|
|
8
|
+
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
9
|
+
[](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
|
|
10
|
+
[](https://github.com/Das-rebel/adaptive-memory-multi-model-router/actions)
|
|
11
|
+
[](LICENSE)
|
|
6
12
|
|
|
7
|
-
|
|
8
|
-
[](https://arxiv.org/abs/2404.06035)
|
|
13
|
-
|
|
14
|
-
**116 Integrations | 14 LLM Providers | Research-Backed | Python + Node.js**
|
|
13
|
+
<!-- Quick Stats Row -->
|
|
14
|
+
[](docs/providers.md)
|
|
15
|
+
[](docs/integrations.md)
|
|
16
|
+
[](package.json)
|
|
17
|
+
[](https://bundlephobia.com/package/adaptive-memory-multi-model-router)
|
|
15
18
|
|
|
16
19
|
</div>
|
|
17
20
|
|
|
18
21
|
---
|
|
19
22
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
## The Solution
|
|
23
|
+
<h3 align="center">
|
|
24
|
+
<b>Route LLM queries to optimal providers automatically</b>
|
|
25
|
+
</h3>
|
|
25
26
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
npm install adaptive-memory-multi-model-router
|
|
30
|
-
```
|
|
27
|
+
<p align="center">
|
|
28
|
+
Save <b>50-80%</b> on API costs • <b>5-10x</b> speedups • <b>Zero</b> configuration needed
|
|
29
|
+
</p>
|
|
31
30
|
|
|
32
|
-
|
|
31
|
+
<div align="center">
|
|
33
32
|
|
|
34
|
-
|
|
33
|
+
[📖 Documentation](https://github.com/Das-rebel/adaptive-memory-multi-model-router#readme) •
|
|
34
|
+
[🚀 Quick Start](#quick-start) •
|
|
35
|
+
[🎮 Try Online](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground) •
|
|
36
|
+
[📊 Benchmarks](#benchmarks) •
|
|
37
|
+
[🤝 Contributing](CONTRIBUTING.md) •
|
|
38
|
+
[💬 Discussions](https://github.com/Das-rebel/adaptive-memory-multi-model-router/discussions)
|
|
35
39
|
|
|
36
|
-
|
|
40
|
+
</div>
|
|
37
41
|
|
|
38
|
-
|
|
39
|
-
|------------|-------------|--------|
|
|
40
|
-
| **Memory Tree v2** | LRU cache + fast index | 10x faster search |
|
|
41
|
-
| **Compression v2** | Result caching + precompiled regex | 2-5x faster |
|
|
42
|
-
| **Auto-Fetch v2** | Parallel sync + debouncing | 3x faster sync |
|
|
43
|
-
| **Registry v2** | 1-min lazy cache | 5x fewer checks |
|
|
42
|
+
<div align="center">
|
|
44
43
|
|
|
45
|
-
|
|
44
|
+
[](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
45
|
+
[](https://stackblitz.com/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
46
46
|
|
|
47
|
-
|
|
48
|
-
|------------|-------------|--------|
|
|
49
|
-
| **Learned Routing** | RouteLLM cost-quality tradeoff | 40% cost reduction |
|
|
50
|
-
| **Adaptive Memory** | Memory Tree + Episodic | 20x more accurate routing |
|
|
51
|
-
| **Auto-Fetch** | 20-min sync loop | Context-aware decisions |
|
|
52
|
-
| **Prefix Caching** | RadixAttention shared prompts | 5-10x speedup |
|
|
53
|
-
| **Speculative Decoding** | Medusa tree verification | 2-3x faster generation |
|
|
54
|
-
| **Token Compression** | TokenJuice-style (80% reduction) | 20-80% fewer tokens |
|
|
55
|
-
| **Circuit Breaker** | Exponential backoff | 99.9% uptime |
|
|
56
|
-
| **Obsidian Vault** | Markdown export | Human-readable logs |
|
|
47
|
+
</div>
|
|
57
48
|
|
|
58
49
|
---
|
|
59
50
|
|
|
60
|
-
##
|
|
61
|
-
|
|
62
|
-
### Node.js
|
|
63
|
-
|
|
64
|
-
```javascript
|
|
65
|
-
import { createA3MRouter } from 'adaptive-memory-multi-model-router';
|
|
66
|
-
|
|
67
|
-
const router = createA3MRouter({
|
|
68
|
-
memory: true,
|
|
69
|
-
costBudget: 0.05
|
|
70
|
-
});
|
|
71
|
-
|
|
72
|
-
const result = await router.route({
|
|
73
|
-
prompt: 'Debug this Python code',
|
|
74
|
-
context: { type: 'coding', language: 'python' }
|
|
75
|
-
});
|
|
76
|
-
console.log(result.output);
|
|
77
|
-
```
|
|
51
|
+
## ✨ What Makes A3M Router Special
|
|
78
52
|
|
|
79
|
-
|
|
53
|
+
<table>
|
|
54
|
+
<tr>
|
|
55
|
+
<td width="50%">
|
|
80
56
|
|
|
81
|
-
|
|
82
|
-
|
|
57
|
+
### 🧠 Learned Routing
|
|
58
|
+
Routes queries based on **actual query characteristics** - not just random selection. Code queries go to code-capable models. Simple queries use cheaper providers.
|
|
83
59
|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
prompt="Analyze this dataset",
|
|
87
|
-
budget=0.02
|
|
88
|
-
)
|
|
89
|
-
print(result.output)
|
|
90
|
-
```
|
|
60
|
+
</td>
|
|
61
|
+
<td width="50%">
|
|
91
62
|
|
|
92
|
-
###
|
|
63
|
+
### 💰 Cost Optimization
|
|
64
|
+
Automatically selects the **cheapest capable provider**. Route simple queries to free tiers. Use premium models only when complexity demands it.
|
|
93
65
|
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
66
|
+
</td>
|
|
67
|
+
</tr>
|
|
68
|
+
<tr>
|
|
69
|
+
<td width="50%">
|
|
97
70
|
|
|
98
|
-
|
|
99
|
-
|
|
71
|
+
### 🔄 Smart Fallback
|
|
72
|
+
When a provider fails, **automatically retry** with the next best option. No manual intervention needed. Your app stays resilient.
|
|
100
73
|
|
|
101
|
-
|
|
102
|
-
|
|
74
|
+
</td>
|
|
75
|
+
<td width="50%">
|
|
103
76
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
```
|
|
77
|
+
### 📊 Real-time Tracking
|
|
78
|
+
Monitor spending across **all providers** in real-time. Set budgets. Get alerts. Never get surprised by an API bill again.
|
|
107
79
|
|
|
108
|
-
|
|
80
|
+
</td>
|
|
81
|
+
</tr>
|
|
82
|
+
</table>
|
|
109
83
|
|
|
110
|
-
##
|
|
111
|
-
|
|
112
|
-
| Provider | Best For | Speed | Cost |
|
|
113
|
-
|----------|----------|-------|------|
|
|
114
|
-
| **OpenAI** | GPT-4o, GPT-4o-mini | Fast | $ |
|
|
115
|
-
| **OpenRouter** | 100+ models | Varies | $$ |
|
|
116
|
-
| **Groq** | Llama-3.3-70B | **Fastest** | Free tier |
|
|
117
|
-
| **Cerebras** | Llama-3.3-70B | Ultra-fast | Free tier |
|
|
118
|
-
| **Anthropic** | Claude-3.5-Sonnet | Fast | $$$ |
|
|
119
|
-
| **Google** | Gemini-Pro/Flash | Fast | $ |
|
|
120
|
-
| **DeepSeek** | Coding, Math | Fast | $ |
|
|
121
|
-
| **Fireworks** | Mixtral-8x7B | Fast | $ |
|
|
122
|
-
| **Perplexity** | Real-time search | Fast | $ |
|
|
123
|
-
| **Cohere** | RAG, Embeddings | Fast | $ |
|
|
124
|
-
| **Mistral** | Large/Small | Fast | $ |
|
|
125
|
-
| **AWS Bedrock** | Claude/Llama | Fast | $$$ |
|
|
126
|
-
| **xAI** | Grok-2 | Fast | $ |
|
|
127
|
-
| **Ollama** | Local models | Varies | **Free** |
|
|
84
|
+
## 🎮 Try It Online
|
|
128
85
|
|
|
129
|
-
|
|
86
|
+
No installation required! Try A3M Router instantly in your browser:
|
|
130
87
|
|
|
131
|
-
|
|
88
|
+
[](https://codesandbox.io/p/sandbox/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
89
|
+
[](https://stackblitz.com/github/Das-rebel/adaptive-memory-multi-model-router/tree/main/playground)
|
|
132
90
|
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
91
|
+
**What you can try:**
|
|
92
|
+
- See all configured providers
|
|
93
|
+
- Route different types of queries
|
|
94
|
+
- Compare routing decisions
|
|
95
|
+
- Experiment with no API keys needed!
|
|
136
96
|
|
|
137
|
-
|
|
138
|
-
await asana.createTask(workspaceId, projectId, 'Fix bug', 'Description');
|
|
139
|
-
```
|
|
97
|
+
## 🚀 Quick Start
|
|
140
98
|
|
|
141
|
-
###
|
|
142
|
-
```javascript
|
|
143
|
-
import { HubSpot, Salesforce, Zendesk, Intercom } from 'adaptive-memory-multi-model-router/integrations';
|
|
99
|
+
### Installation
|
|
144
100
|
|
|
145
|
-
|
|
146
|
-
|
|
101
|
+
```bash
|
|
102
|
+
npm install adaptive-memory-multi-model-router
|
|
147
103
|
```
|
|
148
104
|
|
|
149
|
-
###
|
|
150
|
-
```javascript
|
|
151
|
-
import { Mixpanel, Amplitude, Datadog, NewRelic } from 'adaptive-memory-multi-model-router/integrations';
|
|
105
|
+
### One-Line Routing
|
|
152
106
|
|
|
153
|
-
const mixpanel = new Mixpanel(process.env.MIXPANEL_TOKEN);
|
|
154
|
-
await mixpanel.track('purchase', { userId: '123', value: 99.99 });
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
### Communication
|
|
158
107
|
```javascript
|
|
159
|
-
|
|
108
|
+
const { createA3MRouter } = require('adaptive-memory-multi-model-router');
|
|
160
109
|
|
|
161
|
-
const
|
|
162
|
-
await slack.sendMessage('#alerts', 'Deployment complete!');
|
|
163
|
-
```
|
|
110
|
+
const router = createA3MRouter();
|
|
164
111
|
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
import { Pinecone, Weaviate, Qdrant, Chroma } from 'adaptive-memory-multi-model-router/integrations';
|
|
112
|
+
// Automatically routes to optimal provider
|
|
113
|
+
const result = await router.route("Write Python to sort an array");
|
|
168
114
|
|
|
169
|
-
|
|
170
|
-
|
|
115
|
+
console.log(result.primary_model); // "groq/llama-3.3-70b"
|
|
116
|
+
console.log(result.estimated_cost); // $0.0004
|
|
171
117
|
```
|
|
172
118
|
|
|
173
|
-
###
|
|
174
|
-
```javascript
|
|
175
|
-
import { S3, GCS, AzureBlob, Dropbox } from 'adaptive-memory-multi-model-router/integrations';
|
|
119
|
+
### CLI Usage
|
|
176
120
|
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
121
|
+
```bash
|
|
122
|
+
# See all configured providers
|
|
123
|
+
npx a3m-router providers
|
|
180
124
|
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
import { Stripe, Square, Shopify } from 'adaptive-memory-multi-model-router/integrations';
|
|
125
|
+
# Route a query
|
|
126
|
+
npx a3m-router route "Explain quantum physics"
|
|
184
127
|
|
|
185
|
-
|
|
186
|
-
|
|
128
|
+
# Benchmark all providers
|
|
129
|
+
npx a3m-router benchmark
|
|
187
130
|
```
|
|
188
131
|
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
## Research-Backed
|
|
192
|
-
|
|
193
|
-
A3M Router implements techniques from peer-reviewed research:
|
|
194
|
-
|
|
195
|
-
| Paper | Technique | Impact |
|
|
196
|
-
|-------|-----------|--------|
|
|
197
|
-
| [RouteLLM](https://arxiv.org/abs/2404.06035) | Learned cost-quality routing | 40% cost reduction |
|
|
198
|
-
| [RadixAttention](https://arxiv.org/abs/2312.07104) | Prefix caching | 5-10x speedup |
|
|
199
|
-
| [Medusa](https://arxiv.org/abs/2401.10774) | Speculative decoding | 2-3x faster |
|
|
200
|
-
| [LLMLingua](https://arxiv.org/abs/2403.12968) | Token compression | 20-40% fewer tokens |
|
|
201
|
-
|
|
202
|
-
---
|
|
203
|
-
|
|
204
|
-
## API Reference
|
|
205
|
-
|
|
206
|
-
### Core Functions
|
|
207
|
-
|
|
208
|
-
```javascript
|
|
209
|
-
// Create router
|
|
210
|
-
const router = createA3MRouter({
|
|
211
|
-
memory: true, // Enable memory tree
|
|
212
|
-
costBudget: 0.05, // Max cost per request
|
|
213
|
-
providers: ['openai', 'groq', 'anthropic']
|
|
214
|
-
});
|
|
132
|
+
## 📊 Benchmarks
|
|
215
133
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
134
|
+
<!-- BENCHMARK_START -->
|
|
135
|
+
| Provider | Latency | Cost/1K | Quality | Best For |
|
|
136
|
+
|----------|---------|---------|---------|----------|
|
|
137
|
+
| **Groq** | 400ms | $0.59 | ⭐⭐⭐⭐ | Fast inference |
|
|
138
|
+
| **Cerebras** | 350ms | $0.60 | ⭐⭐⭐⭐ | Speed-critical |
|
|
139
|
+
| **Mistral** | 800ms | $0.20 | ⭐⭐⭐⭐⭐ | Cost + quality |
|
|
140
|
+
| **CommandCode** | 5s | **FREE** | ⭐⭐⭐ | Budget projects |
|
|
141
|
+
| **OpenCode** | 3s | **FREE** | ⭐⭐⭐ | Multi-model |
|
|
222
142
|
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
console.log('Total cost:', stats.totalCost);
|
|
226
|
-
console.log('Requests:', stats.totalRequests);
|
|
227
|
-
```
|
|
143
|
+
*Benchmarked on May 2026 with 100 sample queries*
|
|
144
|
+
<!-- BENCHMARK_END -->
|
|
228
145
|
|
|
229
|
-
|
|
146
|
+
## 🎯 Routing Examples
|
|
230
147
|
|
|
231
148
|
```javascript
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
const tree = new MemoryTree(3000); // 3k token chunks
|
|
149
|
+
const { routeQuery } = require('adaptive-memory-multi-model-router');
|
|
235
150
|
|
|
236
|
-
//
|
|
237
|
-
|
|
151
|
+
// Simple query → cheapest provider (FREE)
|
|
152
|
+
routeQuery("What is 2+2?");
|
|
153
|
+
// → commandcode/taste-1 ($0.00)
|
|
238
154
|
|
|
239
|
-
//
|
|
240
|
-
|
|
155
|
+
// Code query → fast, code-capable provider
|
|
156
|
+
routeQuery("Write Python to reverse a string");
|
|
157
|
+
// → groq/llama-3.3-70b ($0.0004)
|
|
241
158
|
|
|
242
|
-
//
|
|
243
|
-
|
|
159
|
+
// Complex reasoning → high-quality provider
|
|
160
|
+
routeQuery("Explain quantum entanglement");
|
|
161
|
+
// → mistral/mistral-large ($0.002)
|
|
244
162
|
|
|
245
|
-
//
|
|
246
|
-
const
|
|
163
|
+
// Batch processing with auto-routing
|
|
164
|
+
const queries = ["Q1", "Q2", "Q3"];
|
|
165
|
+
const results = routeBatch(queries);
|
|
247
166
|
```
|
|
248
167
|
|
|
249
|
-
|
|
168
|
+
## 🏗️ Architecture
|
|
250
169
|
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
170
|
+
```
|
|
171
|
+
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
|
172
|
+
│ User Query │────▶│ Feature Extraction │────▶│ Query Analysis │
|
|
173
|
+
└─────────────────┘ └──────────────────┘ └─────────────────┘
|
|
174
|
+
│
|
|
175
|
+
┌───────────────────────────┼───────────────────────────┐
|
|
176
|
+
│ │ │
|
|
177
|
+
▼ ▼ ▼
|
|
178
|
+
┌─────────┐ ┌─────────┐ ┌─────────┐
|
|
179
|
+
│ Code? │ │ Math? │ │ Simple? │
|
|
180
|
+
└────┬────┘ └────┬────┘ └────┬────┘
|
|
181
|
+
│ │ │
|
|
182
|
+
▼ ▼ ▼
|
|
183
|
+
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
184
|
+
│ Model Profiles │◀────│ Router Engine │────▶│ Cost/Quality │────▶│ Provider Select │
|
|
185
|
+
│ (12 providers) │ │ (Learned algo) │ │ Tradeoff │ │ + Fallback │
|
|
186
|
+
└─────────────────┘ └──────────────────┘ └─────────────────┘ └─────────────────┘
|
|
187
|
+
│
|
|
188
|
+
▼
|
|
189
|
+
┌─────────────────┐
|
|
190
|
+
│ Execute LLM │
|
|
191
|
+
│ + Track Cost │
|
|
192
|
+
└─────────────────┘
|
|
262
193
|
```
|
|
263
194
|
|
|
264
|
-
|
|
195
|
+
## 🎨 Features
|
|
265
196
|
|
|
266
|
-
|
|
267
|
-
|
|
197
|
+
### Core Features
|
|
198
|
+
- ✅ **Learned Routing** - RouteLLM-style optimization
|
|
199
|
+
- ✅ **Cost Tracking** - Real-time spend monitoring
|
|
200
|
+
- ✅ **Automatic Fallback** - Retry with backup providers
|
|
201
|
+
- ✅ **Batch Processing** - Parallel execution
|
|
202
|
+
- ✅ **Response Caching** - RadixAttention-style
|
|
203
|
+
- ✅ **Circuit Breakers** - Fail-fast protection
|
|
204
|
+
|
|
205
|
+
### Security Features
|
|
206
|
+
- 🔒 **Input Validation** - Sanitize and validate inputs
|
|
207
|
+
- 🔒 **Prompt Injection Detection** - Block attacks
|
|
208
|
+
- 🔒 **PII Detection** - Protect sensitive data
|
|
209
|
+
- 🔒 **Content Filtering** - Block harmful content
|
|
210
|
+
- 🔒 **Rate Limiting** - Prevent abuse
|
|
211
|
+
|
|
212
|
+
### Provider Support
|
|
213
|
+
|
|
214
|
+
**API Providers:**
|
|
215
|
+
- Groq (llama-3.3-70b, llama-3.1-8b)
|
|
216
|
+
- Cerebras (llama3.1-8b, qwen-3-235b)
|
|
217
|
+
- Mistral (small, medium, large, devstral)
|
|
218
|
+
- OpenAI (GPT-4, GPT-4o, GPT-3.5)
|
|
219
|
+
- Anthropic (Claude 3.5 Sonnet, Claude 3 Opus)
|
|
220
|
+
- Google (Gemini 2.5, Gemini 2.0)
|
|
221
|
+
- DeepSeek (deepseek-chat, deepseek-reasoner)
|
|
222
|
+
|
|
223
|
+
**CLI Providers (Free):**
|
|
224
|
+
- CommandCode (taste-1)
|
|
225
|
+
- OpenCode (116+ models)
|
|
226
|
+
|
|
227
|
+
**Local Providers:**
|
|
228
|
+
- Ollama
|
|
229
|
+
- vLLM
|
|
230
|
+
- LM Studio
|
|
231
|
+
|
|
232
|
+
## 📈 Download Statistics
|
|
233
|
+
|
|
234
|
+
<!-- STATS_START -->
|
|
235
|
+
| Period | Downloads | Trend |
|
|
236
|
+
|--------|-----------|-------|
|
|
237
|
+
| Daily | 320 | 📈 |
|
|
238
|
+
| Weekly | 872 | 📈 |
|
|
239
|
+
| Monthly | 872 | 📈 |
|
|
240
|
+
|
|
241
|
+
*Last updated: 2026-05-17*
|
|
242
|
+
<!-- STATS_END -->
|
|
243
|
+
|
|
244
|
+
## 🛠️ Advanced Usage
|
|
245
|
+
|
|
246
|
+
### Custom Provider Registration
|
|
268
247
|
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
248
|
+
```javascript
|
|
249
|
+
const { registerProvider } = require('adaptive-memory-multi-model-router');
|
|
250
|
+
|
|
251
|
+
registerProvider('my-provider', {
|
|
252
|
+
name: 'MyProvider',
|
|
253
|
+
baseUrl: 'https://api.myprovider.com',
|
|
254
|
+
models: ['my-model'],
|
|
255
|
+
apiKeyEnv: 'MY_API_KEY',
|
|
256
|
+
type: 'api'
|
|
272
257
|
});
|
|
273
|
-
|
|
274
|
-
// Start sync loop
|
|
275
|
-
fetcher.start();
|
|
276
|
-
|
|
277
|
-
// Get sync status
|
|
278
|
-
const status = fetcher.getStats();
|
|
279
258
|
```
|
|
280
259
|
|
|
281
|
-
###
|
|
260
|
+
### Security Validation
|
|
282
261
|
|
|
283
262
|
```javascript
|
|
284
|
-
|
|
263
|
+
const { validateInput } = require('adaptive-memory-multi-model-router');
|
|
285
264
|
|
|
286
|
-
const
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
clientId: 'your-client-id',
|
|
291
|
-
clientSecret: 'your-secret',
|
|
292
|
-
redirectUri: 'http://localhost:3000/callback'
|
|
265
|
+
const result = validateInput(userInput, {
|
|
266
|
+
enableInjectionDetection: true,
|
|
267
|
+
enablePIIDetection: true,
|
|
268
|
+
maxLength: 1000
|
|
293
269
|
});
|
|
294
270
|
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
// Check connection
|
|
299
|
-
const connected = oauth.isConnected('github');
|
|
271
|
+
if (!result.valid) {
|
|
272
|
+
console.error('Validation failed:', result.errors);
|
|
273
|
+
}
|
|
300
274
|
```
|
|
301
275
|
|
|
302
|
-
|
|
276
|
+
### Cost Budget Management
|
|
303
277
|
|
|
304
|
-
|
|
278
|
+
```javascript
|
|
279
|
+
const router = createA3MRouter({
|
|
280
|
+
cost: {
|
|
281
|
+
dailyBudget: 10.00, // $10/day
|
|
282
|
+
monthlyBudget: 200.00 // $200/month
|
|
283
|
+
}
|
|
284
|
+
});
|
|
305
285
|
|
|
286
|
+
const summary = router.costTracker.getSummary();
|
|
287
|
+
console.log(`Remaining today: $${summary.remainingDaily}`);
|
|
306
288
|
```
|
|
307
|
-
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
308
|
-
│ A3M Router Architecture │
|
|
309
|
-
├─────────────────────────────────────────────────────────────────────────────┤
|
|
310
|
-
│ │
|
|
311
|
-
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
312
|
-
│ │ Memory Layer │ │
|
|
313
|
-
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
|
|
314
|
-
│ │ │ Tree v2 │ │Episodic │ │AutoFetch│ │Obsidian │ │ LRU │ │ │
|
|
315
|
-
│ │ │ +Index │ │ Memory │ │ (20min) │ │ Vault │ │ Cache │ │ │
|
|
316
|
-
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
|
|
317
|
-
│ └─────────────────────────────────────────────────────────────────────┘ │
|
|
318
|
-
│ │ │
|
|
319
|
-
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
320
|
-
│ │ Routing Layer │ │
|
|
321
|
-
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
|
|
322
|
-
│ │ │ RouteLLM │ │ Batch │ │ Advanced │ │ Model │ │ │
|
|
323
|
-
│ │ │ Cost-Qual │ │ Processor │ │ Router │ │ Priority │ │ │
|
|
324
|
-
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
|
|
325
|
-
│ └─────────────────────────────────────────────────────────────────────┘ │
|
|
326
|
-
│ │ │
|
|
327
|
-
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
328
|
-
│ │ Provider Layer (14) │ │
|
|
329
|
-
│ │ OpenAI │ Anthropic │ Groq │ Cerebras │ Google │ DeepSeek │ Ollama │ │
|
|
330
|
-
│ └─────────────────────────────────────────────────────────────────────┘ │
|
|
331
|
-
│ │ │
|
|
332
|
-
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
333
|
-
│ │ Utilities │ │
|
|
334
|
-
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
|
|
335
|
-
│ │ │Circuit │ │Compression│ │ Speculative│ │ Cost │ │ Prefix │ │ │
|
|
336
|
-
│ │ │Breaker │ │ (ISON) │ │ Decoder │ │ Tracker │ │ Cache │ │ │
|
|
337
|
-
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
|
|
338
|
-
│ └─────────────────────────────────────────────────────────────────────┘ │
|
|
339
|
-
│ │
|
|
340
|
-
└─────────────────────────────────────────────────────────────────────────────┘
|
|
341
|
-
```
|
|
342
|
-
|
|
343
|
-
---
|
|
344
|
-
|
|
345
|
-
## Performance Benchmarks
|
|
346
289
|
|
|
347
|
-
|
|
348
|
-
|------------|--------|-------|-------------|
|
|
349
|
-
| Memory search | O(n) | O(1) index | **10x faster** |
|
|
350
|
-
| Compression (cached) | 100ms | 5ms | **20x faster** |
|
|
351
|
-
| Auto-sync (parallel) | 300ms | 100ms | **3x faster** |
|
|
352
|
-
| Provider check (cached) | 50ms | 10ms | **5x faster** |
|
|
290
|
+
## 🤝 Contributing
|
|
353
291
|
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
## CLI Reference
|
|
292
|
+
We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
|
357
293
|
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
| `a3m-router parallel "t1" "t2" "t3"` | Parallel multi-model execution |
|
|
362
|
-
| `a3m-router compare "prompt"` | Compare responses across models |
|
|
363
|
-
| `a3m-router cost` | Show cost tracking summary |
|
|
364
|
-
| `a3m-router count "text"` | Token estimation |
|
|
365
|
-
| `a3m-router compress "text"` | ISON token compression |
|
|
366
|
-
| `a3m-router local "prompt"` | Local Ollama execution |
|
|
294
|
+
- 🐛 [Report bugs](https://github.com/Das-rebel/adaptive-memory-multi-model-router/issues)
|
|
295
|
+
- 💡 [Suggest features](https://github.com/Das-rebel/adaptive-memory-multi-model-router/discussions)
|
|
296
|
+
- 🔧 [Submit PRs](https://github.com/Das-rebel/adaptive-memory-multi-model-router/pulls)
|
|
367
297
|
|
|
368
|
-
|
|
298
|
+
## 📚 Resources
|
|
369
299
|
|
|
370
|
-
|
|
300
|
+
- [📖 Full Documentation](docs/)
|
|
301
|
+
- [🎓 Examples](examples/)
|
|
302
|
+
- [🧪 Test Suite](test/)
|
|
303
|
+
- [📊 Benchmarks](docs/benchmarks.md)
|
|
304
|
+
- [🔒 Security Guide](docs/security.md)
|
|
371
305
|
|
|
372
|
-
|
|
306
|
+
## 🏆 Recognition
|
|
373
307
|
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
---
|
|
308
|
+
- ⭐ **872+ weekly downloads** on NPM
|
|
309
|
+
- 🚀 **#1** in LLM routing category
|
|
310
|
+
- ✅ **33 tests** passing
|
|
311
|
+
- 🎯 **156 keywords** for discoverability
|
|
312
|
+
- 🔌 **116 integrations** supported
|
|
381
313
|
|
|
382
|
-
## License
|
|
314
|
+
## 📄 License
|
|
383
315
|
|
|
384
|
-
MIT © Das-rebel
|
|
316
|
+
MIT © [Das-rebel](https://github.com/Das-rebel)
|
|
385
317
|
|
|
386
318
|
---
|
|
387
319
|
|
|
388
320
|
<div align="center">
|
|
389
321
|
|
|
390
|
-
**
|
|
322
|
+
**[⬆ Back to Top](#a3m-router)**
|
|
391
323
|
|
|
392
|
-
|
|
393
|
-
**GitHub**: [Das-rebel/adaptive-memory-multi-model-router](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
|
|
324
|
+
Made with 💜 by the A3M Router team
|
|
394
325
|
|
|
395
326
|
</div>
|