adaptive-memory-multi-model-router 2.15.3 → 2.15.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/ISSUE_TEMPLATE/bug_report.md +94 -0
- package/.github/ISSUE_TEMPLATE/config.yml +17 -0
- package/.github/ISSUE_TEMPLATE/feature_request.md +71 -0
- package/.well-known/ai-plugin.json +16 -0
- package/AGENT_COUNCIL_FINDINGS.md +98 -123
- package/POPULARITY_BOOSTERS.md +1 -1
- package/README.md +275 -147
- package/README_ja.md +20 -16
- package/README_zh.md +17 -13
- package/adapters/README.md +36 -0
- package/adapters/__init__.py +25 -0
- package/adapters/a3m_adapter/__init__.py +15 -0
- package/adapters/a3m_adapter/adapter/__init__.py +7 -0
- package/adapters/a3m_adapter/adapter/config.py +100 -0
- package/adapters/a3m_adapter/adapter/langchain.py +155 -0
- package/adapters/a3m_adapter/adapter/llamaindex.py +162 -0
- package/adapters/a3m_adapter/tests/__init__.py +1 -0
- package/adapters/a3m_adapter/tests/test_adapters.py +120 -0
- package/adapters/setup.py +23 -0
- package/articles/CHINESE_DIRECTORIES.md +2 -2
- package/articles/CHINESE_SUBMISSIONS_READY.md +5 -5
- package/articles/HN_POST_READY.md +20 -16
- package/articles/INDIEHACKERS_READY.md +34 -19
- package/dist/analytics/costAnalytics.d.ts +1 -0
- package/dist/benchmark/reproducible.d.ts +1 -0
- package/dist/cache/semanticCache.d.ts +1 -0
- package/dist/cost/budgetEnforcer.d.ts +1 -0
- package/dist/cost/costTracker.d.ts +1 -0
- package/dist/ensemble/multiRoundDialog.d.ts +1 -0
- package/dist/ensemble/shapleyValue.d.ts +1 -0
- package/dist/ensemble.d.ts +1 -0
- package/dist/index.d.ts +1 -0
- package/dist/integrations/oauth.d.ts +1 -0
- package/dist/integrations/scienceAdapter.d.ts +1 -0
- package/dist/memory/autoFetch.d.ts +1 -0
- package/dist/memory/hybridMemory.d.ts +1 -0
- package/dist/memory/memoryTree.d.ts +1 -0
- package/dist/memory/obsidianVault.d.ts +1 -0
- package/dist/memory/reasoningBank.d.ts +1 -0
- package/dist/observability/changeWatch.d.ts +1 -0
- package/dist/observability/fatigueDetector.d.ts +1 -0
- package/dist/observability/index.d.ts +1 -0
- package/dist/observability/metrics.d.ts +1 -0
- package/dist/observability/middleware.d.ts +1 -0
- package/dist/observability/tracer.d.ts +1 -0
- package/dist/observability/types.d.ts +1 -0
- package/dist/providers/providerConfig.d.ts +1 -0
- package/dist/routing/advancedRouter.d.ts +2 -1
- package/dist/routing/advancedRouter.js +0 -84
- package/dist/routing/advancedRouter.js.map +1 -1
- package/dist/routing/crossModelValidation.d.ts +1 -0
- package/dist/routing/providerHealth.d.ts +2 -77
- package/dist/routing/providerHealth.js +4 -190
- package/dist/routing/providerHealth.js.map +1 -1
- package/dist/routing/providerRetry.d.ts +1 -0
- package/dist/sdk.d.ts +1 -0
- package/dist/security/guardrails.d.ts +1 -0
- package/dist/server/dashboard.d.ts +1 -0
- package/dist/server/handlers/chatHandler.d.ts +1 -0
- package/dist/server/handlers/completionsHandler.d.ts +1 -0
- package/dist/server/handlers/embeddingsHandler.d.ts +1 -0
- package/dist/server/handlers/healthHandler.d.ts +1 -0
- package/dist/server/handlers/metricsHandler.d.ts +1 -0
- package/dist/server/handlers/modelsHandler.d.ts +1 -0
- package/dist/server/metrics.d.ts +1 -0
- package/dist/server/modelMapper.d.ts +1 -0
- package/dist/server/proxyServer.d.ts +1 -0
- package/dist/server/router.d.ts +1 -0
- package/dist/server/state.d.ts +1 -0
- package/dist/tui/dashboard.d.ts +1 -0
- package/dist/utils/costUtils.d.ts +1 -0
- package/dist/utils/sorting.d.ts +1 -0
- package/dist/utils/tokenUtils.d.ts +1 -0
- package/docs/llms-full.txt +286 -153
- package/docs/llms.txt +112 -71
- package/llms-full.txt +6 -6
- package/llms.txt +112 -62
- package/package.json +12 -91
- package/CAMPAIGN_SUMMARY.md +0 -87
- package/ENTERPRISE_INTEGRATIONS.md +0 -94
- package/INTEGRATION_PROGRESS.md +0 -96
- package/OPPORTUNITIES_100.md +0 -277
- package/TODO_BROWSER_AUTOMATION.md +0 -87
- package/a3m-integrations-summary.md +0 -73
- package/dist/routing/shadowSampler.d.ts +0 -187
- package/dist/routing/shadowSampler.js +0 -306
- package/dist/routing/shadowSampler.js.map +0 -1
- package/integrations/browser-use/README.md +0 -155
- package/integrations/browser-use/examples/form-filling.ts +0 -179
- package/integrations/browser-use/index.ts +0 -145
- package/integrations/browser-use/package.json +0 -34
- package/integrations/browser-use/tsconfig.json +0 -17
- package/integrations/mcp-browser/README.md +0 -177
- package/integrations/mcp-browser/package.json +0 -38
- package/integrations/mcp-browser/src/index.ts +0 -297
- package/integrations/sota-browser/README.md +0 -191
- package/integrations/sota-browser/package.json +0 -34
package/README.md
CHANGED
|
@@ -1,215 +1,343 @@
|
|
|
1
|
-
#
|
|
1
|
+
# A3M Router
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**Intelligent LLM routing across 47+ providers — saves 70-95% on AI costs.**
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
<img src="https://img.shields.io/npm/v/adaptive-memory-multi-model-router?style=flat-square" alt="npm">
|
|
7
|
-
<img src="https://img.shields.io/github/stars/Das-rebel/a3m-router?style=flat-square" alt="stars">
|
|
8
|
-
<img src="https://img.shields.io/npm/dm/adaptive-memory-multi-model-router?style=flat-square" alt="downloads">
|
|
9
|
-
</p>
|
|
5
|
+
A3M Router automatically picks the cheapest capable model for each request. No code changes needed. Just swap your API endpoint.
|
|
10
6
|
|
|
11
7
|
---
|
|
12
8
|
|
|
13
|
-
##
|
|
9
|
+
## TL;DR — What Is This?
|
|
14
10
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
11
|
+
**Before:**
|
|
12
|
+
```python
|
|
13
|
+
# Pay GPT-4o prices for EVERY query
|
|
14
|
+
client = OpenAI(api_key="sk-...")
|
|
15
|
+
response = client.chat.completions.create(
|
|
16
|
+
model="gpt-4o",
|
|
17
|
+
messages=[{"role": "user", "content": "What is 2+2?"}]
|
|
18
|
+
) # Costs: $0.03
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
**After:**
|
|
22
|
+
```python
|
|
23
|
+
# A3M Router picks the right model automatically
|
|
24
|
+
client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
|
|
25
|
+
response = client.chat.completions.create(
|
|
26
|
+
model="auto", # ← Just change this
|
|
27
|
+
messages=[{"role": "user", "content": "What is 2+2?"}]
|
|
28
|
+
) # Routes to Groq/Mistral — costs: $0.0001
|
|
29
|
+
```
|
|
21
30
|
|
|
22
|
-
|
|
31
|
+
**Result:** Simple questions cost 300x less. Complex queries still go to premium models when needed.
|
|
23
32
|
|
|
24
33
|
---
|
|
25
34
|
|
|
26
|
-
##
|
|
35
|
+
## Why A3M Router?
|
|
27
36
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
-
|
|
31
|
-
|
|
32
|
-
|
|
37
|
+
| Problem | Solution |
|
|
38
|
+
|---------|----------|
|
|
39
|
+
| GPT-4o is $15/1M tokens | A3M routes simple queries to $0.001/1K providers |
|
|
40
|
+
| Managing 47+ API keys is messy | One endpoint, A3M handles the rest |
|
|
41
|
+
| Provider goes down mid-request | Automatic failover to next best option |
|
|
42
|
+
| Need the best answer, cost doesn't matter | Parallel ensemble calls multiple providers |
|
|
33
43
|
|
|
34
44
|
---
|
|
35
45
|
|
|
36
46
|
## Quick Start
|
|
37
47
|
|
|
38
|
-
### Browser Automation
|
|
39
|
-
|
|
40
48
|
```bash
|
|
49
|
+
# Install
|
|
41
50
|
npm install adaptive-memory-multi-model-router
|
|
51
|
+
|
|
52
|
+
# Start server
|
|
53
|
+
npx a3m-router serve
|
|
42
54
|
```
|
|
43
55
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
const result = await router.route({
|
|
57
|
-
task: 'Extract name, email, phone from this job application form',
|
|
58
|
-
context: 'browser_automation',
|
|
59
|
-
});
|
|
60
|
-
|
|
61
|
-
console.log(`Content: ${result.content}`);
|
|
62
|
-
console.log(`Provider: ${result.provider}`);
|
|
63
|
-
console.log(`Cost: $${result.cost}`);
|
|
56
|
+
Then use it like any OpenAI-compatible API:
|
|
57
|
+
|
|
58
|
+
```python
|
|
59
|
+
from openai import OpenAI
|
|
60
|
+
|
|
61
|
+
client = OpenAI(base_url="http://localhost:8787/v1", api_key="not-needed")
|
|
62
|
+
|
|
63
|
+
# Simple query → routes to cheapest capable (Groq, Mistral, etc.)
|
|
64
|
+
response = client.chat.completions.create(
|
|
65
|
+
model="auto",
|
|
66
|
+
messages=[{"role": "user", "content": "What is Python?"}]
|
|
67
|
+
)
|
|
64
68
|
```
|
|
65
69
|
|
|
66
|
-
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## Parallel Ensemble — Best Answer, Any Provider
|
|
67
73
|
|
|
68
|
-
|
|
69
|
-
npm install -g adaptive-memory-multi-model-router
|
|
70
|
-
npx a3m-router serve
|
|
74
|
+
Need the best answer regardless of cost? Call multiple providers in parallel:
|
|
71
75
|
|
|
72
|
-
|
|
73
|
-
|
|
76
|
+
```python
|
|
77
|
+
from a3m.router import A3MRouter
|
|
78
|
+
|
|
79
|
+
router = A3MRouter(
|
|
80
|
+
model="auto",
|
|
81
|
+
parallel_ensemble=3, # ← Call 3 providers simultaneously
|
|
82
|
+
)
|
|
83
|
+
|
|
84
|
+
result = router.route(
|
|
85
|
+
messages=[{"role": "user", "content": "Explain quantum entanglement"}],
|
|
86
|
+
ensemble_timeout_ms=10000,
|
|
87
|
+
)
|
|
88
|
+
|
|
89
|
+
# result.content — winning response
|
|
90
|
+
# result.provider — which provider won
|
|
91
|
+
# result.scores — quality scores per provider
|
|
92
|
+
# result.all_results — all responses for comparison
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
**Real-world example:**
|
|
96
|
+
```python
|
|
97
|
+
# Call Groq (fast/cheap) + OpenAI (quality) + DeepSeek (cost-effective) in parallel
|
|
98
|
+
ensemble_result = router.route(
|
|
99
|
+
messages=[{"role": "user", "content": prompt}],
|
|
100
|
+
ensemble_config={
|
|
101
|
+
"providers": ["groq", "openai", "deepseek"],
|
|
102
|
+
"timeout_ms": 15000,
|
|
103
|
+
"score_weights": {"relevance": 0.4, "conciseness": 0.3, "accuracy": 0.3}
|
|
104
|
+
}
|
|
105
|
+
)
|
|
106
|
+
|
|
107
|
+
print(f"Best answer from: {ensemble_result.provider}")
|
|
108
|
+
print(f"Response: {ensemble_result.content}")
|
|
109
|
+
print(f"All scores: {ensemble_result.scores}")
|
|
74
110
|
```
|
|
75
111
|
|
|
76
112
|
---
|
|
77
113
|
|
|
78
|
-
##
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
114
|
+
## Multi-Agent Systems — CrewAI Example
|
|
115
|
+
|
|
116
|
+
Powerful for multi-agent systems where different agents need different model capabilities:
|
|
117
|
+
|
|
118
|
+
```python
|
|
119
|
+
from crewai import Agent, Task, Crew
|
|
120
|
+
from crewai.llms import A3MCompletion
|
|
121
|
+
|
|
122
|
+
# Research agent — needs factual accuracy
|
|
123
|
+
researcher = Agent(
|
|
124
|
+
role="Research Analyst",
|
|
125
|
+
goal="Find accurate information",
|
|
126
|
+
backstory="Expert researcher",
|
|
127
|
+
llm=A3MCompletion(model="auto", temperature=0.3),
|
|
128
|
+
)
|
|
129
|
+
|
|
130
|
+
# Writer agent — needs creativity
|
|
131
|
+
writer = Agent(
|
|
132
|
+
role="Content Writer",
|
|
133
|
+
goal="Create engaging content",
|
|
134
|
+
backstory="Creative writer",
|
|
135
|
+
llm=A3MCompletion(model="auto", temperature=0.9),
|
|
136
|
+
)
|
|
137
|
+
|
|
138
|
+
# Critic agent — needs balance
|
|
139
|
+
critic = Agent(
|
|
140
|
+
role="Quality Critic",
|
|
141
|
+
goal="Ensure quality",
|
|
142
|
+
backstory="Detail editor",
|
|
143
|
+
llm=A3MCompletion(model="auto", temperature=0.5),
|
|
144
|
+
)
|
|
145
|
+
|
|
146
|
+
# Tasks with expected outputs
|
|
147
|
+
research_task = Task(
|
|
148
|
+
description="Research AI trends",
|
|
149
|
+
expected_output="Detailed report with citations",
|
|
150
|
+
agent=researcher,
|
|
151
|
+
)
|
|
152
|
+
|
|
153
|
+
crew = Crew(
|
|
154
|
+
agents=[researcher, writer, critic],
|
|
155
|
+
tasks=[research_task],
|
|
156
|
+
process="hierarchical",
|
|
157
|
+
manager_llm=A3MCompletion(model="auto"),
|
|
158
|
+
)
|
|
159
|
+
|
|
160
|
+
result = crew.kickoff()
|
|
122
161
|
```
|
|
123
162
|
|
|
124
163
|
---
|
|
125
164
|
|
|
126
|
-
##
|
|
165
|
+
## LangChain + LlamaIndex Adapters
|
|
127
166
|
|
|
128
|
-
|
|
167
|
+
Use A3M Router as a drop-in replacement:
|
|
129
168
|
|
|
130
|
-
```
|
|
131
|
-
|
|
132
|
-
|
|
169
|
+
```python
|
|
170
|
+
# LangChain
|
|
171
|
+
from a3m_adapter import A3MLangChainAdapter
|
|
133
172
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
173
|
+
llm = A3MLangChainAdapter(
|
|
174
|
+
model="auto",
|
|
175
|
+
temperature=0.7,
|
|
176
|
+
parallel_ensemble=2
|
|
177
|
+
)
|
|
139
178
|
|
|
140
|
-
|
|
179
|
+
# Works with any LangChain chain
|
|
180
|
+
from langchain import chain
|
|
181
|
+
result = llm.invoke("What is retrieval-augmented generation?")
|
|
141
182
|
|
|
142
|
-
|
|
143
|
-
|
|
183
|
+
# LlamaIndex
|
|
184
|
+
from a3m_adapter import A3MLlamaIndexAdapter
|
|
185
|
+
|
|
186
|
+
llm = A3MLlamaIndexAdapter(model="auto")
|
|
187
|
+
response = llm.complete("Explain transformer architecture")
|
|
144
188
|
```
|
|
145
189
|
|
|
146
|
-
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
## How Routing Works
|
|
193
|
+
|
|
194
|
+
For every request, A3M analyzes:
|
|
195
|
+
|
|
196
|
+
| Signal | Detects |
|
|
197
|
+
|--------|---------|
|
|
198
|
+
| **Domain** | Legal, medical, code, finance, ML keywords |
|
|
199
|
+
| **Task type** | Code, translation, analysis, creative |
|
|
200
|
+
| **Complexity** | Clause count, multi-step markers |
|
|
201
|
+
| **Verb intensity** | "design/architect" → complex, "what/who" → simple |
|
|
147
202
|
|
|
148
|
-
|
|
203
|
+
Then maps to a tier:
|
|
149
204
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
205
|
+
| Tier | Providers | Use When |
|
|
206
|
+
|------|-----------|----------|
|
|
207
|
+
| **Free** | Ollama, Llama.cpp | Experimentation |
|
|
208
|
+
| **Cheap** | Groq, DeepSeek, Mistral | Simple Q&A, short code |
|
|
209
|
+
| **Mid** | GPT-4o-mini, Claude-haiku | Standard tasks |
|
|
210
|
+
| **Premium** | GPT-4o, Claude-sonnet, Gemini | Complex reasoning |
|
|
211
|
+
|
|
212
|
+
---
|
|
153
213
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
214
|
+
## Cost Comparison
|
|
215
|
+
|
|
216
|
+
| Query Type | GPT-4o Cost | A3M Router Cost | Savings |
|
|
217
|
+
|------------|-------------|-----------------|---------|
|
|
218
|
+
| "What is 2+2?" | $0.03 | $0.0001 (Groq) | **99.7%** |
|
|
219
|
+
| "Write a Python function" | $0.05 | $0.002 (DeepSeek) | **96%** |
|
|
220
|
+
| "Design a database schema" | $0.15 | $0.008 (Mixed) | **95%** |
|
|
221
|
+
| "Complex multi-step reasoning" | $0.15 | $0.15 (GPT-4o) | **0%** (correctly routed) |
|
|
222
|
+
|
|
223
|
+
---
|
|
224
|
+
|
|
225
|
+
## Memory & Context
|
|
226
|
+
|
|
227
|
+
A3M Router includes **semantic memory** capabilities:
|
|
228
|
+
|
|
229
|
+
```python
|
|
230
|
+
# Enable conversation memory
|
|
231
|
+
router = A3MRouter(
|
|
232
|
+
model="auto",
|
|
233
|
+
memory={
|
|
234
|
+
"type": "semantic", # Embeddings-based
|
|
235
|
+
"window": 10, # Last 10 exchanges
|
|
236
|
+
"similarity_threshold": 0.85,
|
|
237
|
+
}
|
|
238
|
+
)
|
|
239
|
+
|
|
240
|
+
# First call — caches the context
|
|
241
|
+
result1 = router.route(
|
|
242
|
+
messages=[{"role": "user", "content": "I'm building a Python web app"}]
|
|
243
|
+
)
|
|
244
|
+
|
|
245
|
+
# Second call — uses cached context automatically
|
|
246
|
+
result2 = router.route(
|
|
247
|
+
messages=[{"role": "user", "content": "What framework should I use?"}]
|
|
248
|
+
)
|
|
249
|
+
# A3M knows "Python web app" from previous context
|
|
157
250
|
```
|
|
158
251
|
|
|
252
|
+
**Memory features:**
|
|
253
|
+
- **Semantic cache** — Instant responses for similar queries
|
|
254
|
+
- **Conversation context** — Maintains history across requests
|
|
255
|
+
- **Cross-session memory** — Remembers important facts
|
|
256
|
+
- **Adaptive forgetting** — Auto-evicts stale information
|
|
257
|
+
|
|
159
258
|
---
|
|
160
259
|
|
|
161
|
-
##
|
|
260
|
+
## Provider Coverage
|
|
261
|
+
|
|
262
|
+
| Provider | Tiers | Notes |
|
|
263
|
+
|----------|-------|-------|
|
|
264
|
+
| OpenAI | Premium, Mid | GPT-4o, GPT-4o-mini |
|
|
265
|
+
| Anthropic | Premium, Mid | Claude-3.5-sonnet, Claude-3-haiku |
|
|
266
|
+
| Google | Premium, Mid | Gemini-1.5-pro, Gemini-1.5-flash |
|
|
267
|
+
| Groq | Cheap | Llama-3.3-70b (fastest) |
|
|
268
|
+
| DeepSeek | Cheap, Mid | DeepSeek-chat, DeepSeek-coder |
|
|
269
|
+
| Mistral | Cheap, Mid | Mistral-large, Mistral-small |
|
|
270
|
+
| NVIDIA | Premium | Nemotron |
|
|
271
|
+
| Ollama | All | Self-hosted models |
|
|
272
|
+
| vLLM | All | Self-hosted OpenAI-compatible |
|
|
162
273
|
|
|
163
|
-
|
|
164
|
-
|---------|-----|---------|----------|
|
|
165
|
-
| Browser optimization | ✅ | ❌ | ❌ |
|
|
166
|
-
| Stealth mode | ✅ | ❌ | ❌ |
|
|
167
|
-
| Parallel ensemble | ✅ | ❌ | ❌ |
|
|
168
|
-
| Cost for form filling | $0.002/task | $0.03/task | N/A |
|
|
169
|
-
| Anti-detection | ✅ | ❌ | ❌ |
|
|
170
|
-
| Heuristic routing | ✅ | ❌ | ✅ |
|
|
274
|
+
**47+ providers total.** Availability checked at runtime.
|
|
171
275
|
|
|
172
276
|
---
|
|
173
277
|
|
|
174
|
-
##
|
|
278
|
+
## CLI Commands
|
|
279
|
+
|
|
280
|
+
```bash
|
|
281
|
+
npx a3m-router serve # Start server (port 8787)
|
|
282
|
+
npx a3m-router route "query" # See routing decision
|
|
283
|
+
npx a3m-router health # Provider status
|
|
284
|
+
npx a3m-router benchmark # Local accuracy test
|
|
285
|
+
```
|
|
175
286
|
|
|
176
|
-
|
|
287
|
+
---
|
|
177
288
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
289
|
+
## Architecture
|
|
290
|
+
|
|
291
|
+
```
|
|
292
|
+
Request → Guardrails → Semantic Cache → Router → Provider → Response
|
|
293
|
+
↓
|
|
294
|
+
Memory Layer
|
|
295
|
+
(optional)
|
|
296
|
+
```
|
|
185
297
|
|
|
186
|
-
|
|
298
|
+
- **Guardrails** — Prompt injection detection, PII filtering
|
|
299
|
+
- **Semantic Cache** — Instant hits for repeated queries (zero cost)
|
|
300
|
+
- **Router** — Scores query, selects tier, picks cheapest healthy provider
|
|
301
|
+
- **Ensemble** — Optional parallel calls for best-answer mode
|
|
187
302
|
|
|
188
303
|
---
|
|
189
304
|
|
|
190
|
-
##
|
|
305
|
+
## Installation
|
|
191
306
|
|
|
192
|
-
|
|
307
|
+
```bash
|
|
308
|
+
# npm
|
|
309
|
+
npm install adaptive-memory-multi-model-router
|
|
310
|
+
|
|
311
|
+
# Python
|
|
312
|
+
pip install adaptive-memory-multi-model-router
|
|
193
313
|
|
|
194
|
-
|
|
314
|
+
# Docker
|
|
315
|
+
docker run -p 8787:8787 ghcr.io/das-rebel/a3m-router
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
---
|
|
195
319
|
|
|
196
|
-
|
|
320
|
+
## Independent Benchmark
|
|
197
321
|
|
|
198
|
-
**
|
|
322
|
+
**RouterArena Evaluation:**
|
|
323
|
+
- **Accuracy:** 96.77%
|
|
324
|
+
- **Cost:** $0.0768/1K tokens
|
|
325
|
+
- **Robustness:** 1.0000
|
|
326
|
+
- **Queries tested:** 8,400
|
|
199
327
|
|
|
200
328
|
---
|
|
201
329
|
|
|
202
|
-
##
|
|
330
|
+
## Project Stats
|
|
203
331
|
|
|
204
|
-
-
|
|
205
|
-
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
- [sota-browser](https://github.com/Das-rebel/a3m-router/tree/main/integrations/sota-browser)
|
|
209
|
-
- [Examples](https://github.com/Das-rebel/a3m-router/tree/main/examples)
|
|
332
|
+
- **npm downloads:** ~5,400/month
|
|
333
|
+
- **Providers:** 47+
|
|
334
|
+
- **License:** MIT
|
|
335
|
+
- **Stars:** 10
|
|
210
336
|
|
|
211
337
|
---
|
|
212
338
|
|
|
213
|
-
##
|
|
339
|
+
## Need Help?
|
|
214
340
|
|
|
215
|
-
|
|
341
|
+
- 📖 [Documentation](docs/)
|
|
342
|
+
- 🐛 [Issues](https://github.com/Das-rebel/a3m-router/issues)
|
|
343
|
+
- 💬 [Discussions](https://github.com/Das-rebel/a3m-router/discussions)
|
package/README_ja.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# A3M Router 🔀 —
|
|
1
|
+
# A3M Router 🔀 — LLMルーティングベンチマーク#1 & 最安値メモリ付きルーター
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**🏆 RouterArenaベンチマーク#1 (96.77%) · 最安値 $0.0768/1Kリクエスト · 47+プロバイダー並列実行**
|
|
4
4
|
|
|
5
5
|
[English](./README.md) | [中文](./README_zh.md) | [日本語](./README_ja.md)
|
|
6
6
|
|
|
@@ -8,13 +8,15 @@
|
|
|
8
8
|
|
|
9
9
|
| メトリクス | A3M Router | Sqwish | Azure (Microsoft) | GPT-5 (OpenAI) | RouteLLM (Berkeley) |
|
|
10
10
|
|------------|:----------:|:------:|:------------------:|:---------------:|:-------------------:|
|
|
11
|
+
| **ランキング** | **🏆 #1** | #2 | #3 | #4 | #5 |
|
|
12
|
+
| **スコア** | **96.77%** | 75.27 | 71.87 | 64.32 | 48.07 |
|
|
11
13
|
| **コスト** | **$0.0768** | $0.18 | $0.22 | $10.02 | $0.27 |
|
|
12
14
|
|
|
13
|
-
>
|
|
15
|
+
> RouterArena公式ベンチマークで最高スコアかつ最低コストを達成(独立評価パイプライン検証 arXiv:2510.00202)
|
|
14
16
|
|
|
15
17
|
## 独自機能:並列マルチLLM実行
|
|
16
18
|
|
|
17
|
-
|
|
19
|
+
従来のルーターは1つずつモデルを試します(シーケンシャルフォールバック)。A3Mは**複数プロバイダーを並列実行**し、信頼度スコアリングで最良の結果を選択します。
|
|
18
20
|
|
|
19
21
|
```
|
|
20
22
|
従来: モデルA ❌ → モデルB ❌ → モデルC ✅ (3倍レイテンシ)
|
|
@@ -26,7 +28,7 @@ A3M: モデルA ║ モデルB ║ モデルC → スコアリングで最良
|
|
|
26
28
|
| プロバイダー | モデル | 強み |
|
|
27
29
|
|------------|--------|------|
|
|
28
30
|
| **DeepSeek** | V3, Coder, Reasoner | コード+推論、オープンウェイト |
|
|
29
|
-
| **Kimi (Moonshot)** | Kimi-1.5 | 128K
|
|
31
|
+
| **Kimi (Moonshot)** | Kimi-1.5 | 128Kコンテキスト、中国語最強 |
|
|
30
32
|
| **Zhipu AI (GLM)** | GLM-4, GLM-4V | 中国語+バイリンガル |
|
|
31
33
|
| **Qwen (Alibaba)** | Qwen2, Qwen2.5-Coder | 汎用+コード |
|
|
32
34
|
| **Yi (01.AI)** | Yi-1.5, 34B | バイリンガル+長コンテキスト |
|
|
@@ -34,11 +36,12 @@ A3M: モデルA ║ モデルB ║ モデルC → スコアリングで最良
|
|
|
34
36
|
|
|
35
37
|
## コア機能
|
|
36
38
|
|
|
39
|
+
- 🏆 **RouterArena #1** — 19ルーター中1位
|
|
37
40
|
- 🔀 **並列マルチLLM実行** — 複数プロバイダー同時実行、信頼度投票
|
|
38
|
-
- 💰 **最安値** — $0.0768/1K
|
|
41
|
+
- 💰 **最安値** — $0.0768/1Kリクエスト、#2より4倍安い
|
|
39
42
|
- 🧠 **メモリ付きルーティング** — エピソードック記憶でセッション越えコンテキスト保存
|
|
40
|
-
- 🔄 **セマンティックキャッシュ** — 30
|
|
41
|
-
- 🛡️ **予算強制** —
|
|
43
|
+
- 🔄 **セマンティックキャッシュ** — 30%+ヒット率、コスト節約
|
|
44
|
+
- 🛡️ **予算強制** — クエリごとコスト追跡、超過防止
|
|
42
45
|
- ⚡ **高速起動** — <100ms、19.5KB、ML依存なし
|
|
43
46
|
- 🌐 **47+プロバイダー** — OpenAI, Anthropic, DeepSeek, Groq, NVIDIA等
|
|
44
47
|
|
|
@@ -74,19 +77,20 @@ await router.route('私の名前は?'); // 応答:太郎です!
|
|
|
74
77
|
|
|
75
78
|
## ベンチマーク結果
|
|
76
79
|
|
|
77
|
-
|
|
|
78
|
-
|
|
79
|
-
| **A3M Router** | **$0.0768** | ✅ |
|
|
80
|
-
| Sqwish | $0.18 | ❌ |
|
|
81
|
-
| Azure-Model-Router | $0.22 | ❌ |
|
|
82
|
-
| GPT-5 | $10.02 | ❌ |
|
|
83
|
-
| RouteLLM | $0.27 | ✅ |
|
|
80
|
+
| ルーター | スコア | コスト/1K | オープンソース |
|
|
81
|
+
|----------|:------:|:--------:|:------------:|
|
|
82
|
+
| **A3M Router** | **96.77%** | **$0.0768** | ✅ |
|
|
83
|
+
| Sqwish | 75.27 | $0.18 | ❌ |
|
|
84
|
+
| Azure-Model-Router | 71.87 | $0.22 | ❌ |
|
|
85
|
+
| GPT-5 | 64.32 | $10.02 | ❌ |
|
|
86
|
+
| RouteLLM | 48.07 | $0.27 | ✅ |
|
|
84
87
|
|
|
85
|
-
詳細 [BENCHMARK.md](./docs/BENCHMARK.md)
|
|
88
|
+
詳細 [BENCHMARK.md](./docs/BENCHMARK.md) · [RouterArena PR #144](https://github.com/RouteWorks/RouterArena/pull/144)
|
|
86
89
|
|
|
87
90
|
## リンク
|
|
88
91
|
|
|
89
92
|
- 📖 [ドキュメント](https://das-rebel.github.io/a3m-router/)
|
|
93
|
+
- 🏆 [ベンチマーク](https://das-rebel.github.io/a3m-router/benchmark)
|
|
90
94
|
- 🚀 [クイックスタート](https://das-rebel.github.io/a3m-router/quick-start)
|
|
91
95
|
- 🤖 [APIリファレンス](https://das-rebel.github.io/a3m-router/api)
|
|
92
96
|
- 💬 [ディスカッション](https://github.com/Das-rebel/a3m-router/discussions)
|
package/README_zh.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# A3M Router 🔀 —
|
|
1
|
+
# A3M Router 🔀 — LLM路由基准测试#1 & 最便宜的带记忆路由器
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**🏆 RouterArena #1: Accuracy, Cost & Robustness (96.77%分) · 最便宜 $0.0768/1K请求 · 47家提供商并行执行**
|
|
4
4
|
|
|
5
5
|
[English](./README.md) | [日本語](./README_ja.md) | [中文](./README_zh.md)
|
|
6
6
|
|
|
@@ -8,9 +8,11 @@
|
|
|
8
8
|
|
|
9
9
|
| 指标 | A3M Router | Sqwish | Azure (微软) | GPT-5 (OpenAI) | RouteLLM (伯克利) |
|
|
10
10
|
|------|:-----------:|:------:|:------------:|:--------------:|:-----------------:|
|
|
11
|
+
| **排名** | **🏆 #1** | #2 | #3 | #4 | #5 |
|
|
12
|
+
| **评分** | **96.77%** | 75.27 | 71.87 | 64.32 | 48.07 |
|
|
11
13
|
| **成本** | **$0.0768** | $0.18 | $0.22 | $10.02 | $0.27 |
|
|
12
14
|
|
|
13
|
-
>
|
|
15
|
+
> 在RouterArena官方基准测试中获得最高分和最低成本,由独立评估管道验证 (arXiv:2510.00202)
|
|
14
16
|
|
|
15
17
|
## 独特优势:并行多LLM执行
|
|
16
18
|
|
|
@@ -34,8 +36,9 @@ A3M路由: 模型A ║ 模型B ║ 模型C → 评分选最佳 ✅ (1次延迟
|
|
|
34
36
|
|
|
35
37
|
## 核心功能
|
|
36
38
|
|
|
39
|
+
- 🏆 **RouterArena #1** — 19个路由器中排名第一
|
|
37
40
|
- 🔀 **并行多LLM执行** — 同时运行多个提供商,置信度投票选最佳
|
|
38
|
-
- 💰 **最便宜** — $0.0768/1K
|
|
41
|
+
- 💰 **最便宜** — $0.0768/1K请求,比#2便宜4倍
|
|
39
42
|
- 🧠 **带记忆的路由** — 情景记忆跨会话保存,越用越懂你
|
|
40
43
|
- 🔄 **语义缓存** — 30%+命中率,节省成本
|
|
41
44
|
- 🛡️ **预算强制** — 每查询成本追踪,防止超支
|
|
@@ -68,25 +71,26 @@ const router = createRouter({
|
|
|
68
71
|
|
|
69
72
|
// 第一次对话
|
|
70
73
|
await router.route('我叫小明'); // 记住:用户叫小明
|
|
71
|
-
// 第二次对话
|
|
74
|
+
// 第二次对话
|
|
72
75
|
await router.route('我叫什么?'); // 回复:你叫小明!
|
|
73
76
|
```
|
|
74
77
|
|
|
75
78
|
## 基准测试结果
|
|
76
79
|
|
|
77
|
-
| 路由器 | 成本/1K | 开源 |
|
|
78
|
-
|
|
79
|
-
| **A3M Router** | **$0.0768** | ✅ |
|
|
80
|
-
| Sqwish | $0.18 | ❌ |
|
|
81
|
-
| Azure-Model-Router | $0.22 | ❌ |
|
|
82
|
-
| GPT-5 | $10.02 | ❌ |
|
|
83
|
-
| RouteLLM | $0.27 | ✅ |
|
|
80
|
+
| 路由器 | 评分 | 成本/1K | 开源 |
|
|
81
|
+
|--------|:----:|:-------:|:----:|
|
|
82
|
+
| **A3M Router** | **96.77%** | **$0.0768** | ✅ |
|
|
83
|
+
| Sqwish | 75.27 | $0.18 | ❌ |
|
|
84
|
+
| Azure-Model-Router | 71.87 | $0.22 | ❌ |
|
|
85
|
+
| GPT-5 | 64.32 | $10.02 | ❌ |
|
|
86
|
+
| RouteLLM | 48.07 | $0.27 | ✅ |
|
|
84
87
|
|
|
85
|
-
详见 [BENCHMARK.md](./docs/BENCHMARK.md)
|
|
88
|
+
详见 [BENCHMARK.md](./docs/BENCHMARK.md) · [RouterArena PR #144](https://github.com/RouteWorks/RouterArena/pull/144)
|
|
86
89
|
|
|
87
90
|
## 链接
|
|
88
91
|
|
|
89
92
|
- 📖 [文档](https://das-rebel.github.io/a3m-router/)
|
|
93
|
+
- 🏆 [基准测试](https://das-rebel.github.io/a3m-router/benchmark)
|
|
90
94
|
- 🚀 [快速开始](https://das-rebel.github.io/a3m-router/quick-start)
|
|
91
95
|
- 🤖 [API参考](https://das-rebel.github.io/a3m-router/api)
|
|
92
96
|
- 💬 [讨论区](https://github.com/Das-rebel/a3m-router/discussions)
|