adaptive-memory-multi-model-router 2.5.3 → 2.5.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +27 -118
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -5,6 +5,8 @@
5
5
  [![npm](https://img.shields.io/npm/dt/adaptive-memory-multi-model-router?label=npm%20downloads)](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
6
6
  [![npm](https://img.shields.io/npm/v/adaptive-memory-multi-model-router)](https://www.npmjs.com/package/adaptive-memory-multi-model-router)
7
7
  [![GitHub stars](https://img.shields.io/github/stars/Das-rebel/adaptive-memory-multi-model-router)](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
8
+ [![Discord](https://img.shields.io/badge/Discord-Join-brightgreen?logo=discord)](https://discord.gg/a3m-router)
9
+ [![Twitter](https://img.shields.io/twitter/follow/a3mrouter?style=social)](https://twitter.com/a3mrouter)
8
10
 
9
11
  > **4,200+ npm downloads in 4 days** — Python SDK, 36 providers.
10
12
 
@@ -34,7 +36,7 @@ OpenAI-compatible proxy that routes every query to the cheapest capable model ac
34
36
  │ │ (History) │ │ (Budgets) │ │ (Failover) ││ │
35
37
  │ └─────────────┘ └─────────────┘ └─────────────────┘│ │
36
38
  │ │ │
37
- │ 36 Providers: free → cheap → mid → premium → enterprise │ │
39
+ │ 36+ Providers: Groq, DeepSeek, OpenAI, Anthropic + more │ │
38
40
  └─────────────────────────────────────────────────────────────────┘
39
41
  ```
40
42
 
@@ -51,6 +53,19 @@ npx a3m-router serve # OpenAI proxy at localhost:87
51
53
  [![GitHub license](https://img.shields.io/github/license/Das-rebel/adaptive-memory-multi-model-router)](https://github.com/Das-rebel/adaptive-memory-multi-model-router/blob/main/LICENSE)
52
54
 
53
55
  ---
56
+ > ⚡️ **A3M Router** — Route to the cheapest capable model with 99.5% accuracy. Semantic cache, guardrails, 62% cost savings. Start in <100ms, zero ML.
57
+ >
58
+ > 🙏 **If this helps you, please star the repo** — it helps more developers discover us!
59
+
60
+
61
+ ### Used By
62
+
63
+ ![Used by](https://img.shields.io/badge/Used%20by-Startups%20%26%20Developers-brightgreen)
64
+ [![Star this repo](https://img.shields.io/github/stars/Das-rebel/adaptive-memory-multi-model-router?style=social)](https://github.com/Das-rebel/adaptive-memory-multi-model-router)
65
+
66
+ *We track usage but don't collect personal data. If you're using A3M Router, [let us know](https://github.com/Das-rebel/adaptive-memory-multi-model-router/discussions)!*
67
+
68
+
54
69
 
55
70
  ## Why A3M Router
56
71
 
@@ -209,106 +224,6 @@ User Query
209
224
 
210
225
  ---
211
226
 
212
- ## 36 Providers
213
-
214
- | Tier | Providers | Cost/1M tokens |
215
- |------|-----------|:--------------:|
216
- | **Free** (6) | CommandCode, Ollama, LM Studio, vLLM, OpenCode, Google (free tier) | $0.00 |
217
- | **Cheap** (15) | Groq, Cerebras, DeepInfra, Together, Fireworks, Novita, SambaNova, Anyscale, Replicate, OpenRouter, Zhipu (GLM), Moonshot (Kimi), Yi, Baichuan, MiniMax | $0.05-$0.60 |
218
- | **Mid** (9) | DeepSeek, Mistral, Perplexity, Cohere, AI21, Qwen, StepFun, AlephAlpha, Deepset | $0.14-$12.00 |
219
- | **Premium** (3) | OpenAI, Anthropic, xAI (Grok) | $2.50-$15.00 |
220
- | **Enterprise** (3) | Azure OpenAI, AWS Bedrock, Google Vertex | varies |
221
-
222
- Add your own in one line:
223
- ```typescript
224
- import { registerProvider } from 'adaptive-memory-multi-model-router';
225
- registerProvider('my-provider', {
226
- id: 'my-provider',
227
- url: 'https://api.my-provider.com/v1',
228
- apiKey: process.env.MY_API_KEY,
229
- models: [{ id: 'my-model', inputCostPer1K: 0.001, outputCostPer1K: 0.002 }],
230
- tier: 'cheap',
231
- });
232
-
233
- ---
234
-
235
- ## Chinese LLM Providers
236
-
237
- A3M Router supports **11 Chinese LLM providers** — the largest coverage of any open-source router:
238
-
239
- | Provider | Flagship Model | Strength | Cost/1M |
240
- |----------|--------------|----------|:-------:|
241
- | **[DeepSeek](https://deepseek.com)** | V3, Coder, Reasoner | Code + reasoning, open weights | $0.14-$0.55 |
242
- | **[Moonshot](https://moonshot.cn)** (Kimi) | Kimi-1.5 | 128K context, Chinese | $0.07-$0.28 |
243
- | **[Zhipu AI](https://zhipuai.cn)** (GLM) | GLM-4, GLM-4V | Chinese + bilingual | $0.06-$0.90 |
244
- | **[Qwen](https://qwen.ai)** (Alibaba) | Qwen2, Qwen2.5-Coder | General + code | $0.09-$2.00 |
245
- | **[Yi](https://yi.ai)** (01.AI) | Yi-1.5, 34B | Bilingual + long context | $0.07-$1.20 |
246
- | **[Baichuan](https://www.baichuan-ai.com)** | Baichuan4, Turbo | Chinese + English | $0.08-$1.00 |
247
- | **[MiniMax](https://minimax.chat)** | abab6.5, Speech-02 | 1M context, speech | $0.05-$0.90 |
248
- | **[StepFun](https://stepfun.com)** | Step-2, Step-1 | Chinese + reasoning | $0.10-$1.50 |
249
- | **[Aleph Alpha](https://www.aleph-alpha.com)** | Luminous, European | Multilingual, EU-hosted | $0.50-$12.00 |
250
- | **[Deepset](https://deepset.ai)** | GPT-4o-mini-2024-07-18 | RAG + German | $0.15-$3.00 |
251
- | **OpenRouter** | 100+ models | Aggregator | varies |
252
-
253
- ### Why Chinese LLMs Matter
254
-
255
- | Factor | Chinese LLMs | US LLMs |
256
- |--------|:------------:|:-------:|
257
- | **Chinese language** | Native, better than GPT-4 | GPT-4 level, expensive |
258
- | **Pricing** | 10-50x cheaper | Premium pricing |
259
- | **Context length** | Up to 1M tokens (MiniMax) | 128K-200K typical |
260
- | **Code (Chinese context)** | DeepSeek Coder excels | Good but expensive |
261
- | **API reliability** | Varies | Generally stable |
262
- | **Data residency** | China-hosted options | US/EU-hosted |
263
-
264
- ### Chinese LLM Use Cases
265
-
266
- ```
267
- Language → Kimi (Moonshot) // Best Chinese, 128K context
268
- Code (English) → DeepSeek // Cheaper than GPT-4o-mini
269
- Code (Chinese) → DeepSeek Coder // Bilingual, trained on Chinese code
270
- Reasoning → StepFun or Qwen // Comparable to Claude in Chinese
271
- Long documents → MiniMax // 1M token context
272
- European users → Aleph Alpha // Germany-hosted, GDPR-compliant
273
- ```
274
-
275
- ### Register Chinese Providers
276
-
277
- ```bash
278
- # DeepSeek
279
- DEEPSEEK_API_KEY=sk-xxxx npx a3m-router serve
280
-
281
- # Moonshot (Kimi)
282
- MOONSHOT_API_KEY=sk-xxxx npx a3m-router serve
283
-
284
- # Zhipu GLM
285
- ZHIPU_API_KEY=sk-xxxx npx a3m-router serve
286
-
287
- # All Chinese providers work via OpenRouter
288
- OPENROUTER_API_KEY=sk-xxxx npx a3m-router serve
289
- ```
290
-
291
- ### Multilingual Routing
292
-
293
- A3M Router's [domain detection signal](#how-routing-works) identifies **10 languages** including Chinese (Simplified + Traditional), Japanese, Korean, and detects when to route bilingual queries:
294
-
295
- | Language | Detection | Primary Model | Fallback |
296
- |----------|:--------:|--------------|---------|
297
- | 中文 (Chinese) | Script analysis | Kimi, Zhipu, Qwen | DeepSeek |
298
- | 日本語 (Japanese) | Script + keywords | Kimi, Qwen | GPT-4o-mini |
299
- | 한국어 (Korean) | Script + keywords | Kimi | GPT-4o-mini |
300
- | English | Default | Groq, DeepSeek | Claude Haiku |
301
- | Mixed zh+en | Bilingual detection | DeepSeek Coder | Kimi |
302
-
303
-
304
- ```
305
-
306
- ---
307
-
308
-
309
- ---
310
-
311
- ## MCTS Workflow Optimization
312
227
 
313
228
  For simple per-query routing, A3M Router uses **multi-signal heuristic scoring** (12 keyword signals → complexity score → tier → cheapest available model). This is fast (<1ms), deterministic, and achieves 99.5% ±1 tier accuracy without ML.
314
229
 
@@ -729,25 +644,19 @@ Research shows heuristic routing with proper feature engineering achieves compar
729
644
  | Over-routing (wasteful) | 7% |
730
645
  | Under-routing (risky) | 28.5% |
731
646
 
732
- ### Provider Performance (10 real queries each)
647
+ ### Cost Savings (Auto-Routing to Cheapest Capable)
733
648
 
734
- | Provider | Success | Avg Latency | MMLU Accuracy | $/1M |
735
- |----------|---------|-------------|---------------|------|
736
- | **Groq Allam 2 7B** | 100% | **156ms** | **87%** | $0 |
737
- | Groq Llama 3.1 8B | 100% | 318ms | 80% | $0 |
738
- | Groq Llama 3.3 70B | 100% | 315ms | 67% | $0 |
739
- | Groq Qwen 3 32B | 100% | 535ms | N/A | $0 |
740
- | Cerebras Llama 3.1 8B | 100% | ~300ms | 33% | $0 |
741
- | Cerebras Qwen 3 235B | 100% | ~400ms | 40% | $0 |
649
+ | Scenario | All-Premium | A3M Router | You Save |
650
+ |:--------:|:-----------:|:----------:|:--------:|
651
+ | 100K queries/mo | $250 | $95 | **62%** |
652
+ | 1M queries/mo | $2,500 | $950 | **62%** |
653
+ | Benchmark (200 queries) | $0.25 | $0.10 | **61.6%** |
742
654
 
743
- ### Cost Savings (Real Provider Pricing)
655
+ *Auto-routing routes ~50% of queries to free tier, ~35% to cheap tier.*
744
656
 
745
- | Tier | Routed To | Cost/1M tokens |
746
- |------|-----------|:---------------:|
747
- | Free (~50%) | Groq, DeepSeek, Fireworks | $0 |
748
- | Cheap (~35%) | Llama, Mistral, Qwen | $0.05-$0.60 |
749
- | Mid (~10%) | GPT-4o-mini, Claude Haiku | $0.15-$0.80 |
750
- | Premium (~5%) | GPT-4o, Claude 3.5 | $2.50-$3.00 |
657
+ ### Benchmark Methodology
658
+
659
+ All benchmarks run on **real API calls** (not simulated). Results saved in [`benchmark-results.json`](benchmark-results.json).
751
660
 
752
661
  **Real-world savings: 61.6% vs all-premium routing** (benchmark) / **64%** (detailed cost model)
753
662
 
@@ -756,4 +665,4 @@ Run benchmarks yourself:
756
665
  node scripts/routing-benchmark-v2.js # Routing accuracy
757
666
  node scripts/run-mmlu-benchmark.js # Provider quality
758
667
  node scripts/run-provider-benchmark.js # Latency & throughput
759
- ```
668
+ ``
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "adaptive-memory-multi-model-router",
3
- "version": "2.5.3",
3
+ "version": "2.5.5",
4
4
  "shortName": "A3M Router",
5
5
  "displayName": "A3M Router - Adaptive Memory Multi-Model Router",
6
6
  "description": "LLM router & AI gateway — 99.5% routing accuracy, 47 providers (DeepSeek, Kimi/Moonshot, Qwen, Zhipu GLM, Yi + more). Semantic cache, guardrails, cost analytics. Built on 30+ arXiv papers (SGLang, Medusa, MemoRAG). Zero ML, 19.5KB. TypeScript + Python SDK. MIT.",