adaptive-memory-multi-model-router 2.9.2 → 2.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +194 -20
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -15,26 +15,30 @@ OpenAI-compatible **LLM gateway & router** that auto-routes every query to the c
15
15
  ### Architecture
16
16
 
17
17
  ```
18
- ┌─────────────────────────────────────────────────────────────────┐
19
- │ A3M Router — Generative Engine │
20
- ├─────────────────────────────────────────────────────────────────┤
21
- │ │
22
- │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
23
- │ │ Guardrails │ → │ Semantic │ → │ Routing Engine │ │
24
- │ │ (Security) │ │ Cache │ │ (Multi-signal │ │
25
- │ │ 17 patterns │ │ (30% hit) │ │ + MCTS) │ │
26
- │ └──────────────┘ └──────────────┘ └────────┬─────────┘ │
27
- │ │ │
28
- │ ┌──────────────────────┬──────────────────────┼────────┐ │
29
- │ │ │ │ │ │
30
- │ ↓ ↓ ↓ │ │
31
- │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐│ │
32
- │ │ MemoryTree │ │ CostTracker│ │ Circuit Breaker ││ │
33
- │ │ (History) │ │ (Budgets) │ │ (Failover) ││ │
34
- │ └─────────────┘ └─────────────┘ └─────────────────┘│ │
35
- │ │ │
36
- │ 47+ Providers: Groq, DeepSeek, Kimi, Qwen, Zhipu, OpenAI, Anthropic + more │ │
37
- └─────────────────────────────────────────────────────────────────┘
18
+ ╔══════════════════════════════════════════════════════════════════╗
19
+ ║ A3M Router — LLM Gateway ║
20
+ ╠══════════════════════════════════════════════════════════════════╣
21
+ ║ ║
22
+ ║ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ ║
23
+ ║ │ Guardrails │ ──▶ │ Cache │ ──▶ │ Router │ ║
24
+ ║ │ 🔒 17x │ │ 💾 30%+ │ │ 🎯 MCTS │ ║
25
+ ║ │ Injection │ │ Hit │ │ Multi-Signal │ ║
26
+ ║ │ PII Detect │ │ Semantic │ │ 12 Signals │ ║
27
+ ║ └─────────────┘ └─────────────┘ └────────┬────────┘ ║
28
+ ║ │ ║
29
+ ║ ┌─────────────────┬──────────────────────────┴──────┐ ║
30
+ ║ │ │ │ ║
31
+ ║ ▼ ▼ ▼ ║
32
+ ║ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐║
33
+ ║ │ MemoryTree │ │ CostTrack │ │ Circuit │║
34
+ ║ │ 🧠 │ │ 💰 │ │ Breaker 🔄 │║
35
+ ║ │ EMA │ │ Budget │ │ 3 Fails → │║
36
+ ║ │ Learning │ │ Alerts │ │ 60s Cooldown│║
37
+ ║ └─────────────┘ └─────────────┘ └─────────────┘║
38
+ ║ ║
39
+ ║ 47+ Providers: Groq · DeepSeek · Kimi · Qwen · Zhipu · Yi · + ║
40
+ ║ OpenAI · Anthropic · Google · Mistral · + ║
41
+ ╚══════════════════════════════════════════════════════════════════╝
38
42
  ```
39
43
 
40
44
 
@@ -168,6 +172,61 @@ curl -s http://localhost:8787/v1/chat/completions \
168
172
 
169
173
  ---
170
174
 
175
+
176
+ ### Terminal Demo
177
+
178
+ ```bash
179
+ $ npx a3m-router serve
180
+ ╔════════════════════════════════════════════════════════════╗
181
+ ║ A3M Router v2.9.2 ║
182
+ ║ 🔀 Intelligent LLM Gateway ║
183
+ ╠════════════════════════════════════════════════════════════╣
184
+ ║ ✅ Proxy: http://localhost:8787 ║
185
+ ║ ✅ Dashboard: http://localhost:8787/dashboard ║
186
+ ║ ✅ Health: http://localhost:8787/health ║
187
+ ╚════════════════════════════════════════════════════════════╝
188
+
189
+ [GROQ] ✅ 145ms | [DEEPSEEK] ✅ 230ms | [KIMI] ✅ 312ms
190
+ [ANTHROPIC] ✅ 520ms | [OPENAI] ✅ 480ms | [QWEN] ✅ 290ms
191
+
192
+ 🧠 Memory: 1,247 queries cached | 💰 Today: $2.34 / $50.00 budget
193
+ ```
194
+
195
+ ```bash
196
+ $ npx a3m-router route "Design a clinical trial for oncology"
197
+
198
+ 🔀 Routing Decision:
199
+ Query: "Design a clinical trial for oncology"
200
+
201
+ 📊 Complexity: 1.00 (premium)
202
+ 🏷️ Tier: premium
203
+
204
+ ✅ Route to: openai/gpt-4o ($2.50/1M tokens)
205
+ 🔄 Fallback: anthropic/claude-3.5-sonnet
206
+
207
+ 💡 Signals: medical(+0.35) + design(+0.20) + multi-step(+0.15)
208
+ ```
209
+
210
+ ```bash
211
+ $ npx a3m-router cost
212
+
213
+ 💰 Cost Analytics (May 2024)
214
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
215
+ Total Spend: $127.45 / $500.00 budget
216
+ Daily Average: $4.27
217
+ Queries: 28,392
218
+
219
+ 📈 By Provider: 📊 By Tier:
220
+ Groq: $42.30 ████████ 33% premium: $89.10 70%
221
+ DeepSeek: $51.20 █████████ 40% mid: $28.90 23%
222
+ Claude: $28.90 █████ 23% cheap: $7.45 6%
223
+ GPT-4o-mini: $5.05 █ 4% free: $2.00 1%
224
+
225
+ 🚨 Budget Alert: Engineering team at 80% ($160 / $200)
226
+ ```
227
+
228
+ ---
229
+
171
230
  ## How It Works — Routing Engine
172
231
 
173
232
  A3M Router combines multi-signal routing, semantic caching, and load balancing to route queries to the cheapest capable model with 99.5% accuracy.
@@ -237,6 +296,73 @@ User Query
237
296
  Result: { model, tier, cost, complexity, reasoning[], fallbackModels[] }
238
297
  ```
239
298
 
299
+ ### Visual Routing Flow
300
+
301
+ ```
302
+ User Query
303
+ │
304
+ ▼
305
+ ┌─────────────────────┐
306
+ │ Guardrails Check │
307
+ │ 🔒 PII / Injection │
308
+ └──────────┬──────────┘
309
+ │
310
+ ✅ Pass?
311
+ / \
312
+ No Yes
313
+ │ │
314
+ ▼ ▼
315
+ [BLOCK] ┌─────────────────┐
316
+ │ Semantic Cache │
317
+ │ 💾 Lookup │
318
+ └────────┬────────┘
319
+ │
320
+ Cache Hit?
321
+ / \
322
+ Yes No
323
+ │ │
324
+ ▼ ▼
325
+ [RETURN] ┌─────────────────┐
326
+ │ │ Route Query │
327
+ │ │ 🎯 12 Signals │
328
+ │ │ Complexity → │
329
+ │ │ Tier │
330
+ │ └────────┬────────┘
331
+ │ │
332
+ │ ▼
333
+ │ ┌─────────────────┐
334
+ │ │ Provider Health │
335
+ │ │ 📊 Scoring │
336
+ │ └────────┬────────┘
337
+ │ │
338
+ │ ▼
339
+ │ ┌─────────────────┐
340
+ │ │ Best Provider │
341
+ │ │ + Fallbacks │
342
+ │ └────────┬────────┘
343
+ │ │
344
+ │ ▼
345
+ │ ┌─────────────────┐
346
+ │ │ Execute LLM │
347
+ │ │ Call │
348
+ │ └────────┬────────┘
349
+ │ │
350
+ │ ▼
351
+ │ ┌─────────────────┐
352
+ │ │ Update Memory │
353
+ │ │ 🧠 EMA Update │
354
+ │ └────────┬────────┘
355
+ │ │
356
+ │ ▼
357
+ │ [RETURN RESPONSE]
358
+ │ │
359
+ └─────────────────┘
360
+ ```
361
+
362
+ ---
363
+
364
+
365
+
240
366
  ### Complexity Examples
241
367
 
242
368
  | Query | Signals Detected | Score | Tier | Route To |
@@ -366,6 +492,54 @@ Router assigns each sub-task to optimal agent, tracks outcomes, learns preferenc
366
492
 
367
493
  ## Features in Detail
368
494
 
495
+ ### Feature Overview
496
+
497
+ ```
498
+ ┌────────────────────────────────────────────────────────────────────────────┐
499
+ │ A3M Router Features │
500
+ ├────────────────────────────────────────────────────────────────────────────┤
501
+ │ │
502
+ │ 🧠 ADAPTIVE MEMORY │ 🎯 INTELLIGENT ROUTING │
503
+ │ ─────────────────── │ ───────────────────── │
504
+ │ • MemoryTree storage │ • 12-keyword signal detection │
505
+ │ • EMA quality scoring │ • 99.5% ±1 tier accuracy │
506
+ │ • Learns from history │ • <1ms routing latency │
507
+ │ • No retraining needed │ • MCTS for complex workflows │
508
+ │ │
509
+ ├────────────────────────────────────────────────────────────────────────────┤
510
+ │ │
511
+ │ 💰 HARD BUDGET ENFORCEMENT │ 🛡️ GUARDRAILS │
512
+ │ ─────────────────────── │ ─────────────────── │
513
+ │ • Per-user/team budgets │ • 17-pattern injection detection │
514
+ │ • Real-time spend tracking │ • PII redaction │
515
+ │ • Alerts at 50/80/100% │ • Content filtering │
516
+ │ • Hard caps (reject when exceeded) │ • Hallucination checks │
517
+ │ │
518
+ ├────────────────────────────────────────────────────────────────────────────┤
519
+ │ │
520
+ │ 🔄 INTELLIGENT FAILOVER │ 💾 SEMANTIC CACHE │
521
+ │ ─────────────────────── │ ─────────────────── │
522
+ │ • Provider health scoring │ • Embedding-based lookup │
523
+ │ • Circuit breaker (3 fails) │ • Configurable similarity threshold │
524
+ │ • Automatic fallback chain │ • Per-route TTL │
525
+ │ • Chinese provider handling │ • 30%+ cache hit rate │
526
+ │ │
527
+ ├────────────────────────────────────────────────────────────────────────────┤
528
+ │ │
529
+ │ ⚡ PER-PROVIDER RETRY │ 📊 COST ANALYTICS │
530
+ │ ───────────────────── │ ─────────────────── │
531
+ │ • Custom timeout per model │ • Per-provider breakdown │
532
+ │ • Exponential backoff │ • Budget vs actual dashboard │
533
+ │ • 429 rate limit handling │ • Projected savings │
534
+ │ • Jitter to prevent storms │ • Monthly/yearly reports │
535
+ │ │
536
+ └────────────────────────────────────────────────────────────────────────────┘
537
+ ```
538
+
539
+ ---
540
+
541
+
542
+
369
543
  ### 🧠 Adaptive Memory & Learning
370
544
 
371
545
  **How Memory Works**
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "adaptive-memory-multi-model-router",
3
- "version": "2.9.2",
3
+ "version": "2.10.0",
4
4
  "shortName": "A3M Router",
5
5
  "displayName": "A3M Router - Adaptive Memory Multi-Model Router",
6
6
  "description": "LLM router & AI gateway — 99.5% routing accuracy, 47 providers (DeepSeek, Kimi/Moonshot, Qwen, Zhipu GLM, Yi + more). Semantic cache, guardrails, cost analytics. Built on 30+ arXiv papers (SGLang, Medusa, MemoRAG). Zero ML, 19.5KB. TypeScript + Python SDK. MIT.",