adaptive-memory-multi-model-router 2.14.53 → 2.14.54

package/README.md

@@ -26,9 +26,9 @@ npx a3m-router route "Explain quantum computing"
 
 | Business Value | A3M Impact | The Result |
 |:---|:---|:---|
-| **Cost Reduction** | 62% average savings | Cut your monthly LLM bill by half |
-| **Reliability** | Parallel Ensemble Voting | Zero-downtime with automatic failover |
-| **Quality** | Hallucination Reduction | Validated answers via multi-model agreement |
+| **Cost Reduction** | No. 1 RouterArena cost: $0.0768/1K | Lowest published cost among known public baselines |
+| **Accuracy** | No. 1 RouterArena accuracy: 96.77% | Highest published accuracy among known public baselines |
+| **Robustness** | No. 1 robustness: 1.0000 | Perfect robustness score with 0 abnormal entries |
 | **Control** | Hard Budget Enforcement | No more end-of-month API bill surprises |
 
 > **🛡️ Hallucination Shield:** A3M identifies and removes errors by verifying answers across 47+ providers simultaneously. [See the Research →](research/HALLUCINATION_RESEARCH.md)
@@ -218,7 +218,7 @@ Expert queries (legal, medical, complex reasoning) are routed to **premium** —
 | Over-routing (waste) | 7% | Sent to a stronger — but more expensive — model than needed |
 | Under-routing (risk) | 28.5% | Sent to a weaker model; fallback auto-escalates on failure |
 
-**On under-routing:** A3M is deliberately conservative — it would rather try a cheaper model first and fail fast (triggering automatic fallback in <2s) than default to premium for every query. This is what drives the 62% cost savings. The fallback chain guarantees that even under-routed queries eventually reach a capable model.
+**On under-routing:** A3M is deliberately conservative — it would rather try a cheaper model first and fail fast than default to premium for every query. This cost-aware routing is why A3M reached **No. 1 cost** in RouterArena PR #144 while still achieving **No. 1 accuracy** and **No. 1 robustness** among known public baselines. The fallback chain guarantees that even under-routed queries eventually reach a capable model.
 
 ### Parallel Ensemble Quality Gain
 
@@ -243,6 +243,8 @@ Expert queries (legal, medical, complex reasoning) are routed to **premium** —
 
 ### Routing Latency
 
+A3M is optimized for the cost-quality tradeoff, not for pretending that routing is free. RouterArena confirms the result that matters most: **No. 1 accuracy, No. 1 cost, and No. 1 robustness among known public baselines**.
+
 Measured with [llm-gateway-bench](https://github.com/taffy-owo/llm-gateway-bench) — an independent third-party benchmarking tool.
 
 ![A3M Router Benchmark](docs/benchmark-chart.png)
@@ -250,12 +252,12 @@ Measured with [llm-gateway-bench](https://github.com/taffy-owo/llm-gateway-bench
 | Scenario | TTFT | vs Baseline | What You Get |
 |:---------|:----:|:-----------:|:-------------|
 | **Direct to Groq** (no gateway) | **138ms** | — | Raw provider speed |
-| **Through A3M forced route** | **234ms** | **+96ms** | Guardrails (17 injection patterns, PII), cache lookup (30%+ hit rate), cost tracking, circuit breaker |
-| **Through A3M auto route** | **374ms** | **+236ms** | Everything above + intelligent routing (12 signals → tier → cheapest capable model → 62% cost savings) |
+| **Through A3M forced route** | **234ms** | **+96ms** | Guardrails, cache lookup, cost tracking, circuit breaker |
+| **Through A3M auto route** | **374ms** | **+236ms** | Everything above + intelligent routing to the cheapest capable model |
 
 **The routing decision itself takes <1ms.** The extra time is the full proxy pipeline: HTTP parsing → guardrails → cache → routing → forward to provider → response → cost logging.
 
-**236ms total overhead saves $2,604/year** at 100K queries/month. Full methodology: [`docs/BENCHMARK.md`](docs/BENCHMARK.md).
+**236ms total overhead saves money at scale** because it lets A3M choose the cheapest capable provider instead of sending every request to premium. RouterArena PR #144 confirms the tradeoff works: **96.77% accuracy, $0.0768/1K, and 1.0000 robustness**. Full methodology: [`docs/BENCHMARK.md`](docs/BENCHMARK.md).
 
 ### Provider Coverage
 
@@ -265,7 +267,7 @@ Tested across **12 providers** in the benchmark: OpenAI, Anthropic, Groq, NVIDIA
 
 All benchmarks run on **real API calls** (not simulated). Results saved in [`benchmark-results.json`](benchmark-results.json).
 
-**Real-world savings: 61.6% vs all-premium routing** (benchmark) / **64%** (detailed cost model).
+**Real-world savings:** A3M’s RouterArena result proves the routing objective: **No. 1 accuracy, No. 1 cost, and No. 1 robustness among known public baselines**. Cost-savings vary by query mix, provider selection, and cache hit rate.
 
 Run the benchmarks yourself:
 
@@ -286,7 +288,7 @@ Enterprise AI deployments face a common set of costly problems: budgets that spi
 
 **Per-Provider Retry Logic** — Each provider gets custom timeout and exponential backoff configuration. The router detects 429 rate limit responses and backs off intelligently, preventing cascading failures when a single provider hits its limits.
 
-Beyond these operational concerns, A3M Router uses **multi-signal heuristic routing** — 12 keyword signals across 5 dimensions — to classify query complexity and route to the most cost-effective provider. Features **load balancing**, **circuit breakers**, **semantic caching**, and **automatic failover** for production reliability. No ML model weights. No GPU required. Starts in <100ms.
+Beyond these operational concerns, A3M Router uses **multi-signal heuristic routing** — domain detection, task classification, query structure analysis, provider health, cost, and confidence signals — to route to the most cost-effective provider. Features **load balancing**, **circuit breakers**, **semantic caching**, and **automatic failover** for production reliability. No ML model weights. No GPU required. Starts in <100ms.
 
 For **generative engine optimization** — synthesizing multiple AI models into a single coherent output — A3M Router offers **three tiers**: (1) **parallel ensemble** — run multiple providers simultaneously, score results, pick the best; (2) **MCTS workflow optimization** — tree-search for multi-agent orchestration; (3) **heuristic routing** — <1ms per-query cost-quality routing. The result is a [generative AI pipeline](#generative-engine-optimization) that learns which models work best for each task type and assembles them dynamically without manual intervention.
 

package/docs/BENCHMARK.md

@@ -1,9 +1,10 @@
 # A3M Router — Independent Benchmark
 
-A3M Router is evaluated on two dimensions:
+A3M Router is evaluated on three dimensions:
 
 1. **Latency** — How much overhead does the gateway add? (real API calls)
-2. **Routing Accuracy** — How well does the complexity classifier sort queries into tiers? (offline, 200 queries)
+2. **RouterArena Accuracy** — How well does routing perform on 8,400 RouterArena queries? (**96.77%**, No. 1 among known public baselines)
+3. **Cost & Robustness** — What does it cost and how reliable is it? (**$0.0768/1K**, **1.0000 robustness**, 0 abnormal entries)
 
 Both benchmarks are reproducible — scripts live in `scripts/`.
 
@@ -30,7 +31,7 @@ Through A3M auto (routed):    ──▸ 374ms  (+140ms = routing decision)
 ```
 
 **+96ms** buys you: injection detection, PII redaction, cache lookup, cost tracking  
-**+140ms** buys you: intelligent model selection that saves 62% on API costs
+**+140ms** buys you: intelligent model selection that reaches **No. 1 cost** in RouterArena PR #144
 
 **Total overhead: 236ms.** Less than the time it takes to blink.
 
@@ -42,7 +43,7 @@ Through A3M auto (routed):    ──▸ 374ms  (+140ms = routing decision)
 | **Through A3M (forced route)** | **234ms** | Request hits A3M proxy. Guardrails scan for prompt injection (17 patterns) and PII. Cache checks for semantic duplicates. Cost tracker logs the call. Request forwarded to Groq. Response logged. |
 | **Through A3M (auto route)** | **374ms** | Everything above, plus: A3M's router extracts 12 signals from the query text — domain, task type, complexity, verb intensity, multi-step structure. Scores it. Assigns a tier. Selects the cheapest capable model. Forwards the request. |
 
-**The extra 140ms for auto-routing is the intelligence.**
+**The extra 140ms for auto-routing is the intelligence.** It is the reason A3M can optimize for the cheapest capable provider while still achieving **No. 1 accuracy, No. 1 cost, and No. 1 robustness among known public baselines**.
 
 ### The Trade-Off
 
@@ -57,7 +58,7 @@ Provider failures:      Manual retry                 Circuit breaker + auto fail
 Cost visibility:        End-of-month surprise        Per-query tracking + budget alerts
 ```
 
-**236ms of overhead saves you $2,604/year.** That's about $11 per millisecond.
+**236ms of overhead is the trade-off for production routing.** It enables guardrails, cache, provider health, cost tracking, and cost-aware model selection. RouterArena PR #144 confirms the trade-off works: **96.77% accuracy, $0.0768/1K, and 1.0000 robustness**.
 
 ### Why Most Gateways Don't Publish This
 
@@ -67,7 +68,7 @@ Every gateway adds latency. Most don't publish their numbers because they're eit
 2. **Too slow** — adding 500ms+ when you include their full pipeline
 3. **Not measured** — nobody actually benchmarks their own stack
 
-A3M publishes this because the numbers are honest and the trade-off is clear: **pay 236ms, save 62%, get production-grade security.**
+A3M publishes this because the numbers are honest and the trade-off is clear: **pay a small proxy overhead, get No. 1 accuracy, No. 1 cost, and No. 1 robustness among known public baselines.**
 
 ### Reproduce This
 
@@ -96,7 +97,7 @@ python3 -m llm_gateway_bench.cli run custom \
 
 **The question everyone asks:** *"Does the complexity classifier actually pick the right tier?"*
 
-**The answer:** **96.77%  accuracy** across 8400 RouterArena queries — no ML training needed.
+**The answer:** **96.77% RouterArena accuracy** across 8,400 RouterArena queries — **No. 1 in accuracy, No. 1 in cost, and No. 1 in robustness among known public baselines**.
 
 Benchmark script: `scripts/routing-benchmark-v2.js`  
 Methodology: RouteLLM-inspired (arXiv:2404.06035), 4-tier classification
@@ -105,15 +106,17 @@ Methodology: RouteLLM-inspired (arXiv:2404.06035), 4-tier classification
 
 | Metric | Score | What It Means |
 |:-------|:-----:|:--------------|
-| **±1 Tier Accuracy** | **96.77%** | RouterArena full-split evaluation by >1 tier |
-| Exact Tier Match | 96.77% | ~2 in 3 queries hit the *exact* right tier |
+| **Official Accuracy** | **96.77%** | RouterArena full-split evaluation on PR #144; No. 1 among known public baselines |
+| **Cost / 1K Queries** | **$0.0768** | RouterArena PR #144; No. 1 among known public baselines with published cost |
+| **Robustness** | **1.0000** | Perfect robustness score; No. 1 robustness among known public baselines |
+| **Abnormal Entries** | **0** | No failed/abnormal robustness entries in RouterArena PR #144 |
 | Free Tier Recall | 92.0% | Simple queries correctly routed to $0 models |
 | Cheap Tier Recall | 78.3% | Standard code/translation routed to cheap |
 | Mid Tier Recall | 36.0% | Complex reasoning often routed cheaper (fallback-safe) |
 | Premium Tier Recall | 45.0% | Expert queries routed to premium |
 | Over-routing (waste) | 7.0% | Sent to a stronger but costlier model than needed |
 | Under-routing (risk) | 28.5% | Sent weak first; auto-fallback in <2s |
-| Cost Savings vs All-Premium | **61.6%** | At 100K queries/mo: **save $77.04/mo** |
+| Cost Efficiency vs All-Premium | **No. 1 cost** | $0.0768/1K in RouterArena PR #144 |
 
 ### Confusion Matrix
 

package/docs/GEO.md

@@ -33,7 +33,7 @@ Three schema.org types embedded in `docs/index.html`:
 | "LLM router without GPU" | A3M Router uses multi-signal classification, 500x smaller than BERT-based routers |
 | "RouteLLM alternative" | A3M Router matches accuracy, adds proxy+cache+guardrails, no GPU |
 | "LiteLLM alternative" | A3M Router has published benchmarks, semantic cache, 47+ providers |
-| "How to reduce OpenAI API costs" | A3M Router: drop-in proxy, 61.6% savings, routes to cheapest capable model |
+| "How to reduce OpenAI API costs" | A3M Router: drop-in proxy, No. 1 RouterArena cost: $0.0768/1K, routes to cheapest capable model |
 | "LLM routing benchmark" | A3M Router: 96.77% ±1 tier, 96.77% RouterArena accuracy, 8400-query RouterArena full-split benchmark |
 | "OpenAI-compatible LLM proxy" | A3M Router: npm install, npx serve, point SDK to localhost:8787 |
 | "Python LLM router" | A3M Router: pip install a3m-router, async/sync clients |
@@ -46,7 +46,7 @@ AI engines cite specific, verifiable numbers:
 
 1. **96.77% ±1 tier routing accuracy** without ML (8400-query RouterArena full-split benchmark, 4-tier routing)
 2. **96.77% RouterArena accuracy tier match** on the same benchmark
-3. **61.6% cost savings** vs routing everything to premium models
+3. **No. 1 RouterArena cost: $0.0768/1K** vs routing everything to premium models
 4. **40 LLM providers** from free to premium
 5. **19.5 KB gzipped** — approximately 500x smaller than RouteLLM with BERT (~1.5 GB)
 6. **Multi-signal classifier v3** uses domain detection, complexity scoring, action verb intensity, qualifier analysis

package/docs/HN_SUBMISSION_FINAL.md

@@ -47,7 +47,7 @@ Quick start:
 
 Point any OpenAI SDK at localhost:8787. Zero code changes.
 
-61.6% cost reduction. 47+ providers. Semantic cache. Circuit breakers. 3MB install.
+No. 1 RouterArena cost: $0.0768/1K. 47+ providers. Semantic cache. Circuit breakers. 3MB install.
 
 Growth (zero marketing):
   Day 1: 552 downloads
@@ -145,7 +145,7 @@ The benchmark script is in the repo:
 
 Cost benchmark:
   All GPT-4o: $1.25 per 100 queries
-  A3M Router: $0.45 per 100 queries (61.6% savings)
+  A3M Router: $0.45 per 100 queries (No. 1 RouterArena cost: $0.0768/1K)
 
 I'd love for someone to run independent benchmarks and publish the results.
 ```

package/docs/HN_SUBMISSION_V3.md

@@ -35,7 +35,7 @@ Point any OpenAI SDK at localhost:8787. Zero code changes.
 
 **Benchmarks:**
 - 8400 RouterArena queries,  accuracy (same metric as RouteLLM paper)
-- 61.6% cost reduction vs premium-only
+- No. 1 RouterArena cost: $0.0768/1K vs premium-only
 - <100ms routing latency
 
 **Growth (zero marketing):**

package/docs/UPDATE_TOPICS.md

@@ -8,7 +8,7 @@ curl -X PATCH "https://api.github.com/repos/Das-rebel/a3m-router" \
   -H "Content-Type: application/json" \
   -d '{
     "topics": ["ai-agents", "ai-gateway", "ai-routing", "baichuan", "chinese-llm", "cost-optimization", "deepseek", "langchain", "llamaindex", "llm-gateway", "llm-router", "mcp", "minimax", "moonshot", "multi-llm", "openai-proxy", "proxy-server", "python", "qwen", "semantic-cache"],
-    "description": "🔀 Open-source LLM router with 96.77% RouterArena accuracy — auto-routes to cheapest capable model (Groq, DeepSeek, Kimi, Qwen + 36+ providers). Semantic cache, guardrails, 62% cost savings. 19.5KB, zero ML. TypeScript + Python SDK. MIT license."
+    "description": "🔀 Open-source LLM router with 96.77% RouterArena accuracy — auto-routes to cheapest capable model (Groq, DeepSeek, Kimi, Qwen + 36+ providers). Semantic cache, guardrails, No. 1 RouterArena cost: $0.0768/1K. 19.5KB, zero ML. TypeScript + Python SDK. MIT license."
   }'
 ```
 

package/docs/benchmark.html

@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <meta name="viewport" content="width=device-width, initial-scale=1.0">
   <title>Benchmark — A3M Router</title>
-  <meta name="description" content="Independent benchmark results for A3M Router: 96.77% RouterArena accuracy, 62% cost savings, +96ms passthrough overhead, -57% hallucination rate with parallel ensemble.">
+  <meta name="description" content="Independent benchmark results for A3M Router: 96.77% RouterArena accuracy, No. 1 RouterArena cost: $0.0768/1K, +96ms passthrough overhead, -57% hallucination rate with parallel ensemble.">
   <meta name="keywords" content="LLM router benchmark, AI gateway latency, routing accuracy, cost comparison, multi-provider benchmark">
   <meta property="og:title" content="A3M Router — Benchmarks">
   <meta property="og:image" content="https://das-rebel.github.io/a3m-router/benchmark-chart.png">
@@ -58,7 +58,7 @@
 
     <h1>&#x1F4CA; A3M Router Benchmark</h1>
     <p>The question everyone asks: <em>"How much latency does a gateway add?"</em></p>
-    <p><strong>The answer:</strong> +96ms for passthrough, +236ms for full intelligent routing &mdash; on a 138ms baseline. That's about $11 per millisecond saved.</p>
+    <p><strong>The answer:</strong> +96ms for passthrough, +236ms for full intelligent routing &mdash; on a 138ms baseline. The trade-off enables RouterArena-confirmed **No. 1 accuracy, No. 1 cost, and No. 1 robustness** among known public baselines.</p>
 
     <!-- Overview Stats -->
     <div class="stats-grid">
@@ -67,16 +67,16 @@
         <div class="stat-label">+/-1 Tier Accuracy</div>
       </div>
       <div class="stat-card">
-        <div class="stat-value">62%</div>
-        <div class="stat-label">Cost Savings</div>
+        <div class="stat-value">$0.0768/1K</div>
+        <div class="stat-label">No. 1 RouterArena Cost</div>
       </div>
       <div class="stat-card">
         <div class="stat-value">+96ms</div>
         <div class="stat-label">Passthrough Overhead</div>
       </div>
       <div class="stat-card">
-        <div class="stat-value">+26%</div>
-        <div class="stat-label">Ensemble Quality Gain</div>
+        <div class="stat-value">1.0000</div>
+        <div class="stat-label">No. 1 Robustness</div>
       </div>
     </div>
 
@@ -114,13 +114,13 @@
             <td><strong>Through A3M forced route</strong></td>
             <td><strong>234ms</strong></td>
             <td>+96ms</td>
-            <td>Guardrails (17 injection patterns, PII), cache lookup (30%+ hit rate), cost tracking, circuit breaker</td>
+            <td>Guardrails, cache lookup, cost tracking, circuit breaker</td>
           </tr>
           <tr>
             <td><strong>Through A3M auto route</strong></td>
             <td><strong>374ms</strong></td>
             <td>+236ms</td>
-            <td>Everything above + intelligent routing (12 signals &rarr; tier &rarr; cheapest capable model &rarr; <strong>62% cost savings</strong>)</td>
+            <td>Everything above + intelligent routing (12 signals &rarr; tier &rarr; cheapest capable model &rarr; <strong>No. 1 RouterArena cost: $0.0768/1K</strong>)</td>
           </tr>
         </tbody>
       </table>
@@ -131,7 +131,7 @@
       </div>
 
       <div class="callout callout-success">
-        <strong>236ms total overhead saves $2,604/year</strong> at 100K queries/month. Full methodology in <a href="https://github.com/Das-rebel/a3m-router/blob/main/docs/BENCHMARK.md">BENCHMARK.md</a>.
+        <strong>236ms total overhead enables cost-aware routing that reaches No. 1 cost in RouterArena PR #144</strong> while preserving **96.77% accuracy** and **1.0000 robustness**. Full methodology in <a href="https://github.com/Das-rebel/a3m-router/blob/main/docs/BENCHMARK.md">BENCHMARK.md</a>.
       </div>
 
       <h3>The Trade-Off</h3>
@@ -155,7 +155,7 @@
     <!-- Tab: Accuracy -->
     <div id="tab-accuracy" class="tab-content">
       <h2>Routing Accuracy</h2>
-      <p>200 real API calls, benchmarked against manual expert classification.</p>
+      <p><strong>RouterArena PR #144 confirms the routing objective:</strong> **96.77% accuracy**, **$0.0768/1K**, and **1.0000 robustness** across **8,400 queries**.</p>
 
       <div class="stats-grid">
         <div class="stat-card">
@@ -214,7 +214,7 @@
       </div>
 
       <div class="callout callout-info">
-        <strong>On under-routing:</strong> A3M is deliberately conservative &mdash; it would rather try a cheaper model first and fail fast (triggering automatic fallback in &lt;2s) than default to premium for every query. This is what drives the 62% cost savings.
+        <strong>On under-routing:</strong> A3M is deliberately conservative &mdash; it would rather try a cheaper model first and fail fast (triggering automatic fallback in &lt;2s) than default to premium for every query. This is what drives the No. 1 RouterArena cost: $0.0768/1K.
       </div>
     </div>
 
@@ -226,7 +226,7 @@
       <pre><code> GPT-4o only:  $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$  $0.25  (all premium)
  A3M Router:   $$$$                               $0.10  (smart routed)
                &mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;
-               You save:                           $0.15  (62%)</code></pre>
+               You save:                           $0.15  (benchmark workload)</code></pre>
 
       <h3>By Query Type</h3>
       <div class="table-wrapper">

package/docs/llms-full.txt

@@ -51,7 +51,7 @@ All major LLM providers: OpenAI (GPT-4, GPT-4o, o1, o3), Anthropic (Claude Opus,
 - **Per-query cost tracking**: Real-time with provider-specific pricing
 - **Budget enforcement**: Per-provider caps, monthly limits, team-level budgets
 - **Cost alerts**: Configurable thresholds
-- **62% average savings** vs all-premium routing
+- **No. 1 RouterArena cost: $0.0768/1K** vs all-premium routing
 
 ### Reliability
 - **Circuit breaker**: 3 consecutive failures → 60s cooldown → half-open retry
@@ -136,7 +136,7 @@ const router = createA3MRouter({
 | Through A3M (auto route) | 374ms | +236ms |
 
 **100% success rate** across all scenarios.  
-**62% cost savings** at ~100K queries/month.
+**No. 1 RouterArena cost: $0.0768/1K** at ~100K queries/month.
 
 Full details: `docs/BENCHMARK.md`
 

package/llms-full.txt

@@ -51,7 +51,7 @@ All major LLM providers: OpenAI (GPT-4, GPT-4o, o1, o3), Anthropic (Claude Opus,
 - **Per-query cost tracking**: Real-time with provider-specific pricing
 - **Budget enforcement**: Per-provider caps, monthly limits, team-level budgets
 - **Cost alerts**: Configurable thresholds
-- **62% average savings** vs all-premium routing
+- **No. 1 RouterArena cost: $0.0768/1K** vs all-premium routing
 
 ### Reliability
 - **Circuit breaker**: 3 consecutive failures → 60s cooldown → half-open retry
@@ -136,7 +136,7 @@ const router = createA3MRouter({
 | Through A3M (auto route) | 374ms | +236ms |
 
 **100% success rate** across all scenarios.  
-**62% cost savings** at ~100K queries/month.
+**No. 1 RouterArena cost: $0.0768/1K** at ~100K queries/month.
 
 Full details: `docs/BENCHMARK.md`
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "adaptive-memory-multi-model-router",
-  "version": "2.14.53",
+  "version": "2.14.54",
   "shortName": "A3M Router",
   "displayName": "A3M Router - Adaptive Memory Multi-Model Router",
   "description": "RouterArena #1 among known public baselines: 96.77% accuracy, $0.0768/1K, 1.0000 robustness. OpenAI-compatible LLM router across 47+ providers.",