adaptive-memory-multi-model-router 2.14.49 → 2.14.52

package/docs/ROUTING_RUBRIC.md

@@ -0,0 +1,197 @@
+# A3M Router — Routing Quality Rubric
+
+Five dimensions, each measured against real evidence from production routing data. The composite score drives the pulse metric and surfaces where routing quality degrades.
+
+## Formula
+
+```
+composite_score = 0.30 × RoutingAccuracy
+                + 0.25 × CostEfficiency
+                + 0.20 × Latency
+                + 0.15 × ErrorHandling
+                + 0.10 × CacheHitRate
+```
+
+**Weight justification:**
+- **30% Accuracy** — Getting the right provider for the right query is the primary function. Everything else is secondary.
+- **25% Cost Efficiency** — The core value proposition. If accuracy is perfect but costs are high, we failed at the value prop.
+- **20% Latency** — Developer experience. A router that's slow gets bypassed regardless of accuracy.
+- **15% Error Handling** — Reliability under provider failures. Matters most in production.
+- **10% Cache Hit Rate** — Bonus optimization. Only matters at scale.
+
+---
+
+## 1. Routing Accuracy (30%)
+
+*"Did the router send the query to the right tier?"*
+
+### Scoring
+
+| Score | Criterion |
+|-------|-----------|
+| 90-100 | >95% within ±1 tier. RouterArena score above 70. Fewer than 1 in 20 queries misrouted by more than one tier. |
+| 75-89 | 85-95% within ±1 tier. RouterArena score 60-70. Occasional over-tiering on simple queries. |
+| 60-74 | 70-85% within ±1 tier. RouterArena score 50-60. Noticeable over-tiering on medium queries. |
+| 45-59 | 50-70% within ±1 tier. Frequent misrouting on complex/expert queries. |
+| <45 | <50% within ±1 tier. Router is essentially random. Major overhaul needed. |
+
+### Evidence to capture
+
+- **RouteLLM comparison** — where RouteLLM routes vs A3M (reference benchmark)
+- **Tier confusion matrix** — which query types cause the most over/under-tiering
+- **RouterArena score** — the single-number benchmark (current: 70.32)
+- **Golden route deviation** — percentage of queries where A3M disagrees with golden route
+
+### Common failure patterns
+
+| Pattern | Fix |
+|---------|-----|
+| All queries go to free tier (0% to mid/premium) | Add confidence floor. If no provider has confidence > 0.5, fallback to premium |
+| Code queries misrouted to creative models | Strengthen code-detection signals (``` blocks, function syntax) |
+| Legal/medical routed to cheap models | Add domain detection for 5 safety-critical domains |
+| Ambiguous queries bounce between tiers | Implement query-type confidence threshold |
+
+### Dollar Impact
+
+```
+Wasted = (MismatchCount × AvgCostDelta) 
+AvgCostDelta = |ActualCost - OptimalCost|
+```
+
+---
+
+## 2. Cost Efficiency (25%)
+
+*"Did the router save money compared to all-premium routing?"*
+
+### Scoring
+
+| Score | Savings vs All-Premium | CPP (Cost Per Query) |
+|-------|----------------------|---------------------|
+| 90-100 | >70% savings | <$0.001/query |
+| 75-89 | 50-70% savings | $0.001-$0.003/query |
+| 60-74 | 30-50% savings | $0.003-$0.006/query |
+| 45-59 | 15-30% savings | $0.006-$0.01/query |
+| <45 | <15% savings | >$0.01/query |
+
+### Evidence to capture
+
+- **Cost per query** over the measurement window
+- **Savings vs all-premium** — total cost if every query went to GPT-4o
+- **Free tier utilization** — % of queries handled by free/cheap providers
+- **Budget cap hits** — how often budget enforcement is triggered
+- **Provider cost breakdown** — cost per provider
+
+### Common failure patterns
+
+| Pattern | Fix |
+|---------|-----|
+| Everything routes to free (0% accuracy) | Add quality floor to cost optimization |
+| Budget cap tripped too often | Increase budget cap or reduce free-tier usage |
+| Premium providers selected for trivial queries | Lower confidence threshold for mid-tier |
+
+### Dollar Impact
+
+```
+Savings = (TotalQueryCount × AvgPremiumCost) - ActualTotalCost
+MonthlySavings = Savings × (30 / MeasurementDays)
+```
+
+---
+
+## 3. Latency (20%)
+
+*"How fast is the router decision?"*
+
+### Scoring (P95 Latency)
+
+| Score | P95 Latency | Overhead vs Direct |
+|-------|------------|-------------------|
+| 90-100 | <200ms | <50ms overhead |
+| 75-89 | 200-500ms | 50-100ms overhead |
+| 60-74 | 500-1000ms | 100-200ms overhead |
+| 45-59 | 1-3s | 200-500ms overhead |
+| <45 | >3s | >500ms overhead |
+
+### Evidence to capture
+
+- **P50, P95, P99 latency** — distribution
+- **Routing decision overhead** — time spent in routing logic vs provider response
+- **Slowest providers** — top 5 by latency
+- **Cache response time** — cached vs uncached query time
+
+---
+
+## 4. Error Handling (15%)
+
+*"How well does the router handle failures?"*
+
+### Scoring
+
+| Score | Criterion |
+|-------|-----------|
+| 90-100 | 0 unhandled failures. All provider failures caught by circuit breaker. Graceful fallback 100% of the time. |
+| 75-89 | <1% unhandled failures. Circuit breaker catches most issues. Fallback succeeds >95%. |
+| 60-74 | 1-3% unhandled failures. Occasional circuit breaker misses. Fallback succeeds >80%. |
+| 45-59 | 3-10% unhandled failures. Circuit breaker coverage gaps. Fallback degrades. |
+| <45 | >10% unhandled failures. Critical reliability issues. |
+
+### Evidence to capture
+
+- **Circuit breaker trips** — how many times each provider was disabled
+- **Fallback success rate** — % of attempts where fallback succeeded
+- **Unhandled failures** — queries that returned no response
+- **Provider health score** — current health of each provider
+
+### Common failure patterns
+
+| Pattern | Fix |
+|---------|-----|
+| Circuit breaker never fires (wasteful retries) | Lower threshold for circuit breaker trip |
+| Circuit breaker fires too often | Increase threshold, add validation before trip |
+| All providers fail simultaneously | Add cold-start provider as emergency fallback |
+
+---
+
+## 5. Cache Hit Rate (10%)
+
+*"How often does semantic cache avoid a duplicate provider call?"*
+
+### Scoring
+
+| Score | Cache Hit Rate |
+|-------|---------------|
+| 90-100 | >40% |
+| 75-89 | 30-40% |
+| 60-74 | 20-30% |
+| 45-59 | 10-20% |
+| <45 | <10% |
+
+### Evidence to capture
+
+- **Global cache hit rate** — across all queries
+- **Per-query-type cache rate** — which query types benefit most
+- **Cache latency savings** — total time saved by cache hits
+- **Cache cost savings** — how much money cache saved
+
+---
+
+## Composite Score Bands
+
+| Band | Score | Meaning |
+|------|-------|---------|
+| 🟢 Excellent | 85-100 | Production-ready. Fine-tune edge cases. |
+| 🟡 Good | 70-84 | Working well. Some optimization opportunities. |
+| 🟠 Fair | 55-69 | Functional but needs attention. |
+| 🔴 Poor | 40-54 | Quality issues. Investigate root cause. |
+| ⚫ Critical | <40 | Router needs significant work. |
+
+## Usage
+
+Calculate after every 100 queries or at least once per week:
+
+```bash
+a3m-router metrics          # Quick pulse
+a3m-router metrics --full   # Full rubric with all dimensions
+a3m-router metrics --export  # Raw JSON for analysis
+```

package/docs/SEO_AUDIT.md

@@ -0,0 +1,186 @@
+# SEO Audit: A3M Router (adaptive-memory-multi-model-router)
+
+**Date:** 2026-05-18 (Updated)
+**Package:** adaptive-memory-multi-model-router
+**NPM URL:** https://www.npmjs.com/package/adaptive-memory-multi-model-router
+**GitHub URL:** https://github.com/Das-rebel/a3m-router
+
+---
+
+## 1. Keyword Research
+
+### Primary Keywords (benchmark-driven, high intent)
+
+| Keyword | Est. Monthly Volume | Competition | Intent | Priority |
+|---------|---------------------|-------------|--------|----------|
+| `llm router benchmark` | 1,200-2,000 | Low | Commercial | P0 |
+| `llm routing accuracy` | 800-1,500 | Low | Informational | P0 |
+| `routellm alternative` | 1,500-3,000 | Low-Medium | Commercial | P0 |
+| `litellm alternative` | 1,500-3,000 | Low-Medium | Commercial | P0 |
+| `llm cost optimization` | 800-1,500 | Low | Commercial | P0 |
+| `openai proxy free` | 2,000-4,000 | Medium | Transactional | P0 |
+| `llm gateway open source` | 1,000-2,000 | Low-Medium | Commercial | P0 |
+
+### Long-Tail Keywords (FAQ/content targets)
+
+| Keyword | Est. Monthly Volume | Competition | Intent | Priority |
+|---------|---------------------|-------------|--------|----------|
+| `how to reduce openai api costs` | 1,500-3,000 | Low | Informational | P0 |
+| `llm routing without gpu` | 300-600 | Very Low | Informational | P0 |
+| `lightweight llm router` | 500-1,000 | Low | Commercial | P0 |
+| `keyword-based llm routing` | 100-300 | Very Low | Informational | P1 |
+| `drop-in openai proxy` | 300-600 | Low | Commercial | P0 |
+| `free llm proxy` | 800-1,500 | Low | Transactional | P0 |
+| `cheapest openai api alternative` | 500-1,000 | Low | Commercial | P0 |
+
+### Competitive/Comparison Keywords (HIGH VALUE)
+
+| Keyword | Est. Monthly Volume | Competition | Priority |
+|---------|---------------------|-------------|----------|
+| `routellm alternative` | 1,500-3,000 | Low-Medium | P0 |
+| `litellm alternative` | 1,500-3,000 | Low-Medium | P0 |
+| `a3m router vs litellm` | 50-100 | Very Low | P1 |
+| `a3m router vs routellm` | 50-100 | Very Low | P1 |
+| `openrouter alternative` | 200-400 | Low | P1 |
+| `portkey alternative` | 100-200 | Very Low | P2 |
+
+### Secondary Keywords
+
+| Keyword | Est. Monthly Volume | Competition | Priority |
+|---------|---------------------|-------------|----------|
+| `ai gateway` | 5,000-8,000 | High | P1 |
+| `model routing` | 500-1,000 | Low | P1 |
+| `llm proxy` | 1,000-2,000 | Low-Medium | P1 |
+| `openai compatible proxy` | 500-1,000 | Low | P1 |
+| `llm load balancer` | 300-800 | Low | P1 |
+| `llm provider comparison` | 1,000-2,000 | Medium | P1 |
+
+---
+
+## 2. Key Messages (use everywhere)
+
+1. **"82.5% routing accuracy without ML"** — Lead metric, differentiator
+2. **"Matches RouteLLM BERT within 2.5%"** — Competitive positioning
+3. **"30x more efficient than GPU-based routing"** — Efficiency story
+4. **"Only router besides RouteLLM with published benchmarks"** — Trust signal
+5. **"245% growth, 2,775 downloads in 3 days"** — Social proof
+
+---
+
+## 3. Competitive Positioning
+
+### RouteLLM Alternative (HIGH VALUE)
+
+"RouteLLM alternative" is our highest-value keyword because:
+- RouteLLM users are actively looking for alternatives (GPU cost, complexity)
+- We have a direct benchmark comparison (within 2.5%)
+- We offer features RouteLLM lacks (proxy, cache, guardrails)
+
+**Positioning:** "A3M Router matches RouteLLM BERT within 2.5% — without GPU. Plus proxy, cache, guardrails."
+
+### LiteLLM Alternative (HIGH VALUE)
+
+"LiteLLM alternative" captures users who want:
+- Published routing benchmarks
+- Zero-config setup
+- Built-in semantic caching
+
+**Positioning:** "A3M Router is the only LiteLLM alternative with published routing benchmarks (82.5% accuracy)."
+
+### Competitive Table
+
+| Competitor | NPM Weekly Downloads | Our Edge |
+|------------|---------------------|----------|
+| litellm | ~80,000 | Published benchmarks, zero-config, semantic cache |
+| openrouter-sdk | ~5,000 | Self-hosted, no middleman fees, published accuracy |
+| portkey-ai | ~3,000 | Open-source, free, no signup, benchmarks |
+| routellm | ~1,000 | No GPU needed, proxy included, 39 providers |
+
+---
+
+## 4. On-Page SEO Checklist
+
+### docs-site/index.html
+
+| Element | Status | Target |
+|---------|--------|--------|
+| Title tag | UPDATED | "A3M Router — 82.5% Routing Accuracy Without ML \| Matches RouteLLM" |
+| Meta description | UPDATED | 30x efficiency story with accuracy metric |
+| Keywords meta | UPDATED | All 12 primary/long-tail keywords |
+| H1 tag | UPDATED | "LLM Routing That Matches GPU Models — Without GPU" |
+| Stats section | UPDATED | Leads with 82.5% accuracy, 2.5% gap, 30x efficiency |
+| FAQ schema | UPDATED | 8 questions targeting AI search queries |
+| OG tags | UPDATED | Benchmark-first messaging |
+| Twitter cards | UPDATED | Benchmark-first messaging |
+
+### Content Structure (H-tag hierarchy)
+
+```
+H1: LLM Routing That Matches GPU Models — Without GPU
+H2: Intelligent LLM Routing (feature)
+H2: Cost Optimization (feature)
+H2: Smart Fallback & Retry (feature)
+H2: Real-time Analytics (feature)
+H2: Security Guardrails (feature)
+H2: Semantic Cache (feature)
+H2: LLM Provider Pricing Tiers (section)
+H3: Free/Budget/Mid/Premium Tier
+H2: Quick Start: LLM Routing in 30 Seconds
+H2: Frequently Asked Questions
+H3: What is LLM routing accuracy?
+H3: How does keyword-based routing compare to ML routing?
+H3: What is the best lightweight LLM router?
+H3: How to reduce OpenAI API costs?
+H3: How does A3M Router compare to RouteLLM?
+H3: How does A3M Router compare to LiteLLM?
+```
+
+---
+
+## 5. Technical SEO
+
+### robots.txt (UPDATED)
+- Allows full crawling
+- Explicitly allows docs/, assets/, llms.txt, README.md
+- Sitemap reference included
+- Blocks /node_modules/, /dist/, /test/, /src/, /.git/
+
+### sitemap.xml (UPDATED)
+- 11 URLs including all key pages
+- New: GEO.md, SEO_AUDIT.md, CONFIGURATION.md, INTEGRATIONS.md, benchmark-results.json, llms.txt
+- Priority weighting: homepage (1.0) > GitHub (0.9) > NPM (0.9) > docs (0.7-0.8)
+
+### llms.txt (UPDATED)
+- Leads with benchmark story (82.5% accuracy)
+- Includes comparison table vs RouteLLM/LiteLLM
+- Structured data section for AI extraction
+- All 5 key messages included
+
+---
+
+## 6. GEO (Generative Engine Optimization)
+
+See `docs/GEO.md` for full GEO strategy. Key elements:
+
+1. **FAQ format** answering AI-searchable questions
+2. **Comparison tables** with verifiable data AI engines cite
+3. **Structured key-value block** for direct AI extraction
+4. **Target AI queries** mapped to A3M Router answers
+
+---
+
+## 7. Action Items
+
+- [x] Update docs-site/index.html title, meta, H1, stats, FAQ
+- [x] Update FAQ schema with benchmark-focused questions
+- [x] Update OG/Twitter cards with benchmark messaging
+- [x] Update llms.txt with benchmark story
+- [x] Create docs/GEO.md with AI search optimization
+- [x] Update docs/SEO_AUDIT.md with new keywords
+- [x] Update public/sitemap.xml with all key pages
+- [x] Update public/robots.txt with better crawling rules
+- [x] Update package.json keywords (optimized)
+- [ ] Create OG banner image with benchmark metrics
+- [ ] Write comparison articles (A3M vs RouteLLM, vs LiteLLM)
+- [ ] Submit sitemap to Google Search Console
+- [ ] Set up Google Search Console for das-rebel.github.io

package/docs/SOCIAL_LISTENING.md

@@ -0,0 +1,219 @@
+# A3M Router — Social Listening & Reply Playbook
+
+> "Set up Google Alerts for competitors → find discussions about routing/cost → craft reply that converts"
+> — Vault insight, score 29.3
+
+## 1. Monitoring Setup
+
+### Google Alerts (free)
+Set up alerts for these keywords. Frequency: "As it happens."
+
+| Alert | Keyword | Why |
+|-------|---------|-----|
+| **A** | `"LLM routing" OR "model routing"` | Direct mention of the space |
+| **B** | `"AI gateway" OR "LLM gateway"` | Competitor category |
+| **C** | `"LiteLLM" OR "portkey" OR "route LLM"` | Competitor names |
+| **D** | `"switch between LLMs" OR "multi-model"` | Pain point search |
+| **E** | `"LLM too expensive" OR "API costs"` | Pain point — cost |
+| **F** | `"open source LLM router"` | Direct search intent |
+
+### F5bot (free tier)
+Monitor Hacker News for:
+- `llm router` `model routing` `ai gateway` `openrouter` `litellm` `route llm`
+
+Setup: https://f5bot.com — enter keywords, get email alerts.
+
+### ReplyGuy (paid, ~$15/mo)
+Automated reply system for Reddit, HN, X, YouTube:
+- Monitors keywords across platforms
+- Suggests replies that are contextual, not spammy
+- Setup: https://replyguy.com
+
+### Morebirds (HN specific)
+Monitors Hacker News for keywords and competitors:
+- https://morebirds.com
+
+---
+
+## 2. Keywords to Monitor (Priority Order)
+
+### Tier 1: High Intent (reply immediately)
+```
+"llm routing" "model routing" "ai gateway" "llm gateway" "routing llm"
+"open source router" "llm router alternative" "best model router"
+"switch between models" "model fallback" "intelligent routing"
+```
+
+### Tier 2: Pain Point (reply with empathy + solution)
+```
+"llm costs too high" "gpt too expensive" "api cost too much" 
+"openai bill too high" "reduce llm costs" "need cheaper llm"
+"paying too much for ai" "api costs killing me"
+```
+
+### Tier 3: Competitor (reply as comparison, not attack)
+```
+"LitellLm" "Portkey" "RouteLLM" "OpenRouter" "Helicone" 
+"LangSmith" "AI Gateway (Cloudflare)" "GenAI Gateway"
+```
+When someone has issues with these, offer A3M as a working alternative.
+
+### Tier 4: Educational (reply with knowledge, mention tool naturally)
+```
+"how to choose llm" "which model to use" "compare models"
+"llm benchmark" "model evaluation" "provider comparison"
+```
+
+---
+
+## 3. Reply Templates
+
+### Template 1: Cost Pain
+**Trigger:** Someone says "my OpenAI bill is too high" or "LLM API costs are crazy"
+
+**Platform:** HN, Reddit, X
+
+**Reply:**
+```
+We were in the same boat — $800/month on GPT-4. Built A3M Router to route smart queries to cheaper models and keep hard ones on premium.
+
+Same answers (RouterArena #1 at 70.32). Cost dropped to ~$5.
+
+Open source, MIT. Run it yourself:
+  npx a3m-router route "your query"
+
+Or verify: npx a3m-router benchmark --reproducible
+```
+
+### Template 2: "Which tool should I use?"
+**Trigger:** Someone comparing LiteLLM, Portkey, RouteLLM
+
+**Platform:** Reddit, HN
+
+**Reply:**
+```
+We evaluated all of them before building A3M. What we wanted:
+- Open source (MIT)
+- Actually routes to cheapest capable model, not just round-robin
+- Works with existing OpenAI SDK
+- Has a reproducible benchmark
+
+A3M Router hits all of those. #1 on RouterArena (70.32). Costs $0.047/1K vs GPT-5 at $10/1K.
+
+npx a3m-router route "test it out"
+```
+
+### Template 3: "Building an AI gateway"
+**Trigger:** Someone asks how to build multi-model routing, or shows their architecture
+
+**Platform:** HN, Reddit, DevTo
+
+**Reply:**
+```
+We built something similar. Spent months on it. Eventually open-sourced it as A3M Router.
+
+Biggest lessons:
+1. Cost-based routing saves 200x vs always-pick-premium
+2. Quality scores persist across sessions (memory)
+3. Cache + parallel execution cut latency 3x
+
+The whole thing is MIT on GitHub. Beats GPT-5 on RouterArena.
+
+npx a3m-router benchmark --reproducible
+```
+
+### Template 4: "My provider is rate-limiting / failing"
+**Trigger:** Someone complaining about OpenAI/Groq/Claude rate limits or failures
+
+**Platform:** HN, Reddit, X
+
+**Reply:**
+```
+A3M Router handles this automatically — fallback to next available provider when one fails or throttles.
+
+47+ providers. Automatic failover. Same response format.
+
+Open source: npx a3m-router route "try it"
+```
+
+### Template 5: "Looking for alternatives"
+**Trigger:** Someone asking for alternatives to a specific tool or service
+
+**Platform:** HN, Reddit, X
+
+**Reply:**
+```
+If you're evaluating options, A3M Router is worth a look:
+- MIT licensed (not source-available)
+- RouterArena #1 (70.32)
+- Same API as OpenAI SDK
+- $0.047/1K vs $10/1K for GPT-5
+
+npx a3m-router route "test" or npx a3m-router benchmark --reproducible
+```
+
+### Template 6: "Model comparison question"
+**Trigger:** Someone asking which model is best for task X
+
+**Platform:** HN, Reddit
+
+**Reply:**
+```
+A3M Router actually solves this — it routes each query to the best model based on: complexity, cost budget, latency needs, and past quality scores.
+
+You define 47+ providers and it picks automatically. Results tracked in memory so it gets smarter over time.
+
+npx a3m-router recommend "coding"   # See what it would pick
+npx a3m-router route "test it"       # Route a real query
+```
+
+### Template 7: Show HN / Launches (competitor)
+**Trigger:** A competitor launches on HN or Product Hunt
+
+**Platform:** HN comments
+
+**Reply:**
+```
+Cool project! Curious how it compares on RouterArena. We got 70.32 — would love to see benchmarks head-to-head.
+
+For anyone evaluating, A3M Router is open source (MIT) with a reproducible benchmark:
+npx a3m-router benchmark --reproducible
+```
+
+---
+
+## 4. Cadence
+
+| Frequency | Action | Time |
+|-----------|--------|------|
+| **Daily (5 min)** | Check Google Alerts + F5bot notifications | Morning |
+| **Daily (10 min)** | Scan HN for relevant threads | 8-10am ET |
+| **Every 2 days** | Check Reddit for keyword matches | Random |
+| **Weekly** | Write 1 educational post on DevTo/blog | Weekend |
+| **Bi-weekly** | Review tracking table, adjust templates | Sunday |
+
+### Golden Rules
+1. **Never pitch in top-level posts** — only reply when relevant
+2. **First sentence = empathy/understanding**, not self-promo
+3. **Always include an action they can take** (a command to run)
+4. **Never copy-paste** — adapt template to the specific conversation
+5. **No URLs in first reply** unless asked (appears spammy)
+
+---
+
+## 5. Tracking Table
+
+| Date | Platform | URL | Template | Reply | Clicks/Installs |
+|------|----------|-----|----------|-------|-----------------|
+| | | | | | |
+| | | | | | |
+
+Keep a running log. Review weekly to see which templates convert best.
+
+---
+
+## 6. Success Metric
+
+Goal: **10 replies per week → 5 conversations → 1 GitHub star or npm install**
+
+At this rate: 50 stars/month, 250 npm installs/month from social listening alone.