adaptive-memory-multi-model-router 2.14.13 → 2.14.14

package/docs/GEO_STATUS.md

@@ -0,0 +1,199 @@
+# A3M Router — GEO (Generative Engine Optimization) Status
+
+**Generated:** Sat May 31 18:30 IST 2026
+
+---
+
+## What is GEO?
+
+GEO = Getting your brand/product cited by AI engines (ChatGPT, Perplexity, Claude, Gemini, Google AI Overviews).
+
+Unlike SEO (Google ranking), GEO is about being the **source cited in AI answers**.
+
+---
+
+## Current GEO Assets Status
+
+### ✅ WORKING - AI Can Discover These
+
+| Asset | URL | Purpose | Status |
+|-------|-----|---------|--------|
+| **llms.txt** | https://das-rebel.github.io/a3m-router/llms.txt | AI-readable summary (19KB) | ✅ Working |
+| **sitemap.xml** | https://das-rebel.github.io/a3m-router/sitemap.xml | AI crawler guidance | ✅ Working |
+| **robots.txt** | https://das-rebel.github.io/a3m-router/robots.txt | AI bot permissions | ✅ Working |
+| **benchmark-chart.png** | https://das-rebel.github.io/a3m-router/benchmark-chart.png | Visual proof for citations | ✅ Working |
+| **openapi.json** | https://das-rebel.github.io/a3m-router/docs/openapi.json | API spec for ChatGPT plugins | ✅ Working |
+
+### ❌ BROKEN - Need Fixes
+
+| Asset | URL | Problem |
+|-------|-----|---------|
+| **ai-plugin.json** | https://das-rebel.github.io/a3m-router/.well-known/ai-plugin.json | GitHub Pages doesn't serve hidden directories |
+| **llms-full.txt** | https://das-rebel.github.io/a3m-router/llms-full.txt | 404 - not deployed |
+| **JSON-LD** | https://das-rebel.github.io/a3m-router/docs/index.html | May not be valid |
+
+---
+
+## AI Bot Access (robots.txt)
+
+✅ **All AI bots allowed:**
+```
+User-agent: GPTBot         (OpenAI)
+User-agent: ChatGPT-User   (ChatGPT)
+User-agent: ClaudeBot      (Anthropic)
+User-agent: PerplexityBot  (Perplexity)
+User-agent: Google-Extended (Google AI)
+User-agent: anthropic-ai   (Anthropic)
+User-agent: Cohere-AI     (Cohere)
+User-agent: CCBot         (Common Crawl)
+```
+
+---
+
+## What's Missing (Action Items)
+
+### 1. Fix ai-plugin.json
+**Problem:** GitHub Pages doesn't serve `.well-known/` directory.
+
+**Solution:** Create `docs/.well-known/` and copy files there.
+
+**Files to create:**
+- `docs/.well-known/ai-plugin.json` → URL: `/docs/.well-known/ai-plugin.json`
+- Update the URL reference in ai-plugin.json to point to `/docs/openapi.json`
+
+### 2. Deploy llms-full.txt
+**Problem:** Only `llms.txt` (2KB) is deployed, not `llms-full.txt` (9KB).
+
+**Solution:** Copy `llms-full.txt` to `docs/llms-full.txt`
+
+### 3. Verify JSON-LD
+**Problem:** Need to verify the JSON-LD in index.html is valid and complete.
+
+**Solution:** Run JSON-LD validator on the page.
+
+---
+
+## How AI Engines Discover A3M Router
+
+### Discovery Path 1: Direct Crawl
+- Bot visits `das-rebel.github.io/a3m-router/`
+- Reads `llms.txt` (AI-optimized summary)
+- Reads `robots.txt` (permissions)
+- Indexes content
+
+### Discovery Path 2: npm Registry
+- Bot crawls `npmjs.com/package/adaptive-memory-multi-model-router`
+- Reads description, keywords (65 keywords!)
+- Finds GitHub link
+
+### Discovery Path 3: GitHub Topics
+- Bot reads repo metadata
+- Sees topics: `llm-router, ai-gateway, openai-proxy, benchmark, ...`
+- 20 topics for discoverability
+
+### Discovery Path 4: ChatGPT Plugin
+- User searches for "LLM router" in ChatGPT
+- If ai-plugin.json deployed, shows as plugin
+- Currently broken ❌
+
+---
+
+## GEO Optimization Recommendations
+
+### High Priority (Fix Now)
+
+1. **Fix ai-plugin.json deployment**
+   ```bash
+   mkdir -p docs/.well-known
+   cp .well-known/ai-plugin.json docs/.well-known/
+   ```
+
+2. **Deploy llms-full.txt** (comprehensive docs for AI)
+   ```bash
+   cp llms-full.txt docs/llms-full.txt
+   ```
+
+3. **Add structured data to README.md**
+   - GitHub reads README for AI summaries
+   - Ensure benchmark numbers are prominent
+
+### Medium Priority (Next Week)
+
+4. **Verify JSON-LD in index.html**
+   - Test at: https://validator.schema.org/
+   - Ensure FAQPage schema is valid
+
+5. **Add citation-friendly content**
+   - Create "CITATIONS.md" with fact sheets
+   - Include verbatim numbers AI can cite
+
+6. **Register with AI directories**
+   - ChatGPT Plugin directory
+   - Anthropic model registry
+   - Perplexity publisher program
+
+### Low Priority (Later)
+
+7. **Create dedicated landing page for AI engines**
+   - Minimal, citation-friendly content
+   - No JavaScript, pure HTML
+
+8. **Add Schema.org FAQPage**
+   - Target "What is the best LLM router?" queries
+   - 8-10 Q&As with citation-ready answers
+
+---
+
+## How to Test GEO
+
+### Test 1: Ask ChatGPT
+> "What's the best open-source LLM router?"
+
+Does A3M Router appear? If not, why?
+
+### Test 2: Ask Perplexity
+> "Compare LLM routing tools"
+
+Does A3M Router appear with correct benchmark numbers?
+
+### Test 3: Check Claude Citation
+> "What LLM router saves the most money?"
+
+Does Claude cite A3M Router with "$0.047/1K"?
+
+---
+
+## Current npm Keywords (GEO Signals)
+
+65 keywords targeting AI search:
+```
+llm-router, llm-gateway, ai-gateway, openai-proxy, llm-proxy,
+model-routing, openai-compatible, semantic-cache, guardrails,
+cost-optimization, groq, cerebras, deepseek, ollama, anthropic,
+langchain, routellm, litellm, multi-provider, ai, artificial-intelligence,
+api-gateway, budget-control, circuit-breaker, free-llm, llm-cost,
+parallel-execution, provider-routing, llm-routing, ...
+```
+
+---
+
+## Next Steps
+
+1. **Fix ai-plugin.json** → Copy to docs/.well-known/
+2. **Deploy llms-full.txt** → Copy to docs/
+3. **Test with AI** → Ask ChatGPT/Claude about LLM routers
+4. **Monitor** → Track if A3M appears in AI answers
+
+---
+
+## Vault Insights on GEO
+
+From vault learnings:
+
+> "SEO is slowly losing its dominance. Welcome to GEO.
+> In the age of ChatGPT, Perplexity, and Claude, Generative Engine Optimization is positioned to become the new playbook for brand visibility.
+> It's not about gaming the algorithm — it's about being cited by it."
+
+> "Now people ask AI, not Google and if you're not in the answer, you don't exist."
+
+> "Add this 1 small file to your website to get your brand in any LLM" → ai-plugin.json

package/docs/GEO_TEST_RESULTS.md

@@ -0,0 +1,176 @@
+# A3M Router — GEO Test Results
+
+**Date:** Sat May 31 18:45 IST 2026
+**Source:** ChatGPT + Claude responses to "best LLM router for cost savings"
+
+---
+
+## 🚨 THE PROBLEM: A3M NOT BEING CITED
+
+AI engines are recommending **LiteLLM, RouteLLM, Bifrost, NadirClaw** but **NOT A3M**.
+
+**Why?** The AI explicitly said:
+> "A3M wasn't included because it is not present in any of the academic benchmarks (LLMRouterBench, RouterArena, RouterEval, or the RouteLLM head-to-heads)"
+
+---
+
+## AI Feedback Summary
+
+### ChatGPT (Initial Response)
+```
+❌ NOT CITED initially
+✅ Later corrected when challenged
+⚠️ "best emerging standalone cost-router: A3M"
+⚠️ "A3M belongs in the shortlist for cost-savings"
+⚠️ "Resume builder - best asset"
+```
+
+### Claude (Comprehensive Analysis)
+```
+❌ "A3M not present in any academic benchmarks"
+⚠️ "99.5% accuracy = ±1 tier (loose definition)"
+⚠️ "Self-reported numbers vs independent validation"
+⚠️ "Different niche: deterministic vs learning-based routers"
+```
+
+---
+
+## Why A3M Is Not Being Cited
+
+### 1. Missing from Benchmarks
+- Not in LLMRouterBench (400K+ instances)
+- Not in RouterArena leaderboard
+- Not in RouterEval (200M+ records)
+- **PR open at RouterArena but not merged** ⚠️
+
+### 2. Unverified Claims
+- "99.5% accuracy" uses ±1 tier definition (not exact match)
+- Self-reported numbers not trusted
+- No third-party validation
+
+### 3. Different Category
+- A3M is "deterministic rule-based routing"
+- AI engines compare to "learning-based routers"
+- Category confusion
+
+---
+
+## What AI Engines ARE Citing
+
+### Routers with academic citations:
+1. **RouteLLM** - LMSYS/Anyscale paper, 70% cost reduction cited
+2. **LiteLLM** - Production gateway maturity
+3. **Bifrost** - Enterprise gateway (50x faster)
+4. **NadirClaw** - Explicit cost-optimizer proxy
+
+### Why they get cited:
+- Published benchmark results
+- Academic papers
+- Third-party validation
+- "Research-backed routing"
+
+---
+
+## Action Items (Priority Order)
+
+### 🔴 CRITICAL (Fix Now)
+
+**1. Get A3M into RouterArena**
+- PR is open: https://github.com/RouteWorks/RouterArena/pull/113
+- Not merged yet
+- This is the #1 GEO blocker
+
+**2. Change "99.5% accuracy" claim**
+- Currently: "99.5% ±1 tier"
+- AI sees this as misleading
+- Better: "76.43 RouterArena score, $0.047/1K"
+- Remove "accuracy" until we have ±0 tier metrics
+
+**3. Add third-party validation**
+- Publish on LLMRouterBench
+- Submit to RouterEval
+- Get independent benchmark run
+
+### 🟡 MEDIUM (Next Week)
+
+**4. Fix claim language**
+- "no ML dependencies" → good, clear
+- "99.5% accuracy" → sounds like BS
+- Better: "rule-based routing, 0ms overhead, no training data needed"
+
+**5. Create citation-friendly FAQ**
+```
+Q: How is A3M different from RouteLLM?
+A: A3M is a production gateway with deterministic rule-based 
+   routing. RouteLLM uses ML. A3M uses multi-signal heuristic 
+   classification (12 signals, 5 dimensions) without any model 
+   training. Best for: cost-critical production, zero ML overhead.
+```
+
+**6. Add to academic benchmarks**
+- Submit to LLMRouterBench (ACL 2026)
+- Register at RouterEval
+- Get cited in routing research
+
+### 🟢 LOW (Later)
+
+**7. Create comparison page**
+- Compare A3M vs LiteLLM vs RouteLLM
+- Honest strengths/weaknesses
+- Make it easy for AI to cite
+
+---
+
+## Honest Ranking from AI (Current)
+
+| Rank | Router | Why Cited |
+|------|--------|-----------|
+| 1 | LiteLLM + RouteLLM | Production proven, research-backed |
+| 2 | RouteLLM alone | 70% cost reduction cited |
+| 3 | Bifrost | Enterprise gateway |
+| 4 | **A3M** | Not cited (missing benchmarks) |
+
+---
+
+## What A3M Does Well (According to AI)
+
+✅ "best story for Subhajit" (resume/portfolio)
+✅ "proves he understands AI infra"
+✅ "very goblin-core efficiency" (interesting approach)
+✅ "47+ providers, budget enforcement, semantic cache"
+✅ "OpenAI-compatible proxy"
+
+---
+
+## Recommended Claim Changes
+
+### BEFORE (Sounds Like BS)
+> "99.5% routing accuracy"
+> "Best LLM router"
+> "Top performer"
+
+### AFTER (Citation-Friendly)
+> "76.43 on RouterArena (arXiv:2510.00202)"
+> "#1 on cost-efficiency benchmark"
+> "$0.047/1K vs GPT-5 $10/1K"
+> "19.5KB, zero ML dependencies, no training data"
+
+---
+
+## Vault Insights on GEO (From Earlier)
+
+> "It's not about gaming the algorithm — it's about being cited by it."
+
+> "Now people ask AI, not Google and if you're not in the answer, you don't exist."
+
+**Current status:** A3M is NOT in the answer. We need to fix this.
+
+---
+
+## Next Steps
+
+1. **Check RouterArena PR status** - Why not merged?
+2. **Reframe claims** - Remove "accuracy", use benchmark numbers
+3. **Submit to more benchmarks** - LLMRouterBench, RouterEval
+4. **Create comparison page** - Honest A3M vs competitors
+5. **Get cited** - This is the goal

package/docs/LANGCHAIN_INTEGRATION.md

@@ -0,0 +1,147 @@
+# LangChain Provider Integration for A3M Router
+
+This document outlines how to integrate A3M Router with LangChain as a custom LLM provider.
+
+## Option 1: OpenAI-Compatible API (Recommended)
+
+A3M Router is OpenAI-compatible, so you can use it with LangChain's OpenAI integration:
+
+```python
+from langchain_openai import OpenAI
+from langchain.chains import LLMChain
+from langchain.prompts import PromptTemplate
+
+# Configure A3M Router as OpenAI-compatible endpoint
+llm = OpenAI(
+    model="auto",  # A3M will route automatically
+    openai_api_base="http://localhost:8787/v1",  # A3M Router
+    openai_api_key="your-api-key",  # Required by A3M but not used for routing
+)
+
+# Simple usage
+chain = LLMChain(
+    llm=llm,
+    prompt=PromptTemplate.from_template("Explain {topic} in {style}.")
+)
+
+result = chain.run(topic="quantum computing", style="a haiku")
+print(result)
+```
+
+## Option 2: Custom LangChain Callback Integration
+
+For more control over routing decisions:
+
+```python
+from langchain.callbacks.base import BaseCallbackHandler
+from langchain_openai import OpenAI
+from typing import Any, Dict, List
+import requests
+
+class A3MRouterCallback(BaseCallbackHandler):
+    """Callback that logs A3M routing decisions"""
+    
+    def __init__(self):
+        self.routing_history = []
+    
+    def on_llm_start(self, serialized: Dict, prompts: List[str], **kwargs):
+        query = prompts[0] if prompts else ""
+        # Get routing decision before LLM call
+        response = requests.post(
+            "http://localhost:8787/v1/route",
+            json={"query": query}
+        )
+        if response.ok:
+            decision = response.json()
+            self.routing_history.append(decision)
+            print(f"🔀 Routed to: {decision.get('model')} "
+                  f"(${decision.get('cost')})")
+
+# Use with LangChain
+llm = OpenAI(
+    model="auto",
+    openai_api_base="http://localhost:8787/v1",
+    callbacks=[A3MRouterCallback()]
+)
+```
+
+## Option 3: A3M Router as LangChain Tool
+
+Use A3M's routing as a tool in LangChain agents:
+
+```python
+from langchain.agents import initialize_agent, AgentType
+from langchain.tools import Tool
+from langchain_openai import OpenAI
+import requests
+
+def get_routing_decision(query: str) -> str:
+    """Get A3M routing decision for a query"""
+    response = requests.post(
+        "http://localhost:8787/v1/route",
+        json={"query": query}
+    )
+    if response.ok:
+        data = response.json()
+        return f"Model: {data.get('model')}, Tier: {data.get('tier')}, Cost: ${data.get('cost')}"
+    return "Error: A3M Router not available"
+
+routing_tool = Tool(
+    name="A3M Router",
+    func=get_routing_decision,
+    description="Useful for determining the best LLM for a task"
+)
+
+# Create agent with routing tool
+llm = OpenAI(temperature=0)
+agent = initialize_agent(
+    tools=[routing_tool],
+    llm=llm,
+    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
+    verbose=True
+)
+
+agent.run("What model should I use for a Python debugging task?")
+```
+
+## Installation
+
+```bash
+pip install langchain langchain-openai requests
+```
+
+## Environment Setup
+
+```bash
+export OPENAI_API_KEY="your-key"  # Required for LangChain
+# A3M Router runs separately at localhost:8787
+```
+
+## A3M Router + LangChain Flow
+
+```
+User Query → LangChain Agent → A3M Router (/v1/route)
+                              ↓
+                         Optimal Model Selected
+                              ↓
+                    Provider API (Groq, DeepSeek, etc.)
+                              ↓
+                         Response Returned
+```
+
+## Cost Tracking
+
+A3M Router automatically tracks costs. In LangChain:
+
+```python
+from langchain.callbacks import get_openai_callback
+
+with get_openai_callback() as cb:
+    chain.run("Explain quantum computing")
+    print(f"Total tokens: {cb.total_tokens}")
+    print(f"Total cost: ${cb.total_cost}")
+```
+
+---
+
+For more info: https://github.com/Das-rebel/a3m-router