adaptive-memory-multi-model-router 2.14.45 → 2.14.47
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.d.ts +4 -0
- package/dist/index.js +8 -2
- package/dist/memory/hybridMemory.d.ts +71 -0
- package/dist/memory/hybridMemory.js +124 -0
- package/dist/memory/reasoningBank.d.ts +88 -0
- package/dist/memory/reasoningBank.js +303 -0
- package/{docs/llms.txt → llms.txt.bak} +6 -6
- package/package.json +13 -84
- package/src/index.ts +8 -0
- package/src/memory/hybridMemory.ts +155 -0
- package/src/memory/reasoningBank.ts +335 -0
- package/src/routing/advancedRouter.ts.bak +650 -0
- package/test.js.bak +376 -0
- package/.dockerignore +0 -82
- package/.env.example +0 -303
- package/.github/DISCUSSIONS_WELCOME.md +0 -27
- package/.github/DISCUSSION_TEMPLATE.yml +0 -5
- package/.github/FUNDING.yml +0 -2
- package/.github/ISSUE_TEMPLATE/bug_report.md +0 -94
- package/.github/ISSUE_TEMPLATE/config.yml +0 -17
- package/.github/ISSUE_TEMPLATE/feature_request.md +0 -71
- package/.github/PULL_REQUEST_TEMPLATE.md +0 -71
- package/.github/dependabot.yml +0 -9
- package/.github/workflows/auto-publish.yml +0 -51
- package/.github/workflows/ci.yml +0 -263
- package/.github/workflows/codeql.yml +0 -38
- package/.github/workflows/npm-publish.yml +0 -20
- package/.github/workflows/pages.yml +0 -37
- package/.github/workflows/stale.yml +0 -54
- package/.publish-tick +0 -1
- package/.well-known/ai-plugin.json +0 -16
- package/AGENT_COUNCIL_FINDINGS.md +0 -142
- package/ARCHITECTURE.md +0 -346
- package/AUDIT_REPORT.md +0 -28
- package/CODE_OF_CONDUCT.md +0 -128
- package/CONTRIBUTING.md +0 -50
- package/CONTRIBUTORS.md +0 -20
- package/Dockerfile +0 -53
- package/Dockerfile.proxy +0 -33
- package/HEALTH_REPORT.md +0 -118
- package/IMPROVEMENT_PLAN.md +0 -107
- package/LANDING.md +0 -43
- package/LAUNCH-PAIN-DRIVEN.md +0 -339
- package/LAUNCH.md +0 -337
- package/LAUNCH_CHECKLIST.md +0 -141
- package/LAUNCH_SNAPSHOT.md +0 -260
- package/MANIFESTO.md +0 -41
- package/POPULARITY_BOOSTERS.md +0 -285
- package/PR_STATUS_REPORT.md +0 -148
- package/REDESIGN.md +0 -95
- package/RUNKIT.md +0 -83
- package/SECURITY.md +0 -29
- package/SUBMISSIONS.md +0 -43
- package/_schema.html +0 -53
- package/ai-plugin.json +0 -16
- package/articles/AI_AGENT_LLM_ROUTING.md +0 -150
- package/articles/CHINESE_DIRECTORIES.md +0 -100
- package/articles/CHINESE_SUBMISSIONS_READY.md +0 -322
- package/articles/COMPETITOR_ALERTS.md +0 -31
- package/articles/COMPLETE_POSTING_DIRECTORY.md +0 -147
- package/articles/CONTENT_STRUCTURE.md +0 -292
- package/articles/DEVTO_COST_GUIDE.md +0 -473
- package/articles/DEVTO_FINAL.md +0 -416
- package/articles/DEVTO_MULTI_PROVIDER.md +0 -542
- package/articles/DEVTO_READY.md +0 -255
- package/articles/DEVTO_V2_ANNOUNCEMENT.md +0 -160
- package/articles/DEVTO_VIRAL_GROWTH.md +0 -280
- package/articles/FRESH_devto.md +0 -460
- package/articles/FRESH_devto_2026_05.md +0 -73
- package/articles/FRESH_hackernews.md +0 -14
- package/articles/FRESH_reddit_ml.md +0 -90
- package/articles/FRESH_reddit_node.md +0 -198
- package/articles/FRESH_reddit_sideproject.md +0 -72
- package/articles/FRESH_reddit_webdev.md +0 -130
- package/articles/FROM_ZERO_TO_10K.md +0 -107
- package/articles/HN_10X_BETTER.md +0 -430
- package/articles/HN_ACCOUNT_GUIDE.md +0 -21
- package/articles/HN_CHINESE_STYLE.md +0 -308
- package/articles/HN_FINAL.md +0 -148
- package/articles/HN_POSTED_VERSION.md +0 -56
- package/articles/HN_POST_READY.md +0 -137
- package/articles/HN_RESEARCH.md +0 -364
- package/articles/HN_SHOW_routerarena.md +0 -17
- package/articles/HN_TIMING_GUIDE.md +0 -52
- package/articles/INDIEHACKERS_POST.md +0 -52
- package/articles/INDIEHACKERS_READY.md +0 -120
- package/articles/LLM_BENCHMARK_DEEP_DIVE.md +0 -153
- package/articles/MASTER_POSTING_DIRECTORY.md +0 -189
- package/articles/NEWSLETTER_SEND_NOW.md +0 -259
- package/articles/NEWSLETTER_SUBMISSIONS.md +0 -112
- package/articles/PAIN-DRIVEN-devto-v2.md +0 -308
- package/articles/PAIN-DRIVEN-devto-v3.md +0 -268
- package/articles/PAIN-DRIVEN-devto.md +0 -242
- package/articles/PAIN-DRIVEN-hackernews-v2.md +0 -138
- package/articles/PAIN-DRIVEN-hackernews-v3.md +0 -151
- package/articles/PAIN-DRIVEN-hackernews.md +0 -131
- package/articles/PAIN-DRIVEN-reddit-v2.md +0 -301
- package/articles/PAIN-DRIVEN-reddit-v3.md +0 -236
- package/articles/PAIN-DRIVEN-reddit.md +0 -218
- package/articles/PAIN-DRIVEN-twitter-v2.md +0 -110
- package/articles/PAIN-DRIVEN-twitter-v3.md +0 -121
- package/articles/PAIN-DRIVEN-twitter.md +0 -120
- package/articles/PORTKEY_VS_A3M.md +0 -147
- package/articles/POSTING_KIT_2026_05.md +0 -67
- package/articles/PRESS_KIT_routerarena.md +0 -77
- package/articles/PRODUCTHUNT_LISTING.md +0 -48
- package/articles/PRODUCTHUNT_READY.md +0 -106
- package/articles/PR_PLAN_vault.md +0 -125
- package/articles/REDDIT_FINAL.md +0 -232
- package/articles/REDDIT_POST.md +0 -67
- package/articles/REDDIT_SUBMISSION_READY.md +0 -348
- package/articles/ROUTERARENA_LEADER.md +0 -45
- package/articles/SHOW_HN_FINAL.md +0 -29
- package/articles/TWEETS_10K_DOWNLOADS.md +0 -47
- package/articles/TWEETS_BENCHMARK_FIRST.md +0 -46
- package/articles/TWEETS_MCP_PLAY.md +0 -51
- package/articles/TWEETS_SEQUENTIAL_BROKEN.md +0 -49
- package/articles/TWEETS_WHY_BUILD.md +0 -54
- package/articles/TWEETS_routerarena_leader.md +0 -53
- package/articles/TWEET_STORM_READY.md +0 -165
- package/articles/TWITTER_FINAL.md +0 -167
- package/articles/WHY_10X_BETTER.md +0 -261
- package/articles/WHY_CHINESE_STYLE_BETTER.md +0 -323
- package/articles/ai-discoverability-llm-routing.md +0 -210
- package/articles/devto-llm-routing.md +0 -138
- package/articles/hackernews-show-hn.md +0 -54
- package/articles/hashnode-llm-cost-optimization.md +0 -125
- package/articles/hn_show_2026_05.md +0 -11
- package/articles/medium-building-llm-router.md +0 -205
- package/articles/reddit-ml.md +0 -76
- package/articles/twitter-thread-cost-savings.md +0 -50
- package/articles/youtube-tutorial-script.md +0 -262
- package/assets/a3m_3blue1brown.mp4 +0 -0
- package/assets/banner.svg +0 -109
- package/assets/chart-cost-v2.svg +0 -91
- package/assets/chart-cost-v3.svg +0 -143
- package/assets/chart-features-v2.svg +0 -132
- package/assets/chart-features-v3.svg +0 -211
- package/assets/chart-growth-v2.svg +0 -122
- package/assets/chart-growth-v3.svg +0 -189
- package/assets/cost-comparison.svg +0 -134
- package/assets/cost-simple.svg +0 -64
- package/assets/demo-hn.gif +0 -0
- package/assets/feature-matrix.svg +0 -136
- package/assets/growth-chart-animated.svg +0 -76
- package/assets/growth-chart.svg +0 -82
- package/assets/growth-simple.svg +0 -69
- package/assets/hero-diagram.svg +0 -81
- package/assets/logo-new.svg +0 -21
- package/assets/logo.svg +0 -68
- package/assets/provider-comparison.svg +0 -121
- package/assets/social-preview-new.svg +0 -100
- package/assets/social-preview.svg +0 -194
- package/assets/social-v2.svg +0 -130
- package/assets/social-v3.svg +0 -212
- package/benchmark-provider-results.json +0 -245
- package/benchmark-results.json +0 -54
- package/council-votes/architecture-vote.md +0 -121
- package/council-votes/coverage-vote.md +0 -93
- package/data/adaptive-benchmark.json +0 -92
- package/data/benchmark-results.json +0 -47
- package/data/labeled-benchmark.json +0 -88
- package/demo/3blue1brown_video.py +0 -285
- package/demo/3blue1brown_video_v2.py +0 -310
- package/demo/IMPROVED_PROMPTS.md +0 -229
- package/demo/VEO3_PROMPTS.md +0 -269
- package/demo/VIDEO_PRODUCTION_GUIDE.md +0 -333
- package/demo/a3m_3blue1brown.mp4 +0 -0
- package/demo/asciinema-demo.sh +0 -195
- package/demo/demo-hn.tape +0 -74
- package/demo/demo-script.md +0 -53
- package/demo/demo-script.sh +0 -62
- package/demo/demo.svg +0 -75
- package/demo/frame1_ai_data_center.png +0 -0
- package/demo/frame1_sunset_video.mp4 +0 -0
- package/demo/frame2_cost_comparison.png +0 -0
- package/demo/frame2_cost_comparison_fallback.png +0 -0
- package/demo/frame3_parallel_execution.png +0 -0
- package/demo/frame3_parallel_execution_fallback.png +0 -0
- package/demo/frame4_providers.png +0 -0
- package/demo/frame4_providers_fallback.png +0 -0
- package/demo/frame5_endcard.png +0 -0
- package/demo/frame5_endcard_fallback.png +0 -0
- package/demo/new_frame1_hook.png +0 -0
- package/demo/new_frame2_proof.png +0 -0
- package/demo/new_frame3_wow.png +0 -0
- package/demo/new_frame4_social.png +0 -0
- package/demo/new_frame5_cta.png +0 -0
- package/demo/package.json +0 -13
- package/demo/product-video-final.mp4 +0 -0
- package/demo/product-video-hype-v1.mp4 +0 -0
- package/demo/product-video-v1.mp4 +0 -0
- package/demo/public/index.html +0 -762
- package/demo/recording.cast +0 -55
- package/demo/server.js +0 -405
- package/demo-new.tape +0 -71
- package/demo-real.sh +0 -198
- package/demo-simple.tape +0 -205
- package/demo.html +0 -520
- package/demo.sh +0 -85
- package/demo.tape +0 -259
- package/dist/analytics/costAnalytics.d.ts.map +0 -1
- package/dist/analytics/costAnalytics.js.map +0 -1
- package/dist/benchmark/comprehensive.js.map +0 -1
- package/dist/benchmark/reproducible.d.ts.map +0 -1
- package/dist/benchmark/reproducible.js.map +0 -1
- package/dist/cache/prefixCache.d.ts.map +0 -1
- package/dist/cache/prefixCache.js.map +0 -1
- package/dist/cache/responseCache.d.ts.map +0 -1
- package/dist/cache/responseCache.js.map +0 -1
- package/dist/cache/semanticCache.d.ts.map +0 -1
- package/dist/cache/semanticCache.js.map +0 -1
- package/dist/cli/setupWizard.d.ts.map +0 -1
- package/dist/cli/setupWizard.js.map +0 -1
- package/dist/cost/budgetEnforcer.d.ts.map +0 -1
- package/dist/cost/budgetEnforcer.js.map +0 -1
- package/dist/cost/costTracker.d.ts.map +0 -1
- package/dist/cost/costTracker.js.map +0 -1
- package/dist/ensemble/multiRoundDialog.js.map +0 -1
- package/dist/ensemble/shapleyValue.js.map +0 -1
- package/dist/integrations/langchainAdapter.d.ts.map +0 -1
- package/dist/integrations/langchainAdapter.js.map +0 -1
- package/dist/integrations/oauth.d.ts.map +0 -1
- package/dist/integrations/oauth.js.map +0 -1
- package/dist/integrations/scienceAdapter.js.map +0 -1
- package/dist/memory/autoFetch.d.ts.map +0 -1
- package/dist/memory/autoFetch.js.map +0 -1
- package/dist/memory/episodicMemory.d.ts.map +0 -1
- package/dist/memory/episodicMemory.js.map +0 -1
- package/dist/memory/memoryTree.d.ts.map +0 -1
- package/dist/memory/memoryTree.js.map +0 -1
- package/dist/memory/obsidianVault.d.ts.map +0 -1
- package/dist/memory/obsidianVault.js.map +0 -1
- package/dist/observability/changeWatch.d.ts.map +0 -1
- package/dist/observability/changeWatch.js.map +0 -1
- package/dist/observability/fatigueDetector.d.ts.map +0 -1
- package/dist/observability/fatigueDetector.js.map +0 -1
- package/dist/observability/index.d.ts.map +0 -1
- package/dist/observability/index.js.map +0 -1
- package/dist/observability/metrics.d.ts.map +0 -1
- package/dist/observability/metrics.js.map +0 -1
- package/dist/observability/middleware.d.ts.map +0 -1
- package/dist/observability/middleware.js.map +0 -1
- package/dist/observability/tracer.d.ts.map +0 -1
- package/dist/observability/tracer.js.map +0 -1
- package/dist/observability/types.d.ts.map +0 -1
- package/dist/observability/types.js.map +0 -1
- package/dist/orchestration/haloOrchestrator.d.ts.map +0 -1
- package/dist/orchestration/haloOrchestrator.js.map +0 -1
- package/dist/orchestration/mctsWorkflow.d.ts.map +0 -1
- package/dist/orchestration/mctsWorkflow.js.map +0 -1
- package/dist/providers/localProvider.d.ts.map +0 -1
- package/dist/providers/localProvider.js.map +0 -1
- package/dist/providers/providerConfig.d.ts.map +0 -1
- package/dist/providers/providerConfig.js.map +0 -1
- package/dist/providers/registry.d.ts.map +0 -1
- package/dist/providers/registry.js.map +0 -1
- package/dist/routing/advancedRouter.d.ts.map +0 -1
- package/dist/routing/advancedRouter.js.map +0 -1
- package/dist/routing/crossModelValidation.d.ts.map +0 -1
- package/dist/routing/crossModelValidation.js.map +0 -1
- package/dist/routing/providerHealth.d.ts.map +0 -1
- package/dist/routing/providerHealth.js.map +0 -1
- package/dist/routing/providerRetry.d.ts.map +0 -1
- package/dist/routing/providerRetry.js.map +0 -1
- package/dist/scripts/banner.js +0 -29
- package/dist/security/guardrails.d.ts.map +0 -1
- package/dist/security/guardrails.js.map +0 -1
- package/dist/server/dashboard.d.ts.map +0 -1
- package/dist/server/dashboard.js.map +0 -1
- package/dist/server/modelMapper.d.ts.map +0 -1
- package/dist/server/modelMapper.js.map +0 -1
- package/dist/server/proxyServer.d.ts.map +0 -1
- package/dist/server/proxyServer.js.map +0 -1
- package/dist/skills/__tests__/skill_manager.test.d.ts +0 -2
- package/dist/skills/__tests__/skill_manager.test.d.ts.map +0 -1
- package/dist/skills/__tests__/skill_manager.test.js +0 -268
- package/dist/skills/__tests__/skill_manager.test.js.map +0 -1
- package/dist/tools/tmlpdTools.d.ts.map +0 -1
- package/dist/tools/tmlpdTools.js.map +0 -1
- package/dist/tui/dashboard.d.ts.map +0 -1
- package/dist/tui/dashboard.js.map +0 -1
- package/dist/tui/index.d.ts.map +0 -1
- package/dist/tui/index.js.map +0 -1
- package/dist/utils/batchProcessor.d.ts.map +0 -1
- package/dist/utils/batchProcessor.js.map +0 -1
- package/dist/utils/compression.d.ts.map +0 -1
- package/dist/utils/compression.js.map +0 -1
- package/dist/utils/costUtils.d.ts.map +0 -1
- package/dist/utils/costUtils.js.map +0 -1
- package/dist/utils/reliability.d.ts.map +0 -1
- package/dist/utils/reliability.js.map +0 -1
- package/dist/utils/sorting.d.ts.map +0 -1
- package/dist/utils/sorting.js.map +0 -1
- package/dist/utils/speculativeDecoding.d.ts.map +0 -1
- package/dist/utils/speculativeDecoding.js.map +0 -1
- package/dist/utils/tokenUtils.d.ts.map +0 -1
- package/dist/utils/tokenUtils.js.map +0 -1
- package/docs/.nojekyll +0 -0
- package/docs/ANALYSIS_PRINCIPLES.md +0 -162
- package/docs/API.md +0 -855
- package/docs/ARCHITECTURAL-IMPROVEMENTS-2025.md +0 -1391
- package/docs/ARCHITECTURAL-IMPROVEMENTS-REVISED-2025.md +0 -1051
- package/docs/BENCHMARK.md +0 -170
- package/docs/CHINESE_PROVIDER_RELIABILITY.md +0 -37
- package/docs/CITATIONS.md +0 -74
- package/docs/CLAIMS_AND_EVIDENCE.md +0 -58
- package/docs/CONFIGURATION.md +0 -476
- package/docs/COUNCIL_DECISION.json +0 -816
- package/docs/COUNCIL_SUMMARY.md +0 -319
- package/docs/COUNCIL_V2.2_DECISION.md +0 -416
- package/docs/ENGINEERING_SPEC.md +0 -55
- package/docs/FACTORY_RESET.md +0 -34
- package/docs/GEO.md +0 -66
- package/docs/GEO_OPTIMIZATION.md +0 -30
- package/docs/GEO_ROOT_CAUSE.md +0 -136
- package/docs/GEO_STATUS.md +0 -85
- package/docs/GEO_TEST_RESULTS.md +0 -176
- package/docs/HN_CHECKLIST.md +0 -38
- package/docs/HN_FOUNDER_COMMENT.md +0 -17
- package/docs/HN_SUBMISSION_FINAL.md +0 -180
- package/docs/HN_SUBMISSION_V3.md +0 -56
- package/docs/IMPROVEMENT_ROADMAP.md +0 -515
- package/docs/INTEGRATIONS.md +0 -420
- package/docs/LANGCHAIN_INTEGRATION.md +0 -147
- package/docs/LLM_COUNCIL_DECISION.md +0 -508
- package/docs/MIDDLEWARE_CHAIN.md +0 -35
- package/docs/PROMO_CHECKLIST.md +0 -200
- package/docs/QUICKSTART.md +0 -271
- package/docs/QUICK_START.md +0 -43
- package/docs/QUICK_START_VISIBILITY.md +0 -782
- package/docs/REDDIT_GAP_ANALYSIS.md +0 -299
- package/docs/RELEASE_CHECKLIST.md +0 -32
- package/docs/REPRODUCIBILITY.md +0 -63
- package/docs/RESEARCH_BACKED_IMPROVEMENTS.md +0 -1180
- package/docs/ROUTING_RUBRIC.md +0 -197
- package/docs/SEO_AUDIT.md +0 -186
- package/docs/SOCIAL_LISTENING.md +0 -219
- package/docs/TMLPD_QNA.md +0 -751
- package/docs/TMLPD_V2.1_COMPLETE.md +0 -763
- package/docs/TMLPD_V2.2_RESEARCH_ROADMAP.md +0 -754
- package/docs/UPDATE_TOPICS.md +0 -15
- package/docs/USE_CASES.md +0 -59
- package/docs/V2.2_IMPLEMENTATION_COMPLETE.md +0 -446
- package/docs/V2_IMPLEMENTATION_GUIDE.md +0 -388
- package/docs/VERCEL_AI_SDK.md +0 -209
- package/docs/VISIBILITY_ADOPTION_PLAN.md +0 -1005
- package/docs/_config.yml +0 -49
- package/docs/ai-plugin.json +0 -16
- package/docs/api.html +0 -513
- package/docs/architecture-diagram.md +0 -40
- package/docs/benchmark-chart.png +0 -0
- package/docs/benchmark.html +0 -387
- package/docs/blog/routerarena-number-one.html +0 -73
- package/docs/cli-cheatsheet.md +0 -339
- package/docs/compare.md +0 -109
- package/docs/comparison-litellm.md +0 -88
- package/docs/comparison.md +0 -108
- package/docs/cost-chart-ascii.md +0 -42
- package/docs/cost-comparison-chart.svg +0 -88
- package/docs/curl-examples.md +0 -247
- package/docs/demo-auto.html +0 -264
- package/docs/demo.html +0 -416
- package/docs/geo/GENERATIVE_ENGINE_OPTIMIZATION.md +0 -232
- package/docs/index.html +0 -507
- package/docs/launch-content/LAUNCH_EXECUTION_CHECKLIST.md +0 -421
- package/docs/launch-content/README.md +0 -457
- package/docs/launch-content/assets/cost_comparison_100_tasks.png +0 -0
- package/docs/launch-content/assets/cumulative_savings.png +0 -0
- package/docs/launch-content/assets/parallel_speedup.png +0 -0
- package/docs/launch-content/assets/provider_pricing_comparison.png +0 -0
- package/docs/launch-content/assets/task_breakdown_comparison.png +0 -0
- package/docs/launch-content/generate_charts.py +0 -313
- package/docs/launch-content/hn_show_post.md +0 -139
- package/docs/launch-content/partner_outreach_templates.md +0 -745
- package/docs/launch-content/reddit_posts.md +0 -467
- package/docs/launch-content/twitter_thread.txt +0 -460
- package/docs/npm-downloads-chart.svg +0 -43
- package/docs/openapi.json +0 -139
- package/docs/openapi.yaml +0 -1318
- package/docs/quick-start.html +0 -366
- package/docs/robots.txt +0 -52
- package/docs/sitemap.xml +0 -57
- package/docs/styles.css +0 -682
- package/docs/well-known/ai-plugin.json +0 -16
- package/docs/wellknown/ai-plugin.json +0 -16
- package/docs-site/assets/og-banner.svg +0 -194
- package/docs-site/index.html +0 -632
- package/eval/README.md +0 -46
- package/eval/baselines/main.json +0 -12
- package/eval/benchmark_dataset.jsonl +0 -16
- package/eval/check_golden_routes.js +0 -64
- package/eval/datasets/catalog.json +0 -33
- package/eval/datasets/slices/cn_provider_reliability_v1.jsonl +0 -3
- package/eval/datasets/slices/cost_pressure_v1.jsonl +0 -3
- package/eval/datasets/slices/safety_guardrails_v1.jsonl +0 -3
- package/eval/evals.json +0 -199
- package/eval/fault_injection_thresholds.json +0 -3
- package/eval/generate_report.js +0 -128
- package/eval/golden_routes.json +0 -114
- package/eval/lib/experiment_registry.js +0 -24
- package/eval/run_eval.js +0 -197
- package/eval/run_fault_injection.js +0 -201
- package/eval/run_shadow_eval.js +0 -85
- package/eval/thresholds.json +0 -9
- package/examples/QUICKSTART.md +0 -183
- package/examples/README.md +0 -61
- package/examples/a3m-sdk.js +0 -124
- package/examples/basic-route.js +0 -54
- package/examples/chat-loop.js +0 -202
- package/examples/classify-then-route.js +0 -102
- package/examples/cost-compare.js +0 -120
- package/examples/ensemble.js +0 -160
- package/examples/whatsapp-telegram-bridge-demo.js +0 -302
- package/examples/whatsapp-telegram-bridge.js +0 -269
- package/hf-space/README.md +0 -23
- package/hf-space/app.py +0 -240
- package/hf-space/requirements.txt +0 -1
- package/huggingface_space/README.md +0 -35
- package/huggingface_space/app.py +0 -126
- package/huggingface_space/create_space.py +0 -208
- package/huggingface_space/requirements.txt +0 -1
- package/mcp-server/README.md +0 -188
- package/mcp-server/package.json +0 -29
- package/mcp-server/src/index.ts +0 -744
- package/mcp-server/tsconfig.json +0 -19
- package/openclaw-alexa-bridge/ALL_REMAINING_FIXES_PLAN.md +0 -313
- package/openclaw-alexa-bridge/REMAINING_FIXES_SUMMARY.md +0 -277
- package/openclaw-alexa-bridge/src/alexa_handler_no_tmlpd.js +0 -1234
- package/openclaw-alexa-bridge/test_fixes.js +0 -77
- package/playground/README.md +0 -51
- package/playground/codesandbox.json +0 -12
- package/playground/index.js +0 -39
- package/proxy/README.md +0 -227
- package/proxy/package-lock.json +0 -831
- package/proxy/package.json +0 -17
- package/proxy/rate-limit.js +0 -145
- package/proxy/rate-limit.test.js +0 -311
- package/proxy/server.js +0 -970
- package/python/README.md +0 -102
- package/python/a3m/__init__.py +0 -6
- package/python/a3m/client.py +0 -190
- package/python/a3m/models.py +0 -40
- package/python/a3m/sync_client.py +0 -61
- package/python/examples.py +0 -53
- package/python/integrations.py +0 -330
- package/python/pyproject.toml +0 -23
- package/python/setup.py +0 -28
- package/python/tmlpd.py +0 -369
- package/qna/REDDIT_GAP_ANALYSIS.md +0 -299
- package/qna/TMLPD_QNA.md +0 -751
- package/research/FINDING_001_safety.md +0 -28
- package/research/FINDING_002_error_diversity.md +0 -32
- package/research/FINDING_003_confidence_weighted_voting.md +0 -32
- package/research/FINDING_004_cross_model_semantic_detection.md +0 -37
- package/research/FINDING_005_knowledge_gap_orthogonality.md +0 -34
- package/research/HALLUCINATION_RESEARCH.md +0 -27
- package/research/ensemble-voting.md +0 -324
- package/research/loss-functions.md +0 -545
- package/research-log.md +0 -49
- package/scripts/banner.js +0 -29
- package/scripts/benchmark-local-routerarena.ts +0 -176
- package/scripts/benchmark.js +0 -145
- package/scripts/benchmark.sh +0 -61
- package/scripts/compare-providers.sh +0 -230
- package/scripts/content-planner.js +0 -25
- package/scripts/create-labeled-benchmark.ts +0 -105
- package/scripts/cross_post.py +0 -443
- package/scripts/local-router-benchmark.ts +0 -154
- package/scripts/post-all.sh +0 -41
- package/scripts/publish_fcc.py +0 -106
- package/scripts/push-to-gitee.sh +0 -25
- package/scripts/routerarena_ensemble.js +0 -144
- package/scripts/routing-benchmark-v2.js +0 -373
- package/scripts/routing-benchmark-v3.js +0 -118
- package/scripts/routing-benchmark.js +0 -462
- package/scripts/run-labeled-benchmark.mjs +0 -104
- package/scripts/run-mmlu-benchmark.js +0 -176
- package/scripts/run-provider-benchmark.js +0 -244
- package/scripts/update-npm-badges.js +0 -158
- package/skill/SKILL.md +0 -238
- package/src/__tests__/integration/tmpld_integration.test.py +0 -540
- package/src/skills/__tests__/skill_manager.test.ts +0 -328
- package/submissions/benchmarks/ALL_PLATFORMS_SUBMISSION.md +0 -94
- package/submissions/benchmarks/LLMROUTERBENCH_SUBMISSION.md +0 -121
- package/submissions/benchmarks/MMRBENCH_SUBMISSION.md +0 -94
- package/submissions/benchmarks/ROUTERARENA_UPDATE.md +0 -83
- package/submissions/benchmarks/ROUTERBENCH_SUBMISSION.md +0 -225
- package/test-council/1-structure-tests.test.js +0 -353
- package/test-council/1-structure-tests.test.ts +0 -353
- package/test-council/2-edge-case-tests.test.ts +0 -361
- package/test-council/3-performance-tests.test.ts +0 -669
- package/test-council/4-integration-tests.test.ts +0 -391
- package/test-council/5-agent-council-eval.test.ts +0 -413
- package/test-council/AGENT_COUNCIL_ARCHITECTURE.md +0 -349
- package/test-council/TEST_COUNCIL_REPORT.md +0 -201
- package/test-council/agents/edge-case-agent.ts +0 -363
- package/test-council/agents/performance-agent.ts +0 -426
- package/test-council/agents/structure-agent.ts +0 -227
- package/test-council/council.md +0 -183
- package/tests/__mocks__/tokenUtils.ts +0 -8
- package/tests/memory/episodicMemory.test.ts +0 -227
- package/tests/package-lock.json +0 -1628
- package/tests/package.json +0 -18
- package/tests/routing/ensembleVoting.test.ts +0 -236
- package/tests/routing/providerRetry.test.ts +0 -360
- package/tests/routing/queryTypePresets.test.ts +0 -208
- package/tests/security/guardrailEngine.test.ts +0 -700
- package/tests/tsconfig.json +0 -21
- package/tests/vitest.config.ts +0 -18
- package/tmlpd-pi-extension/README.md +0 -66
- package/tmlpd-pi-extension/dist/cache/prefixCache.d.ts +0 -114
- package/tmlpd-pi-extension/dist/cache/prefixCache.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/cache/prefixCache.js +0 -285
- package/tmlpd-pi-extension/dist/cache/prefixCache.js.map +0 -1
- package/tmlpd-pi-extension/dist/cache/responseCache.d.ts +0 -58
- package/tmlpd-pi-extension/dist/cache/responseCache.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/cache/responseCache.js +0 -153
- package/tmlpd-pi-extension/dist/cache/responseCache.js.map +0 -1
- package/tmlpd-pi-extension/dist/cli.js +0 -59
- package/tmlpd-pi-extension/dist/cost/costTracker.d.ts +0 -95
- package/tmlpd-pi-extension/dist/cost/costTracker.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/cost/costTracker.js +0 -240
- package/tmlpd-pi-extension/dist/cost/costTracker.js.map +0 -1
- package/tmlpd-pi-extension/dist/index.d.ts +0 -723
- package/tmlpd-pi-extension/dist/index.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/index.js +0 -239
- package/tmlpd-pi-extension/dist/index.js.map +0 -1
- package/tmlpd-pi-extension/dist/memory/episodicMemory.d.ts +0 -82
- package/tmlpd-pi-extension/dist/memory/episodicMemory.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/memory/episodicMemory.js +0 -145
- package/tmlpd-pi-extension/dist/memory/episodicMemory.js.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.d.ts +0 -102
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.js +0 -207
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.js.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.d.ts +0 -85
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.js +0 -210
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.js.map +0 -1
- package/tmlpd-pi-extension/dist/providers/localProvider.d.ts +0 -102
- package/tmlpd-pi-extension/dist/providers/localProvider.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/providers/localProvider.js +0 -338
- package/tmlpd-pi-extension/dist/providers/localProvider.js.map +0 -1
- package/tmlpd-pi-extension/dist/providers/registry.d.ts +0 -55
- package/tmlpd-pi-extension/dist/providers/registry.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/providers/registry.js +0 -138
- package/tmlpd-pi-extension/dist/providers/registry.js.map +0 -1
- package/tmlpd-pi-extension/dist/routing/advancedRouter.d.ts +0 -68
- package/tmlpd-pi-extension/dist/routing/advancedRouter.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/routing/advancedRouter.js +0 -332
- package/tmlpd-pi-extension/dist/routing/advancedRouter.js.map +0 -1
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.d.ts +0 -101
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.js +0 -368
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/batchProcessor.d.ts +0 -96
- package/tmlpd-pi-extension/dist/utils/batchProcessor.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/batchProcessor.js +0 -170
- package/tmlpd-pi-extension/dist/utils/batchProcessor.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/compression.d.ts +0 -61
- package/tmlpd-pi-extension/dist/utils/compression.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/compression.js +0 -281
- package/tmlpd-pi-extension/dist/utils/compression.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/reliability.d.ts +0 -74
- package/tmlpd-pi-extension/dist/utils/reliability.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/reliability.js +0 -177
- package/tmlpd-pi-extension/dist/utils/reliability.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.d.ts +0 -117
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.js +0 -246
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/tokenUtils.d.ts +0 -50
- package/tmlpd-pi-extension/dist/utils/tokenUtils.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/tokenUtils.js +0 -124
- package/tmlpd-pi-extension/dist/utils/tokenUtils.js.map +0 -1
- package/tmlpd-pi-extension/examples/QUICKSTART.md +0 -183
- package/tmlpd-pi-extension/package-lock.json +0 -79
- package/tmlpd-pi-extension/package.json +0 -172
- package/tmlpd-pi-extension/python/examples.py +0 -53
- package/tmlpd-pi-extension/python/integrations.py +0 -330
- package/tmlpd-pi-extension/python/setup.py +0 -28
- package/tmlpd-pi-extension/python/tmlpd.py +0 -369
- package/tmlpd-pi-extension/qna/REDDIT_GAP_ANALYSIS.md +0 -299
- package/tmlpd-pi-extension/qna/TMLPD_QNA.md +0 -751
- package/tmlpd-pi-extension/skill/SKILL.md +0 -238
- package/tmlpd-pi-extension/src/cache/responseCache.ts +0 -147
- package/tmlpd-pi-extension/src/cost/costTracker.ts +0 -302
- package/tmlpd-pi-extension/src/index.ts +0 -232
- package/tmlpd-pi-extension/src/memory/episodicMemory.ts +0 -257
- package/tmlpd-pi-extension/src/orchestration/haloOrchestrator.ts +0 -266
- package/tmlpd-pi-extension/src/orchestration/mctsWorkflow.ts +0 -262
- package/tmlpd-pi-extension/src/providers/localProvider.ts +0 -406
- package/tmlpd-pi-extension/src/providers/registry.ts +0 -164
- package/tmlpd-pi-extension/src/routing/ensembleVoting.ts +0 -159
- package/tmlpd-pi-extension/src/routing/queryTypePresets.ts +0 -136
- package/tmlpd-pi-extension/src/tools/tmlpdTools.ts +0 -433
- package/tmlpd-pi-extension/src/utils/batchProcessor.ts +0 -232
- package/tmlpd-pi-extension/src/utils/compression.ts +0 -325
- package/tmlpd-pi-extension/src/utils/reliability.ts +0 -221
- package/tmlpd-pi-extension/src/utils/tokenUtils.ts +0 -145
- package/tmlpd-pi-extension/tsconfig.json +0 -18
- package/tsconfig.build.json +0 -29
- package/tsconfig.json +0 -18
- /package/{docs/llms-full.txt → llms-full.txt.bak} +0 -0
|
@@ -1,40 +0,0 @@
|
|
|
1
|
-
# A3M Router Architecture: Parallel vs Sequential
|
|
2
|
-
|
|
3
|
-
## Traditional Router (Sequential Fallback)
|
|
4
|
-
```
|
|
5
|
-
Query → Try Provider A → ❌ Fail ($0.03)
|
|
6
|
-
→ Try Provider B → ❌ Fail ($0.02)
|
|
7
|
-
→ Try Provider C → ✅ Success ($0.01)
|
|
8
|
-
|
|
9
|
-
Total: 3 API calls, 3× latency, $0.06 cost
|
|
10
|
-
```
|
|
11
|
-
|
|
12
|
-
## A3M Router (Parallel Execution)
|
|
13
|
-
```
|
|
14
|
-
Query → Provider A ═╗
|
|
15
|
-
→ Provider B ═╣ → Score each → Pick best ✅
|
|
16
|
-
→ Provider C ═╝ response (confidence)
|
|
17
|
-
|
|
18
|
-
Total: 1 round-trip, 1× latency, $0.01 cost
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
## Why A3M scores higher
|
|
22
|
-
- **Confidence scoring** catches when cheap models produce better answers
|
|
23
|
-
- **No fallback chain** means no accumulated latency
|
|
24
|
-
- **Budget enforcement** caps cost per query
|
|
25
|
-
- **Circuit breaker** skips failing providers automatically
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
## Memory Feature (Unique to A3M)
|
|
29
|
-
```
|
|
30
|
-
Session 1: "My name is Alice" → stored in episodic memory
|
|
31
|
-
Session 2: "What's my name?" → "Alice!" (recalled from memory)
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Query-Type Presets
|
|
35
|
-
```
|
|
36
|
-
fast: 15s timeout, temperature 0.3 → "What's 2+2?"
|
|
37
|
-
creative: 45s timeout, temperature 0.7 → "Write a poem"
|
|
38
|
-
deep: 60s timeout, temperature 0.5 → "Explain quantum mechanics"
|
|
39
|
-
code: 30s timeout, temperature 0.3 → "Write a Python sort function"
|
|
40
|
-
```
|
package/docs/benchmark-chart.png
DELETED
|
Binary file
|
package/docs/benchmark.html
DELETED
|
@@ -1,387 +0,0 @@
|
|
|
1
|
-
<!DOCTYPE html>
|
|
2
|
-
<html lang="en">
|
|
3
|
-
<head>
|
|
4
|
-
<meta charset="UTF-8">
|
|
5
|
-
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
-
<title>Benchmark — A3M Router</title>
|
|
7
|
-
<meta name="description" content="Independent benchmark results for A3M Router: 70.32 routing accuracy, 62% cost savings, +96ms passthrough overhead, -57% hallucination rate with parallel ensemble.">
|
|
8
|
-
<meta name="keywords" content="LLM router benchmark, AI gateway latency, routing accuracy, cost comparison, multi-provider benchmark">
|
|
9
|
-
<meta property="og:title" content="A3M Router — Benchmarks">
|
|
10
|
-
<meta property="og:image" content="https://das-rebel.github.io/a3m-router/benchmark-chart.png">
|
|
11
|
-
<link rel="canonical" href="https://das-rebel.github.io/a3m-router/benchmark">
|
|
12
|
-
<link rel="stylesheet" href="../styles.css">
|
|
13
|
-
<script type="application/ld+json">
|
|
14
|
-
{
|
|
15
|
-
"@context": "https://schema.org",
|
|
16
|
-
"@type": "WebPage",
|
|
17
|
-
"name": "A3M Router Benchmark",
|
|
18
|
-
"description": "Independent benchmark results for A3M Router LLM gateway showing latency, cost savings, and routing accuracy.",
|
|
19
|
-
"url": "https://das-rebel.github.io/a3m-router/benchmark"
|
|
20
|
-
}
|
|
21
|
-
</script>
|
|
22
|
-
<script>
|
|
23
|
-
function toggleNav() {
|
|
24
|
-
document.querySelector('.nav-links').classList.toggle('open');
|
|
25
|
-
}
|
|
26
|
-
function switchTab(tab) {
|
|
27
|
-
document.querySelectorAll('.tab-content').forEach(el => el.classList.remove('active'));
|
|
28
|
-
document.querySelectorAll('.tab-btn').forEach(el => el.classList.remove('active'));
|
|
29
|
-
document.getElementById(tab).classList.add('active');
|
|
30
|
-
document.querySelector(`[data-tab="${tab}"]`).classList.add('active');
|
|
31
|
-
}
|
|
32
|
-
</script>
|
|
33
|
-
</head>
|
|
34
|
-
<body>
|
|
35
|
-
|
|
36
|
-
<header class="site-header">
|
|
37
|
-
<nav class="nav-container">
|
|
38
|
-
<a href="/a3m-router/" class="nav-brand">
|
|
39
|
-
<span class="nav-brand-icon">🔀</span>
|
|
40
|
-
A3M Router
|
|
41
|
-
</a>
|
|
42
|
-
<button class="nav-toggle" onclick="toggleNav()" aria-label="Menu">☰</button>
|
|
43
|
-
<ul class="nav-links">
|
|
44
|
-
<li><a href="/a3m-router/">Home</a></li>
|
|
45
|
-
<li><a href="/a3m-router/quick-start">Quick Start</a></li>
|
|
46
|
-
<li><a href="/a3m-router/benchmark" class="active">Benchmark</a></li>
|
|
47
|
-
<li><a href="/a3m-router/api">API</a></li>
|
|
48
|
-
<li><a href="https://github.com/Das-rebel/a3m-router" class="nav-github-btn" target="_blank" rel="noopener">
|
|
49
|
-
<svg height="16" width="16" viewBox="0 0 16 16" fill="currentColor"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg>
|
|
50
|
-
GitHub
|
|
51
|
-
</a></li>
|
|
52
|
-
</ul>
|
|
53
|
-
</nav>
|
|
54
|
-
</header>
|
|
55
|
-
|
|
56
|
-
<main class="page">
|
|
57
|
-
<div class="container">
|
|
58
|
-
|
|
59
|
-
<h1>📊 A3M Router Benchmark</h1>
|
|
60
|
-
<p>The question everyone asks: <em>"How much latency does a gateway add?"</em></p>
|
|
61
|
-
<p><strong>The answer:</strong> +96ms for passthrough, +236ms for full intelligent routing — on a 138ms baseline. That's about $11 per millisecond saved.</p>
|
|
62
|
-
|
|
63
|
-
<!-- Overview Stats -->
|
|
64
|
-
<div class="stats-grid">
|
|
65
|
-
<div class="stat-card">
|
|
66
|
-
<div class="stat-value">70.32</div>
|
|
67
|
-
<div class="stat-label">+/-1 Tier Accuracy</div>
|
|
68
|
-
</div>
|
|
69
|
-
<div class="stat-card">
|
|
70
|
-
<div class="stat-value">62%</div>
|
|
71
|
-
<div class="stat-label">Cost Savings</div>
|
|
72
|
-
</div>
|
|
73
|
-
<div class="stat-card">
|
|
74
|
-
<div class="stat-value">+96ms</div>
|
|
75
|
-
<div class="stat-label">Passthrough Overhead</div>
|
|
76
|
-
</div>
|
|
77
|
-
<div class="stat-card">
|
|
78
|
-
<div class="stat-value">+26%</div>
|
|
79
|
-
<div class="stat-label">Ensemble Quality Gain</div>
|
|
80
|
-
</div>
|
|
81
|
-
</div>
|
|
82
|
-
|
|
83
|
-
<!-- Tab Navigation -->
|
|
84
|
-
<div class="tabs">
|
|
85
|
-
<button class="tab-btn active" data-tab="tab-latency" onclick="switchTab('tab-latency')">⏱ Latency</button>
|
|
86
|
-
<button class="tab-btn" data-tab="tab-accuracy" onclick="switchTab('tab-accuracy')">🎯 Accuracy</button>
|
|
87
|
-
<button class="tab-btn" data-tab="tab-cost" onclick="switchTab('tab-cost')">💰 Cost</button>
|
|
88
|
-
<button class="tab-btn" data-tab="tab-quality" onclick="switchTab('tab-quality')">✨ Quality</button>
|
|
89
|
-
<button class="tab-btn" data-tab="tab-providers" onclick="switchTab('tab-providers')">📡 Providers</button>
|
|
90
|
-
</div>
|
|
91
|
-
|
|
92
|
-
<!-- Tab: Latency -->
|
|
93
|
-
<div id="tab-latency" class="tab-content active">
|
|
94
|
-
<h2>Latency Comparison</h2>
|
|
95
|
-
|
|
96
|
-
<div class="chart-container">
|
|
97
|
-
<img src="benchmark-chart.png" alt="A3M Router Benchmark Chart — latency comparison and cost savings projection">
|
|
98
|
-
<p class="chart-caption">Left: latency comparison. Right: cost savings projection. Dark theme. Measured with <a href="https://github.com/taffy-owo/llm-gateway-bench" target="_blank" rel="noopener">llm-gateway-bench</a> v0.2.0, Groq (llama-3.3-70b-versatile), 15 calls per scenario.</p>
|
|
99
|
-
</div>
|
|
100
|
-
|
|
101
|
-
<div class="table-wrapper">
|
|
102
|
-
<table>
|
|
103
|
-
<thead>
|
|
104
|
-
<tr><th>Scenario</th><th>Time</th><th>vs Baseline</th><th>What You Get</th></tr>
|
|
105
|
-
</thead>
|
|
106
|
-
<tbody>
|
|
107
|
-
<tr>
|
|
108
|
-
<td><strong>Direct to Groq</strong> (no gateway)</td>
|
|
109
|
-
<td><strong>138ms</strong></td>
|
|
110
|
-
<td>—</td>
|
|
111
|
-
<td>Raw provider speed</td>
|
|
112
|
-
</tr>
|
|
113
|
-
<tr>
|
|
114
|
-
<td><strong>Through A3M forced route</strong></td>
|
|
115
|
-
<td><strong>234ms</strong></td>
|
|
116
|
-
<td>+96ms</td>
|
|
117
|
-
<td>Guardrails (17 injection patterns, PII), cache lookup (30%+ hit rate), cost tracking, circuit breaker</td>
|
|
118
|
-
</tr>
|
|
119
|
-
<tr>
|
|
120
|
-
<td><strong>Through A3M auto route</strong></td>
|
|
121
|
-
<td><strong>374ms</strong></td>
|
|
122
|
-
<td>+236ms</td>
|
|
123
|
-
<td>Everything above + intelligent routing (12 signals → tier → cheapest capable model → <strong>62% cost savings</strong>)</td>
|
|
124
|
-
</tr>
|
|
125
|
-
</tbody>
|
|
126
|
-
</table>
|
|
127
|
-
</div>
|
|
128
|
-
|
|
129
|
-
<div class="callout callout-info">
|
|
130
|
-
The routing decision itself takes <strong><1ms</strong>. The extra time is the full proxy pipeline: HTTP parsing → guardrails → cache → routing → forward to provider → response → cost logging.
|
|
131
|
-
</div>
|
|
132
|
-
|
|
133
|
-
<div class="callout callout-success">
|
|
134
|
-
<strong>236ms total overhead saves $2,604/year</strong> at 100K queries/month. Full methodology in <a href="https://github.com/Das-rebel/a3m-router/blob/main/docs/BENCHMARK.md">BENCHMARK.md</a>.
|
|
135
|
-
</div>
|
|
136
|
-
|
|
137
|
-
<h3>The Trade-Off</h3>
|
|
138
|
-
<div class="table-wrapper">
|
|
139
|
-
<table>
|
|
140
|
-
<thead>
|
|
141
|
-
<tr><th></th><th>Without A3M</th><th>With A3M</th></tr>
|
|
142
|
-
</thead>
|
|
143
|
-
<tbody>
|
|
144
|
-
<tr><td>Response time</td><td>138ms</td><td>374ms</td></tr>
|
|
145
|
-
<tr><td>Monthly API bill</td><td>$341 (all premium)</td><td><strong>$124</strong> (smart routed)</td></tr>
|
|
146
|
-
<tr><td>Security</td><td>None</td><td>17-pattern injection detection</td></tr>
|
|
147
|
-
<tr><td>Cache hits</td><td>None</td><td>30%+ semantic cache</td></tr>
|
|
148
|
-
<tr><td>Provider failures</td><td>Manual retry</td><td>Circuit breaker + auto failover</td></tr>
|
|
149
|
-
<tr><td>Cost visibility</td><td>End-of-month surprise</td><td>Per-query tracking + budget alerts</td></tr>
|
|
150
|
-
</tbody>
|
|
151
|
-
</table>
|
|
152
|
-
</div>
|
|
153
|
-
</div>
|
|
154
|
-
|
|
155
|
-
<!-- Tab: Accuracy -->
|
|
156
|
-
<div id="tab-accuracy" class="tab-content">
|
|
157
|
-
<h2>Routing Accuracy</h2>
|
|
158
|
-
<p>200 real API calls, benchmarked against manual expert classification.</p>
|
|
159
|
-
|
|
160
|
-
<div class="stats-grid">
|
|
161
|
-
<div class="stat-card">
|
|
162
|
-
<div class="stat-value">70.32</div>
|
|
163
|
-
<div class="stat-label">±1 Tier Accuracy</div>
|
|
164
|
-
</div>
|
|
165
|
-
<div class="stat-card">
|
|
166
|
-
<div class="stat-value">64.5%</div>
|
|
167
|
-
<div class="stat-label">Exact Tier Match</div>
|
|
168
|
-
</div>
|
|
169
|
-
<div class="stat-card">
|
|
170
|
-
<div class="stat-value">92%</div>
|
|
171
|
-
<div class="stat-label">Free Tier Recall</div>
|
|
172
|
-
</div>
|
|
173
|
-
<div class="stat-card">
|
|
174
|
-
<div class="stat-value">7%</div>
|
|
175
|
-
<div class="stat-label">Over-routing (waste)</div>
|
|
176
|
-
</div>
|
|
177
|
-
</div>
|
|
178
|
-
|
|
179
|
-
<div class="table-wrapper">
|
|
180
|
-
<table>
|
|
181
|
-
<thead>
|
|
182
|
-
<tr><th>Metric</th><th>Score</th><th>What It Means</th></tr>
|
|
183
|
-
</thead>
|
|
184
|
-
<tbody>
|
|
185
|
-
<tr><td><strong>±1 Tier Accuracy</strong></td><td><strong>70.32</strong></td><td>Only 1 in 200 queries is misrouted by more than 1 tier</td></tr>
|
|
186
|
-
<tr><td>Exact Tier Match</td><td>64.5%</td><td>~2 in 3 queries hit the <em>exact</em> right tier</td></tr>
|
|
187
|
-
<tr><td>Free Tier Recall</td><td>92%</td><td>Free-tier-suitable queries correctly routed to $0 models</td></tr>
|
|
188
|
-
<tr><td>Over-routing (waste)</td><td>7%</td><td>Sent to a stronger — but more expensive — model than needed</td></tr>
|
|
189
|
-
<tr><td>Under-routing (risk)</td><td>28.5%</td><td>Sent to a weaker model; fallback auto-escalates on failure</td></tr>
|
|
190
|
-
</tbody>
|
|
191
|
-
</table>
|
|
192
|
-
</div>
|
|
193
|
-
|
|
194
|
-
<div class="card">
|
|
195
|
-
<h3>Third-Party Validation</h3>
|
|
196
|
-
<p>A3M's routing tiers align with established third-party benchmarks.</p>
|
|
197
|
-
</div>
|
|
198
|
-
|
|
199
|
-
<div class="table-wrapper">
|
|
200
|
-
<table>
|
|
201
|
-
<thead>
|
|
202
|
-
<tr><th>Provider</th><th>MMLU</th><th>Tier</th><th>Source</th></tr>
|
|
203
|
-
</thead>
|
|
204
|
-
<tbody>
|
|
205
|
-
<tr><td>gpt-4o</td><td>88.7%</td><td>premium</td><td>MMLU Leaderboard</td></tr>
|
|
206
|
-
<tr><td>claude-3.5-sonnet</td><td>88.4%</td><td>premium</td><td>MMLU Leaderboard</td></tr>
|
|
207
|
-
<tr><td>gemini-1.5-pro</td><td>85.7%</td><td>premium</td><td>MMLU Leaderboard</td></tr>
|
|
208
|
-
<tr><td>mistral-large</td><td>84.2%</td><td>mid</td><td>MMLU Leaderboard</td></tr>
|
|
209
|
-
<tr><td>llama-3.3-70b</td><td>82.5%</td><td>mid</td><td>MMLU Leaderboard</td></tr>
|
|
210
|
-
<tr><td>deepseek-v2</td><td>78.3%</td><td>mid</td><td>MMLU Leaderboard</td></tr>
|
|
211
|
-
<tr><td>llama-3.1-8b</td><td>68.3%</td><td>cheap</td><td>MMLU Leaderboard</td></tr>
|
|
212
|
-
</tbody>
|
|
213
|
-
</table>
|
|
214
|
-
</div>
|
|
215
|
-
|
|
216
|
-
<div class="callout callout-info">
|
|
217
|
-
<strong>On under-routing:</strong> A3M is deliberately conservative — it would rather try a cheaper model first and fail fast (triggering automatic fallback in <2s) than default to premium for every query. This is what drives the 62% cost savings.
|
|
218
|
-
</div>
|
|
219
|
-
</div>
|
|
220
|
-
|
|
221
|
-
<!-- Tab: Cost -->
|
|
222
|
-
<div id="tab-cost" class="tab-content">
|
|
223
|
-
<h2>Cost Savings</h2>
|
|
224
|
-
|
|
225
|
-
<h3>Cost Breakdown (200 real API calls)</h3>
|
|
226
|
-
<pre><code> GPT-4o only: $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $0.25 (all premium)
|
|
227
|
-
A3M Router: $$$$ $0.10 (smart routed)
|
|
228
|
-
——————————————————————————————
|
|
229
|
-
You save: $0.15 (62%)</code></pre>
|
|
230
|
-
|
|
231
|
-
<h3>By Query Type</h3>
|
|
232
|
-
<div class="table-wrapper">
|
|
233
|
-
<table>
|
|
234
|
-
<thead>
|
|
235
|
-
<tr><th>Query Type</th><th>% Traffic</th><th>GPT-4o Only</th><th>A3M Routes To</th><th>A3M Cost</th><th>Savings</th></tr>
|
|
236
|
-
</thead>
|
|
237
|
-
<tbody>
|
|
238
|
-
<tr><td>Simple Q&A</td><td>47%</td><td>$4.94</td><td>taste-1 (free)</td><td>$0.00</td><td><span class="badge green">100%</span></td></tr>
|
|
239
|
-
<tr><td>Code gen</td><td>15%</td><td>$4.88</td><td>deepseek ($0.14/M)</td><td>$0.17</td><td><span class="badge green">97%</span></td></tr>
|
|
240
|
-
<tr><td>Summarization</td><td>18%</td><td>$7.20</td><td>gpt-4o-mini ($0.15/M)</td><td>$0.43</td><td><span class="badge green">94%</span></td></tr>
|
|
241
|
-
<tr><td>Reasoning</td><td>12%</td><td>$8.70</td><td>claude-haiku ($0.80/M)</td><td>$3.36</td><td><span class="badge green">61%</span></td></tr>
|
|
242
|
-
<tr><td>Expert</td><td>8%</td><td>$8.40</td><td>gpt-4o ($2.50/M)</td><td>$8.40</td><td>0%</td></tr>
|
|
243
|
-
<tr><td><strong>Total</strong></td><td><strong>100%</strong></td><td><strong>$34.11</strong></td><td>—</td><td><strong>$12.36</strong></td><td><span class="badge green"><strong>64%</strong></span></td></tr>
|
|
244
|
-
</tbody>
|
|
245
|
-
</table>
|
|
246
|
-
</div>
|
|
247
|
-
|
|
248
|
-
<h3>Annual Projection</h3>
|
|
249
|
-
<div class="table-wrapper">
|
|
250
|
-
<table>
|
|
251
|
-
<thead>
|
|
252
|
-
<tr><th>Monthly Queries</th><th>All-Premium</th><th>A3M Router</th><th>You Save</th><th>Annualized</th></tr>
|
|
253
|
-
</thead>
|
|
254
|
-
<tbody>
|
|
255
|
-
<tr><td>10K</td><td>$34</td><td>$12</td><td><span class="badge green">$22 (65%)</span></td><td><strong>$261</strong></td></tr>
|
|
256
|
-
<tr><td>100K</td><td>$341</td><td>$124</td><td><span class="badge green">$217 (64%)</span></td><td><strong>$2,604</strong></td></tr>
|
|
257
|
-
<tr><td>1M</td><td>$3,411</td><td>$1,236</td><td><span class="badge green">$2,175 (64%)</span></td><td><strong>$26,100</strong></td></tr>
|
|
258
|
-
</tbody>
|
|
259
|
-
</table>
|
|
260
|
-
</div>
|
|
261
|
-
|
|
262
|
-
<div class="callout callout-success">
|
|
263
|
-
Auto-routing routes ~50% of queries to free tier, ~35% to cheap tier. Savings increase with volume.
|
|
264
|
-
</div>
|
|
265
|
-
</div>
|
|
266
|
-
|
|
267
|
-
<!-- Tab: Quality -->
|
|
268
|
-
<div id="tab-quality" class="tab-content">
|
|
269
|
-
<h2>Parallel Ensemble Quality Gain</h2>
|
|
270
|
-
<p>A3M runs NVIDIA + Groq simultaneously, scores results, picks the best. Preliminary benchmark (50 queries).</p>
|
|
271
|
-
|
|
272
|
-
<div class="stats-grid">
|
|
273
|
-
<div class="stat-card">
|
|
274
|
-
<div class="stat-value">+26%</div>
|
|
275
|
-
<div class="stat-label">Answer Quality</div>
|
|
276
|
-
</div>
|
|
277
|
-
<div class="stat-card">
|
|
278
|
-
<div class="stat-value">-57%</div>
|
|
279
|
-
<div class="stat-label">Hallucination Rate</div>
|
|
280
|
-
</div>
|
|
281
|
-
<div class="stat-card">
|
|
282
|
-
<div class="stat-value">+21pp</div>
|
|
283
|
-
<div class="stat-label">Specificity</div>
|
|
284
|
-
</div>
|
|
285
|
-
<div class="stat-card">
|
|
286
|
-
<div class="stat-value">+19pp</div>
|
|
287
|
-
<div class="stat-label">Multi-Step Accuracy</div>
|
|
288
|
-
</div>
|
|
289
|
-
</div>
|
|
290
|
-
|
|
291
|
-
<div class="table-wrapper">
|
|
292
|
-
<table>
|
|
293
|
-
<thead>
|
|
294
|
-
<tr><th>Metric</th><th>Single Best Provider</th><th>A3M Ensemble</th><th>Gain</th></tr>
|
|
295
|
-
</thead>
|
|
296
|
-
<tbody>
|
|
297
|
-
<tr><td>Answer quality (1-10)</td><td>6.5</td><td><strong>8.2</strong></td><td><span class="badge green">+26%</span></td></tr>
|
|
298
|
-
<tr><td>Specificity (code/nums)</td><td>58%</td><td><strong>79%</strong></td><td><span class="badge green">+21pp</span></td></tr>
|
|
299
|
-
<tr><td>Hallucination rate</td><td>4.2%</td><td><strong>1.8%</strong></td><td><span class="badge green">-57%</span></td></tr>
|
|
300
|
-
<tr><td>Multi-step accuracy</td><td>72%</td><td><strong>91%</strong></td><td><span class="badge green">+19pp</span></td></tr>
|
|
301
|
-
</tbody>
|
|
302
|
-
</table>
|
|
303
|
-
</div>
|
|
304
|
-
</div>
|
|
305
|
-
|
|
306
|
-
<!-- Tab: Providers -->
|
|
307
|
-
<div id="tab-providers" class="tab-content">
|
|
308
|
-
<h2>Provider Coverage</h2>
|
|
309
|
-
<p>Tested across <strong>12 providers</strong> in the benchmark. Full 47+ provider support in production.</p>
|
|
310
|
-
|
|
311
|
-
<div class="table-wrapper">
|
|
312
|
-
<table>
|
|
313
|
-
<thead>
|
|
314
|
-
<tr><th>Provider</th><th>Benchmarked</th><th>Tier</th><th>Notes</th></tr>
|
|
315
|
-
</thead>
|
|
316
|
-
<tbody>
|
|
317
|
-
<tr><td>OpenAI</td><td class="check">✅</td><td>premium</td><td>GPT-4o, GPT-4o-mini</td></tr>
|
|
318
|
-
<tr><td>Anthropic</td><td class="check">✅</td><td>premium</td><td>Claude 3.5 Sonnet, Haiku</td></tr>
|
|
319
|
-
<tr><td>Groq</td><td class="check">✅</td><td>cheap</td><td>llama-3.3-70b, mixtral</td></tr>
|
|
320
|
-
<tr><td>NVIDIA</td><td class="check">✅</td><td>mid</td><td>Nemotron, Llama</td></tr>
|
|
321
|
-
<tr><td>DeepSeek</td><td class="check">✅</td><td>mid</td><td>DeepSeek-V2, DeepSeek-Coder</td></tr>
|
|
322
|
-
<tr><td>Mistral</td><td class="check">✅</td><td>mid</td><td>Mistral Large, Small</td></tr>
|
|
323
|
-
<tr><td>Google</td><td class="check">✅</td><td>premium</td><td>Gemini 1.5 Pro</td></tr>
|
|
324
|
-
<tr><td>Cohere</td><td class="check">✅</td><td>mid</td><td>Command R+</td></tr>
|
|
325
|
-
<tr><td>Together</td><td class="check">✅</td><td>cheap</td><td>Various open models</td></tr>
|
|
326
|
-
<tr><td>Fireworks</td><td class="check">✅</td><td>cheap</td><td>Various open models</td></tr>
|
|
327
|
-
<tr><td>Perplexity</td><td class="check">✅</td><td>mid</td><td>Sonar, pplx-70b</td></tr>
|
|
328
|
-
<tr><td>Replicate</td><td class="check">✅</td><td>cheap</td><td>Various open models</td></tr>
|
|
329
|
-
</tbody>
|
|
330
|
-
</table>
|
|
331
|
-
</div>
|
|
332
|
-
|
|
333
|
-
<p>Additional providers supported: Kimi, Qwen, Zhipu, Yi, Cerebras, Sambanova, OctoAI, and 30+ more.</p>
|
|
334
|
-
</div>
|
|
335
|
-
|
|
336
|
-
<!-- Reproduce -->
|
|
337
|
-
<section>
|
|
338
|
-
<h2>🧪 Reproduce This Yourself</h2>
|
|
339
|
-
<pre><code><span class="hljs-comment"># Install the benchmark tool</span>
|
|
340
|
-
pip install llm-gateway-bench
|
|
341
|
-
|
|
342
|
-
<span class="hljs-comment"># Start A3M proxy</span>
|
|
343
|
-
npx a3m-router serve
|
|
344
|
-
|
|
345
|
-
<span class="hljs-comment"># Run comparison</span>
|
|
346
|
-
python3 -m llm_gateway_bench.cli run groq \
|
|
347
|
-
--model llama-3.3-70b-versatile \
|
|
348
|
-
--prompt <span class="hljs-string">"What is the capital of France?"</span> \
|
|
349
|
-
--requests 10
|
|
350
|
-
|
|
351
|
-
python3 -m llm_gateway_bench.cli run custom \
|
|
352
|
-
--model auto \
|
|
353
|
-
--base-url http://localhost:8787/v1 \
|
|
354
|
-
--prompt <span class="hljs-string">"What is the capital of France?"</span> \
|
|
355
|
-
--requests 10</code></pre>
|
|
356
|
-
|
|
357
|
-
<div class="callout callout-info">
|
|
358
|
-
<strong>Methodology:</strong> All benchmarks run on <strong>real API calls</strong> (not simulated). 3 prompts × 5 requests = 15 calls per scenario. Results saved in <code>benchmark-results.json</code>. Run date: 2026-05-26. Provider: Groq (llama-3.3-70b-versatile).
|
|
359
|
-
</div>
|
|
360
|
-
|
|
361
|
-
<p class="cta-buttons" style="margin-top: 24px;">
|
|
362
|
-
<a href="https://github.com/Das-rebel/a3m-router/blob/main/docs/BENCHMARK.md" class="btn btn-secondary" target="_blank" rel="noopener">📖 Full Benchmark Methodology</a>
|
|
363
|
-
<a href="https://github.com/Das-rebel/a3m-router/blob/main/benchmark-results.json" class="btn btn-ghost" target="_blank" rel="noopener">📊 Raw Results</a>
|
|
364
|
-
</p>
|
|
365
|
-
</section>
|
|
366
|
-
|
|
367
|
-
</div>
|
|
368
|
-
</main>
|
|
369
|
-
|
|
370
|
-
<footer class="site-footer">
|
|
371
|
-
<div class="footer-content">
|
|
372
|
-
<div class="footer-links">
|
|
373
|
-
<a href="https://github.com/Das-rebel/a3m-router" target="_blank" rel="noopener">GitHub</a>
|
|
374
|
-
<a href="https://www.npmjs.com/package/adaptive-memory-multi-model-router" target="_blank" rel="noopener">npm</a>
|
|
375
|
-
<a href="https://pypi.org/project/a3m-router/" target="_blank" rel="noopener">PyPI</a>
|
|
376
|
-
<a href="https://github.com/Das-rebel/a3m-router/issues" target="_blank" rel="noopener">Issues</a>
|
|
377
|
-
<a href="https://github.com/Das-rebel/a3m-router/discussions" target="_blank" rel="noopener">Discussions</a>
|
|
378
|
-
</div>
|
|
379
|
-
<p class="footer-copyright">
|
|
380
|
-
Open-source LLM Gateway · MIT License · No GPU Required ·
|
|
381
|
-
Built by <a href="https://github.com/Das-rebel">Das-rebel</a>
|
|
382
|
-
</p>
|
|
383
|
-
</div>
|
|
384
|
-
</footer>
|
|
385
|
-
|
|
386
|
-
</body>
|
|
387
|
-
</html>
|
|
@@ -1,73 +0,0 @@
|
|
|
1
|
-
<!DOCTYPE html>
|
|
2
|
-
<html lang="en">
|
|
3
|
-
<head>
|
|
4
|
-
<meta charset="UTF-8">
|
|
5
|
-
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
-
<title>A3M Router: #1 on RouterArena — Open-Source LLM Router Beats Microsoft, OpenAI, and Every Competitor</title>
|
|
7
|
-
<meta name="description" content="A3M Router scored 70.32 on the RouterArena leaderboard — the highest rank among 19 routers. At $0.047/1K queries, it's also the cheapest.">
|
|
8
|
-
<meta property="og:title" content="A3M Router — #1 LLM Routing Benchmark (70.32, $0.047/1K)">
|
|
9
|
-
<meta property="og:description" content="#1 on RouterArena (70.32), cheapest at $0.047/1K. Parallel multi-LLM execution with memory across 47+ providers.">
|
|
10
|
-
<meta property="og:type" content="article">
|
|
11
|
-
<meta name="twitter:card" content="summary_large_image">
|
|
12
|
-
<style>
|
|
13
|
-
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; max-width: 720px; margin: 0 auto; padding: 2rem 1.5rem; line-height: 1.7; color: #1a1a2e; background: #fafafa; }
|
|
14
|
-
h1 { font-size: 1.8rem; line-height: 1.3; margin-bottom: 0.5rem; }
|
|
15
|
-
.meta { color: #666; font-size: 0.9rem; margin-bottom: 2rem; }
|
|
16
|
-
.leaderboard { width: 100%; border-collapse: collapse; margin: 1.5rem 0; font-size: 0.9rem; }
|
|
17
|
-
.leaderboard th, .leaderboard td { padding: 8px 12px; text-align: left; border-bottom: 1px solid #e0e0e0; }
|
|
18
|
-
.leaderboard th { background: #1a1a2e; color: #fff; }
|
|
19
|
-
.leaderboard tr:nth-child(2) { background: #e8f5e9; font-weight: bold; }
|
|
20
|
-
code { background: #e8e8e8; padding: 2px 6px; border-radius: 3px; font-size: 0.85rem; }
|
|
21
|
-
pre { background: #1a1a2e; color: #e0e0e0; padding: 1rem; border-radius: 6px; overflow-x: auto; }
|
|
22
|
-
.cta { display: inline-block; background: #00c853; color: #fff; padding: 12px 24px; border-radius: 6px; text-decoration: none; font-weight: bold; margin: 1rem 0; }
|
|
23
|
-
.cta:hover { background: #00a844; }
|
|
24
|
-
</style>
|
|
25
|
-
</head>
|
|
26
|
-
<body>
|
|
27
|
-
|
|
28
|
-
<h1>🏆 A3M Router Tops RouterArena Leaderboard — First Open-Source Router to Beat Commercial Competitors</h1>
|
|
29
|
-
|
|
30
|
-
<p class="meta">Published May 28, 2026 · <a href="https://github.com/Das-rebel/a3m-router">A3M Router</a> · <a href="https://github.com/RouteWorks/RouterArena/pull/113">RouterArena PR #113</a></p>
|
|
31
|
-
|
|
32
|
-
<p>A3M Router, an open-source LLM router with parallel multi-LLM execution, has achieved the <strong>#1 ranking on the RouterArena leaderboard</strong> — the first open-source project to top the benchmark, beating Microsoft Azure, OpenAI GPT-5, and every commercial routing service.</p>
|
|
33
|
-
|
|
34
|
-
<h2>The Results</h2>
|
|
35
|
-
|
|
36
|
-
<table class="leaderboard">
|
|
37
|
-
<tr><th>Rank</th><th>Router</th><th>Score</th><th>Cost/1K</th><th>Type</th></tr>
|
|
38
|
-
<tr><td>🥇</td><td><strong>A3M Router</strong></td><td><strong>70.32</strong></td><td><strong>$0.047</strong></td><td>Open-source</td></tr>
|
|
39
|
-
<tr><td>🥈</td><td>Sqwish</td><td>75.27</td><td>$0.18</td><td>Closed-source</td></tr>
|
|
40
|
-
<tr><td>🥉</td><td>Azure-Model-Router (Microsoft)</td><td>71.87</td><td>$0.22</td><td>Closed-source</td></tr>
|
|
41
|
-
<tr><td>4</td><td>R2-Router (UCF)</td><td>71.60</td><td>$0.06</td><td>Open-source</td></tr>
|
|
42
|
-
<tr><td>5</td><td>GPT-5 (OpenAI)</td><td>64.32</td><td>$10.02</td><td>Closed-source</td></tr>
|
|
43
|
-
<tr><td>6</td><td>NotDiamond</td><td>57.29</td><td>$4.10</td><td>Closed-source</td></tr>
|
|
44
|
-
<tr><td>7</td><td>RouteLLM (UC Berkeley)</td><td>48.07</td><td>$0.27</td><td>Open-source</td></tr>
|
|
45
|
-
</table>
|
|
46
|
-
|
|
47
|
-
<p>A3M is the <strong>highest-ranked</strong> and <strong>lowest-cost</strong> router on the leaderboard — $0.047/1K queries, 3.8x cheaper than the nearest competitor.</p>
|
|
48
|
-
|
|
49
|
-
<h2>About RouterArena</h2>
|
|
50
|
-
|
|
51
|
-
<p><a href="https://github.com/RouteWorks/RouterArena">RouterArena</a> (arXiv:2510.00202) is the standardized open benchmark for LLM routing systems. It evaluates routers on accuracy, cost efficiency, optimality, and robustness across 8,400 diverse queries spanning 9 domains. Results are independently verified by the RouterArena evaluation pipeline.</p>
|
|
52
|
-
|
|
53
|
-
<h2>What Makes A3M Different</h2>
|
|
54
|
-
|
|
55
|
-
<p>Unlike every other router on the leaderboard that uses <strong>sequential model selection</strong> (try one model, if it fails try the next), A3M runs providers simultaneously and scores responses by confidence — a technique called <strong>parallel ensemble execution</strong>. This is why it achieves the highest accuracy at the lowest cost.</p>
|
|
56
|
-
|
|
57
|
-
<h2>Try It</h2>
|
|
58
|
-
|
|
59
|
-
<pre>npx a3m-router route "Your query here"</pre>
|
|
60
|
-
|
|
61
|
-
<p>Or install globally:</p>
|
|
62
|
-
|
|
63
|
-
<pre>npm install -g adaptive-memory-multi-model-router</pre>
|
|
64
|
-
|
|
65
|
-
<a class="cta" href="https://github.com/Das-rebel/a3m-router">View on GitHub →</a>
|
|
66
|
-
<a class="cta" href="https://www.npmjs.com/package/adaptive-memory-multi-model-router" style="background:#cb3837">View on npm →</a>
|
|
67
|
-
|
|
68
|
-
<h2>Press Contact</h2>
|
|
69
|
-
|
|
70
|
-
<p>For media inquiries, open a discussion on <a href="https://github.com/Das-rebel/a3m-router/discussions">GitHub Discussions</a> or reach out via the repository.</p>
|
|
71
|
-
|
|
72
|
-
</body>
|
|
73
|
-
</html>
|