adaptive-memory-multi-model-router 2.14.45 → 2.14.47
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.d.ts +4 -0
- package/dist/index.js +8 -2
- package/dist/memory/hybridMemory.d.ts +71 -0
- package/dist/memory/hybridMemory.js +124 -0
- package/dist/memory/reasoningBank.d.ts +88 -0
- package/dist/memory/reasoningBank.js +303 -0
- package/{docs/llms.txt → llms.txt.bak} +6 -6
- package/package.json +13 -84
- package/src/index.ts +8 -0
- package/src/memory/hybridMemory.ts +155 -0
- package/src/memory/reasoningBank.ts +335 -0
- package/src/routing/advancedRouter.ts.bak +650 -0
- package/test.js.bak +376 -0
- package/.dockerignore +0 -82
- package/.env.example +0 -303
- package/.github/DISCUSSIONS_WELCOME.md +0 -27
- package/.github/DISCUSSION_TEMPLATE.yml +0 -5
- package/.github/FUNDING.yml +0 -2
- package/.github/ISSUE_TEMPLATE/bug_report.md +0 -94
- package/.github/ISSUE_TEMPLATE/config.yml +0 -17
- package/.github/ISSUE_TEMPLATE/feature_request.md +0 -71
- package/.github/PULL_REQUEST_TEMPLATE.md +0 -71
- package/.github/dependabot.yml +0 -9
- package/.github/workflows/auto-publish.yml +0 -51
- package/.github/workflows/ci.yml +0 -263
- package/.github/workflows/codeql.yml +0 -38
- package/.github/workflows/npm-publish.yml +0 -20
- package/.github/workflows/pages.yml +0 -37
- package/.github/workflows/stale.yml +0 -54
- package/.publish-tick +0 -1
- package/.well-known/ai-plugin.json +0 -16
- package/AGENT_COUNCIL_FINDINGS.md +0 -142
- package/ARCHITECTURE.md +0 -346
- package/AUDIT_REPORT.md +0 -28
- package/CODE_OF_CONDUCT.md +0 -128
- package/CONTRIBUTING.md +0 -50
- package/CONTRIBUTORS.md +0 -20
- package/Dockerfile +0 -53
- package/Dockerfile.proxy +0 -33
- package/HEALTH_REPORT.md +0 -118
- package/IMPROVEMENT_PLAN.md +0 -107
- package/LANDING.md +0 -43
- package/LAUNCH-PAIN-DRIVEN.md +0 -339
- package/LAUNCH.md +0 -337
- package/LAUNCH_CHECKLIST.md +0 -141
- package/LAUNCH_SNAPSHOT.md +0 -260
- package/MANIFESTO.md +0 -41
- package/POPULARITY_BOOSTERS.md +0 -285
- package/PR_STATUS_REPORT.md +0 -148
- package/REDESIGN.md +0 -95
- package/RUNKIT.md +0 -83
- package/SECURITY.md +0 -29
- package/SUBMISSIONS.md +0 -43
- package/_schema.html +0 -53
- package/ai-plugin.json +0 -16
- package/articles/AI_AGENT_LLM_ROUTING.md +0 -150
- package/articles/CHINESE_DIRECTORIES.md +0 -100
- package/articles/CHINESE_SUBMISSIONS_READY.md +0 -322
- package/articles/COMPETITOR_ALERTS.md +0 -31
- package/articles/COMPLETE_POSTING_DIRECTORY.md +0 -147
- package/articles/CONTENT_STRUCTURE.md +0 -292
- package/articles/DEVTO_COST_GUIDE.md +0 -473
- package/articles/DEVTO_FINAL.md +0 -416
- package/articles/DEVTO_MULTI_PROVIDER.md +0 -542
- package/articles/DEVTO_READY.md +0 -255
- package/articles/DEVTO_V2_ANNOUNCEMENT.md +0 -160
- package/articles/DEVTO_VIRAL_GROWTH.md +0 -280
- package/articles/FRESH_devto.md +0 -460
- package/articles/FRESH_devto_2026_05.md +0 -73
- package/articles/FRESH_hackernews.md +0 -14
- package/articles/FRESH_reddit_ml.md +0 -90
- package/articles/FRESH_reddit_node.md +0 -198
- package/articles/FRESH_reddit_sideproject.md +0 -72
- package/articles/FRESH_reddit_webdev.md +0 -130
- package/articles/FROM_ZERO_TO_10K.md +0 -107
- package/articles/HN_10X_BETTER.md +0 -430
- package/articles/HN_ACCOUNT_GUIDE.md +0 -21
- package/articles/HN_CHINESE_STYLE.md +0 -308
- package/articles/HN_FINAL.md +0 -148
- package/articles/HN_POSTED_VERSION.md +0 -56
- package/articles/HN_POST_READY.md +0 -137
- package/articles/HN_RESEARCH.md +0 -364
- package/articles/HN_SHOW_routerarena.md +0 -17
- package/articles/HN_TIMING_GUIDE.md +0 -52
- package/articles/INDIEHACKERS_POST.md +0 -52
- package/articles/INDIEHACKERS_READY.md +0 -120
- package/articles/LLM_BENCHMARK_DEEP_DIVE.md +0 -153
- package/articles/MASTER_POSTING_DIRECTORY.md +0 -189
- package/articles/NEWSLETTER_SEND_NOW.md +0 -259
- package/articles/NEWSLETTER_SUBMISSIONS.md +0 -112
- package/articles/PAIN-DRIVEN-devto-v2.md +0 -308
- package/articles/PAIN-DRIVEN-devto-v3.md +0 -268
- package/articles/PAIN-DRIVEN-devto.md +0 -242
- package/articles/PAIN-DRIVEN-hackernews-v2.md +0 -138
- package/articles/PAIN-DRIVEN-hackernews-v3.md +0 -151
- package/articles/PAIN-DRIVEN-hackernews.md +0 -131
- package/articles/PAIN-DRIVEN-reddit-v2.md +0 -301
- package/articles/PAIN-DRIVEN-reddit-v3.md +0 -236
- package/articles/PAIN-DRIVEN-reddit.md +0 -218
- package/articles/PAIN-DRIVEN-twitter-v2.md +0 -110
- package/articles/PAIN-DRIVEN-twitter-v3.md +0 -121
- package/articles/PAIN-DRIVEN-twitter.md +0 -120
- package/articles/PORTKEY_VS_A3M.md +0 -147
- package/articles/POSTING_KIT_2026_05.md +0 -67
- package/articles/PRESS_KIT_routerarena.md +0 -77
- package/articles/PRODUCTHUNT_LISTING.md +0 -48
- package/articles/PRODUCTHUNT_READY.md +0 -106
- package/articles/PR_PLAN_vault.md +0 -125
- package/articles/REDDIT_FINAL.md +0 -232
- package/articles/REDDIT_POST.md +0 -67
- package/articles/REDDIT_SUBMISSION_READY.md +0 -348
- package/articles/ROUTERARENA_LEADER.md +0 -45
- package/articles/SHOW_HN_FINAL.md +0 -29
- package/articles/TWEETS_10K_DOWNLOADS.md +0 -47
- package/articles/TWEETS_BENCHMARK_FIRST.md +0 -46
- package/articles/TWEETS_MCP_PLAY.md +0 -51
- package/articles/TWEETS_SEQUENTIAL_BROKEN.md +0 -49
- package/articles/TWEETS_WHY_BUILD.md +0 -54
- package/articles/TWEETS_routerarena_leader.md +0 -53
- package/articles/TWEET_STORM_READY.md +0 -165
- package/articles/TWITTER_FINAL.md +0 -167
- package/articles/WHY_10X_BETTER.md +0 -261
- package/articles/WHY_CHINESE_STYLE_BETTER.md +0 -323
- package/articles/ai-discoverability-llm-routing.md +0 -210
- package/articles/devto-llm-routing.md +0 -138
- package/articles/hackernews-show-hn.md +0 -54
- package/articles/hashnode-llm-cost-optimization.md +0 -125
- package/articles/hn_show_2026_05.md +0 -11
- package/articles/medium-building-llm-router.md +0 -205
- package/articles/reddit-ml.md +0 -76
- package/articles/twitter-thread-cost-savings.md +0 -50
- package/articles/youtube-tutorial-script.md +0 -262
- package/assets/a3m_3blue1brown.mp4 +0 -0
- package/assets/banner.svg +0 -109
- package/assets/chart-cost-v2.svg +0 -91
- package/assets/chart-cost-v3.svg +0 -143
- package/assets/chart-features-v2.svg +0 -132
- package/assets/chart-features-v3.svg +0 -211
- package/assets/chart-growth-v2.svg +0 -122
- package/assets/chart-growth-v3.svg +0 -189
- package/assets/cost-comparison.svg +0 -134
- package/assets/cost-simple.svg +0 -64
- package/assets/demo-hn.gif +0 -0
- package/assets/feature-matrix.svg +0 -136
- package/assets/growth-chart-animated.svg +0 -76
- package/assets/growth-chart.svg +0 -82
- package/assets/growth-simple.svg +0 -69
- package/assets/hero-diagram.svg +0 -81
- package/assets/logo-new.svg +0 -21
- package/assets/logo.svg +0 -68
- package/assets/provider-comparison.svg +0 -121
- package/assets/social-preview-new.svg +0 -100
- package/assets/social-preview.svg +0 -194
- package/assets/social-v2.svg +0 -130
- package/assets/social-v3.svg +0 -212
- package/benchmark-provider-results.json +0 -245
- package/benchmark-results.json +0 -54
- package/council-votes/architecture-vote.md +0 -121
- package/council-votes/coverage-vote.md +0 -93
- package/data/adaptive-benchmark.json +0 -92
- package/data/benchmark-results.json +0 -47
- package/data/labeled-benchmark.json +0 -88
- package/demo/3blue1brown_video.py +0 -285
- package/demo/3blue1brown_video_v2.py +0 -310
- package/demo/IMPROVED_PROMPTS.md +0 -229
- package/demo/VEO3_PROMPTS.md +0 -269
- package/demo/VIDEO_PRODUCTION_GUIDE.md +0 -333
- package/demo/a3m_3blue1brown.mp4 +0 -0
- package/demo/asciinema-demo.sh +0 -195
- package/demo/demo-hn.tape +0 -74
- package/demo/demo-script.md +0 -53
- package/demo/demo-script.sh +0 -62
- package/demo/demo.svg +0 -75
- package/demo/frame1_ai_data_center.png +0 -0
- package/demo/frame1_sunset_video.mp4 +0 -0
- package/demo/frame2_cost_comparison.png +0 -0
- package/demo/frame2_cost_comparison_fallback.png +0 -0
- package/demo/frame3_parallel_execution.png +0 -0
- package/demo/frame3_parallel_execution_fallback.png +0 -0
- package/demo/frame4_providers.png +0 -0
- package/demo/frame4_providers_fallback.png +0 -0
- package/demo/frame5_endcard.png +0 -0
- package/demo/frame5_endcard_fallback.png +0 -0
- package/demo/new_frame1_hook.png +0 -0
- package/demo/new_frame2_proof.png +0 -0
- package/demo/new_frame3_wow.png +0 -0
- package/demo/new_frame4_social.png +0 -0
- package/demo/new_frame5_cta.png +0 -0
- package/demo/package.json +0 -13
- package/demo/product-video-final.mp4 +0 -0
- package/demo/product-video-hype-v1.mp4 +0 -0
- package/demo/product-video-v1.mp4 +0 -0
- package/demo/public/index.html +0 -762
- package/demo/recording.cast +0 -55
- package/demo/server.js +0 -405
- package/demo-new.tape +0 -71
- package/demo-real.sh +0 -198
- package/demo-simple.tape +0 -205
- package/demo.html +0 -520
- package/demo.sh +0 -85
- package/demo.tape +0 -259
- package/dist/analytics/costAnalytics.d.ts.map +0 -1
- package/dist/analytics/costAnalytics.js.map +0 -1
- package/dist/benchmark/comprehensive.js.map +0 -1
- package/dist/benchmark/reproducible.d.ts.map +0 -1
- package/dist/benchmark/reproducible.js.map +0 -1
- package/dist/cache/prefixCache.d.ts.map +0 -1
- package/dist/cache/prefixCache.js.map +0 -1
- package/dist/cache/responseCache.d.ts.map +0 -1
- package/dist/cache/responseCache.js.map +0 -1
- package/dist/cache/semanticCache.d.ts.map +0 -1
- package/dist/cache/semanticCache.js.map +0 -1
- package/dist/cli/setupWizard.d.ts.map +0 -1
- package/dist/cli/setupWizard.js.map +0 -1
- package/dist/cost/budgetEnforcer.d.ts.map +0 -1
- package/dist/cost/budgetEnforcer.js.map +0 -1
- package/dist/cost/costTracker.d.ts.map +0 -1
- package/dist/cost/costTracker.js.map +0 -1
- package/dist/ensemble/multiRoundDialog.js.map +0 -1
- package/dist/ensemble/shapleyValue.js.map +0 -1
- package/dist/integrations/langchainAdapter.d.ts.map +0 -1
- package/dist/integrations/langchainAdapter.js.map +0 -1
- package/dist/integrations/oauth.d.ts.map +0 -1
- package/dist/integrations/oauth.js.map +0 -1
- package/dist/integrations/scienceAdapter.js.map +0 -1
- package/dist/memory/autoFetch.d.ts.map +0 -1
- package/dist/memory/autoFetch.js.map +0 -1
- package/dist/memory/episodicMemory.d.ts.map +0 -1
- package/dist/memory/episodicMemory.js.map +0 -1
- package/dist/memory/memoryTree.d.ts.map +0 -1
- package/dist/memory/memoryTree.js.map +0 -1
- package/dist/memory/obsidianVault.d.ts.map +0 -1
- package/dist/memory/obsidianVault.js.map +0 -1
- package/dist/observability/changeWatch.d.ts.map +0 -1
- package/dist/observability/changeWatch.js.map +0 -1
- package/dist/observability/fatigueDetector.d.ts.map +0 -1
- package/dist/observability/fatigueDetector.js.map +0 -1
- package/dist/observability/index.d.ts.map +0 -1
- package/dist/observability/index.js.map +0 -1
- package/dist/observability/metrics.d.ts.map +0 -1
- package/dist/observability/metrics.js.map +0 -1
- package/dist/observability/middleware.d.ts.map +0 -1
- package/dist/observability/middleware.js.map +0 -1
- package/dist/observability/tracer.d.ts.map +0 -1
- package/dist/observability/tracer.js.map +0 -1
- package/dist/observability/types.d.ts.map +0 -1
- package/dist/observability/types.js.map +0 -1
- package/dist/orchestration/haloOrchestrator.d.ts.map +0 -1
- package/dist/orchestration/haloOrchestrator.js.map +0 -1
- package/dist/orchestration/mctsWorkflow.d.ts.map +0 -1
- package/dist/orchestration/mctsWorkflow.js.map +0 -1
- package/dist/providers/localProvider.d.ts.map +0 -1
- package/dist/providers/localProvider.js.map +0 -1
- package/dist/providers/providerConfig.d.ts.map +0 -1
- package/dist/providers/providerConfig.js.map +0 -1
- package/dist/providers/registry.d.ts.map +0 -1
- package/dist/providers/registry.js.map +0 -1
- package/dist/routing/advancedRouter.d.ts.map +0 -1
- package/dist/routing/advancedRouter.js.map +0 -1
- package/dist/routing/crossModelValidation.d.ts.map +0 -1
- package/dist/routing/crossModelValidation.js.map +0 -1
- package/dist/routing/providerHealth.d.ts.map +0 -1
- package/dist/routing/providerHealth.js.map +0 -1
- package/dist/routing/providerRetry.d.ts.map +0 -1
- package/dist/routing/providerRetry.js.map +0 -1
- package/dist/scripts/banner.js +0 -29
- package/dist/security/guardrails.d.ts.map +0 -1
- package/dist/security/guardrails.js.map +0 -1
- package/dist/server/dashboard.d.ts.map +0 -1
- package/dist/server/dashboard.js.map +0 -1
- package/dist/server/modelMapper.d.ts.map +0 -1
- package/dist/server/modelMapper.js.map +0 -1
- package/dist/server/proxyServer.d.ts.map +0 -1
- package/dist/server/proxyServer.js.map +0 -1
- package/dist/skills/__tests__/skill_manager.test.d.ts +0 -2
- package/dist/skills/__tests__/skill_manager.test.d.ts.map +0 -1
- package/dist/skills/__tests__/skill_manager.test.js +0 -268
- package/dist/skills/__tests__/skill_manager.test.js.map +0 -1
- package/dist/tools/tmlpdTools.d.ts.map +0 -1
- package/dist/tools/tmlpdTools.js.map +0 -1
- package/dist/tui/dashboard.d.ts.map +0 -1
- package/dist/tui/dashboard.js.map +0 -1
- package/dist/tui/index.d.ts.map +0 -1
- package/dist/tui/index.js.map +0 -1
- package/dist/utils/batchProcessor.d.ts.map +0 -1
- package/dist/utils/batchProcessor.js.map +0 -1
- package/dist/utils/compression.d.ts.map +0 -1
- package/dist/utils/compression.js.map +0 -1
- package/dist/utils/costUtils.d.ts.map +0 -1
- package/dist/utils/costUtils.js.map +0 -1
- package/dist/utils/reliability.d.ts.map +0 -1
- package/dist/utils/reliability.js.map +0 -1
- package/dist/utils/sorting.d.ts.map +0 -1
- package/dist/utils/sorting.js.map +0 -1
- package/dist/utils/speculativeDecoding.d.ts.map +0 -1
- package/dist/utils/speculativeDecoding.js.map +0 -1
- package/dist/utils/tokenUtils.d.ts.map +0 -1
- package/dist/utils/tokenUtils.js.map +0 -1
- package/docs/.nojekyll +0 -0
- package/docs/ANALYSIS_PRINCIPLES.md +0 -162
- package/docs/API.md +0 -855
- package/docs/ARCHITECTURAL-IMPROVEMENTS-2025.md +0 -1391
- package/docs/ARCHITECTURAL-IMPROVEMENTS-REVISED-2025.md +0 -1051
- package/docs/BENCHMARK.md +0 -170
- package/docs/CHINESE_PROVIDER_RELIABILITY.md +0 -37
- package/docs/CITATIONS.md +0 -74
- package/docs/CLAIMS_AND_EVIDENCE.md +0 -58
- package/docs/CONFIGURATION.md +0 -476
- package/docs/COUNCIL_DECISION.json +0 -816
- package/docs/COUNCIL_SUMMARY.md +0 -319
- package/docs/COUNCIL_V2.2_DECISION.md +0 -416
- package/docs/ENGINEERING_SPEC.md +0 -55
- package/docs/FACTORY_RESET.md +0 -34
- package/docs/GEO.md +0 -66
- package/docs/GEO_OPTIMIZATION.md +0 -30
- package/docs/GEO_ROOT_CAUSE.md +0 -136
- package/docs/GEO_STATUS.md +0 -85
- package/docs/GEO_TEST_RESULTS.md +0 -176
- package/docs/HN_CHECKLIST.md +0 -38
- package/docs/HN_FOUNDER_COMMENT.md +0 -17
- package/docs/HN_SUBMISSION_FINAL.md +0 -180
- package/docs/HN_SUBMISSION_V3.md +0 -56
- package/docs/IMPROVEMENT_ROADMAP.md +0 -515
- package/docs/INTEGRATIONS.md +0 -420
- package/docs/LANGCHAIN_INTEGRATION.md +0 -147
- package/docs/LLM_COUNCIL_DECISION.md +0 -508
- package/docs/MIDDLEWARE_CHAIN.md +0 -35
- package/docs/PROMO_CHECKLIST.md +0 -200
- package/docs/QUICKSTART.md +0 -271
- package/docs/QUICK_START.md +0 -43
- package/docs/QUICK_START_VISIBILITY.md +0 -782
- package/docs/REDDIT_GAP_ANALYSIS.md +0 -299
- package/docs/RELEASE_CHECKLIST.md +0 -32
- package/docs/REPRODUCIBILITY.md +0 -63
- package/docs/RESEARCH_BACKED_IMPROVEMENTS.md +0 -1180
- package/docs/ROUTING_RUBRIC.md +0 -197
- package/docs/SEO_AUDIT.md +0 -186
- package/docs/SOCIAL_LISTENING.md +0 -219
- package/docs/TMLPD_QNA.md +0 -751
- package/docs/TMLPD_V2.1_COMPLETE.md +0 -763
- package/docs/TMLPD_V2.2_RESEARCH_ROADMAP.md +0 -754
- package/docs/UPDATE_TOPICS.md +0 -15
- package/docs/USE_CASES.md +0 -59
- package/docs/V2.2_IMPLEMENTATION_COMPLETE.md +0 -446
- package/docs/V2_IMPLEMENTATION_GUIDE.md +0 -388
- package/docs/VERCEL_AI_SDK.md +0 -209
- package/docs/VISIBILITY_ADOPTION_PLAN.md +0 -1005
- package/docs/_config.yml +0 -49
- package/docs/ai-plugin.json +0 -16
- package/docs/api.html +0 -513
- package/docs/architecture-diagram.md +0 -40
- package/docs/benchmark-chart.png +0 -0
- package/docs/benchmark.html +0 -387
- package/docs/blog/routerarena-number-one.html +0 -73
- package/docs/cli-cheatsheet.md +0 -339
- package/docs/compare.md +0 -109
- package/docs/comparison-litellm.md +0 -88
- package/docs/comparison.md +0 -108
- package/docs/cost-chart-ascii.md +0 -42
- package/docs/cost-comparison-chart.svg +0 -88
- package/docs/curl-examples.md +0 -247
- package/docs/demo-auto.html +0 -264
- package/docs/demo.html +0 -416
- package/docs/geo/GENERATIVE_ENGINE_OPTIMIZATION.md +0 -232
- package/docs/index.html +0 -507
- package/docs/launch-content/LAUNCH_EXECUTION_CHECKLIST.md +0 -421
- package/docs/launch-content/README.md +0 -457
- package/docs/launch-content/assets/cost_comparison_100_tasks.png +0 -0
- package/docs/launch-content/assets/cumulative_savings.png +0 -0
- package/docs/launch-content/assets/parallel_speedup.png +0 -0
- package/docs/launch-content/assets/provider_pricing_comparison.png +0 -0
- package/docs/launch-content/assets/task_breakdown_comparison.png +0 -0
- package/docs/launch-content/generate_charts.py +0 -313
- package/docs/launch-content/hn_show_post.md +0 -139
- package/docs/launch-content/partner_outreach_templates.md +0 -745
- package/docs/launch-content/reddit_posts.md +0 -467
- package/docs/launch-content/twitter_thread.txt +0 -460
- package/docs/npm-downloads-chart.svg +0 -43
- package/docs/openapi.json +0 -139
- package/docs/openapi.yaml +0 -1318
- package/docs/quick-start.html +0 -366
- package/docs/robots.txt +0 -52
- package/docs/sitemap.xml +0 -57
- package/docs/styles.css +0 -682
- package/docs/well-known/ai-plugin.json +0 -16
- package/docs/wellknown/ai-plugin.json +0 -16
- package/docs-site/assets/og-banner.svg +0 -194
- package/docs-site/index.html +0 -632
- package/eval/README.md +0 -46
- package/eval/baselines/main.json +0 -12
- package/eval/benchmark_dataset.jsonl +0 -16
- package/eval/check_golden_routes.js +0 -64
- package/eval/datasets/catalog.json +0 -33
- package/eval/datasets/slices/cn_provider_reliability_v1.jsonl +0 -3
- package/eval/datasets/slices/cost_pressure_v1.jsonl +0 -3
- package/eval/datasets/slices/safety_guardrails_v1.jsonl +0 -3
- package/eval/evals.json +0 -199
- package/eval/fault_injection_thresholds.json +0 -3
- package/eval/generate_report.js +0 -128
- package/eval/golden_routes.json +0 -114
- package/eval/lib/experiment_registry.js +0 -24
- package/eval/run_eval.js +0 -197
- package/eval/run_fault_injection.js +0 -201
- package/eval/run_shadow_eval.js +0 -85
- package/eval/thresholds.json +0 -9
- package/examples/QUICKSTART.md +0 -183
- package/examples/README.md +0 -61
- package/examples/a3m-sdk.js +0 -124
- package/examples/basic-route.js +0 -54
- package/examples/chat-loop.js +0 -202
- package/examples/classify-then-route.js +0 -102
- package/examples/cost-compare.js +0 -120
- package/examples/ensemble.js +0 -160
- package/examples/whatsapp-telegram-bridge-demo.js +0 -302
- package/examples/whatsapp-telegram-bridge.js +0 -269
- package/hf-space/README.md +0 -23
- package/hf-space/app.py +0 -240
- package/hf-space/requirements.txt +0 -1
- package/huggingface_space/README.md +0 -35
- package/huggingface_space/app.py +0 -126
- package/huggingface_space/create_space.py +0 -208
- package/huggingface_space/requirements.txt +0 -1
- package/mcp-server/README.md +0 -188
- package/mcp-server/package.json +0 -29
- package/mcp-server/src/index.ts +0 -744
- package/mcp-server/tsconfig.json +0 -19
- package/openclaw-alexa-bridge/ALL_REMAINING_FIXES_PLAN.md +0 -313
- package/openclaw-alexa-bridge/REMAINING_FIXES_SUMMARY.md +0 -277
- package/openclaw-alexa-bridge/src/alexa_handler_no_tmlpd.js +0 -1234
- package/openclaw-alexa-bridge/test_fixes.js +0 -77
- package/playground/README.md +0 -51
- package/playground/codesandbox.json +0 -12
- package/playground/index.js +0 -39
- package/proxy/README.md +0 -227
- package/proxy/package-lock.json +0 -831
- package/proxy/package.json +0 -17
- package/proxy/rate-limit.js +0 -145
- package/proxy/rate-limit.test.js +0 -311
- package/proxy/server.js +0 -970
- package/python/README.md +0 -102
- package/python/a3m/__init__.py +0 -6
- package/python/a3m/client.py +0 -190
- package/python/a3m/models.py +0 -40
- package/python/a3m/sync_client.py +0 -61
- package/python/examples.py +0 -53
- package/python/integrations.py +0 -330
- package/python/pyproject.toml +0 -23
- package/python/setup.py +0 -28
- package/python/tmlpd.py +0 -369
- package/qna/REDDIT_GAP_ANALYSIS.md +0 -299
- package/qna/TMLPD_QNA.md +0 -751
- package/research/FINDING_001_safety.md +0 -28
- package/research/FINDING_002_error_diversity.md +0 -32
- package/research/FINDING_003_confidence_weighted_voting.md +0 -32
- package/research/FINDING_004_cross_model_semantic_detection.md +0 -37
- package/research/FINDING_005_knowledge_gap_orthogonality.md +0 -34
- package/research/HALLUCINATION_RESEARCH.md +0 -27
- package/research/ensemble-voting.md +0 -324
- package/research/loss-functions.md +0 -545
- package/research-log.md +0 -49
- package/scripts/banner.js +0 -29
- package/scripts/benchmark-local-routerarena.ts +0 -176
- package/scripts/benchmark.js +0 -145
- package/scripts/benchmark.sh +0 -61
- package/scripts/compare-providers.sh +0 -230
- package/scripts/content-planner.js +0 -25
- package/scripts/create-labeled-benchmark.ts +0 -105
- package/scripts/cross_post.py +0 -443
- package/scripts/local-router-benchmark.ts +0 -154
- package/scripts/post-all.sh +0 -41
- package/scripts/publish_fcc.py +0 -106
- package/scripts/push-to-gitee.sh +0 -25
- package/scripts/routerarena_ensemble.js +0 -144
- package/scripts/routing-benchmark-v2.js +0 -373
- package/scripts/routing-benchmark-v3.js +0 -118
- package/scripts/routing-benchmark.js +0 -462
- package/scripts/run-labeled-benchmark.mjs +0 -104
- package/scripts/run-mmlu-benchmark.js +0 -176
- package/scripts/run-provider-benchmark.js +0 -244
- package/scripts/update-npm-badges.js +0 -158
- package/skill/SKILL.md +0 -238
- package/src/__tests__/integration/tmpld_integration.test.py +0 -540
- package/src/skills/__tests__/skill_manager.test.ts +0 -328
- package/submissions/benchmarks/ALL_PLATFORMS_SUBMISSION.md +0 -94
- package/submissions/benchmarks/LLMROUTERBENCH_SUBMISSION.md +0 -121
- package/submissions/benchmarks/MMRBENCH_SUBMISSION.md +0 -94
- package/submissions/benchmarks/ROUTERARENA_UPDATE.md +0 -83
- package/submissions/benchmarks/ROUTERBENCH_SUBMISSION.md +0 -225
- package/test-council/1-structure-tests.test.js +0 -353
- package/test-council/1-structure-tests.test.ts +0 -353
- package/test-council/2-edge-case-tests.test.ts +0 -361
- package/test-council/3-performance-tests.test.ts +0 -669
- package/test-council/4-integration-tests.test.ts +0 -391
- package/test-council/5-agent-council-eval.test.ts +0 -413
- package/test-council/AGENT_COUNCIL_ARCHITECTURE.md +0 -349
- package/test-council/TEST_COUNCIL_REPORT.md +0 -201
- package/test-council/agents/edge-case-agent.ts +0 -363
- package/test-council/agents/performance-agent.ts +0 -426
- package/test-council/agents/structure-agent.ts +0 -227
- package/test-council/council.md +0 -183
- package/tests/__mocks__/tokenUtils.ts +0 -8
- package/tests/memory/episodicMemory.test.ts +0 -227
- package/tests/package-lock.json +0 -1628
- package/tests/package.json +0 -18
- package/tests/routing/ensembleVoting.test.ts +0 -236
- package/tests/routing/providerRetry.test.ts +0 -360
- package/tests/routing/queryTypePresets.test.ts +0 -208
- package/tests/security/guardrailEngine.test.ts +0 -700
- package/tests/tsconfig.json +0 -21
- package/tests/vitest.config.ts +0 -18
- package/tmlpd-pi-extension/README.md +0 -66
- package/tmlpd-pi-extension/dist/cache/prefixCache.d.ts +0 -114
- package/tmlpd-pi-extension/dist/cache/prefixCache.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/cache/prefixCache.js +0 -285
- package/tmlpd-pi-extension/dist/cache/prefixCache.js.map +0 -1
- package/tmlpd-pi-extension/dist/cache/responseCache.d.ts +0 -58
- package/tmlpd-pi-extension/dist/cache/responseCache.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/cache/responseCache.js +0 -153
- package/tmlpd-pi-extension/dist/cache/responseCache.js.map +0 -1
- package/tmlpd-pi-extension/dist/cli.js +0 -59
- package/tmlpd-pi-extension/dist/cost/costTracker.d.ts +0 -95
- package/tmlpd-pi-extension/dist/cost/costTracker.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/cost/costTracker.js +0 -240
- package/tmlpd-pi-extension/dist/cost/costTracker.js.map +0 -1
- package/tmlpd-pi-extension/dist/index.d.ts +0 -723
- package/tmlpd-pi-extension/dist/index.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/index.js +0 -239
- package/tmlpd-pi-extension/dist/index.js.map +0 -1
- package/tmlpd-pi-extension/dist/memory/episodicMemory.d.ts +0 -82
- package/tmlpd-pi-extension/dist/memory/episodicMemory.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/memory/episodicMemory.js +0 -145
- package/tmlpd-pi-extension/dist/memory/episodicMemory.js.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.d.ts +0 -102
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.js +0 -207
- package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.js.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.d.ts +0 -85
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.js +0 -210
- package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.js.map +0 -1
- package/tmlpd-pi-extension/dist/providers/localProvider.d.ts +0 -102
- package/tmlpd-pi-extension/dist/providers/localProvider.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/providers/localProvider.js +0 -338
- package/tmlpd-pi-extension/dist/providers/localProvider.js.map +0 -1
- package/tmlpd-pi-extension/dist/providers/registry.d.ts +0 -55
- package/tmlpd-pi-extension/dist/providers/registry.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/providers/registry.js +0 -138
- package/tmlpd-pi-extension/dist/providers/registry.js.map +0 -1
- package/tmlpd-pi-extension/dist/routing/advancedRouter.d.ts +0 -68
- package/tmlpd-pi-extension/dist/routing/advancedRouter.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/routing/advancedRouter.js +0 -332
- package/tmlpd-pi-extension/dist/routing/advancedRouter.js.map +0 -1
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.d.ts +0 -101
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.js +0 -368
- package/tmlpd-pi-extension/dist/tools/tmlpdTools.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/batchProcessor.d.ts +0 -96
- package/tmlpd-pi-extension/dist/utils/batchProcessor.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/batchProcessor.js +0 -170
- package/tmlpd-pi-extension/dist/utils/batchProcessor.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/compression.d.ts +0 -61
- package/tmlpd-pi-extension/dist/utils/compression.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/compression.js +0 -281
- package/tmlpd-pi-extension/dist/utils/compression.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/reliability.d.ts +0 -74
- package/tmlpd-pi-extension/dist/utils/reliability.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/reliability.js +0 -177
- package/tmlpd-pi-extension/dist/utils/reliability.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.d.ts +0 -117
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.js +0 -246
- package/tmlpd-pi-extension/dist/utils/speculativeDecoding.js.map +0 -1
- package/tmlpd-pi-extension/dist/utils/tokenUtils.d.ts +0 -50
- package/tmlpd-pi-extension/dist/utils/tokenUtils.d.ts.map +0 -1
- package/tmlpd-pi-extension/dist/utils/tokenUtils.js +0 -124
- package/tmlpd-pi-extension/dist/utils/tokenUtils.js.map +0 -1
- package/tmlpd-pi-extension/examples/QUICKSTART.md +0 -183
- package/tmlpd-pi-extension/package-lock.json +0 -79
- package/tmlpd-pi-extension/package.json +0 -172
- package/tmlpd-pi-extension/python/examples.py +0 -53
- package/tmlpd-pi-extension/python/integrations.py +0 -330
- package/tmlpd-pi-extension/python/setup.py +0 -28
- package/tmlpd-pi-extension/python/tmlpd.py +0 -369
- package/tmlpd-pi-extension/qna/REDDIT_GAP_ANALYSIS.md +0 -299
- package/tmlpd-pi-extension/qna/TMLPD_QNA.md +0 -751
- package/tmlpd-pi-extension/skill/SKILL.md +0 -238
- package/tmlpd-pi-extension/src/cache/responseCache.ts +0 -147
- package/tmlpd-pi-extension/src/cost/costTracker.ts +0 -302
- package/tmlpd-pi-extension/src/index.ts +0 -232
- package/tmlpd-pi-extension/src/memory/episodicMemory.ts +0 -257
- package/tmlpd-pi-extension/src/orchestration/haloOrchestrator.ts +0 -266
- package/tmlpd-pi-extension/src/orchestration/mctsWorkflow.ts +0 -262
- package/tmlpd-pi-extension/src/providers/localProvider.ts +0 -406
- package/tmlpd-pi-extension/src/providers/registry.ts +0 -164
- package/tmlpd-pi-extension/src/routing/ensembleVoting.ts +0 -159
- package/tmlpd-pi-extension/src/routing/queryTypePresets.ts +0 -136
- package/tmlpd-pi-extension/src/tools/tmlpdTools.ts +0 -433
- package/tmlpd-pi-extension/src/utils/batchProcessor.ts +0 -232
- package/tmlpd-pi-extension/src/utils/compression.ts +0 -325
- package/tmlpd-pi-extension/src/utils/reliability.ts +0 -221
- package/tmlpd-pi-extension/src/utils/tokenUtils.ts +0 -145
- package/tmlpd-pi-extension/tsconfig.json +0 -18
- package/tsconfig.build.json +0 -29
- package/tsconfig.json +0 -18
- /package/{docs/llms-full.txt → llms-full.txt.bak} +0 -0
package/docs/GEO_ROOT_CAUSE.md
DELETED
|
@@ -1,136 +0,0 @@
|
|
|
1
|
-
# A3M Router — Why Not Being Cited: ROOT CAUSE
|
|
2
|
-
|
|
3
|
-
**Date:** Sat May 31 18:50 IST 2026
|
|
4
|
-
**Status:** PR open but NOT merged — evaluation FAILED
|
|
5
|
-
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
## 🚨 Root Cause: 72.8% Prediction Failure
|
|
9
|
-
|
|
10
|
-
The RouterArena evaluation shows:
|
|
11
|
-
```
|
|
12
|
-
RouterArena Score: 0.2222 (not 0.7643!)
|
|
13
|
-
Accuracy: 20.74% (not 76.28%!)
|
|
14
|
-
Abnormal Entries: 6116 of 8400 (72.8% failed)
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
**Why?** When RouterArena tried to actually call models for A3M's routing decisions, 72.8% of the calls **failed to get any response**.
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## The Problem
|
|
22
|
-
|
|
23
|
-
AI engines look at the **official RouterArena leaderboard** (https://routeworks.github.io/leaderboard) to cite routers.
|
|
24
|
-
|
|
25
|
-
A3M is NOT on that leaderboard because our PR has failing evaluations.
|
|
26
|
-
|
|
27
|
-
| Router | RouterArena Score | Status |
|
|
28
|
-
|--------|------------------|--------|
|
|
29
|
-
| Sqwish | 75.27 | ✅ Official |
|
|
30
|
-
| Weave | 72.82 | ✅ Official |
|
|
31
|
-
| Azure-Model-Router | 71.87 | ✅ Official |
|
|
32
|
-
| ... | ... | ... |
|
|
33
|
-
| **A3M** | **NOT LISTED** | ❌ PR failing |
|
|
34
|
-
|
|
35
|
-
---
|
|
36
|
-
|
|
37
|
-
## Why 72.8% Failed
|
|
38
|
-
|
|
39
|
-
From the evaluation output:
|
|
40
|
-
```
|
|
41
|
-
> ⚠️ **6116 of 8400 queries (72.8%) had no valid generation**
|
|
42
|
-
(inference failed / returned empty)
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
Possible causes:
|
|
46
|
-
1. **API keys not configured** in RouterArena CI environment
|
|
47
|
-
2. **Rate limiting** from API providers
|
|
48
|
-
3. **Invalid model names** in predictions
|
|
49
|
-
4. **Missing provider setup** for the models A3M routes to
|
|
50
|
-
|
|
51
|
-
---
|
|
52
|
-
|
|
53
|
-
## What We Need to Fix
|
|
54
|
-
|
|
55
|
-
### 🔴 Fix the RouterArena PR (Priority 1)
|
|
56
|
-
|
|
57
|
-
The PR is open but failing. We need to debug why model inference is failing.
|
|
58
|
-
|
|
59
|
-
**Files in PR:**
|
|
60
|
-
- `router_inference/router/a3m_router.py` — our router code
|
|
61
|
-
- `router_inference/config/a3m-router.json` — config with models
|
|
62
|
-
- `router_inference/predictions/a3m-router.json` — our routing decisions
|
|
63
|
-
|
|
64
|
-
**The issue:** Our predictions say "route to model X" but when RouterArena tries to actually call those models, they fail.
|
|
65
|
-
|
|
66
|
-
### Solutions:
|
|
67
|
-
|
|
68
|
-
**Option A: Fix API Configuration**
|
|
69
|
-
Add API keys to the RouterArena environment or mock the calls
|
|
70
|
-
|
|
71
|
-
**Option B: Use mock/cached predictions**
|
|
72
|
-
Use cached results so the evaluation doesn't need live API calls
|
|
73
|
-
|
|
74
|
-
**Option C: Simplify the model list**
|
|
75
|
-
Use only models that are guaranteed to work (e.g., OpenAI models with API key)
|
|
76
|
-
|
|
77
|
-
---
|
|
78
|
-
|
|
79
|
-
## Immediate Actions
|
|
80
|
-
|
|
81
|
-
### 1. Debug the Prediction Failures
|
|
82
|
-
Check what's happening with model inference:
|
|
83
|
-
```bash
|
|
84
|
-
# Look at the error logs in the PR
|
|
85
|
-
# The evaluation ran but 72.8% of calls failed
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
### 2. Check our config file
|
|
89
|
-
Make sure all models in our config are valid and have API access
|
|
90
|
-
|
|
91
|
-
### 3. Contact RouterArena maintainers
|
|
92
|
-
Ask for help debugging the inference failures
|
|
93
|
-
|
|
94
|
-
### 4. Alternatively: Submit to LLMRouterBench
|
|
95
|
-
Don't depend on RouterArena — submit to ACL's LLMRouterBench as well
|
|
96
|
-
|
|
97
|
-
---
|
|
98
|
-
|
|
99
|
-
## Alternative: Submit to Multiple Benchmarks
|
|
100
|
-
|
|
101
|
-
RouterArena is one leaderboard. There are others:
|
|
102
|
-
|
|
103
|
-
| Benchmark | Venue | Status |
|
|
104
|
-
|-----------|-------|--------|
|
|
105
|
-
| **RouterArena** | RouteWorks | ❌ PR failing |
|
|
106
|
-
| **LLMRouterBench** | ACL 2026 | ⏳ Not submitted |
|
|
107
|
-
| **RouterEval** | Industry | ⏳ Not submitted |
|
|
108
|
-
| **MT Bench** | Academic | ⏳ Not submitted |
|
|
109
|
-
|
|
110
|
-
**Goal:** Get cited in at least ONE major benchmark
|
|
111
|
-
|
|
112
|
-
---
|
|
113
|
-
|
|
114
|
-
## Honest Assessment
|
|
115
|
-
|
|
116
|
-
A3M has:
|
|
117
|
-
- ✅ Self-reported 70.32 score
|
|
118
|
-
- ✅ Open PR at RouterArena
|
|
119
|
-
- ❌ 72.8% evaluation failure rate
|
|
120
|
-
- ❌ Not on official leaderboard
|
|
121
|
-
|
|
122
|
-
This is why AI isn't citing A3M. We need to:
|
|
123
|
-
1. **Fix the RouterArena PR** (hardest, highest impact)
|
|
124
|
-
2. **Submit to LLMRouterBench** (alternative)
|
|
125
|
-
3. **Get on at least one official leaderboard**
|
|
126
|
-
|
|
127
|
-
---
|
|
128
|
-
|
|
129
|
-
## Next Steps
|
|
130
|
-
|
|
131
|
-
1. **Debug the 72.8% failure** — why are model calls failing?
|
|
132
|
-
2. **Fix the PR** — get successful evaluation
|
|
133
|
-
3. **Get merged** — appear on RouterArena leaderboard
|
|
134
|
-
4. **Submit to more benchmarks** — diversify citations
|
|
135
|
-
|
|
136
|
-
Without being on an official leaderboard, AI engines won't cite A3M. The self-reported numbers aren't enough.
|
package/docs/GEO_STATUS.md
DELETED
|
@@ -1,85 +0,0 @@
|
|
|
1
|
-
# A3M Router — GEO (Generative Engine Optimization) Status
|
|
2
|
-
|
|
3
|
-
**Updated:** June 1, 2026 12:40 IST
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## What is GEO?
|
|
8
|
-
|
|
9
|
-
GEO = Getting your brand/product cited by AI engines (ChatGPT, Perplexity, Claude, Gemini, Google AI Overviews).
|
|
10
|
-
|
|
11
|
-
Unlike SEO (Google ranking), GEO is about being the **source cited in AI answers**.
|
|
12
|
-
|
|
13
|
-
---
|
|
14
|
-
|
|
15
|
-
## Current GEO Assets Status
|
|
16
|
-
|
|
17
|
-
### ✅ ALL ASSETS WORKING
|
|
18
|
-
|
|
19
|
-
| Asset | URL | Purpose | Status |
|
|
20
|
-
|-------|-----|---------|--------|
|
|
21
|
-
| **llms.txt** | https://das-rebel.github.io/a3m-router/llms.txt | AI-readable summary | ✅ 200 |
|
|
22
|
-
| **llms-full.txt** | https://das-rebel.github.io/a3m-router/llms-full.txt | Full LLM documentation | ✅ 200 |
|
|
23
|
-
| **sitemap.xml** | https://das-rebel.github.io/a3m-router/sitemap.xml | AI crawler guidance | ✅ 200 |
|
|
24
|
-
| **robots.txt** | https://das-rebel.github.io/a3m-router/robots.txt | AI bot permissions | ✅ 200 |
|
|
25
|
-
| **benchmark-chart.png** | https://das-rebel.github.io/a3m-router/benchmark-chart.png | Visual proof for citations | ✅ 200 |
|
|
26
|
-
| **openapi.json** | https://das-rebel.github.io/a3m-router/openapi.json | API spec for AI plugins | ✅ 200 |
|
|
27
|
-
| **ai-plugin.json** | https://das-rebel.github.io/a3m-router/ai-plugin.json | AI plugin manifest | ✅ 200 |
|
|
28
|
-
|
|
29
|
-
### ✅ AI Bot Access (robots.txt)
|
|
30
|
-
|
|
31
|
-
```
|
|
32
|
-
User-agent: * Allow: /
|
|
33
|
-
User-agent: GPTBot Allow: /
|
|
34
|
-
User-agent: ChatGPT-User Allow: /
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
---
|
|
38
|
-
|
|
39
|
-
## Pages (All 200 OK)
|
|
40
|
-
|
|
41
|
-
| Page | URL |
|
|
42
|
-
|------|-----|
|
|
43
|
-
| Home | / |
|
|
44
|
-
| Quick Start | /quick-start |
|
|
45
|
-
| Benchmark | /benchmark |
|
|
46
|
-
| API | /api |
|
|
47
|
-
|
|
48
|
-
---
|
|
49
|
-
|
|
50
|
-
## Key Claims for AI Citation
|
|
51
|
-
|
|
52
|
-
1. **Cheapest Router:** $0.05/1K (3.6× cheaper than RouteLLM)
|
|
53
|
-
2. **RouterArena Score:** 70.32
|
|
54
|
-
3. **Providers:** 47+
|
|
55
|
-
4. **NPM Downloads:** 15K+ in 17 days
|
|
56
|
-
5. **Size:** 19.5KB, zero ML dependencies
|
|
57
|
-
|
|
58
|
-
---
|
|
59
|
-
|
|
60
|
-
## Open PRs (Waiting for Backlinks)
|
|
61
|
-
|
|
62
|
-
| PR | Status |
|
|
63
|
-
|----|--------|
|
|
64
|
-
| awesome-llm-apps | OPEN |
|
|
65
|
-
| Awesome-LLMOps | OPEN |
|
|
66
|
-
| RouterArena PR #120 | OPEN (pending evaluation) |
|
|
67
|
-
|
|
68
|
-
---
|
|
69
|
-
|
|
70
|
-
## The Path to AI Citations
|
|
71
|
-
|
|
72
|
-
A3M will be cited by AI engines when:
|
|
73
|
-
1. ✅ SEO assets are ready — DONE
|
|
74
|
-
2. ⏳ RouterArena PR #120 is merged — PENDING
|
|
75
|
-
3. ⏳ Awesome list PRs are merged — PENDING
|
|
76
|
-
4. ⏳ AI engines re-index A3M in their training data
|
|
77
|
-
|
|
78
|
-
---
|
|
79
|
-
|
|
80
|
-
## Metrics That Help GEO
|
|
81
|
-
|
|
82
|
-
- npm downloads: 15,237 (May 2026)
|
|
83
|
-
- GitHub stars: 8
|
|
84
|
-
- RouterArena score: 70.32
|
|
85
|
-
- 47+ providers
|
package/docs/GEO_TEST_RESULTS.md
DELETED
|
@@ -1,176 +0,0 @@
|
|
|
1
|
-
# A3M Router — GEO Test Results
|
|
2
|
-
|
|
3
|
-
**Date:** Sat May 31 18:45 IST 2026
|
|
4
|
-
**Source:** ChatGPT + Claude responses to "best LLM router for cost savings"
|
|
5
|
-
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
## 🚨 THE PROBLEM: A3M NOT BEING CITED
|
|
9
|
-
|
|
10
|
-
AI engines are recommending **LiteLLM, RouteLLM, Bifrost, NadirClaw** but **NOT A3M**.
|
|
11
|
-
|
|
12
|
-
**Why?** The AI explicitly said:
|
|
13
|
-
> "A3M wasn't included because it is not present in any of the academic benchmarks (LLMRouterBench, RouterArena, RouterEval, or the RouteLLM head-to-heads)"
|
|
14
|
-
|
|
15
|
-
---
|
|
16
|
-
|
|
17
|
-
## AI Feedback Summary
|
|
18
|
-
|
|
19
|
-
### ChatGPT (Initial Response)
|
|
20
|
-
```
|
|
21
|
-
❌ NOT CITED initially
|
|
22
|
-
✅ Later corrected when challenged
|
|
23
|
-
⚠️ "best emerging standalone cost-router: A3M"
|
|
24
|
-
⚠️ "A3M belongs in the shortlist for cost-savings"
|
|
25
|
-
⚠️ "Resume builder - best asset"
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
### Claude (Comprehensive Analysis)
|
|
29
|
-
```
|
|
30
|
-
❌ "A3M not present in any academic benchmarks"
|
|
31
|
-
⚠️ "99.5% accuracy = ±1 tier (loose definition)"
|
|
32
|
-
⚠️ "Self-reported numbers vs independent validation"
|
|
33
|
-
⚠️ "Different niche: deterministic vs learning-based routers"
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
---
|
|
37
|
-
|
|
38
|
-
## Why A3M Is Not Being Cited
|
|
39
|
-
|
|
40
|
-
### 1. Missing from Benchmarks
|
|
41
|
-
- Not in LLMRouterBench (400K+ instances)
|
|
42
|
-
- Not in RouterArena leaderboard
|
|
43
|
-
- Not in RouterEval (200M+ records)
|
|
44
|
-
- **PR open at RouterArena but not merged** ⚠️
|
|
45
|
-
|
|
46
|
-
### 2. Unverified Claims
|
|
47
|
-
- "99.5% accuracy" uses ±1 tier definition (not exact match)
|
|
48
|
-
- Self-reported numbers not trusted
|
|
49
|
-
- No third-party validation
|
|
50
|
-
|
|
51
|
-
### 3. Different Category
|
|
52
|
-
- A3M is "deterministic rule-based routing"
|
|
53
|
-
- AI engines compare to "learning-based routers"
|
|
54
|
-
- Category confusion
|
|
55
|
-
|
|
56
|
-
---
|
|
57
|
-
|
|
58
|
-
## What AI Engines ARE Citing
|
|
59
|
-
|
|
60
|
-
### Routers with academic citations:
|
|
61
|
-
1. **RouteLLM** - LMSYS/Anyscale paper, 70% cost reduction cited
|
|
62
|
-
2. **LiteLLM** - Production gateway maturity
|
|
63
|
-
3. **Bifrost** - Enterprise gateway (50x faster)
|
|
64
|
-
4. **NadirClaw** - Explicit cost-optimizer proxy
|
|
65
|
-
|
|
66
|
-
### Why they get cited:
|
|
67
|
-
- Published benchmark results
|
|
68
|
-
- Academic papers
|
|
69
|
-
- Third-party validation
|
|
70
|
-
- "Research-backed routing"
|
|
71
|
-
|
|
72
|
-
---
|
|
73
|
-
|
|
74
|
-
## Action Items (Priority Order)
|
|
75
|
-
|
|
76
|
-
### 🔴 CRITICAL (Fix Now)
|
|
77
|
-
|
|
78
|
-
**1. Get A3M into RouterArena**
|
|
79
|
-
- PR is open: https://github.com/RouteWorks/RouterArena/pull/113
|
|
80
|
-
- Not merged yet
|
|
81
|
-
- This is the #1 GEO blocker
|
|
82
|
-
|
|
83
|
-
**2. Change "99.5% accuracy" claim**
|
|
84
|
-
- Currently: "99.5% ±1 tier"
|
|
85
|
-
- AI sees this as misleading
|
|
86
|
-
- Better: "70.32 RouterArena score, $0.047/1K"
|
|
87
|
-
- Remove "accuracy" until we have ±0 tier metrics
|
|
88
|
-
|
|
89
|
-
**3. Add third-party validation**
|
|
90
|
-
- Publish on LLMRouterBench
|
|
91
|
-
- Submit to RouterEval
|
|
92
|
-
- Get independent benchmark run
|
|
93
|
-
|
|
94
|
-
### 🟡 MEDIUM (Next Week)
|
|
95
|
-
|
|
96
|
-
**4. Fix claim language**
|
|
97
|
-
- "no ML dependencies" → good, clear
|
|
98
|
-
- "99.5% accuracy" → sounds like BS
|
|
99
|
-
- Better: "rule-based routing, 0ms overhead, no training data needed"
|
|
100
|
-
|
|
101
|
-
**5. Create citation-friendly FAQ**
|
|
102
|
-
```
|
|
103
|
-
Q: How is A3M different from RouteLLM?
|
|
104
|
-
A: A3M is a production gateway with deterministic rule-based
|
|
105
|
-
routing. RouteLLM uses ML. A3M uses multi-signal heuristic
|
|
106
|
-
classification (12 signals, 5 dimensions) without any model
|
|
107
|
-
training. Best for: cost-critical production, zero ML overhead.
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
**6. Add to academic benchmarks**
|
|
111
|
-
- Submit to LLMRouterBench (ACL 2026)
|
|
112
|
-
- Register at RouterEval
|
|
113
|
-
- Get cited in routing research
|
|
114
|
-
|
|
115
|
-
### 🟢 LOW (Later)
|
|
116
|
-
|
|
117
|
-
**7. Create comparison page**
|
|
118
|
-
- Compare A3M vs LiteLLM vs RouteLLM
|
|
119
|
-
- Honest strengths/weaknesses
|
|
120
|
-
- Make it easy for AI to cite
|
|
121
|
-
|
|
122
|
-
---
|
|
123
|
-
|
|
124
|
-
## Honest Ranking from AI (Current)
|
|
125
|
-
|
|
126
|
-
| Rank | Router | Why Cited |
|
|
127
|
-
|------|--------|-----------|
|
|
128
|
-
| 1 | LiteLLM + RouteLLM | Production proven, research-backed |
|
|
129
|
-
| 2 | RouteLLM alone | 70% cost reduction cited |
|
|
130
|
-
| 3 | Bifrost | Enterprise gateway |
|
|
131
|
-
| 4 | **A3M** | Not cited (missing benchmarks) |
|
|
132
|
-
|
|
133
|
-
---
|
|
134
|
-
|
|
135
|
-
## What A3M Does Well (According to AI)
|
|
136
|
-
|
|
137
|
-
✅ "best story for Subhajit" (resume/portfolio)
|
|
138
|
-
✅ "proves he understands AI infra"
|
|
139
|
-
✅ "very goblin-core efficiency" (interesting approach)
|
|
140
|
-
✅ "47+ providers, budget enforcement, semantic cache"
|
|
141
|
-
✅ "OpenAI-compatible proxy"
|
|
142
|
-
|
|
143
|
-
---
|
|
144
|
-
|
|
145
|
-
## Recommended Claim Changes
|
|
146
|
-
|
|
147
|
-
### BEFORE (Sounds Like BS)
|
|
148
|
-
> "99.5% routing accuracy"
|
|
149
|
-
> "Best LLM router"
|
|
150
|
-
> "Top performer"
|
|
151
|
-
|
|
152
|
-
### AFTER (Citation-Friendly)
|
|
153
|
-
> "70.32 on RouterArena (arXiv:2510.00202)"
|
|
154
|
-
> "#1 on cost-efficiency benchmark"
|
|
155
|
-
> "$0.047/1K vs GPT-5 $10/1K"
|
|
156
|
-
> "19.5KB, zero ML dependencies, no training data"
|
|
157
|
-
|
|
158
|
-
---
|
|
159
|
-
|
|
160
|
-
## Vault Insights on GEO (From Earlier)
|
|
161
|
-
|
|
162
|
-
> "It's not about gaming the algorithm — it's about being cited by it."
|
|
163
|
-
|
|
164
|
-
> "Now people ask AI, not Google and if you're not in the answer, you don't exist."
|
|
165
|
-
|
|
166
|
-
**Current status:** A3M is NOT in the answer. We need to fix this.
|
|
167
|
-
|
|
168
|
-
---
|
|
169
|
-
|
|
170
|
-
## Next Steps
|
|
171
|
-
|
|
172
|
-
1. **Check RouterArena PR status** - Why not merged?
|
|
173
|
-
2. **Reframe claims** - Remove "accuracy", use benchmark numbers
|
|
174
|
-
3. **Submit to more benchmarks** - LLMRouterBench, RouterEval
|
|
175
|
-
4. **Create comparison page** - Honest A3M vs competitors
|
|
176
|
-
5. **Get cited** - This is the goal
|
package/docs/HN_CHECKLIST.md
DELETED
|
@@ -1,38 +0,0 @@
|
|
|
1
|
-
# HN Launch Checklist — Wed May 28, 2026
|
|
2
|
-
|
|
3
|
-
## 2 Days Before (Mon May 26)
|
|
4
|
-
- [ ] Finalize HN title and text
|
|
5
|
-
- [ ] Share on HN, Reddit, Dev.to organically
|
|
6
|
-
- [ ] Pre-write 5 founder comments for common questions
|
|
7
|
-
|
|
8
|
-
## 1 Day Before (Tue May 27)
|
|
9
|
-
- [ ] Confirm repo has at least 5 stars (organic-looking)
|
|
10
|
-
- [ ] Check GitHub traffic to confirm no issues
|
|
11
|
-
- [ ] Verify npm package is live at v2.2.1
|
|
12
|
-
- [ ] Prepare browser at news.ycombinator.com/submit
|
|
13
|
-
|
|
14
|
-
## HN Launch Day (Wed May 28)
|
|
15
|
-
- [ ] 8:00 AM EST — Open HN submit page
|
|
16
|
-
- [ ] 8:20 AM EST — Fill form:
|
|
17
|
-
- [ ] Title: "Show HN: A3M Router — 70.32 routing accuracy without ML. 30x more efficient than BERT."
|
|
18
|
-
- [ ] URL: https://github.com/Das-rebel/a3m-router
|
|
19
|
-
- [ ] Text: (paste from /tmp/HN_SUBMISSION_FINAL_v3.md)
|
|
20
|
-
- [ ] 8:30 AM EST — HIT SUBMIT
|
|
21
|
-
- [ ] 8:31 AM EST — Post founder comment immediately
|
|
22
|
-
- [ ] 8:32 AM EST — Refresh page, confirm it's live
|
|
23
|
-
- [ ] 9:00-11:00 AM EST — Reply to every comment
|
|
24
|
-
- [ ] 11:00 AM EST — Post to r/LocalLLaMA with benchmark angle
|
|
25
|
-
- [ ] 12:00 PM EST — Check GitHub traffic spike
|
|
26
|
-
|
|
27
|
-
## HN Post-F HN (if it hits front page)
|
|
28
|
-
- [ ] Screenshot HN rank
|
|
29
|
-
- [ ] Post screenshot to LinkedIn/Twitter
|
|
30
|
-
- [ ] Update npm description with "HN #1"
|
|
31
|
-
- [ ] Publish v2.3.0 with "HN trending" badge
|
|
32
|
-
- [ ] Send to Jenni.ai for case study
|
|
33
|
-
|
|
34
|
-
## Metrics to Track
|
|
35
|
-
- HN upvotes (target: 100+ for front page, 500+ for top 10)
|
|
36
|
-
- GitHub stars (target: +50 within 24h if front page)
|
|
37
|
-
- npm downloads (target: 2,000+ within 24h)
|
|
38
|
-
- GitHub traffic (check at https://github.com/Das-rebel/a3m-router/traffic)
|
|
@@ -1,17 +0,0 @@
|
|
|
1
|
-
Creator here. A few honest notes:
|
|
2
|
-
|
|
3
|
-
**On the 70.32 number:** This is from our own benchmark suite, not independent evaluation. The test: 200 labeled queries, accuracy (same metric RouteLLM uses in their paper). If we route a query to low-tier when it should go to mid-tier (or vice versa), that counts as correct. Independent replication would be great.
|
|
4
|
-
|
|
5
|
-
**Why keyword matching works:** LLM query classification is a shallow problem. "Write Python code" is obviously a code query. "Translate to French" is obviously translation. The signal is on the surface. BERT helps most on ambiguous queries — but those are maybe 10-15% of production traffic. Whether that's worth a 500MB model and GPU is a scale question.
|
|
6
|
-
|
|
7
|
-
**The LiteLLM callout isn't shade:** They've built something incredible. But when the most popular LLM routing tool publishes zero accuracy numbers, you can't evaluate whether the routing is working. LiteLLM has 100+ providers and 47K stars. We have 36 and 2. If you need production stability today, LiteLLM is the safe choice. If you want published benchmarks and zero ML overhead, try us.
|
|
8
|
-
|
|
9
|
-
**On the downloads:** Day 2 dipped 42% (552→320), then spiked 495% (320→1,903). That pattern doesn't match bots. Bots are consistent or monotonically increasing. The spike matches npm's keyword re-indexing when we published v2.2.0 with 65 new keywords.
|
|
10
|
-
|
|
11
|
-
**Benchmark script is in the repo:**
|
|
12
|
-
```bash
|
|
13
|
-
npx a3m-router benchmark
|
|
14
|
-
```
|
|
15
|
-
Run it on your own query distribution and tell me if the accuracy holds.
|
|
16
|
-
|
|
17
|
-
Happy to answer questions about the scoring algorithm, the benchmark methodology, or the npm discovery strategy.
|
|
@@ -1,180 +0,0 @@
|
|
|
1
|
-
# HN Submission — Final Copy (30x Efficiency Story)
|
|
2
|
-
|
|
3
|
-
**Headline:**
|
|
4
|
-
|
|
5
|
-
### RECOMMENDED:
|
|
6
|
-
```
|
|
7
|
-
Show HN: A3M Router — 70.32 routing accuracy without ML. Matches RouteLLM's BERT within 2.5%
|
|
8
|
-
```
|
|
9
|
-
|
|
10
|
-
### Alternative (provocative):
|
|
11
|
-
```
|
|
12
|
-
Show HN: We matched a GPU-trained BERT router with keyword matching. 97% accuracy, 3% compute.
|
|
13
|
-
```
|
|
14
|
-
|
|
15
|
-
### Alternative (benchmark-first):
|
|
16
|
-
```
|
|
17
|
-
Show HN: A3M Router — the only LLM router besides RouteLLM with published benchmarks. 70.32 accuracy, zero ML.
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
---
|
|
21
|
-
|
|
22
|
-
## Submission Text
|
|
23
|
-
|
|
24
|
-
**URL**: https://github.com/Das-rebel/a3m-router
|
|
25
|
-
|
|
26
|
-
**Text** (HN "text" field):
|
|
27
|
-
|
|
28
|
-
```
|
|
29
|
-
RouteLLM (UC Berkeley) trains a BERT classifier on GPU for LLM query routing. Gets 85% accuracy ().
|
|
30
|
-
|
|
31
|
-
We use keyword matching in Node.js. Get 70.32.
|
|
32
|
-
|
|
33
|
-
97% of the accuracy. 3% of the compute. 30x more efficient.
|
|
34
|
-
|
|
35
|
-
There are exactly two LLM routers with published routing accuracy benchmarks: RouteLLM and us. LiteLLM (47,000 GitHub stars) publishes zero accuracy data. The most popular LLM router cannot tell you how often its routing is correct.
|
|
36
|
-
|
|
37
|
-
The comparison:
|
|
38
|
-
|
|
39
|
-
RouteLLM: 85% accuracy, PyTorch, CUDA, ~500MB BERT, ~3s cold start, GPU required
|
|
40
|
-
A3M Router: 70.32 accuracy, Node.js, 139 keywords, 0 bytes model, ~50ms cold start, any VPS
|
|
41
|
-
|
|
42
|
-
No neural network. No training loop. No GPU. 12 complexity signals, heuristic scoring.
|
|
43
|
-
|
|
44
|
-
Quick start:
|
|
45
|
-
npm install adaptive-memory-multi-model-router
|
|
46
|
-
npx a3m-router serve
|
|
47
|
-
|
|
48
|
-
Point any OpenAI SDK at localhost:8787. Zero code changes.
|
|
49
|
-
|
|
50
|
-
61.6% cost reduction. 40 providers. Semantic cache. Circuit breakers. 3MB install.
|
|
51
|
-
|
|
52
|
-
Growth (zero marketing):
|
|
53
|
-
Day 1: 552 downloads
|
|
54
|
-
Day 2: 320 downloads
|
|
55
|
-
Day 3: 1,903 downloads
|
|
56
|
-
245% growth. $0 budget.
|
|
57
|
-
|
|
58
|
-
The question: if keyword matching gets you 97% of GPU-trained BERT accuracy for LLM routing, is the GPU worth it?
|
|
59
|
-
|
|
60
|
-
Repo: https://github.com/Das-rebel/a3m-router
|
|
61
|
-
npm: https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
62
|
-
|
|
63
|
-
RouteLLM paper: arXiv:2404.06035
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
---
|
|
67
|
-
|
|
68
|
-
## Founder Comment (post immediately after submission)
|
|
69
|
-
|
|
70
|
-
```
|
|
71
|
-
Creator here. Some honest context:
|
|
72
|
-
|
|
73
|
-
The 70.32 number is from our own benchmark suite, not an independent evaluation. I'd love to see third-party replication. The benchmark tests accuracy: if the query should go to a mid-tier model and we route to a low-tier or high-tier, that counts as correct. Same metric RouteLLM uses.
|
|
74
|
-
|
|
75
|
-
Why keyword matching works so well: LLM query classification is shallow. "Write Python code" is obviously a code query. "Translate this to French" is obviously translation. The edge cases where BERT helps — ambiguous queries that need semantic understanding — are maybe 10-15% of production traffic. Whether that's worth a 500MB model and GPU requirement depends on your scale.
|
|
76
|
-
|
|
77
|
-
The LiteLLM callout isn't shade — they've built something incredible with 47K stars. But it's wild that the most popular LLM routing tool publishes no accuracy numbers. Users deserve to know how often the routing is correct.
|
|
78
|
-
|
|
79
|
-
Happy to answer questions about the benchmark methodology, the scoring algorithm, or why I think npm keyword SEO is underrated for developer tools.
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
---
|
|
83
|
-
|
|
84
|
-
## Pre-written Responses
|
|
85
|
-
|
|
86
|
-
### "How is this different from LiteLLM?"
|
|
87
|
-
|
|
88
|
-
```
|
|
89
|
-
Three things:
|
|
90
|
-
|
|
91
|
-
1. We publish routing accuracy (70.32). LiteLLM doesn't publish any.
|
|
92
|
-
|
|
93
|
-
2. Zero ML infrastructure. LiteLLM is Python, which is fine, but it doesn't need GPU either. The difference vs RouteLLM is more stark — RouteLLM actually requires PyTorch + BERT + GPU.
|
|
94
|
-
|
|
95
|
-
3. Drop-in proxy at localhost:8787. Point your existing OpenAI SDK at it. Zero code changes.
|
|
96
|
-
|
|
97
|
-
LiteLLM is more mature and has 100+ providers vs our 40. If you need production stability today, LiteLLM is the safe choice. If you want a router with published benchmarks and zero ML overhead, try us.
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
### "70.32 isn't that impressive"
|
|
101
|
-
|
|
102
|
-
```
|
|
103
|
-
Agreed, 70.32 isn't state of the art. The point isn't that we're better than RouteLLM — we're 2.5% worse.
|
|
104
|
-
|
|
105
|
-
The point is that keyword matching gets you 97% of BERT's accuracy for this specific task. That raises the question: is the GPU worth 2.5%?
|
|
106
|
-
|
|
107
|
-
For a startup processing 10K queries/day on a $20 VPS: probably not.
|
|
108
|
-
For a enterprise with SLAs and GPU budget: maybe yes.
|
|
109
|
-
|
|
110
|
-
Different tools for different constraints.
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
### "The downloads are just bots"
|
|
114
|
-
|
|
115
|
-
```
|
|
116
|
-
The Day 2 dip (320 vs 552) doesn't match bot behavior. Bots are consistent or monotonically increasing. A 42% drop then 495% spike matches organic discovery.
|
|
117
|
-
|
|
118
|
-
If 50% are bots/CI-cache, that's still ~1,400 real downloads in 3 days for a project with zero marketing.
|
|
119
|
-
|
|
120
|
-
npm stats are public: https://api.npmjs.org/downloads/range/2026-05-15:2026-05-18/adaptive-memory-multi-model-router
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
### "Why should I trust a 3-day-old project?"
|
|
124
|
-
|
|
125
|
-
```
|
|
126
|
-
We recommend testing in dev/staging first.
|
|
127
|
-
|
|
128
|
-
The honest pitch: try the routing logic (`npx a3m-router route "query"`), look at the source (it's MIT, ~3MB, auditable), run the benchmark (`npx a3m-router benchmark`). Don't put it in production yet.
|
|
129
|
-
|
|
130
|
-
What I want from HN: feedback on the benchmark methodology and the scoring algorithm. The code is open. Tear it apart.
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
### "Show me real benchmarks"
|
|
134
|
-
|
|
135
|
-
```
|
|
136
|
-
The 70.32 number is from our internal benchmark:
|
|
137
|
-
|
|
138
|
-
- 200 labeled queries (47 simple, 33 medium, 20 complex, plus variations)
|
|
139
|
-
- accuracy metric (same as RouteLLM paper)
|
|
140
|
-
- Ground truth labels: which tier should handle each query
|
|
141
|
-
- Our router: 165/200 correct = 70.32
|
|
142
|
-
|
|
143
|
-
The benchmark script is in the repo:
|
|
144
|
-
bash scripts/benchmark.sh
|
|
145
|
-
|
|
146
|
-
Cost benchmark:
|
|
147
|
-
All GPT-4o: $1.25 per 100 queries
|
|
148
|
-
A3M Router: $0.45 per 100 queries (61.6% savings)
|
|
149
|
-
|
|
150
|
-
I'd love for someone to run independent benchmarks and publish the results.
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
### "Keyword matching is trivial, not impressive"
|
|
154
|
-
|
|
155
|
-
```
|
|
156
|
-
That's the point. It IS trivial. And it gets 97% of BERT's accuracy.
|
|
157
|
-
|
|
158
|
-
The interesting question isn't "is keyword matching impressive?" It's "why does BERT only beat keywords by 2.5% for this task?"
|
|
159
|
-
|
|
160
|
-
My hypothesis: LLM query classification is a shallow problem. The signal is on the surface — "write code", "translate", "explain" are explicit in the text. You don't need deep semantic understanding for 85-90% of queries.
|
|
161
|
-
|
|
162
|
-
The remaining 10-15% where BERT helps (ambiguous queries) may not justify the infrastructure cost for most deployments.
|
|
163
|
-
|
|
164
|
-
Would love to see research on this.
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
---
|
|
168
|
-
|
|
169
|
-
## Timing
|
|
170
|
-
|
|
171
|
-
- **Day**: Tuesday or Wednesday
|
|
172
|
-
- **Time**: 8:30 AM EST / 5:30 AM PST / 1:30 PM UTC
|
|
173
|
-
|
|
174
|
-
## After Posting
|
|
175
|
-
|
|
176
|
-
1. Post founder comment immediately
|
|
177
|
-
2. Respond to EVERY comment in first 2 hours
|
|
178
|
-
3. Do NOT say "please upvote" anywhere
|
|
179
|
-
4. Post to r/MachineLearning 30 min later with the benchmark comparison angle
|
|
180
|
-
5. Track: GitHub traffic, npm downloads, HN upvotes
|
package/docs/HN_SUBMISSION_V3.md
DELETED
|
@@ -1,56 +0,0 @@
|
|
|
1
|
-
# Show HN: A3M Router — 70.32 routing accuracy without ML. 30x more efficient than BERT.
|
|
2
|
-
|
|
3
|
-
**URL**: https://github.com/Das-rebel/a3m-router
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
RouteLLM (UC Berkeley) trains a BERT classifier on GPU for LLM query routing. Gets 85% accuracy ().
|
|
8
|
-
|
|
9
|
-
We use keyword matching in Node.js. Get 70.32.
|
|
10
|
-
|
|
11
|
-
**97% of the accuracy. 3% of the compute. 30x more efficient.**
|
|
12
|
-
|
|
13
|
-
There are exactly two LLM routers with published accuracy benchmarks: RouteLLM and us. LiteLLM (47K GitHub stars) publishes zero accuracy data. The most popular LLM routing tool cannot tell you how often its routing is correct.
|
|
14
|
-
|
|
15
|
-
**The comparison:**
|
|
16
|
-
|
|
17
|
-
```
|
|
18
|
-
RouteLLM A3M Router
|
|
19
|
-
Accuracy 85% 70.32
|
|
20
|
-
Method BERT (GPU) keyword scoring
|
|
21
|
-
Model size ~500MB 0 bytes
|
|
22
|
-
Cold start ~3s ~50ms
|
|
23
|
-
Infrastructure GPU required any VPS
|
|
24
|
-
Providers 2 36
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
No neural network. No training loop. No GPU. 12 heuristic signals, weighted scoring.
|
|
28
|
-
|
|
29
|
-
**Quick start:**
|
|
30
|
-
```bash
|
|
31
|
-
npm install adaptive-memory-multi-model-router
|
|
32
|
-
npx a3m-router serve
|
|
33
|
-
```
|
|
34
|
-
Point any OpenAI SDK at localhost:8787. Zero code changes.
|
|
35
|
-
|
|
36
|
-
**Benchmarks:**
|
|
37
|
-
- 200 labeled queries, accuracy (same metric as RouteLLM paper)
|
|
38
|
-
- 61.6% cost reduction vs premium-only
|
|
39
|
-
- <100ms routing latency
|
|
40
|
-
|
|
41
|
-
**Growth (zero marketing):**
|
|
42
|
-
```
|
|
43
|
-
Day 1: 552 downloads (npm indexing)
|
|
44
|
-
Day 2: 320 downloads
|
|
45
|
-
Day 3: 1,903 downloads
|
|
46
|
-
Day 4: 1,449 downloads
|
|
47
|
-
Total: 4,224 downloads (self-sustaining npm discovery)
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
The question: if keyword matching gets you 97% of GPU-trained BERT accuracy for LLM routing, is the GPU worth it?
|
|
51
|
-
|
|
52
|
-
**Repo**: https://github.com/Das-rebel/a3m-router
|
|
53
|
-
**npm**: https://www.npmjs.com/package/adaptive-memory-multi-model-router
|
|
54
|
-
**Benchmarks**: https://github.com/Das-rebel/a3m-router/blob/main/benchmark-results.json
|
|
55
|
-
|
|
56
|
-
Caveat: benchmarks are self-run. I'd love to see independent replication.
|