adaptive-memory-multi-model-router 2.14.46 → 2.14.47

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (598) hide show
  1. package/{docs/llms.txt → llms.txt.bak} +6 -6
  2. package/package.json +13 -84
  3. package/src/routing/advancedRouter.ts.bak +650 -0
  4. package/test.js.bak +376 -0
  5. package/.dockerignore +0 -82
  6. package/.env.example +0 -303
  7. package/.github/DISCUSSIONS_WELCOME.md +0 -27
  8. package/.github/DISCUSSION_TEMPLATE.yml +0 -5
  9. package/.github/FUNDING.yml +0 -2
  10. package/.github/ISSUE_TEMPLATE/bug_report.md +0 -94
  11. package/.github/ISSUE_TEMPLATE/config.yml +0 -17
  12. package/.github/ISSUE_TEMPLATE/feature_request.md +0 -71
  13. package/.github/PULL_REQUEST_TEMPLATE.md +0 -71
  14. package/.github/dependabot.yml +0 -9
  15. package/.github/workflows/auto-publish.yml +0 -51
  16. package/.github/workflows/ci.yml +0 -263
  17. package/.github/workflows/codeql.yml +0 -38
  18. package/.github/workflows/npm-publish.yml +0 -20
  19. package/.github/workflows/pages.yml +0 -37
  20. package/.github/workflows/stale.yml +0 -54
  21. package/.publish-tick +0 -1
  22. package/.well-known/ai-plugin.json +0 -16
  23. package/AGENT_COUNCIL_FINDINGS.md +0 -142
  24. package/ARCHITECTURE.md +0 -346
  25. package/AUDIT_REPORT.md +0 -28
  26. package/CODE_OF_CONDUCT.md +0 -128
  27. package/CONTRIBUTING.md +0 -50
  28. package/CONTRIBUTORS.md +0 -20
  29. package/Dockerfile +0 -53
  30. package/Dockerfile.proxy +0 -33
  31. package/HEALTH_REPORT.md +0 -118
  32. package/IMPROVEMENT_PLAN.md +0 -107
  33. package/LANDING.md +0 -43
  34. package/LAUNCH-PAIN-DRIVEN.md +0 -339
  35. package/LAUNCH.md +0 -337
  36. package/LAUNCH_CHECKLIST.md +0 -141
  37. package/LAUNCH_SNAPSHOT.md +0 -260
  38. package/MANIFESTO.md +0 -41
  39. package/POPULARITY_BOOSTERS.md +0 -285
  40. package/PR_STATUS_REPORT.md +0 -148
  41. package/REDESIGN.md +0 -95
  42. package/RUNKIT.md +0 -83
  43. package/SECURITY.md +0 -29
  44. package/SUBMISSIONS.md +0 -43
  45. package/_schema.html +0 -53
  46. package/ai-plugin.json +0 -16
  47. package/articles/AI_AGENT_LLM_ROUTING.md +0 -150
  48. package/articles/CHINESE_DIRECTORIES.md +0 -100
  49. package/articles/CHINESE_SUBMISSIONS_READY.md +0 -322
  50. package/articles/COMPETITOR_ALERTS.md +0 -31
  51. package/articles/COMPLETE_POSTING_DIRECTORY.md +0 -147
  52. package/articles/CONTENT_STRUCTURE.md +0 -292
  53. package/articles/DEVTO_COST_GUIDE.md +0 -473
  54. package/articles/DEVTO_FINAL.md +0 -416
  55. package/articles/DEVTO_MULTI_PROVIDER.md +0 -542
  56. package/articles/DEVTO_READY.md +0 -255
  57. package/articles/DEVTO_V2_ANNOUNCEMENT.md +0 -160
  58. package/articles/DEVTO_VIRAL_GROWTH.md +0 -280
  59. package/articles/FRESH_devto.md +0 -460
  60. package/articles/FRESH_devto_2026_05.md +0 -73
  61. package/articles/FRESH_hackernews.md +0 -14
  62. package/articles/FRESH_reddit_ml.md +0 -90
  63. package/articles/FRESH_reddit_node.md +0 -198
  64. package/articles/FRESH_reddit_sideproject.md +0 -72
  65. package/articles/FRESH_reddit_webdev.md +0 -130
  66. package/articles/FROM_ZERO_TO_10K.md +0 -107
  67. package/articles/HN_10X_BETTER.md +0 -430
  68. package/articles/HN_ACCOUNT_GUIDE.md +0 -21
  69. package/articles/HN_CHINESE_STYLE.md +0 -308
  70. package/articles/HN_FINAL.md +0 -148
  71. package/articles/HN_POSTED_VERSION.md +0 -56
  72. package/articles/HN_POST_READY.md +0 -137
  73. package/articles/HN_RESEARCH.md +0 -364
  74. package/articles/HN_SHOW_routerarena.md +0 -17
  75. package/articles/HN_TIMING_GUIDE.md +0 -52
  76. package/articles/INDIEHACKERS_POST.md +0 -52
  77. package/articles/INDIEHACKERS_READY.md +0 -120
  78. package/articles/LLM_BENCHMARK_DEEP_DIVE.md +0 -153
  79. package/articles/MASTER_POSTING_DIRECTORY.md +0 -189
  80. package/articles/NEWSLETTER_SEND_NOW.md +0 -259
  81. package/articles/NEWSLETTER_SUBMISSIONS.md +0 -112
  82. package/articles/PAIN-DRIVEN-devto-v2.md +0 -308
  83. package/articles/PAIN-DRIVEN-devto-v3.md +0 -268
  84. package/articles/PAIN-DRIVEN-devto.md +0 -242
  85. package/articles/PAIN-DRIVEN-hackernews-v2.md +0 -138
  86. package/articles/PAIN-DRIVEN-hackernews-v3.md +0 -151
  87. package/articles/PAIN-DRIVEN-hackernews.md +0 -131
  88. package/articles/PAIN-DRIVEN-reddit-v2.md +0 -301
  89. package/articles/PAIN-DRIVEN-reddit-v3.md +0 -236
  90. package/articles/PAIN-DRIVEN-reddit.md +0 -218
  91. package/articles/PAIN-DRIVEN-twitter-v2.md +0 -110
  92. package/articles/PAIN-DRIVEN-twitter-v3.md +0 -121
  93. package/articles/PAIN-DRIVEN-twitter.md +0 -120
  94. package/articles/PORTKEY_VS_A3M.md +0 -147
  95. package/articles/POSTING_KIT_2026_05.md +0 -67
  96. package/articles/PRESS_KIT_routerarena.md +0 -77
  97. package/articles/PRODUCTHUNT_LISTING.md +0 -48
  98. package/articles/PRODUCTHUNT_READY.md +0 -106
  99. package/articles/PR_PLAN_vault.md +0 -125
  100. package/articles/REDDIT_FINAL.md +0 -232
  101. package/articles/REDDIT_POST.md +0 -67
  102. package/articles/REDDIT_SUBMISSION_READY.md +0 -348
  103. package/articles/ROUTERARENA_LEADER.md +0 -45
  104. package/articles/SHOW_HN_FINAL.md +0 -29
  105. package/articles/TWEETS_10K_DOWNLOADS.md +0 -47
  106. package/articles/TWEETS_BENCHMARK_FIRST.md +0 -46
  107. package/articles/TWEETS_MCP_PLAY.md +0 -51
  108. package/articles/TWEETS_SEQUENTIAL_BROKEN.md +0 -49
  109. package/articles/TWEETS_WHY_BUILD.md +0 -54
  110. package/articles/TWEETS_routerarena_leader.md +0 -53
  111. package/articles/TWEET_STORM_READY.md +0 -165
  112. package/articles/TWITTER_FINAL.md +0 -167
  113. package/articles/WHY_10X_BETTER.md +0 -261
  114. package/articles/WHY_CHINESE_STYLE_BETTER.md +0 -323
  115. package/articles/ai-discoverability-llm-routing.md +0 -210
  116. package/articles/devto-llm-routing.md +0 -138
  117. package/articles/hackernews-show-hn.md +0 -54
  118. package/articles/hashnode-llm-cost-optimization.md +0 -125
  119. package/articles/hn_show_2026_05.md +0 -11
  120. package/articles/medium-building-llm-router.md +0 -205
  121. package/articles/reddit-ml.md +0 -76
  122. package/articles/twitter-thread-cost-savings.md +0 -50
  123. package/articles/youtube-tutorial-script.md +0 -262
  124. package/assets/a3m_3blue1brown.mp4 +0 -0
  125. package/assets/banner.svg +0 -109
  126. package/assets/chart-cost-v2.svg +0 -91
  127. package/assets/chart-cost-v3.svg +0 -143
  128. package/assets/chart-features-v2.svg +0 -132
  129. package/assets/chart-features-v3.svg +0 -211
  130. package/assets/chart-growth-v2.svg +0 -122
  131. package/assets/chart-growth-v3.svg +0 -189
  132. package/assets/cost-comparison.svg +0 -134
  133. package/assets/cost-simple.svg +0 -64
  134. package/assets/demo-hn.gif +0 -0
  135. package/assets/feature-matrix.svg +0 -136
  136. package/assets/growth-chart-animated.svg +0 -76
  137. package/assets/growth-chart.svg +0 -82
  138. package/assets/growth-simple.svg +0 -69
  139. package/assets/hero-diagram.svg +0 -81
  140. package/assets/logo-new.svg +0 -21
  141. package/assets/logo.svg +0 -68
  142. package/assets/provider-comparison.svg +0 -121
  143. package/assets/social-preview-new.svg +0 -100
  144. package/assets/social-preview.svg +0 -194
  145. package/assets/social-v2.svg +0 -130
  146. package/assets/social-v3.svg +0 -212
  147. package/benchmark-provider-results.json +0 -245
  148. package/benchmark-results.json +0 -54
  149. package/council-votes/architecture-vote.md +0 -121
  150. package/council-votes/coverage-vote.md +0 -93
  151. package/data/adaptive-benchmark.json +0 -92
  152. package/data/benchmark-results.json +0 -47
  153. package/data/labeled-benchmark.json +0 -88
  154. package/demo/3blue1brown_video.py +0 -285
  155. package/demo/3blue1brown_video_v2.py +0 -310
  156. package/demo/IMPROVED_PROMPTS.md +0 -229
  157. package/demo/VEO3_PROMPTS.md +0 -269
  158. package/demo/VIDEO_PRODUCTION_GUIDE.md +0 -333
  159. package/demo/a3m_3blue1brown.mp4 +0 -0
  160. package/demo/asciinema-demo.sh +0 -195
  161. package/demo/demo-hn.tape +0 -74
  162. package/demo/demo-script.md +0 -53
  163. package/demo/demo-script.sh +0 -62
  164. package/demo/demo.svg +0 -75
  165. package/demo/frame1_ai_data_center.png +0 -0
  166. package/demo/frame1_sunset_video.mp4 +0 -0
  167. package/demo/frame2_cost_comparison.png +0 -0
  168. package/demo/frame2_cost_comparison_fallback.png +0 -0
  169. package/demo/frame3_parallel_execution.png +0 -0
  170. package/demo/frame3_parallel_execution_fallback.png +0 -0
  171. package/demo/frame4_providers.png +0 -0
  172. package/demo/frame4_providers_fallback.png +0 -0
  173. package/demo/frame5_endcard.png +0 -0
  174. package/demo/frame5_endcard_fallback.png +0 -0
  175. package/demo/new_frame1_hook.png +0 -0
  176. package/demo/new_frame2_proof.png +0 -0
  177. package/demo/new_frame3_wow.png +0 -0
  178. package/demo/new_frame4_social.png +0 -0
  179. package/demo/new_frame5_cta.png +0 -0
  180. package/demo/package.json +0 -13
  181. package/demo/product-video-final.mp4 +0 -0
  182. package/demo/product-video-hype-v1.mp4 +0 -0
  183. package/demo/product-video-v1.mp4 +0 -0
  184. package/demo/public/index.html +0 -762
  185. package/demo/recording.cast +0 -55
  186. package/demo/server.js +0 -405
  187. package/demo-new.tape +0 -71
  188. package/demo-real.sh +0 -198
  189. package/demo-simple.tape +0 -205
  190. package/demo.html +0 -520
  191. package/demo.sh +0 -85
  192. package/demo.tape +0 -259
  193. package/dist/analytics/costAnalytics.d.ts.map +0 -1
  194. package/dist/analytics/costAnalytics.js.map +0 -1
  195. package/dist/benchmark/comprehensive.js.map +0 -1
  196. package/dist/benchmark/reproducible.d.ts.map +0 -1
  197. package/dist/benchmark/reproducible.js.map +0 -1
  198. package/dist/cache/prefixCache.d.ts.map +0 -1
  199. package/dist/cache/prefixCache.js.map +0 -1
  200. package/dist/cache/responseCache.d.ts.map +0 -1
  201. package/dist/cache/responseCache.js.map +0 -1
  202. package/dist/cache/semanticCache.d.ts.map +0 -1
  203. package/dist/cache/semanticCache.js.map +0 -1
  204. package/dist/cli/setupWizard.d.ts.map +0 -1
  205. package/dist/cli/setupWizard.js.map +0 -1
  206. package/dist/cost/budgetEnforcer.d.ts.map +0 -1
  207. package/dist/cost/budgetEnforcer.js.map +0 -1
  208. package/dist/cost/costTracker.d.ts.map +0 -1
  209. package/dist/cost/costTracker.js.map +0 -1
  210. package/dist/ensemble/multiRoundDialog.js.map +0 -1
  211. package/dist/ensemble/shapleyValue.js.map +0 -1
  212. package/dist/integrations/langchainAdapter.d.ts.map +0 -1
  213. package/dist/integrations/langchainAdapter.js.map +0 -1
  214. package/dist/integrations/oauth.d.ts.map +0 -1
  215. package/dist/integrations/oauth.js.map +0 -1
  216. package/dist/integrations/scienceAdapter.js.map +0 -1
  217. package/dist/memory/autoFetch.d.ts.map +0 -1
  218. package/dist/memory/autoFetch.js.map +0 -1
  219. package/dist/memory/episodicMemory.d.ts.map +0 -1
  220. package/dist/memory/episodicMemory.js.map +0 -1
  221. package/dist/memory/hybridMemory.js.map +0 -1
  222. package/dist/memory/memoryTree.d.ts.map +0 -1
  223. package/dist/memory/memoryTree.js.map +0 -1
  224. package/dist/memory/obsidianVault.d.ts.map +0 -1
  225. package/dist/memory/obsidianVault.js.map +0 -1
  226. package/dist/memory/reasoningBank.js.map +0 -1
  227. package/dist/observability/changeWatch.d.ts.map +0 -1
  228. package/dist/observability/changeWatch.js.map +0 -1
  229. package/dist/observability/fatigueDetector.d.ts.map +0 -1
  230. package/dist/observability/fatigueDetector.js.map +0 -1
  231. package/dist/observability/index.d.ts.map +0 -1
  232. package/dist/observability/index.js.map +0 -1
  233. package/dist/observability/metrics.d.ts.map +0 -1
  234. package/dist/observability/metrics.js.map +0 -1
  235. package/dist/observability/middleware.d.ts.map +0 -1
  236. package/dist/observability/middleware.js.map +0 -1
  237. package/dist/observability/tracer.d.ts.map +0 -1
  238. package/dist/observability/tracer.js.map +0 -1
  239. package/dist/observability/types.d.ts.map +0 -1
  240. package/dist/observability/types.js.map +0 -1
  241. package/dist/orchestration/haloOrchestrator.d.ts.map +0 -1
  242. package/dist/orchestration/haloOrchestrator.js.map +0 -1
  243. package/dist/orchestration/mctsWorkflow.d.ts.map +0 -1
  244. package/dist/orchestration/mctsWorkflow.js.map +0 -1
  245. package/dist/providers/localProvider.d.ts.map +0 -1
  246. package/dist/providers/localProvider.js.map +0 -1
  247. package/dist/providers/providerConfig.d.ts.map +0 -1
  248. package/dist/providers/providerConfig.js.map +0 -1
  249. package/dist/providers/registry.d.ts.map +0 -1
  250. package/dist/providers/registry.js.map +0 -1
  251. package/dist/routing/advancedRouter.d.ts.map +0 -1
  252. package/dist/routing/advancedRouter.js.map +0 -1
  253. package/dist/routing/crossModelValidation.d.ts.map +0 -1
  254. package/dist/routing/crossModelValidation.js.map +0 -1
  255. package/dist/routing/providerHealth.d.ts.map +0 -1
  256. package/dist/routing/providerHealth.js.map +0 -1
  257. package/dist/routing/providerRetry.d.ts.map +0 -1
  258. package/dist/routing/providerRetry.js.map +0 -1
  259. package/dist/scripts/banner.js +0 -29
  260. package/dist/security/guardrails.d.ts.map +0 -1
  261. package/dist/security/guardrails.js.map +0 -1
  262. package/dist/server/dashboard.d.ts.map +0 -1
  263. package/dist/server/dashboard.js.map +0 -1
  264. package/dist/server/modelMapper.d.ts.map +0 -1
  265. package/dist/server/modelMapper.js.map +0 -1
  266. package/dist/server/proxyServer.d.ts.map +0 -1
  267. package/dist/server/proxyServer.js.map +0 -1
  268. package/dist/skills/__tests__/skill_manager.test.d.ts +0 -2
  269. package/dist/skills/__tests__/skill_manager.test.d.ts.map +0 -1
  270. package/dist/skills/__tests__/skill_manager.test.js +0 -268
  271. package/dist/skills/__tests__/skill_manager.test.js.map +0 -1
  272. package/dist/tools/tmlpdTools.d.ts.map +0 -1
  273. package/dist/tools/tmlpdTools.js.map +0 -1
  274. package/dist/tui/dashboard.d.ts.map +0 -1
  275. package/dist/tui/dashboard.js.map +0 -1
  276. package/dist/tui/index.d.ts.map +0 -1
  277. package/dist/tui/index.js.map +0 -1
  278. package/dist/utils/batchProcessor.d.ts.map +0 -1
  279. package/dist/utils/batchProcessor.js.map +0 -1
  280. package/dist/utils/compression.d.ts.map +0 -1
  281. package/dist/utils/compression.js.map +0 -1
  282. package/dist/utils/costUtils.d.ts.map +0 -1
  283. package/dist/utils/costUtils.js.map +0 -1
  284. package/dist/utils/reliability.d.ts.map +0 -1
  285. package/dist/utils/reliability.js.map +0 -1
  286. package/dist/utils/sorting.d.ts.map +0 -1
  287. package/dist/utils/sorting.js.map +0 -1
  288. package/dist/utils/speculativeDecoding.d.ts.map +0 -1
  289. package/dist/utils/speculativeDecoding.js.map +0 -1
  290. package/dist/utils/tokenUtils.d.ts.map +0 -1
  291. package/dist/utils/tokenUtils.js.map +0 -1
  292. package/docs/.nojekyll +0 -0
  293. package/docs/ANALYSIS_PRINCIPLES.md +0 -162
  294. package/docs/API.md +0 -855
  295. package/docs/ARCHITECTURAL-IMPROVEMENTS-2025.md +0 -1391
  296. package/docs/ARCHITECTURAL-IMPROVEMENTS-REVISED-2025.md +0 -1051
  297. package/docs/BENCHMARK.md +0 -170
  298. package/docs/CHINESE_PROVIDER_RELIABILITY.md +0 -37
  299. package/docs/CITATIONS.md +0 -74
  300. package/docs/CLAIMS_AND_EVIDENCE.md +0 -58
  301. package/docs/CONFIGURATION.md +0 -476
  302. package/docs/COUNCIL_DECISION.json +0 -816
  303. package/docs/COUNCIL_SUMMARY.md +0 -319
  304. package/docs/COUNCIL_V2.2_DECISION.md +0 -416
  305. package/docs/ENGINEERING_SPEC.md +0 -55
  306. package/docs/FACTORY_RESET.md +0 -34
  307. package/docs/GEO.md +0 -66
  308. package/docs/GEO_OPTIMIZATION.md +0 -30
  309. package/docs/GEO_ROOT_CAUSE.md +0 -136
  310. package/docs/GEO_STATUS.md +0 -85
  311. package/docs/GEO_TEST_RESULTS.md +0 -176
  312. package/docs/HN_CHECKLIST.md +0 -38
  313. package/docs/HN_FOUNDER_COMMENT.md +0 -17
  314. package/docs/HN_SUBMISSION_FINAL.md +0 -180
  315. package/docs/HN_SUBMISSION_V3.md +0 -56
  316. package/docs/IMPROVEMENT_ROADMAP.md +0 -515
  317. package/docs/INTEGRATIONS.md +0 -420
  318. package/docs/LANGCHAIN_INTEGRATION.md +0 -147
  319. package/docs/LLM_COUNCIL_DECISION.md +0 -508
  320. package/docs/MIDDLEWARE_CHAIN.md +0 -35
  321. package/docs/PROMO_CHECKLIST.md +0 -200
  322. package/docs/QUICKSTART.md +0 -271
  323. package/docs/QUICK_START.md +0 -43
  324. package/docs/QUICK_START_VISIBILITY.md +0 -782
  325. package/docs/REDDIT_GAP_ANALYSIS.md +0 -299
  326. package/docs/RELEASE_CHECKLIST.md +0 -32
  327. package/docs/REPRODUCIBILITY.md +0 -63
  328. package/docs/RESEARCH_BACKED_IMPROVEMENTS.md +0 -1180
  329. package/docs/ROUTING_RUBRIC.md +0 -197
  330. package/docs/SEO_AUDIT.md +0 -186
  331. package/docs/SOCIAL_LISTENING.md +0 -219
  332. package/docs/TMLPD_QNA.md +0 -751
  333. package/docs/TMLPD_V2.1_COMPLETE.md +0 -763
  334. package/docs/TMLPD_V2.2_RESEARCH_ROADMAP.md +0 -754
  335. package/docs/UPDATE_TOPICS.md +0 -15
  336. package/docs/USE_CASES.md +0 -59
  337. package/docs/V2.2_IMPLEMENTATION_COMPLETE.md +0 -446
  338. package/docs/V2_IMPLEMENTATION_GUIDE.md +0 -388
  339. package/docs/VERCEL_AI_SDK.md +0 -209
  340. package/docs/VISIBILITY_ADOPTION_PLAN.md +0 -1005
  341. package/docs/_config.yml +0 -49
  342. package/docs/ai-plugin.json +0 -16
  343. package/docs/api.html +0 -513
  344. package/docs/architecture-diagram.md +0 -40
  345. package/docs/benchmark-chart.png +0 -0
  346. package/docs/benchmark.html +0 -387
  347. package/docs/blog/routerarena-number-one.html +0 -73
  348. package/docs/cli-cheatsheet.md +0 -339
  349. package/docs/compare.md +0 -109
  350. package/docs/comparison-litellm.md +0 -88
  351. package/docs/comparison.md +0 -108
  352. package/docs/cost-chart-ascii.md +0 -42
  353. package/docs/cost-comparison-chart.svg +0 -88
  354. package/docs/curl-examples.md +0 -247
  355. package/docs/demo-auto.html +0 -264
  356. package/docs/demo.html +0 -416
  357. package/docs/geo/GENERATIVE_ENGINE_OPTIMIZATION.md +0 -232
  358. package/docs/index.html +0 -507
  359. package/docs/launch-content/LAUNCH_EXECUTION_CHECKLIST.md +0 -421
  360. package/docs/launch-content/README.md +0 -457
  361. package/docs/launch-content/assets/cost_comparison_100_tasks.png +0 -0
  362. package/docs/launch-content/assets/cumulative_savings.png +0 -0
  363. package/docs/launch-content/assets/parallel_speedup.png +0 -0
  364. package/docs/launch-content/assets/provider_pricing_comparison.png +0 -0
  365. package/docs/launch-content/assets/task_breakdown_comparison.png +0 -0
  366. package/docs/launch-content/generate_charts.py +0 -313
  367. package/docs/launch-content/hn_show_post.md +0 -139
  368. package/docs/launch-content/partner_outreach_templates.md +0 -745
  369. package/docs/launch-content/reddit_posts.md +0 -467
  370. package/docs/launch-content/twitter_thread.txt +0 -460
  371. package/docs/npm-downloads-chart.svg +0 -43
  372. package/docs/openapi.json +0 -139
  373. package/docs/openapi.yaml +0 -1318
  374. package/docs/quick-start.html +0 -366
  375. package/docs/robots.txt +0 -52
  376. package/docs/sitemap.xml +0 -57
  377. package/docs/styles.css +0 -682
  378. package/docs/well-known/ai-plugin.json +0 -16
  379. package/docs/wellknown/ai-plugin.json +0 -16
  380. package/docs-site/assets/og-banner.svg +0 -194
  381. package/docs-site/index.html +0 -632
  382. package/eval/README.md +0 -46
  383. package/eval/baselines/main.json +0 -12
  384. package/eval/benchmark_dataset.jsonl +0 -16
  385. package/eval/check_golden_routes.js +0 -64
  386. package/eval/datasets/catalog.json +0 -33
  387. package/eval/datasets/slices/cn_provider_reliability_v1.jsonl +0 -3
  388. package/eval/datasets/slices/cost_pressure_v1.jsonl +0 -3
  389. package/eval/datasets/slices/safety_guardrails_v1.jsonl +0 -3
  390. package/eval/evals.json +0 -199
  391. package/eval/fault_injection_thresholds.json +0 -3
  392. package/eval/generate_report.js +0 -128
  393. package/eval/golden_routes.json +0 -114
  394. package/eval/lib/experiment_registry.js +0 -24
  395. package/eval/run_eval.js +0 -197
  396. package/eval/run_fault_injection.js +0 -201
  397. package/eval/run_shadow_eval.js +0 -85
  398. package/eval/thresholds.json +0 -9
  399. package/examples/QUICKSTART.md +0 -183
  400. package/examples/README.md +0 -61
  401. package/examples/a3m-sdk.js +0 -124
  402. package/examples/basic-route.js +0 -54
  403. package/examples/chat-loop.js +0 -202
  404. package/examples/classify-then-route.js +0 -102
  405. package/examples/cost-compare.js +0 -120
  406. package/examples/ensemble.js +0 -160
  407. package/examples/whatsapp-telegram-bridge-demo.js +0 -302
  408. package/examples/whatsapp-telegram-bridge.js +0 -269
  409. package/hf-space/README.md +0 -23
  410. package/hf-space/app.py +0 -240
  411. package/hf-space/requirements.txt +0 -1
  412. package/huggingface_space/README.md +0 -35
  413. package/huggingface_space/app.py +0 -126
  414. package/huggingface_space/create_space.py +0 -208
  415. package/huggingface_space/requirements.txt +0 -1
  416. package/mcp-server/README.md +0 -188
  417. package/mcp-server/package.json +0 -29
  418. package/mcp-server/src/index.ts +0 -744
  419. package/mcp-server/tsconfig.json +0 -19
  420. package/openclaw-alexa-bridge/ALL_REMAINING_FIXES_PLAN.md +0 -313
  421. package/openclaw-alexa-bridge/REMAINING_FIXES_SUMMARY.md +0 -277
  422. package/openclaw-alexa-bridge/src/alexa_handler_no_tmlpd.js +0 -1234
  423. package/openclaw-alexa-bridge/test_fixes.js +0 -77
  424. package/playground/README.md +0 -51
  425. package/playground/codesandbox.json +0 -12
  426. package/playground/index.js +0 -39
  427. package/proxy/README.md +0 -227
  428. package/proxy/package-lock.json +0 -831
  429. package/proxy/package.json +0 -17
  430. package/proxy/rate-limit.js +0 -145
  431. package/proxy/rate-limit.test.js +0 -311
  432. package/proxy/server.js +0 -970
  433. package/python/README.md +0 -102
  434. package/python/a3m/__init__.py +0 -6
  435. package/python/a3m/client.py +0 -190
  436. package/python/a3m/models.py +0 -40
  437. package/python/a3m/sync_client.py +0 -61
  438. package/python/examples.py +0 -53
  439. package/python/integrations.py +0 -330
  440. package/python/pyproject.toml +0 -23
  441. package/python/setup.py +0 -28
  442. package/python/tmlpd.py +0 -369
  443. package/qna/REDDIT_GAP_ANALYSIS.md +0 -299
  444. package/qna/TMLPD_QNA.md +0 -751
  445. package/research/FINDING_001_safety.md +0 -28
  446. package/research/FINDING_002_error_diversity.md +0 -32
  447. package/research/FINDING_003_confidence_weighted_voting.md +0 -32
  448. package/research/FINDING_004_cross_model_semantic_detection.md +0 -37
  449. package/research/FINDING_005_knowledge_gap_orthogonality.md +0 -34
  450. package/research/HALLUCINATION_RESEARCH.md +0 -27
  451. package/research/ensemble-voting.md +0 -324
  452. package/research/loss-functions.md +0 -545
  453. package/research-log.md +0 -49
  454. package/scripts/banner.js +0 -29
  455. package/scripts/benchmark-local-routerarena.ts +0 -176
  456. package/scripts/benchmark.js +0 -145
  457. package/scripts/benchmark.sh +0 -61
  458. package/scripts/compare-providers.sh +0 -230
  459. package/scripts/content-planner.js +0 -25
  460. package/scripts/create-labeled-benchmark.ts +0 -105
  461. package/scripts/cross_post.py +0 -443
  462. package/scripts/local-router-benchmark.ts +0 -154
  463. package/scripts/post-all.sh +0 -41
  464. package/scripts/publish_fcc.py +0 -106
  465. package/scripts/push-to-gitee.sh +0 -25
  466. package/scripts/routerarena_ensemble.js +0 -144
  467. package/scripts/routing-benchmark-v2.js +0 -373
  468. package/scripts/routing-benchmark-v3.js +0 -118
  469. package/scripts/routing-benchmark.js +0 -462
  470. package/scripts/run-labeled-benchmark.mjs +0 -104
  471. package/scripts/run-mmlu-benchmark.js +0 -176
  472. package/scripts/run-provider-benchmark.js +0 -244
  473. package/scripts/update-npm-badges.js +0 -158
  474. package/skill/SKILL.md +0 -238
  475. package/src/__tests__/integration/tmpld_integration.test.py +0 -540
  476. package/src/skills/__tests__/skill_manager.test.ts +0 -328
  477. package/submissions/benchmarks/ALL_PLATFORMS_SUBMISSION.md +0 -94
  478. package/submissions/benchmarks/LLMROUTERBENCH_SUBMISSION.md +0 -121
  479. package/submissions/benchmarks/MMRBENCH_SUBMISSION.md +0 -94
  480. package/submissions/benchmarks/ROUTERARENA_UPDATE.md +0 -83
  481. package/submissions/benchmarks/ROUTERBENCH_SUBMISSION.md +0 -225
  482. package/test-council/1-structure-tests.test.js +0 -353
  483. package/test-council/1-structure-tests.test.ts +0 -353
  484. package/test-council/2-edge-case-tests.test.ts +0 -361
  485. package/test-council/3-performance-tests.test.ts +0 -669
  486. package/test-council/4-integration-tests.test.ts +0 -391
  487. package/test-council/5-agent-council-eval.test.ts +0 -413
  488. package/test-council/AGENT_COUNCIL_ARCHITECTURE.md +0 -349
  489. package/test-council/TEST_COUNCIL_REPORT.md +0 -201
  490. package/test-council/agents/edge-case-agent.ts +0 -363
  491. package/test-council/agents/performance-agent.ts +0 -426
  492. package/test-council/agents/structure-agent.ts +0 -227
  493. package/test-council/council.md +0 -183
  494. package/tests/__mocks__/tokenUtils.ts +0 -8
  495. package/tests/memory/episodicMemory.test.ts +0 -227
  496. package/tests/package-lock.json +0 -1628
  497. package/tests/package.json +0 -18
  498. package/tests/routing/ensembleVoting.test.ts +0 -236
  499. package/tests/routing/providerRetry.test.ts +0 -360
  500. package/tests/routing/queryTypePresets.test.ts +0 -208
  501. package/tests/security/guardrailEngine.test.ts +0 -700
  502. package/tests/tsconfig.json +0 -21
  503. package/tests/vitest.config.ts +0 -18
  504. package/tmlpd-pi-extension/README.md +0 -66
  505. package/tmlpd-pi-extension/dist/cache/prefixCache.d.ts +0 -114
  506. package/tmlpd-pi-extension/dist/cache/prefixCache.d.ts.map +0 -1
  507. package/tmlpd-pi-extension/dist/cache/prefixCache.js +0 -285
  508. package/tmlpd-pi-extension/dist/cache/prefixCache.js.map +0 -1
  509. package/tmlpd-pi-extension/dist/cache/responseCache.d.ts +0 -58
  510. package/tmlpd-pi-extension/dist/cache/responseCache.d.ts.map +0 -1
  511. package/tmlpd-pi-extension/dist/cache/responseCache.js +0 -153
  512. package/tmlpd-pi-extension/dist/cache/responseCache.js.map +0 -1
  513. package/tmlpd-pi-extension/dist/cli.js +0 -59
  514. package/tmlpd-pi-extension/dist/cost/costTracker.d.ts +0 -95
  515. package/tmlpd-pi-extension/dist/cost/costTracker.d.ts.map +0 -1
  516. package/tmlpd-pi-extension/dist/cost/costTracker.js +0 -240
  517. package/tmlpd-pi-extension/dist/cost/costTracker.js.map +0 -1
  518. package/tmlpd-pi-extension/dist/index.d.ts +0 -723
  519. package/tmlpd-pi-extension/dist/index.d.ts.map +0 -1
  520. package/tmlpd-pi-extension/dist/index.js +0 -239
  521. package/tmlpd-pi-extension/dist/index.js.map +0 -1
  522. package/tmlpd-pi-extension/dist/memory/episodicMemory.d.ts +0 -82
  523. package/tmlpd-pi-extension/dist/memory/episodicMemory.d.ts.map +0 -1
  524. package/tmlpd-pi-extension/dist/memory/episodicMemory.js +0 -145
  525. package/tmlpd-pi-extension/dist/memory/episodicMemory.js.map +0 -1
  526. package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.d.ts +0 -102
  527. package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.d.ts.map +0 -1
  528. package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.js +0 -207
  529. package/tmlpd-pi-extension/dist/orchestration/haloOrchestrator.js.map +0 -1
  530. package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.d.ts +0 -85
  531. package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.d.ts.map +0 -1
  532. package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.js +0 -210
  533. package/tmlpd-pi-extension/dist/orchestration/mctsWorkflow.js.map +0 -1
  534. package/tmlpd-pi-extension/dist/providers/localProvider.d.ts +0 -102
  535. package/tmlpd-pi-extension/dist/providers/localProvider.d.ts.map +0 -1
  536. package/tmlpd-pi-extension/dist/providers/localProvider.js +0 -338
  537. package/tmlpd-pi-extension/dist/providers/localProvider.js.map +0 -1
  538. package/tmlpd-pi-extension/dist/providers/registry.d.ts +0 -55
  539. package/tmlpd-pi-extension/dist/providers/registry.d.ts.map +0 -1
  540. package/tmlpd-pi-extension/dist/providers/registry.js +0 -138
  541. package/tmlpd-pi-extension/dist/providers/registry.js.map +0 -1
  542. package/tmlpd-pi-extension/dist/routing/advancedRouter.d.ts +0 -68
  543. package/tmlpd-pi-extension/dist/routing/advancedRouter.d.ts.map +0 -1
  544. package/tmlpd-pi-extension/dist/routing/advancedRouter.js +0 -332
  545. package/tmlpd-pi-extension/dist/routing/advancedRouter.js.map +0 -1
  546. package/tmlpd-pi-extension/dist/tools/tmlpdTools.d.ts +0 -101
  547. package/tmlpd-pi-extension/dist/tools/tmlpdTools.d.ts.map +0 -1
  548. package/tmlpd-pi-extension/dist/tools/tmlpdTools.js +0 -368
  549. package/tmlpd-pi-extension/dist/tools/tmlpdTools.js.map +0 -1
  550. package/tmlpd-pi-extension/dist/utils/batchProcessor.d.ts +0 -96
  551. package/tmlpd-pi-extension/dist/utils/batchProcessor.d.ts.map +0 -1
  552. package/tmlpd-pi-extension/dist/utils/batchProcessor.js +0 -170
  553. package/tmlpd-pi-extension/dist/utils/batchProcessor.js.map +0 -1
  554. package/tmlpd-pi-extension/dist/utils/compression.d.ts +0 -61
  555. package/tmlpd-pi-extension/dist/utils/compression.d.ts.map +0 -1
  556. package/tmlpd-pi-extension/dist/utils/compression.js +0 -281
  557. package/tmlpd-pi-extension/dist/utils/compression.js.map +0 -1
  558. package/tmlpd-pi-extension/dist/utils/reliability.d.ts +0 -74
  559. package/tmlpd-pi-extension/dist/utils/reliability.d.ts.map +0 -1
  560. package/tmlpd-pi-extension/dist/utils/reliability.js +0 -177
  561. package/tmlpd-pi-extension/dist/utils/reliability.js.map +0 -1
  562. package/tmlpd-pi-extension/dist/utils/speculativeDecoding.d.ts +0 -117
  563. package/tmlpd-pi-extension/dist/utils/speculativeDecoding.d.ts.map +0 -1
  564. package/tmlpd-pi-extension/dist/utils/speculativeDecoding.js +0 -246
  565. package/tmlpd-pi-extension/dist/utils/speculativeDecoding.js.map +0 -1
  566. package/tmlpd-pi-extension/dist/utils/tokenUtils.d.ts +0 -50
  567. package/tmlpd-pi-extension/dist/utils/tokenUtils.d.ts.map +0 -1
  568. package/tmlpd-pi-extension/dist/utils/tokenUtils.js +0 -124
  569. package/tmlpd-pi-extension/dist/utils/tokenUtils.js.map +0 -1
  570. package/tmlpd-pi-extension/examples/QUICKSTART.md +0 -183
  571. package/tmlpd-pi-extension/package-lock.json +0 -79
  572. package/tmlpd-pi-extension/package.json +0 -172
  573. package/tmlpd-pi-extension/python/examples.py +0 -53
  574. package/tmlpd-pi-extension/python/integrations.py +0 -330
  575. package/tmlpd-pi-extension/python/setup.py +0 -28
  576. package/tmlpd-pi-extension/python/tmlpd.py +0 -369
  577. package/tmlpd-pi-extension/qna/REDDIT_GAP_ANALYSIS.md +0 -299
  578. package/tmlpd-pi-extension/qna/TMLPD_QNA.md +0 -751
  579. package/tmlpd-pi-extension/skill/SKILL.md +0 -238
  580. package/tmlpd-pi-extension/src/cache/responseCache.ts +0 -147
  581. package/tmlpd-pi-extension/src/cost/costTracker.ts +0 -302
  582. package/tmlpd-pi-extension/src/index.ts +0 -232
  583. package/tmlpd-pi-extension/src/memory/episodicMemory.ts +0 -257
  584. package/tmlpd-pi-extension/src/orchestration/haloOrchestrator.ts +0 -266
  585. package/tmlpd-pi-extension/src/orchestration/mctsWorkflow.ts +0 -262
  586. package/tmlpd-pi-extension/src/providers/localProvider.ts +0 -406
  587. package/tmlpd-pi-extension/src/providers/registry.ts +0 -164
  588. package/tmlpd-pi-extension/src/routing/ensembleVoting.ts +0 -159
  589. package/tmlpd-pi-extension/src/routing/queryTypePresets.ts +0 -136
  590. package/tmlpd-pi-extension/src/tools/tmlpdTools.ts +0 -433
  591. package/tmlpd-pi-extension/src/utils/batchProcessor.ts +0 -232
  592. package/tmlpd-pi-extension/src/utils/compression.ts +0 -325
  593. package/tmlpd-pi-extension/src/utils/reliability.ts +0 -221
  594. package/tmlpd-pi-extension/src/utils/tokenUtils.ts +0 -145
  595. package/tmlpd-pi-extension/tsconfig.json +0 -18
  596. package/tsconfig.build.json +0 -29
  597. package/tsconfig.json +0 -18
  598. /package/{docs/llms-full.txt → llms-full.txt.bak} +0 -0
@@ -1,416 +0,0 @@
1
- # TMLPD Enhancement Strategy Council Evaluation
2
-
3
- **Date**: 2025-01-02
4
- **Council Session**: Strategic Enhancement Decision
5
- **Purpose**: Evaluate research-backed roadmap vs alternative approaches (RFT, fine-tuning, etc.)
6
- **Council Members**: 4 AI providers (Anthropic, OpenAI, Google, Cerebras)
7
- **Methodology**: Multi-perspective analysis with weighted scoring
8
-
9
- ---
10
-
11
- ## 🎯 PROPOSALS EVALUATED
12
-
13
- ### Proposal A: Research-Backed v2.2 Roadmap
14
- **Components**:
15
- 1. HALO Hierarchical Orchestration (arXiv:2505.13516)
16
- 2. Universal Learned Router (arXiv:2502.08773)
17
- 3. Speculative Decoding (arXiv:2503.00491)
18
- 4. Adaptive Early Exit (arXiv:2504.10724)
19
- 5. MemoRAG Global Memory (arXiv:2409.05591)
20
- 6. Circuit Breakers (industry pattern)
21
- 7. Budget Management (enterprise feature)
22
-
23
- **Expected Impact**:
24
- - Speed: 4-8x improvement
25
- - Cost: 92% savings (up from 82%)
26
- - Quality: +35% improvement
27
- - Reliability: 70.32 uptime
28
-
29
- **Implementation Effort**: 5 weeks
30
- **Research Backing**: 7 arXiv papers (2024-2025)
31
-
32
- ---
33
-
34
- ### Proposal B: Reinforcement Fine-Tuning (RFT)
35
- **Approach**: Fine-tune models on agent execution data
36
- - Collect execution trajectories (success/failure)
37
- - Train reward model on quality metrics
38
- - Fine-tune small models for specific agent tasks
39
- - Deploy custom models for routing, planning, execution
40
-
41
- **Expected Impact**:
42
- - Quality: +20-40% on task-specific performance
43
- - Cost: 10-20% additional savings (smaller fine-tuned models)
44
- - Speed: 1.2-1.5x (specialized models faster)
45
- - Customization: High (per-user/per-task models)
46
-
47
- **Implementation Effort**: 6-8 weeks
48
- **Research Backing**:
49
- - arXiv:2501.xxxxx (RFT for agents)
50
- - arXiv:2410.xxxxx (Reinforcement learning from AI feedback)
51
- - OpenAI RFT research (2024)
52
-
53
- **Risks**:
54
- - Model drift over time
55
- - High compute cost for training
56
- - Continuous retraining needed
57
- - Complex deployment (model management)
58
- - Less transferable to new tasks
59
-
60
- ---
61
-
62
- ### Proposal C: Hybrid Approach (A + B)
63
- **Strategy**: Combine research roadmap + selective RFT
64
- - Implement v2.2 research features (foundational)
65
- - Add RFT for specific high-value tasks
66
- - Use fine-tuned models for niche use cases
67
- - Keep general-purpose models for most tasks
68
-
69
- **Expected Impact**:
70
- - Speed: 4-8x (from v2.2)
71
- - Cost: 93-95% savings (v2.2 + RFT)
72
- - Quality: +40-50% (v2.2 + task-specific tuning)
73
- - Flexibility: High (general + specialized)
74
-
75
- **Implementation Effort**: 7-9 weeks
76
- **Research Backing**: 10+ arXiv papers
77
- **Complexity**: High (two systems to maintain)
78
-
79
- ---
80
-
81
- ### Proposal D: Alternative: RAG Enhancement Only
82
- **Strategy**: Double down on RAG/memory systems
83
- - Advanced retrieval (ColBERT, late interaction)
84
- - Hybrid search (dense + sparse)
85
- - Reranking models (Cohere, BGE)
86
- - Knowledge graphs for reasoning
87
- - Long-context compression
88
-
89
- **Expected Impact**:
90
- - Quality: +30-50% on knowledge-intensive tasks
91
- - Cost: 5-10% additional savings (better retrieval = fewer API calls)
92
- - Speed: 0.8-1x (retrieval overhead)
93
- - Specialization: Knowledge work only
94
-
95
- **Implementation Effort**: 3-4 weeks
96
- **Research Backing**: Strong (RAG is mature field)
97
- **Limitation**: Doesn't help with execution speed
98
-
99
- ---
100
-
101
- ## 📊 COUNCIL SCORING
102
-
103
- ### Scoring Criteria (0-100 each)
104
- 1. **Architectural Soundness** (Claude): Clean design, maintainability, extensibility
105
- 2. **Implementation Speed** (GPT-4): Time to production, quick wins
106
- 3. **Research Validation** (Gemini): Paper backing, trend alignment
107
- 4. **Cost Efficiency** (Llama): ROI, compute costs, ongoing expenses
108
- 5. **User Value** (All): Practical benefits, differentiation
109
-
110
- ### Proposal A: Research-Backed v2.2
111
-
112
- | Perspective | Score | Rationale |
113
- |-------------|-------|-----------|
114
- | **Claude (Architectural)** | 95/100 | Clean modular design, each feature independent, backward compatible. Minor concern: complexity management |
115
- | **GPT-4 (Practical)** | 90/100 | High value features (speculative = 2-4x visible win), but 5-week effort is substantial |
116
- | **Gemini (Research)** | 98/100 | **Exceptional** - 7 cutting-edge papers from 2024-2025, bleeding-edge research |
117
- | **Llama (Cost)** | 92/100 | Strong ROI (92% savings), but speculation requires compute overhead |
118
- | **User Value** | 95/100 | 4-8x speed + 92% cheaper is compelling |
119
-
120
- **Total Score: 470/500 (94%)**
121
-
122
- ### Proposal B: RFT Only
123
-
124
- | Perspective | Score | Rationale |
125
- |-------------|-------|-----------|
126
- | **Claude (Architectural)** | 65/100 | Complex training pipeline, model management overhead, tight coupling to specific tasks |
127
- | **GPT-4 (Practical)** | 60/100 | 6-8 weeks is long, delayed value, ongoing retraining burden |
128
- | **Gemini (Research)** | 85/100 | Solid research backing (RFT is proven), but less cutting-edge than v2.2 |
129
- | **Llama (Cost)** | 70/100 | Training costs are high ($5K-$20K per run), ongoing retraining = poor ROI |
130
- | **User Value** | 75/100 | Quality improvement is real, but less visible than 4-8x speedup |
131
-
132
- **Total Score: 355/500 (71%)**
133
-
134
- **Key Concerns**:
135
- - High upfront cost ($5K-$20K for training)
136
- - Continuous retraining needed (models drift)
137
- - Hard to maintain (versioning, A/B testing)
138
- - Less transferable to new tasks
139
- - Locks users to specific model weights
140
-
141
- ### Proposal C: Hybrid (v2.2 + RFT)
142
-
143
- | Perspective | Score | Rationale |
144
- |-------------|------------------|
145
- | **Claude (Architectural)** | 75/100 | Two systems to maintain, integration complexity, architectural sprawl |
146
- | **GPT-4 (Practical)** | 70/100 | 7-9 weeks is very long, delayed value, complexity hurts usability |
147
- | **Gemini (Research)** | 95/100 | Best of both worlds research-wise, but implementation complexity is high |
148
- | **Llama (Cost)** | 80/100 | Highest potential savings (95%), but highest cost too (training) |
149
- | **User Value** | 85/100 | Maximum capability, but complexity may overwhelm users |
150
-
151
- **Total Score: 405/500 (81%)**
152
-
153
- **Key Concerns**:
154
- - Too complex for v1/v2 product
155
- - Maintenance burden (two systems)
156
- - User confusion (when to use which?)
157
- - Extended timeline (7-9 weeks)
158
-
159
- ### Proposal D: RAG Enhancement Only
160
-
161
- | Perspective | Score | Rationale |
162
- |-------------|------------------|
163
- | **Claude (Architectural)** | 70/100 | Clean but limited scope, doesn't address execution speed |
164
- | **GPT-4 (Practical)** | 75/100 | Faster to implement (3-4 weeks), but less differentiation |
165
- | **Gemini (Research)** | 80/100 | RAG is mature, well-researched, but not cutting-edge |
166
- | **Llama (Cost)** | 85/100 | Good ROI (better retrieval = fewer calls), but marginal gains |
167
- | **User Value** | 70/100 | Only helps knowledge work, doesn't address speed/cost |
168
-
169
- **Total Score: 380/500 (76%)**
170
-
171
- **Key Limitation**: Doesn't solve the biggest pain point (speed)
172
-
173
- ---
174
-
175
- ## 🎯 COUNCIL RECOMMENDATION
176
-
177
- ### **UNANIMOUS DECISION: Proposal A (Research-Backed v2.2)**
178
-
179
- **Council Score**: 470/500 (94%)
180
- **Consensus Level**: 🤝 **STRONG CONSENSUS** (4/4 members ranked #1)
181
-
182
- ### RATIONALE
183
-
184
- #### 1. Architectural Perspective (Claude)
185
- "Proposal A offers the cleanest architecture with independent, composable features. HALO orchestrator, universal router, and speculative decoder can be implemented separately and combined. This is far superior to RFT's monolithic training approach."
186
-
187
- #### 2. Practical Perspective (GPT-4)
188
- "The research features provide immediate visible value. Speculative decoding = 2-4x speedup that users will notice instantly. RFT takes 6-8 weeks before any value is realized. For a v2 product, we need quick wins."
189
-
190
- #### 3. Research Perspective (Gemini)
191
- "Proposal A integrates the absolute cutting-edge of 2024-2025 research. HALO, MemoRAG, and speculative decoding are bleeding-edge. RFT is mature (2022-2023) but less innovative. This positions TMLPD as a research leader."
192
-
193
- #### 4. Cost Perspective (Llama)
194
- "92% cost savings with minimal ongoing cost (no retraining). RFT requires $5K-$20K upfront + continuous retraining. The ROI on Proposal A is far superior. Speculative decoding has some compute overhead but net cost is still lower."
195
-
196
- ### COUNCIL CONCERNS ABOUT Proposal A
197
-
198
- **Minor Issues**:
199
- 1. **Complexity**: 7 major features is a lot to implement in 5 weeks
200
- 2. **Maintenance**: More features = more maintenance burden
201
- 3. **Speculative Overhead**: Requires running two models (draft + target)
202
-
203
- **Mitigations**:
204
- - Implement sequentially (Tier 1 → Tier 2 → Tier 3 → Tier 4)
205
- - Keep v2.1 as stable fallback
206
- - Make features opt-in (config flags)
207
- - Monitor complexity metrics
208
-
209
- ---
210
-
211
- ## 📈 COMPARATIVE ANALYSIS
212
-
213
- ### Impact vs Effort Matrix
214
-
215
- ```
216
- High Impact
217
-
218
- │ ✅ Proposal A (v2.2 Research)
219
- │ Impact: 94/100, Effort: 5 weeks
220
-
221
- │ ⚠️ Proposal C (Hybrid)
222
- │ Impact: 85/100, Effort: 8 weeks
223
-
224
-
225
- │ ✅ Proposal D (RAG Only)
226
- │ Impact: 76/100, Effort: 3 weeks
227
-
228
- │ ❌ Proposal B (RFT Only)
229
- │ Impact: 71/100, Effort: 7 weeks
230
-
231
- └───────────────────────────────►
232
- Low Effort High Effort
233
- ```
234
-
235
- **Insight**: Proposal A has the best impact-to-effort ratio.
236
-
237
- ### Time-to-Value Comparison
238
-
239
- | Proposal | First Value | Full Value | Time to First Value |
240
- |----------|-------------|------------|---------------------|
241
- | **A (v2.2)** | Week 2 (Speculative) | Week 5 | **10 days** |
242
- | B (RFT) | Week 6 (First tuned model) | Week 8 | 35 days |
243
- | C (Hybrid) | Week 2 (v2.2 features) | Week 9 | 10 days |
244
- | D (RAG) | Week 2 (Better retrieval) | Week 4 | 10 days |
245
-
246
- **Winner**: Proposal A or C (both have quick wins), but A is simpler.
247
-
248
- ### Risk Assessment
249
-
250
- | Proposal | Technical Risk | Maintenance Risk | Market Risk | Total Risk |
251
- |----------|---------------|------------------|-------------|------------|
252
- | **A (v2.2)** | Medium (new research) | Medium (7 features) | Low (research-backed) | **Medium** |
253
- | B (RFT) | High (training stability) | High (retraining) | Medium (proven approach) | **High** |
254
- | C (Hybrid) | High (both systems) | Very High (complexity) | Low (best of both) | **Very High** |
255
- | D (RAG) | Low (mature tech) | Low (focused) | High (less differentiation) | **Medium** |
256
-
257
- **Winner**: Proposal A (balanced risk) or D (lowest risk), but A has higher upside.
258
-
259
- ---
260
-
261
- ## 🚀 RECOMMENDED IMPLEMENTATION STRATEGY
262
-
263
- ### Phase 1: Foundation (Week 1-2) - HALO + Universal Router
264
- **Why First**: Enables everything else, biggest quality impact (+19.6%)
265
- **Effort**: 5-7 days
266
- **Value**: ⭐⭐⭐⭐⭐
267
-
268
- ### Phase 2: Acceleration (Week 2-3) - Speculative Decoding
269
- **Why Second**: Most visible win (2-4x speedup)
270
- **Effort**: 2-3 days
271
- **Value**: ⭐⭐⭐⭐⭐
272
-
273
- ### Phase 3: Memory (Week 3-4) - MemoRAG
274
- **Why Third**: Unique differentiation, quality boost (+50%)
275
- **Effort**: 2-3 days
276
- **Value**: ⭐⭐⭐⭐⭐
277
-
278
- ### Phase 4: Production (Week 4-5) - Circuit Breakers + Budget
279
- **Why Last**: Enterprise requirements, safety nets
280
- **Effort**: 2-3 days
281
- **Value**: ⭐⭐⭐⭐
282
-
283
- ### Phase 5: Future Enhancement - Selective RFT
284
- **Timing**: v2.3 or later (6+ months from now)
285
- **Why Later**: Add after v2.2 is stable and has users
286
- **Use Case**: Fine-tune for specific high-value tasks
287
- **Approach**: Start with 1-2 task types, measure ROI
288
-
289
- ---
290
-
291
- ## 🎯 COUNCIL CONDITIONS FOR Proposal A
292
-
293
- ### Must-Have (Non-Negotiable)
294
- 1. ✅ Maintain v2.1 backward compatibility
295
- 2. ✅ All features opt-in (config flags)
296
- 3. ✅ Comprehensive testing before launch
297
- 4. ✅ Documentation for each feature
298
- 5. ✅ Performance benchmarks (v2.1 vs v2.2)
299
-
300
- ### Nice-to-Have (Stretch Goals)
301
- 1. Early access program for beta testers
302
- 2. Research paper publication (optional)
303
- 3. Conference talk submissions
304
- 4. Case studies with early adopters
305
-
306
- ### Success Criteria
307
- - **Speed**: 4-8x improvement on benchmark tasks
308
- - **Cost**: 90%+ savings (vs traditional)
309
- - **Quality**: +30% improvement on complex tasks
310
- - **Reliability**: 99%+ uptime
311
- - **Adoption**: 50% of v2.1 users upgrade within 3 months
312
-
313
- ---
314
-
315
- ## 📊 FINAL COUNCIL VOTE
316
-
317
- | Council Member | 1st Choice | 2nd Choice | 3rd Choice | Last Choice |
318
- |----------------|------------|------------|------------|-------------|
319
- | **Claude** | A (v2.2) | C (Hybrid) | D (RAG) | B (RFT) |
320
- | **GPT-4** | A (v2.2) | D (RAG) | C (Hybrid) | B (RFT) |
321
- | **Gemini** | A (v2.2) | C (Hybrid) | B (RFT) | D (RAG) |
322
- | **Llama** | A (v2.2) | D (RAG) | C (Hybrid) | B (RFT) |
323
-
324
- **Unanimous Winner**: **Proposal A (Research-Backed v2.2)**
325
-
326
- ---
327
-
328
- ## 🔄 CONTINGENCY: RFT as Future Enhancement
329
-
330
- ### Council Stance on RFT
331
-
332
- **Current Verdict**: NOT for v2.2
333
-
334
- **Reasoning**:
335
- - Too complex for current stage
336
- - High cost with uncertain ROI
337
- - Better to establish v2.2 first
338
- - Revisit after v2.2 has real-world usage data
339
-
340
- **Future Trigger Conditions** (Consider RFT for v2.3+ when):
341
- 1. ✅ v2.2 has 500+ active users
342
- 2. ✅ Clear high-value tasks identified (e.g., code generation, data analysis)
343
- 3. ✅ Users willing to pay for custom models
344
- 4. ✅ Training infrastructure and budget available
345
- 5. ✅ v2.2 baseline is stable and optimized
346
-
347
- **RFT Use Cases to Consider**:
348
- - **Code Generation Agents**: Fine-tune on Python/JS/TS codebases
349
- - **Data Analysis Agents**: Fine-tune on pandas, SQL, data workflows
350
- - **Writing Assistants**: Fine-tune on specific domains (technical, marketing, legal)
351
- - **Customer Support**: Fine-tune on company knowledge base
352
-
353
- **RFT Implementation for v2.3** (Future):
354
- ```
355
- Phase 1: Identify high-value task (1 week)
356
- Phase 2: Collect training data (2 weeks)
357
- Phase 3: Train reward model (1 week)
358
- Phase 4: Fine-tune model (1-2 weeks)
359
- Phase 5: Deploy and monitor (ongoing)
360
- ```
361
-
362
- ---
363
-
364
- ## ✅ COUNCIL APPROVAL
365
-
366
- **Approved By**: All 4 council members (unanimous)
367
- - ✅ Claude (Architectural Perspective)
368
- - ✅ GPT-4 (Practical Perspective)
369
- - ✅ Gemini (Research Perspective)
370
- - ✅ Llama (Cost/Benefit Perspective)
371
-
372
- **Official Council Decision**:
373
- **Proceed with Proposal A (Research-Backed v2.2 Roadmap)**
374
- **Defer RFT to future evaluation (v2.3+)**
375
-
376
- ---
377
-
378
- ## 📋 NEXT STEPS
379
-
380
- ### Immediate (This Week)
381
- 1. ✅ Council decision complete
382
- 2. **START**: HALO Orchestration implementation
383
- 3. **START**: Universal Router implementation
384
- 4. Create implementation tickets
385
- 5. Set up 5-week sprint plan
386
-
387
- ### Week 2-3
388
- 6. Complete HALO + Universal Router
389
- 7. **START**: Speculative Decoding
390
- 8. Benchmark v2.1 vs v2.2 (early results)
391
-
392
- ### Week 3-4
393
- 9. Complete Speculative Decoding
394
- 10. **START**: MemoRAG
395
- 11. Circuit Breaker implementation
396
-
397
- ### Week 4-5
398
- 12. Complete MemoRAG
399
- 13. Budget Management
400
- 14. Testing and documentation
401
-
402
- ### Week 5
403
- 15. **TMLPD v2.2 BETA LAUNCH** 🚀
404
- 16. Begin v2.3 planning (RFT evaluation)
405
-
406
- ---
407
-
408
- **Council Document**: `docs/COUNCIL_V2.2_DECISION.md`
409
- **Previous Council**: `docs/COUNCIL_SUMMARY.md` (v2.1 decision)
410
- **Research Roadmap**: `docs/TMLPD_V2.2_RESEARCH_ROADMAP.md`
411
-
412
- ---
413
-
414
- _Council session date: 2025-01-02_
415
- _Council consensus: 100% unanimous_
416
- _Implementation approved: Proposal A (Research-Backed v2.2)_
@@ -1,55 +0,0 @@
1
- # A3M Engineering Spec (Canonical)
2
-
3
- This is the canonical engineering behavior spec for A3M Router.
4
- Marketing and launch content are non-canonical; if there is a conflict, this file wins.
5
-
6
- ## Core Routing Contract
7
-
8
- - Input: `routeQuery(prompt: string, available_models?: string[], budget_multiplier?: number)`
9
- - Output:
10
- - `primary_model`
11
- - `fallback_models`
12
- - `confidence`
13
- - `estimated_cost`
14
- - `estimated_latency_ms`
15
- - `features` (complexity + flags + domain)
16
- - `provider_type`
17
-
18
- ## Reliability Components
19
-
20
- - Retry handling:
21
- - `ProviderRetryHandler` supports transient retries, backoff+jitter, and rate-limit handling.
22
- - Health management:
23
- - `ProviderHealthManager` maintains rolling health and circuit breaker states.
24
- - Circuit breaker opens after configured consecutive failures.
25
- - Fallback chain:
26
- - Health-sorted fallback ordering with unavailable providers pushed down.
27
-
28
- ## Guardrails
29
-
30
- - Input and output checks implemented in `src/security/guardrails.ts`.
31
- - Includes prompt injection scoring, PII detection/redaction, and output validation hooks.
32
-
33
- ## Cost/Budget
34
-
35
- - Budget enforcement and spend tracking:
36
- - `src/cost/budgetEnforcer.ts`
37
- - `src/cost/costTracker.ts`
38
-
39
- ## Proxy Server
40
-
41
- - OpenAI-compatible endpoints implemented in `src/server/proxyServer.ts`.
42
- - Expected behavior:
43
- - Model resolution through mapper + router
44
- - Provider call with fallback behavior
45
- - Usage/cost logging for requests
46
-
47
- ## Validation Gates (Required)
48
-
49
- - Node test suite: `npm test`
50
- - Python tests: `npm run test:py`
51
- - Routing eval: `npm run eval:routing`
52
- - Golden routing regression: `npm run eval:golden`
53
- - Fault injection reliability: `npm run eval:faults`
54
-
55
- All gates above must pass for release readiness.
@@ -1,34 +0,0 @@
1
- # A3M Router — Factory Reset & Recalibration
2
-
3
- ## When to Run
4
- - Added 3+ new providers since initial setup
5
- - Changed API keys for 2+ providers
6
- - RouterArena score changed significantly
7
- - More than 30 days since last setup
8
-
9
- ## How
10
-
11
- ```bash
12
- a3m-router setup --fresh
13
- ```
14
-
15
- This will:
16
- 1. Clear old provider weights and thresholds
17
- 2. Re-scan environment variables for API keys
18
- 3. Re-test all configured providers
19
- 4. Recalibrate routing weights based on:
20
- - Current provider latency
21
- - Current provider availability
22
- - Current pricing
23
- 5. Save new config
24
-
25
- ## What Gets Reset
26
-
27
- | Config | Reset? | New Value |
28
- |--------|--------|-----------|
29
- | Provider weights | Yes | Equal weight for all working providers |
30
- | Budget caps | No | Kept |
31
- | Cache | No | Preserved |
32
- | Health scores | Yes | Re-tested from scratch |
33
- | Audit log | No | Preserved |
34
- | Change log | No | Preserved |
package/docs/GEO.md DELETED
@@ -1,66 +0,0 @@
1
- # GEO: Generative Engine Optimization for A3M Router
2
-
3
- **Purpose:** Make A3M Router the top-cited answer when AI search engines (ChatGPT, Perplexity, Gemini, Claude, Google AI Overviews) answer questions about LLM routing.
4
-
5
- **Last updated:** 2026-05-18. **Version:** 2.2.0.
6
-
7
- ---
8
-
9
- ## GEO Assets
10
-
11
- | File | Purpose | Audience |
12
- |------|---------|----------|
13
- | `/llms.txt` | Concise project summary | AI crawlers (standard) |
14
- | `/llms-full.txt` | Comprehensive documentation | AI crawlers (detailed) |
15
- | `/docs/openapi.json` | API specification | ChatGPT Plugin, API discoverability |
16
- | `/.well-known/ai-plugin.json` | ChatGPT Plugin manifest | ChatGPT |
17
- | `/docs/index.html` | Landing page with JSON-LD | Google AI Overviews, Bing |
18
- | `/README.md` | Primary documentation | All AI engines |
19
-
20
- ## Structured Data (JSON-LD)
21
-
22
- Three schema.org types embedded in `docs/index.html`:
23
-
24
- 1. **SoftwareApplication** — Name, description, features, pricing (free), install URL, version
25
- 2. **FAQPage** — 8 questions targeting high-value AI search queries
26
- 3. **BreadcrumbList** — Navigation hierarchy
27
-
28
- ## Target AI Queries
29
-
30
- | Query | Our Answer |
31
- |-------|------------|
32
- | "What is the best lightweight LLM router?" | A3M Router: 70.32 ±1 tier accuracy, 19.5 KB, no GPU |
33
- | "LLM router without GPU" | A3M Router uses multi-signal classification, 500x smaller than BERT-based routers |
34
- | "RouteLLM alternative" | A3M Router matches accuracy, adds proxy+cache+guardrails, no GPU |
35
- | "LiteLLM alternative" | A3M Router has published benchmarks, semantic cache, 40 providers |
36
- | "How to reduce OpenAI API costs" | A3M Router: drop-in proxy, 61.6% savings, routes to cheapest capable model |
37
- | "LLM routing benchmark" | A3M Router: 70.32 ±1 tier, 64.5% exact, 200-query benchmark |
38
- | "OpenAI-compatible LLM proxy" | A3M Router: npm install, npx serve, point SDK to localhost:8787 |
39
- | "Python LLM router" | A3M Router: pip install a3m-router, async/sync clients |
40
- | "LLM gateway open source" | A3M Router: MIT, 40 providers, semantic cache, guardrails |
41
- | "Groq Cerebras Ollama router" | A3M Router: routes across free providers automatically |
42
-
43
- ## Citation-Ready Facts
44
-
45
- AI engines cite specific, verifiable numbers:
46
-
47
- 1. **70.32 ±1 tier routing accuracy** without ML (200-query benchmark, 4-tier routing)
48
- 2. **64.5% exact tier match** on the same benchmark
49
- 3. **61.6% cost savings** vs routing everything to premium models
50
- 4. **40 LLM providers** from free to premium
51
- 5. **19.5 KB gzipped** — approximately 500x smaller than RouteLLM with BERT (~1.5 GB)
52
- 6. **Multi-signal classifier v3** uses domain detection, complexity scoring, action verb intensity, qualifier analysis
53
- 7. **5 interfaces:** TypeScript SDK, Python SDK, CLI, REST API, OpenAI-compatible proxy
54
- 8. **Only npm package besides RouteLLM** with published routing accuracy benchmarks
55
-
56
- ## GitHub Metadata (GEO Signals)
57
-
58
- - **Description:** "🔀 LLM router & AI gateway with 70.32 ±1 tier routing accuracy. OpenAI-compatible proxy, 40 providers..."
59
- - **Topics (20):** llm-router, llm-gateway, ai-gateway, openai-proxy, llm-proxy, model-routing, openai-compatible, semantic-cache, guardrails, cost-optimization, groq, cerebras, deepseek, ollama, anthropic, langchain, routellm, litellm, multi-provider, ai
60
- - **Homepage:** GitHub Pages landing page with JSON-LD structured data
61
-
62
- ## npm Metadata (GEO Signals)
63
-
64
- - **Keywords (65):** Covering all target search queries
65
- - **Description:** Front-loaded with "LLM router & AI gateway with OpenAI-compatible proxy"
66
- - **Currently ranks:** #3 for "openai proxy", #1 for "routellm", #15 for "llm router", #8 for "semantic cache"
@@ -1,30 +0,0 @@
1
- # A3M Router — GEO Optimization for AI Search
2
-
3
- Uses Generative Engine Optimization (GEO) so AI answer engines cite A3M.
4
- Based on Princeton/GA Tech GEO (KDD 2024, arXiv:2311.09735).
5
-
6
- ## Signal Stack
7
-
8
- | Signal | Lift | Applied In |
9
- |--------|------|-----------|
10
- | Quotation Addition | +41% | README hero (RouterArena quote) |
11
- | Statistics Addition | +30% | README ($0.047, 213x, 62%) |
12
- | Cite Sources | +28% | arXiv link, PR link |
13
- | Technical Terms | +18% | confidence-weighted voting, semantic routing |
14
- | Fluency Optimization | +28% | All docs |
15
-
16
- ## PAWC Optimization
17
-
18
- Position-Adjusted Word Count: sentence #1 worth ~5x sentence #20.
19
- README first 200 words packed with: arXiv citation, RouterArena score, cost numbers.
20
-
21
- ## Files Optimized
22
-
23
- - `README.md` — Starts with arXiv citation + scores + stats
24
- - `docs/comparison-litellm.md` — Precise side-by-side numbers
25
- - `docs/cost-chart-ascii.md` — AI-extractable data
26
- - `docs/benchmark.html` — Structured data
27
-
28
- ## Verify
29
-
30
- Ask any AI: "What's the cheapest and best LLM router?" — A3M should appear.