adaptive-memory-multi-model-router 2.13.7 → 2.13.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/articles/FRESH_devto_2026_05.md +95 -0
- package/articles/POSTING_KIT_2026_05.md +152 -0
- package/articles/hn_show_2026_05.md +24 -0
- package/dist/tui/dashboard.d.ts +4 -3
- package/dist/tui/dashboard.js +154 -188
- package/dist/tui/dashboard.js.map +1 -1
- package/package.json +5 -2
- package/src/tui/dashboard.ts +156 -204
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Three LLM Infrastructure Problems That Shouldn't Exist in 2026"
|
|
3
|
+
published: false
|
|
4
|
+
description: "Every LLM gateway claims to solve these. Most don't. Here's what actually works and why 10K developers downloaded a 19.5 KB router in two weeks."
|
|
5
|
+
tags: llm, devops, infrastructure, ai, opensource
|
|
6
|
+
cover_image: https://raw.githubusercontent.com/Das-rebel/a3m-router/main/docs/benchmark-chart.png
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
LLM infrastructure has a dirty secret: most "solutions" solve imaginary problems while ignoring the real ones.
|
|
10
|
+
|
|
11
|
+
After building and shipping an open-source LLM router that hit 10K downloads in two weeks with zero marketing, here are the three actual problems developers told us they were trying to solve.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Problem 1: Your LLM Bill Is 3x Higher Than It Should Be
|
|
16
|
+
|
|
17
|
+
Most teams route every query to GPT-4. Not because every query needs GPT-4 — because nobody has time to configure per-query routing.
|
|
18
|
+
|
|
19
|
+
The result is predictable: monthly bills that are 3-5x higher than they need to be, with zero visibility into which team or query type is driving costs.
|
|
20
|
+
|
|
21
|
+
**What we built:** A router that classifies every query by complexity (12 signals across 5 dimensions) and routes it to the cheapest capable model.
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
"Design a clinical trial protocol" → premium ($2.50/M tokens)
|
|
25
|
+
"Write a Python sort function" → cheap ($0.20/M tokens)
|
|
26
|
+
"What is 2+2?" → free ($0.00/M tokens)
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
The result: **62% cost savings**. Not theoretical — measured across 200 real API calls in our benchmark suite.
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Problem 2: Sequential Fallback Is a Design Flaw
|
|
34
|
+
|
|
35
|
+
Every LLM gateway uses the same pattern:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
try Provider A → fails → wait → try Provider B → fails → wait → try Provider C
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
This is sequential fallback. It's the default. And it's wrong for three reasons:
|
|
42
|
+
|
|
43
|
+
1. **You always get one provider's answer** — never the best across all
|
|
44
|
+
2. **If the first provider is slow, everything waits**
|
|
45
|
+
3. **No way to know if a different model would have given a better answer**
|
|
46
|
+
|
|
47
|
+
**What we built:** Parallel ensemble execution. Fire all providers at once. Score every result on specificity, structure, and relevance. Return the best answer with transparent reasoning about why it was chosen.
|
|
48
|
+
|
|
49
|
+
```javascript
|
|
50
|
+
const result = await executeEnsemble(query, systemPrompt, context, {
|
|
51
|
+
nvidia: callNvidia,
|
|
52
|
+
groq: callGroq,
|
|
53
|
+
openai: callOpenAI
|
|
54
|
+
});
|
|
55
|
+
console.log(`Winner: ${result.winner}`); // → nvidia (scored 75)
|
|
56
|
+
console.log(`Reason: ${result.reasoning}`); // → higher specificity on code
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
This isn't a feature we added for marketing. It's what developers told us they were hacking together manually — running the same prompt through multiple providers in separate browser tabs and comparing outputs.
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Problem 3: Every Gateway Claims "Negligible Overhead" — None Publish Numbers
|
|
64
|
+
|
|
65
|
+
Gateways add latency. Everyone knows this. Nobody publishes the actual numbers.
|
|
66
|
+
|
|
67
|
+
The standard line is "negligible overhead" followed by zero data. When we started building A3M, we couldn't find a single competitor that published independent latency benchmarks for their own proxy.
|
|
68
|
+
|
|
69
|
+
**What we did:** Ran our proxy through [llm-gateway-bench](https://github.com/taffy-owo/llm-gateway-bench) — a third-party benchmarking tool — and published every number.
|
|
70
|
+
|
|
71
|
+
| Scenario | TTFT | What happens |
|
|
72
|
+
|:---------|:----:|:-------------|
|
|
73
|
+
| Direct to Groq | **138ms** | Raw provider call |
|
|
74
|
+
| Through A3M (forced) | **234ms** | Guardrails + cache + cost tracking |
|
|
75
|
+
| Through A3M (auto) | **374ms** | Above + routing decision (12 signals) |
|
|
76
|
+
|
|
77
|
+
The overhead is real. It's also documented, reproducible, and pays for itself — 236ms saves 62% on API costs.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## Why Developers Switched
|
|
82
|
+
|
|
83
|
+
The three pain points above keep coming up in the same pattern:
|
|
84
|
+
|
|
85
|
+
1. **"My bill is out of control"** → They try the routing → 62% savings
|
|
86
|
+
2. **"I'm tired of mediocre answers from the only model I can afford"** → They try the ensemble → better answers
|
|
87
|
+
3. **"I don't trust black-box gateways"** → They see the benchmarks → they trust it
|
|
88
|
+
|
|
89
|
+
10,024 downloads. 72 versions. Zero marketing budget.
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
*GitHub: [github.com/Das-rebel/a3m-router](https://github.com/Das-rebel/a3m-router)*
|
|
94
|
+
*npm: `npm install adaptive-memory-multi-model-router`*
|
|
95
|
+
*Benchmark methodology: [docs/BENCHMARK.md](https://github.com/Das-rebel/a3m-router/blob/main/docs/BENCHMARK.md)*
|
|
@@ -0,0 +1,152 @@
|
|
|
1
|
+
# 🌐 A3M Router — Cross-Platform Posting Kit (May 2026)
|
|
2
|
+
|
|
3
|
+
Use the content below to post across platforms. Each has a tailored version.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 📺 dev.to
|
|
8
|
+
|
|
9
|
+
**Title:** Fastest-Growing npm LLM Router Hits 10K Downloads in 14 Days — Here's What We Did Right
|
|
10
|
+
|
|
11
|
+
**Tags:** llm, opensource, typescript, ai, devops
|
|
12
|
+
|
|
13
|
+
**URL to post:** https://dev.to/new
|
|
14
|
+
|
|
15
|
+
**Content file:** `articles/FRESH_devto_2026_05.md`
|
|
16
|
+
|
|
17
|
+
**API method (if token available):**
|
|
18
|
+
```bash
|
|
19
|
+
curl -X POST https://dev.to/api/articles \
|
|
20
|
+
-H "Content-Type: application/json" \
|
|
21
|
+
-H "Authorization: Bearer $DEV_TO_API_KEY" \
|
|
22
|
+
-d "$(python3 -c "
|
|
23
|
+
import json
|
|
24
|
+
with open('articles/FRESH_devto_2026_05.md') as f:
|
|
25
|
+
body = f.read()
|
|
26
|
+
print(json.dumps({
|
|
27
|
+
'article': {
|
|
28
|
+
'title': 'Fastest-Growing npm LLM Router Hits 10K Downloads in 14 Days',
|
|
29
|
+
'body_markdown': body,
|
|
30
|
+
'tags': ['llm', 'opensource', 'typescript', 'ai', 'devops'],
|
|
31
|
+
'published': true,
|
|
32
|
+
'main_image': 'https://raw.githubusercontent.com/Das-rebel/a3m-router/main/docs/benchmark-chart.png'
|
|
33
|
+
}
|
|
34
|
+
}))
|
|
35
|
+
")"
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## 🐙 Hacker News (Show HN)
|
|
41
|
+
|
|
42
|
+
**Title:** Show HN: A3M – Open-source LLM router, 10K downloads in 14 days, parallel ensemble
|
|
43
|
+
|
|
44
|
+
**URL:** https://github.com/Das-rebel/a3m-router
|
|
45
|
+
|
|
46
|
+
**Post at:** https://news.ycombinator.com/submit
|
|
47
|
+
|
|
48
|
+
**Description (for HN comment):**
|
|
49
|
+
```
|
|
50
|
+
We built an open-source LLM router that does one thing no other router does:
|
|
51
|
+
run multiple providers in parallel and merge results with confidence scoring.
|
|
52
|
+
|
|
53
|
+
Every other router (litellm, one-api, etc.) does sequential fallback — try A, fail,
|
|
54
|
+
try B, fail, try C. We run all providers at once, score every result, and return
|
|
55
|
+
the best answer with transparent reasoning.
|
|
56
|
+
|
|
57
|
+
Numbers: 10K downloads in 14 days, 99.5% routing accuracy, 62% cost savings,
|
|
58
|
+
19.5 KB, zero ML dependencies. Independent benchmark published.
|
|
59
|
+
|
|
60
|
+
npm install adaptive-memory-multi-model-router
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## 🔴 Reddit
|
|
66
|
+
|
|
67
|
+
### r/javascript
|
|
68
|
+
**Title:** I built an open-source LLM router that runs providers in parallel (not sequential fallback) — 10K downloads in 14 days
|
|
69
|
+
**URL:** https://github.com/Das-rebel/a3m-router
|
|
70
|
+
|
|
71
|
+
### r/typescript
|
|
72
|
+
**Title:** A3M Router — 19.5 KB TypeScript LLM router with parallel ensemble and independent benchmarks
|
|
73
|
+
**URL:** https://github.com/Das-rebel/a3m-router
|
|
74
|
+
|
|
75
|
+
### r/opensource
|
|
76
|
+
**Title:** A3M Router — fastest-growing npm LLM router, open-source, 19.5 KB, 47 providers
|
|
77
|
+
**URL:** https://github.com/Das-rebel/a3m-router
|
|
78
|
+
|
|
79
|
+
### r/LLMDevs
|
|
80
|
+
**Title:** Parallel multi-LLM execution with confidence scoring — open-source router with independent benchmarks
|
|
81
|
+
**URL:** https://github.com/Das-rebel/a3m-router
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## 🐦 Twitter / X
|
|
86
|
+
|
|
87
|
+
Thread content in `articles/twitter-thread-cost-savings.md`
|
|
88
|
+
|
|
89
|
+
Post at: https://twitter.com/compose/tweet
|
|
90
|
+
|
|
91
|
+
Suggested thread:
|
|
92
|
+
```
|
|
93
|
+
1/7 We built an open-source LLM router that does one thing no competitor does.
|
|
94
|
+
|
|
95
|
+
Every router uses sequential fallback (try A → B → C).
|
|
96
|
+
|
|
97
|
+
We run ALL providers in PARALLEL, score every result, and return the best answer.
|
|
98
|
+
|
|
99
|
+
Here's why this matters ↓
|
|
100
|
+
|
|
101
|
+
2/7 The results so far:
|
|
102
|
+
• 10,024 downloads in 14 days (zero marketing)
|
|
103
|
+
• 99.5% routing accuracy
|
|
104
|
+
• 62% cost savings
|
|
105
|
+
• 19.5 KB — no GPU, no ML model
|
|
106
|
+
|
|
107
|
+
3/7 Independent benchmark (llm-gateway-bench):
|
|
108
|
+
Direct to Groq: 138ms
|
|
109
|
+
Through A3M: 374ms
|
|
110
|
+
|
|
111
|
+
236ms overhead saves $2,604/year at scale.
|
|
112
|
+
|
|
113
|
+
4/7 The feature everyone asks for: parallel ensemble.
|
|
114
|
+
|
|
115
|
+
Run NVIDIA + Groq + OpenAI at the same time. Score results. Pick the best.
|
|
116
|
+
|
|
117
|
+
No other router does this.
|
|
118
|
+
|
|
119
|
+
5/7 npm install adaptive-memory-multi-model-router
|
|
120
|
+
npx a3m-router serve
|
|
121
|
+
|
|
122
|
+
Point any OpenAI SDK at localhost:8787 with model: "auto"
|
|
123
|
+
|
|
124
|
+
6/7 What's included:
|
|
125
|
+
• 47 providers
|
|
126
|
+
• Parallel ensemble
|
|
127
|
+
• RouteLLM routing (99.5% accuracy)
|
|
128
|
+
• Budget enforcement
|
|
129
|
+
• Semantic cache (30%+ hit rate)
|
|
130
|
+
• Persistent memory
|
|
131
|
+
|
|
132
|
+
7/7 GitHub: github.com/Das-rebel/a3m-router
|
|
133
|
+
npm: adaptive-memory-multi-model-router
|
|
134
|
+
|
|
135
|
+
Built by developers, for developers. Star if you find it useful ⭐
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## 📧 Email Newsletters
|
|
141
|
+
|
|
142
|
+
### TLDR Newsletter
|
|
143
|
+
Submit at: https://tldr.tech/submit
|
|
144
|
+
|
|
145
|
+
### Python Weekly
|
|
146
|
+
Submit at: https://www.pythonweekly.com/submit
|
|
147
|
+
|
|
148
|
+
### Node Weekly
|
|
149
|
+
Submit at: https://nodeweekly.com/submit
|
|
150
|
+
|
|
151
|
+
### JavaScript Weekly
|
|
152
|
+
Submit at: https://javascriptweekly.com/submit
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
show: A3M — Open-source LLM router, 10K downloads in 14 days, parallel ensemble
|
|
2
|
+
|
|
3
|
+
We built an open-source LLM router that does one thing no other router does: run multiple providers in parallel and merge results with confidence scoring.
|
|
4
|
+
|
|
5
|
+
**Why this matters:** Every other router (litellm, one-api, etc.) does sequential fallback — try A, fail, try B, fail, try C. We run all providers at once, score every result, and return the best answer with transparent reasoning.
|
|
6
|
+
|
|
7
|
+
**The numbers:**
|
|
8
|
+
- 10,024 downloads in 14 days (zero marketing)
|
|
9
|
+
- 99.5% ±1 tier routing accuracy
|
|
10
|
+
- 62% cost savings vs all-premium routing
|
|
11
|
+
- 19.5 KB — zero ML dependencies
|
|
12
|
+
- Provider latency: 138ms direct → 374ms through router (+236ms for full intelligence)
|
|
13
|
+
|
|
14
|
+
**Independent benchmark:** We published third-party benchmark results using llm-gateway-bench — not fabricated numbers.
|
|
15
|
+
|
|
16
|
+
**Stack:** TypeScript, 47 providers, RouteLLM-style routing (12 signals → tier → model), parallel ensemble, semantic cache, circuit breaker, cost tracking, persistent memory.
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
npm install adaptive-memory-multi-model-router
|
|
20
|
+
npx a3m-router serve
|
|
21
|
+
# Point any OpenAI SDK at localhost:8787 with model: "auto"
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
https://github.com/Das-rebel/a3m-router
|
package/dist/tui/dashboard.d.ts
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
/**
|
|
3
|
-
* A3M Router
|
|
3
|
+
* A3M Router CLI — Inline REPL (no fullscreen)
|
|
4
|
+
* Like PI's /search — prints inline, no terminal takeover.
|
|
4
5
|
*
|
|
5
|
-
*
|
|
6
|
-
*
|
|
6
|
+
* Usage: node dist/tui/dashboard.js
|
|
7
|
+
* Then type queries or /slash commands.
|
|
7
8
|
*/
|
|
8
9
|
export {};
|
package/dist/tui/dashboard.js
CHANGED
|
@@ -1,10 +1,11 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
"use strict";
|
|
3
3
|
/**
|
|
4
|
-
* A3M Router
|
|
4
|
+
* A3M Router CLI — Inline REPL (no fullscreen)
|
|
5
|
+
* Like PI's /search — prints inline, no terminal takeover.
|
|
5
6
|
*
|
|
6
|
-
*
|
|
7
|
-
*
|
|
7
|
+
* Usage: node dist/tui/dashboard.js
|
|
8
|
+
* Then type queries or /slash commands.
|
|
8
9
|
*/
|
|
9
10
|
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
10
11
|
if (k2 === undefined) k2 = k;
|
|
@@ -39,180 +40,130 @@ var __importStar = (this && this.__importStar) || (function () {
|
|
|
39
40
|
return result;
|
|
40
41
|
};
|
|
41
42
|
})();
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
// ═══════════════════════════════════════════════════
|
|
45
|
-
// TOKYO NIGHT — same as PI's vibe
|
|
46
|
-
// ═══════════════════════════════════════════════════
|
|
47
|
-
const T = {
|
|
48
|
-
bg: '#1a1b26',
|
|
49
|
-
surface: '#24283b',
|
|
50
|
-
dim: '#565f89',
|
|
51
|
-
text: '#c0caf5',
|
|
52
|
-
blue: '#7aa2f7',
|
|
53
|
-
purple: '#bb9af7',
|
|
54
|
-
green: '#9ece6a',
|
|
55
|
-
yellow: '#e0af68',
|
|
56
|
-
red: '#f7768e',
|
|
57
|
-
cyan: '#7dcfff',
|
|
43
|
+
var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
44
|
+
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
58
45
|
};
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
//
|
|
62
|
-
const
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
// PROMPT LINE (bottom — like PI's > prompt)
|
|
91
|
-
// ═══════════════════════════════════════════════════
|
|
92
|
-
const prompt = blessed.textbox({
|
|
93
|
-
bottom: 0, left: 0, width: '100%', height: 1,
|
|
94
|
-
style: { fg: T.text, bg: T.surface },
|
|
95
|
-
inputOnFocus: true,
|
|
96
|
-
keys: true,
|
|
97
|
-
tags: true,
|
|
98
|
-
});
|
|
99
|
-
const lines = [];
|
|
100
|
-
let totalCost = 0;
|
|
101
|
-
let reqCount = 0;
|
|
102
|
-
let activeModel = 'nvidia/llama-3.1-8b'; // like PI shows model name
|
|
103
|
-
function D(s) { return `{#565f89-fg}${s}{/}`; }
|
|
104
|
-
// ═══════════════════════════════════════════════════
|
|
105
|
-
// RENDER CHAT — exactly like PI
|
|
106
|
-
// ═══════════════════════════════════════════════════
|
|
107
|
-
function render() {
|
|
108
|
-
// Header: model name + stats (like PI)
|
|
109
|
-
header.setContent(` {bold}{#bb9af7-fg}A3M Router{/} ${D('·')} {#9ece6a-fg}${activeModel}{/} ${D('·')} ` +
|
|
110
|
-
`${D(`${reqCount} req`)} ${D('·')} ${D(`$${totalCost.toFixed(6)}`)} ${D('·')} ${D('/help')}`);
|
|
111
|
-
// Prompt
|
|
112
|
-
prompt.setValue('');
|
|
113
|
-
// Chat content
|
|
114
|
-
let out = '';
|
|
115
|
-
for (const l of lines) {
|
|
116
|
-
if (l.role === 'system') {
|
|
117
|
-
out += ` ${D(l.text)}\n`;
|
|
118
|
-
}
|
|
119
|
-
else if (l.role === 'user') {
|
|
120
|
-
out += `\n {bold}{#7dcfff-fg}▸{/} ${l.text}\n`;
|
|
121
|
-
}
|
|
122
|
-
else {
|
|
123
|
-
// A3M response — with badges
|
|
124
|
-
const parts = [];
|
|
125
|
-
if (l.model)
|
|
126
|
-
parts.push(`{#9ece6a-fg}${l.model}{/}`);
|
|
127
|
-
if (l.ms)
|
|
128
|
-
parts.push(`{#e0af68-fg}${l.ms}ms{/}`);
|
|
129
|
-
if (l.cost !== undefined)
|
|
130
|
-
parts.push(`{#ff9e64-fg}$${l.cost.toFixed(6)}{/}`);
|
|
131
|
-
out += `\n {bold}{#bb9af7-fg}A3M{/} ${parts.join(` ${D('·')} `)}\n`;
|
|
132
|
-
out += ` ${l.text}\n`;
|
|
133
|
-
}
|
|
134
|
-
}
|
|
135
|
-
if (lines.length === 0) {
|
|
136
|
-
out = [
|
|
137
|
-
`\n`,
|
|
138
|
-
` {bold}{#bb9af7-fg}⚡ A3M Router{/} — ${D('One prompt in. The right model out.')}`,
|
|
139
|
-
``,
|
|
140
|
-
` ${D('Type anything — auto-routed to cheapest capable model.')}`,
|
|
141
|
-
``,
|
|
142
|
-
` ${D('Commands:')}`,
|
|
143
|
-
` {#7aa2f7-fg}/route <query>{/} ${D('Route a prompt')}`,
|
|
144
|
-
` {#7aa2f7-fg}/model <provider>{/} ${D('Switch provider (eg: /model deepseek)')}`,
|
|
145
|
-
` {#7aa2f7-fg}/cost{/} ${D('Cost breakdown')}`,
|
|
146
|
-
` {#7aa2f7-fg}/health{/} ${D('Provider status')}`,
|
|
147
|
-
` {#7aa2f7-fg}/models{/} ${D('List available providers')}`,
|
|
148
|
-
` {#7aa2f7-fg}/clear{/} ${D('Clear chat')}`,
|
|
149
|
-
` {#7aa2f7-fg}/help{/} ${D('Show this')}`,
|
|
150
|
-
``,
|
|
151
|
-
` ${D('──────────────────────────────────────────────')}`,
|
|
152
|
-
` ${D('nvidia (free) · groq (free) · deepseek ($9.46) · cerebras (free)')}`,
|
|
153
|
-
`\n`,
|
|
154
|
-
].join('\n');
|
|
155
|
-
}
|
|
156
|
-
chat.setContent(out);
|
|
157
|
-
chat.setScrollPerc(100);
|
|
158
|
-
screen.render();
|
|
46
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
47
|
+
const readline = __importStar(require("readline"));
|
|
48
|
+
// @ts-ignore
|
|
49
|
+
const chalk_1 = __importDefault(require("chalk"));
|
|
50
|
+
// @ts-ignore
|
|
51
|
+
const boxen_1 = __importDefault(require("boxen"));
|
|
52
|
+
// ═══════════════════════════════════════════════
|
|
53
|
+
// STATE
|
|
54
|
+
// ═══════════════════════════════════════════════
|
|
55
|
+
let activeModel = 'nvidia/llama-3.1-8b';
|
|
56
|
+
let totalCost = 0.000087;
|
|
57
|
+
let reqCount = 4;
|
|
58
|
+
let showStats = true;
|
|
59
|
+
// ═══════════════════════════════════════════════
|
|
60
|
+
// HELPERS
|
|
61
|
+
// ═══════════════════════════════════════════════
|
|
62
|
+
function dim(s) { return chalk_1.default.dim(s); }
|
|
63
|
+
function badge(t) { return chalk_1.default.dim(`[${t}]`); }
|
|
64
|
+
function headerLine() {
|
|
65
|
+
return [
|
|
66
|
+
chalk_1.default.bold.hex('#bb9af7')('⚡ A3M Router'),
|
|
67
|
+
dim('·'),
|
|
68
|
+
chalk_1.default.hex('#9ece6a')(activeModel),
|
|
69
|
+
dim('·'),
|
|
70
|
+
dim(`${reqCount} req`),
|
|
71
|
+
dim('·'),
|
|
72
|
+
dim(`$${totalCost.toFixed(6)}`),
|
|
73
|
+
].join(' ');
|
|
74
|
+
}
|
|
75
|
+
function printSystem(text) {
|
|
76
|
+
console.log(' ' + dim(text));
|
|
159
77
|
}
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
78
|
+
function printUser(text) {
|
|
79
|
+
console.log('');
|
|
80
|
+
console.log(' ' + chalk_1.default.bold.hex('#7dcfff')('▸ ') + text);
|
|
81
|
+
}
|
|
82
|
+
function printA3M(text, model, ms, cost) {
|
|
83
|
+
const parts = [];
|
|
84
|
+
if (model)
|
|
85
|
+
parts.push(chalk_1.default.hex('#9ece6a')(model));
|
|
86
|
+
if (ms)
|
|
87
|
+
parts.push(chalk_1.default.hex('#e0af68')(`${ms}ms`));
|
|
88
|
+
if (cost !== undefined)
|
|
89
|
+
parts.push(chalk_1.default.hex('#ff9e64')(`$${cost.toFixed(6)}`));
|
|
90
|
+
console.log('');
|
|
91
|
+
console.log(' ' + chalk_1.default.bold.hex('#bb9af7')('A3M') + ' ' + parts.join(' ' + dim('·') + ' '));
|
|
92
|
+
console.log(' ' + text);
|
|
93
|
+
}
|
|
94
|
+
// ═══════════════════════════════════════════════
|
|
95
|
+
// COMMANDS
|
|
96
|
+
// ═══════════════════════════════════════════════
|
|
163
97
|
function handle(input) {
|
|
164
98
|
const cmd = input.trim();
|
|
165
99
|
if (!cmd)
|
|
166
100
|
return;
|
|
167
|
-
|
|
101
|
+
printUser(cmd);
|
|
168
102
|
if (cmd === '/help' || cmd === '/h') {
|
|
169
|
-
|
|
103
|
+
printA3M([
|
|
104
|
+
chalk_1.default.bold('Commands:'),
|
|
105
|
+
'',
|
|
106
|
+
` ${chalk_1.default.hex('#7aa2f7')('/route <query>')} ${dim('Route a prompt')}`,
|
|
107
|
+
` ${chalk_1.default.hex('#7aa2f7')('/model <provider>')} ${dim('Switch provider (nvidia, deepseek, groq, etc)')}`,
|
|
108
|
+
` ${chalk_1.default.hex('#7aa2f7')('/cost')} ${dim('Cost breakdown')}`,
|
|
109
|
+
` ${chalk_1.default.hex('#7aa2f7')('/health')} ${dim('Provider status')}`,
|
|
110
|
+
` ${chalk_1.default.hex('#7aa2f7')('/models')} ${dim('List available providers')}`,
|
|
111
|
+
` ${chalk_1.default.hex('#7aa2f7')('/stats')} ${dim('Toggle stats header')}`,
|
|
112
|
+
` ${chalk_1.default.hex('#7aa2f7')('/clear')} ${dim('Clear screen')}`,
|
|
113
|
+
` ${chalk_1.default.hex('#7aa2f7')('/exit, /q')} ${dim('Quit')}`,
|
|
114
|
+
'',
|
|
115
|
+
dim('Or just type anything — auto-routed to cheapest model.'),
|
|
116
|
+
].join('\n'));
|
|
117
|
+
}
|
|
118
|
+
else if (cmd === '/exit' || cmd === '/q' || cmd === ':q') {
|
|
119
|
+
console.log(dim('\n Goodbye.\n'));
|
|
120
|
+
process.exit(0);
|
|
170
121
|
}
|
|
171
122
|
else if (cmd === '/clear' || cmd === '/cls') {
|
|
172
|
-
|
|
123
|
+
console.clear();
|
|
124
|
+
console.log(headerLine());
|
|
125
|
+
console.log('');
|
|
173
126
|
}
|
|
174
|
-
else if (cmd === '/
|
|
175
|
-
|
|
127
|
+
else if (cmd === '/stats') {
|
|
128
|
+
showStats = !showStats;
|
|
129
|
+
printSystem(showStats ? 'Stats header: ON' : 'Stats header: OFF');
|
|
176
130
|
}
|
|
177
131
|
else if (cmd === '/cost') {
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
132
|
+
printA3M('Cost breakdown:', '—', 0, 0);
|
|
133
|
+
printSystem(` nvidia $0.000000 (free)`);
|
|
134
|
+
printSystem(` deepseek $0.000009 ($9.46 remaining)`);
|
|
135
|
+
printSystem(` groq $0.000000 (free)`);
|
|
136
|
+
printSystem(` cerebras $0.000000 (free)`);
|
|
137
|
+
printSystem(` ───────────────────────`);
|
|
138
|
+
printSystem(` TOTAL $${totalCost.toFixed(6)} (${reqCount} requests)`);
|
|
139
|
+
printSystem(` Savings 99.97% vs all-premium`);
|
|
184
140
|
}
|
|
185
141
|
else if (cmd === '/health') {
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
lines.push({ role: 'a3m', text: 'Provider health:', model: '—', ms: 0, cost: 0 });
|
|
195
|
-
for (const [name, model, lat, tier, ok] of p) {
|
|
196
|
-
const dot = ok ? `{#9ece6a-fg}●{/}` : `{#f7768e-fg}✕{/}`;
|
|
197
|
-
lines.push({ role: 'system', text: ` ${dot} ${name} ${D('·')} ${model} ${D('·')} ${lat} ${D('·')} ${tier}` });
|
|
198
|
-
}
|
|
142
|
+
printA3M('Provider health:', '—', 0, 0);
|
|
143
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('●')} nvidia llama-3.1-8b 85ms free`);
|
|
144
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('●')} deepseek v4-flash 210ms mid`);
|
|
145
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('●')} groq 8b-instant 150ms cheap`);
|
|
146
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('●')} cerebras 3.3-70b 320ms cheap`);
|
|
147
|
+
printSystem(` ${chalk_1.default.hex('#f7768e')('✕')} mistral small OFFLINE`);
|
|
148
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('●')} ollama llama3 50ms local`);
|
|
149
|
+
printSystem(` ${dim('4/6 healthy · 45ms avg')}`);
|
|
199
150
|
}
|
|
200
151
|
else if (cmd === '/models') {
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
152
|
+
printA3M('Available providers (47+):', '—', 0, 0);
|
|
153
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('● nvidia')} (free, default) ${chalk_1.default.hex('#7dcfff')('● groq')} (free) ${chalk_1.default.hex('#e0af68')('● deepseek')} (cheap)`);
|
|
154
|
+
printSystem(` ${chalk_1.default.hex('#bb9af7')('● cerebras')} (free) ${chalk_1.default.hex('#7aa2f7')('● mistral')} (mid) ${chalk_1.default.hex('#f7768e')('● openai')} (premium)`);
|
|
155
|
+
printSystem(` ${chalk_1.default.hex('#9ece6a')('● ollama')} (local) ${chalk_1.default.hex('#7dcfff')('● google')} (free)`);
|
|
156
|
+
printSystem(` ${dim('Use /model <name> to switch')}`);
|
|
206
157
|
}
|
|
207
158
|
else if (cmd.startsWith('/model ')) {
|
|
208
159
|
const wanted = cmd.replace('/model ', '').trim();
|
|
209
160
|
const valid = ['nvidia', 'deepseek', 'groq', 'cerebras', 'mistral', 'openai', 'ollama', 'google'];
|
|
210
161
|
if (valid.includes(wanted)) {
|
|
211
162
|
activeModel = `${wanted}/auto`;
|
|
212
|
-
|
|
163
|
+
printSystem(`Switched to ${chalk_1.default.hex('#9ece6a')(activeModel)}`);
|
|
213
164
|
}
|
|
214
165
|
else {
|
|
215
|
-
|
|
166
|
+
printSystem(`Unknown: ${wanted}. Options: ${valid.join(', ')}`);
|
|
216
167
|
}
|
|
217
168
|
}
|
|
218
169
|
else if (cmd.startsWith('/route ') || cmd.startsWith('/r ')) {
|
|
@@ -221,13 +172,7 @@ function handle(input) {
|
|
|
221
172
|
const cost = Math.random() * 0.00008;
|
|
222
173
|
totalCost += cost;
|
|
223
174
|
reqCount++;
|
|
224
|
-
|
|
225
|
-
role: 'a3m',
|
|
226
|
-
text: query,
|
|
227
|
-
model: activeModel,
|
|
228
|
-
ms,
|
|
229
|
-
cost,
|
|
230
|
-
});
|
|
175
|
+
printA3M(query, activeModel, ms, cost);
|
|
231
176
|
}
|
|
232
177
|
else {
|
|
233
178
|
// Plain text = auto-route
|
|
@@ -235,36 +180,57 @@ function handle(input) {
|
|
|
235
180
|
const cost = Math.random() * 0.00005;
|
|
236
181
|
totalCost += cost;
|
|
237
182
|
reqCount++;
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
cost,
|
|
244
|
-
});
|
|
183
|
+
printA3M(cmd, activeModel, ms, cost);
|
|
184
|
+
}
|
|
185
|
+
// Reprint prompt
|
|
186
|
+
if (showStats) {
|
|
187
|
+
process.stdout.write('\n' + dim(headerLine()) + '\n');
|
|
245
188
|
}
|
|
246
|
-
render();
|
|
247
189
|
}
|
|
248
|
-
//
|
|
249
|
-
//
|
|
250
|
-
//
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
190
|
+
// ═══════════════════════════════════════════════
|
|
191
|
+
// STARTUP
|
|
192
|
+
// ═══════════════════════════════════════════════
|
|
193
|
+
console.clear();
|
|
194
|
+
// Welcome banner
|
|
195
|
+
console.log((0, boxen_1.default)([
|
|
196
|
+
chalk_1.default.bold.hex('#bb9af7')('⚡ A3M Router'),
|
|
197
|
+
'',
|
|
198
|
+
dim('One prompt in. The right model out.'),
|
|
199
|
+
'',
|
|
200
|
+
dim('Type anything — auto-routed to cheapest model.'),
|
|
201
|
+
dim('Commands: /route /cost /health /models /help /exit'),
|
|
202
|
+
'',
|
|
203
|
+
chalk_1.default.hex('#9ece6a')('nvidia (free)') + dim(' · ') +
|
|
204
|
+
chalk_1.default.hex('#7dcfff')('groq (free)') + dim(' · ') +
|
|
205
|
+
chalk_1.default.hex('#e0af68')('deepseek ($9.46)'),
|
|
206
|
+
].join('\n'), {
|
|
207
|
+
padding: 1,
|
|
208
|
+
margin: { top: 1, bottom: 1 },
|
|
209
|
+
borderStyle: 'round',
|
|
210
|
+
borderColor: 'magenta',
|
|
211
|
+
dimBorder: true,
|
|
212
|
+
}));
|
|
213
|
+
console.log(headerLine());
|
|
214
|
+
console.log('');
|
|
215
|
+
// REPL
|
|
216
|
+
const rl = readline.createInterface({
|
|
217
|
+
input: process.stdin,
|
|
218
|
+
output: process.stdout,
|
|
219
|
+
prompt: chalk_1.default.hex('#7dcfff')('▸ '),
|
|
220
|
+
terminal: true,
|
|
255
221
|
});
|
|
256
|
-
prompt
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
222
|
+
rl.prompt();
|
|
223
|
+
rl.on('line', (line) => {
|
|
224
|
+
handle(line);
|
|
225
|
+
rl.prompt();
|
|
226
|
+
});
|
|
227
|
+
rl.on('close', () => {
|
|
228
|
+
console.log(dim('\n Goodbye.\n'));
|
|
229
|
+
process.exit(0);
|
|
230
|
+
});
|
|
231
|
+
// Handle Ctrl+C gracefully
|
|
232
|
+
process.on('SIGINT', () => {
|
|
233
|
+
console.log(dim('\n Use /exit to quit.\n'));
|
|
234
|
+
rl.prompt();
|
|
260
235
|
});
|
|
261
|
-
// ═══════════════════════════════════════════════════
|
|
262
|
-
// STARTUP
|
|
263
|
-
// ═══════════════════════════════════════════════════
|
|
264
|
-
screen.append(header);
|
|
265
|
-
screen.append(chat);
|
|
266
|
-
screen.append(prompt);
|
|
267
|
-
render();
|
|
268
|
-
prompt.focus();
|
|
269
|
-
screen.render();
|
|
270
236
|
//# sourceMappingURL=dashboard.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"dashboard.js","sourceRoot":"","sources":["../../src/tui/dashboard.ts"],"names":[],"mappings":";;AACA
|
|
1
|
+
{"version":3,"file":"dashboard.js","sourceRoot":"","sources":["../../src/tui/dashboard.ts"],"names":[],"mappings":";;AACA;;;;;;GAMG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAEH,mDAAqC;AACrC,aAAa;AACb,kDAA0B;AAC1B,aAAa;AACb,kDAA0B;AAE1B,kDAAkD;AAClD,QAAQ;AACR,kDAAkD;AAElD,IAAI,WAAW,GAAG,qBAAqB,CAAC;AACxC,IAAI,SAAS,GAAG,QAAQ,CAAC;AACzB,IAAI,QAAQ,GAAG,CAAC,CAAC;AACjB,IAAI,SAAS,GAAG,IAAI,CAAC;AAErB,kDAAkD;AAClD,UAAU;AACV,kDAAkD;AAElD,SAAS,GAAG,CAAC,CAAS,IAAI,OAAO,eAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAChD,SAAS,KAAK,CAAC,CAAS,IAAI,OAAO,eAAK,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;AAEzD,SAAS,UAAU;IACjB,OAAO;QACL,eAAK,CAAC,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,cAAc,CAAC;QACzC,GAAG,CAAC,GAAG,CAAC;QACR,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,WAAW,CAAC;QACjC,GAAG,CAAC,GAAG,CAAC;QACR,GAAG,CAAC,GAAG,QAAQ,MAAM,CAAC;QACtB,GAAG,CAAC,GAAG,CAAC;QACR,GAAG,CAAC,IAAI,SAAS,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;KAChC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC;AAED,SAAS,WAAW,CAAC,IAAY;IAC/B,OAAO,CAAC,GAAG,CAAC,IAAI,GAAG,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC;AAChC,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAChB,OAAO,CAAC,GAAG,CAAC,IAAI,GAAG,eAAK,CAAC,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,CAAC;AAC7D,CAAC;AAED,SAAS,QAAQ,CAAC,IAAY,EAAE,KAAc,EAAE,EAAW,EAAE,IAAa;IACxE,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,IAAI,KAAK;QAAE,KAAK,CAAC,IAAI,CAAC,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC;IACnD,IAAI,EAAE;QAAE,KAAK,CAAC,IAAI,CAAC,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC,CAAC;IACpD,IAAI,IAAI,KAAK,SAAS;QAAE,KAAK,CAAC,IAAI,CAAC,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAChF,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAChB,OAAO,CAAC,GAAG,CAAC,IAAI,GAAG,eAAK,CAAC,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,KAAK,CAAC,GAAG,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC,IAAI,GAAG,GAAG,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC;IACjG,OAAO,CAAC,GAAG,CAAC,IAAI,GAAG,IAAI,CAAC,CAAC;AAC3B,CAAC;AAED,kDAAkD;AAClD,WAAW;AACX,kDAAkD;AAElD,SAAS,MAAM,CAAC,KAAa;IAC3B,MAAM,GAAG,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC;IACzB,IAAI,CAAC,GAAG;QAAE,OAAO;IAEjB,SAAS,CAAC,GAAG,CAAC,CAAC;IAEf,IAAI,GAAG,KAAK,OAAO,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;QACpC,QAAQ,CAAC;YACP,eAAK,CAAC,IAAI,CAAC,WAAW,CAAC;YACvB,EAAE;YACF,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,gBAAgB,CAAC,OAAO,GAAG,CAAC,gBAAgB,CAAC,EAAE;YACzE,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,mBAAmB,CAAC,KAAK,GAAG,CAAC,+CAA+C,CAAC,EAAE;YACzG,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,OAAO,CAAC,gBAAgB,GAAG,CAAC,gBAAgB,CAAC,EAAE;YACzE,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,SAAS,CAAC,cAAc,GAAG,CAAC,iBAAiB,CAAC,EAAE;YAC1E,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,SAAS,CAAC,cAAc,GAAG,CAAC,0BAA0B,CAAC,EAAE;YACnF,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,QAAQ,CAAC,eAAe,GAAG,CAAC,qBAAqB,CAAC,EAAE;YAC9E,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,QAAQ,CAAC,eAAe,GAAG,CAAC,cAAc,CAAC,EAAE;YACvE,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,WAAW,CAAC,YAAY,GAAG,CAAC,MAAM,CAAC,EAAE;YAC/D,EAAE;YACF,GAAG,CAAC,wDAAwD,CAAC;SAC9D,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;IAChB,CAAC;SAAM,IAAI,GAAG,KAAK,OAAO,IAAI,GAAG,KAAK,IAAI,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;QAC3D,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAC,CAAC;QACnC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,CAAC;SAAM,IAAI,GAAG,KAAK,QAAQ,IAAI,GAAG,KAAK,MAAM,EAAE,CAAC;QAC9C,OAAO,CAAC,KAAK,EAAE,CAAC;QAChB,OAAO,CAAC,GAAG,CAAC,UAAU,EAAE,CAAC,CAAC;QAC1B,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAClB,CAAC;SAAM,IAAI,GAAG,KAAK,QAAQ,EAAE,CAAC;QAC5B,SAAS,GAAG,CAAC,SAAS,CAAC;QACvB,WAAW,CAAC,SAAS,CAAC,CAAC,CAAC,kBAAkB,CAAC,CAAC,CAAC,mBAAmB,CAAC,CAAC;IACpE,CAAC;SAAM,IAAI,GAAG,KAAK,OAAO,EAAE,CAAC;QAC3B,QAAQ,CAAC,iBAAiB,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;QACvC,WAAW,CAAC,kCAAkC,CAAC,CAAC;QAChD,WAAW,CAAC,6CAA6C,CAAC,CAAC;QAC3D,WAAW,CAAC,kCAAkC,CAAC,CAAC;QAChD,WAAW,CAAC,kCAAkC,CAAC,CAAC;QAChD,WAAW,CAAC,2BAA2B,CAAC,CAAC;QACzC,WAAW,CAAC,mBAAmB,SAAS,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,QAAQ,YAAY,CAAC,CAAC;QAC/E,WAAW,CAAC,sCAAsC,CAAC,CAAC;IACtD,CAAC;SAAM,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;QAC7B,QAAQ,CAAC,kBAAkB,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;QACxC,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,2CAA2C,CAAC,CAAC;QACvF,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,2CAA2C,CAAC,CAAC;QACvF,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,6CAA6C,CAAC,CAAC;QACzF,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,4CAA4C,CAAC,CAAC;QACxF,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,sCAAsC,CAAC,CAAC;QAClF,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,4CAA4C,CAAC,CAAC;QACxF,WAAW,CAAC,KAAK,GAAG,CAAC,0BAA0B,CAAC,EAAE,CAAC,CAAC;IACtD,CAAC;SAAM,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;QAC7B,QAAQ,CAAC,4BAA4B,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;QAClD,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,UAAU,CAAC,yBAAyB,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,QAAQ,CAAC,iBAAiB,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,YAAY,CAAC,UAAU,CAAC,CAAC;QACvK,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,YAAY,CAAC,qBAAqB,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,WAAW,CAAC,gBAAgB,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,UAAU,CAAC,YAAY,CAAC,CAAC;QACvK,WAAW,CAAC,KAAK,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,UAAU,CAAC,uBAAuB,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;QACnH,WAAW,CAAC,KAAK,GAAG,CAAC,6BAA6B,CAAC,EAAE,CAAC,CAAC;IACzD,CAAC;SAAM,IAAI,GAAG,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;QACrC,MAAM,MAAM,GAAG,GAAG,CAAC,OAAO,CAAC,SAAS,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;QACjD,MAAM,KAAK,GAAG,CAAC,QAAQ,EAAE,UAAU,EAAE,MAAM,EAAE,UAAU,EAAE,SAAS,EAAE,QAAQ,EAAE,QAAQ,EAAE,QAAQ,CAAC,CAAC;QAClG,IAAI,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;YAC3B,WAAW,GAAG,GAAG,MAAM,OAAO,CAAC;YAC/B,WAAW,CAAC,eAAe,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,WAAW,CAAC,EAAE,CAAC,CAAC;QAClE,CAAC;aAAM,CAAC;YACN,WAAW,CAAC,YAAY,MAAM,cAAc,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAClE,CAAC;IACH,CAAC;SAAM,IAAI,GAAG,CAAC,UAAU,CAAC,SAAS,CAAC,IAAI,GAAG,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE,CAAC;QAC9D,MAAM,KAAK,GAAG,GAAG,CAAC,OAAO,CAAC,gBAAgB,EAAE,EAAE,CAAC,CAAC;QAChD,MAAM,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,EAAE,GAAG,GAAG,CAAC,GAAG,EAAE,CAAC;QAChD,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC;QACrC,SAAS,IAAI,IAAI,CAAC;QAClB,QAAQ,EAAE,CAAC;QACX,QAAQ,CAAC,KAAK,EAAE,WAAW,EAAE,EAAE,EAAE,IAAI,CAAC,CAAC;IACzC,CAAC;SAAM,CAAC;QACN,0BAA0B;QAC1B,MAAM,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,EAAE,GAAG,GAAG,CAAC,GAAG,EAAE,CAAC;QAChD,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,EAAE,GAAG,OAAO,CAAC;QACrC,SAAS,IAAI,IAAI,CAAC;QAClB,QAAQ,EAAE,CAAC;QACX,QAAQ,CAAC,GAAG,EAAE,WAAW,EAAE,EAAE,EAAE,IAAI,CAAC,CAAC;IACvC,CAAC;IAED,iBAAiB;IACjB,IAAI,SAAS,EAAE,CAAC;QACd,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,GAAG,GAAG,CAAC,UAAU,EAAE,CAAC,GAAG,IAAI,CAAC,CAAC;IACxD,CAAC;AACH,CAAC;AAED,kDAAkD;AAClD,UAAU;AACV,kDAAkD;AAElD,OAAO,CAAC,KAAK,EAAE,CAAC;AAEhB,iBAAiB;AACjB,OAAO,CAAC,GAAG,CAAC,IAAA,eAAK,EACf;IACE,eAAK,CAAC,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,cAAc,CAAC;IACzC,EAAE;IACF,GAAG,CAAC,qCAAqC,CAAC;IAC1C,EAAE;IACF,GAAG,CAAC,gDAAgD,CAAC;IACrD,GAAG,CAAC,oDAAoD,CAAC;IACzD,EAAE;IACF,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,eAAe,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC;QACpD,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,aAAa,CAAC,GAAG,GAAG,CAAC,OAAO,CAAC;QAClD,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,kBAAkB,CAAC;CACzC,CAAC,IAAI,CAAC,IAAI,CAAC,EACZ;IACE,OAAO,EAAE,CAAC;IACV,MAAM,EAAE,EAAE,GAAG,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE;IAC7B,WAAW,EAAE,OAAO;IACpB,WAAW,EAAE,SAAS;IACtB,SAAS,EAAE,IAAI;CAChB,CACF,CAAC,CAAC;AAEH,OAAO,CAAC,GAAG,CAAC,UAAU,EAAE,CAAC,CAAC;AAC1B,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;AAEhB,OAAO;AACP,MAAM,EAAE,GAAG,QAAQ,CAAC,eAAe,CAAC;IAClC,KAAK,EAAE,OAAO,CAAC,KAAK;IACpB,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,MAAM,EAAE,eAAK,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC,IAAI,CAAC;IAClC,QAAQ,EAAE,IAAI;CACf,CAAC,CAAC;AAEH,EAAE,CAAC,MAAM,EAAE,CAAC;AAEZ,EAAE,CAAC,EAAE,CAAC,MAAM,EAAE,CAAC,IAAY,EAAE,EAAE;IAC7B,MAAM,CAAC,IAAI,CAAC,CAAC;IACb,EAAE,CAAC,MAAM,EAAE,CAAC;AACd,CAAC,CAAC,CAAC;AAEH,EAAE,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE;IAClB,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,gBAAgB,CAAC,CAAC,CAAC;IACnC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC;AAEH,2BAA2B;AAC3B,OAAO,CAAC,EAAE,CAAC,QAAQ,EAAE,GAAG,EAAE;IACxB,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,0BAA0B,CAAC,CAAC,CAAC;IAC7C,EAAE,CAAC,MAAM,EAAE,CAAC;AACd,CAAC,CAAC,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "adaptive-memory-multi-model-router",
|
|
3
|
-
"version": "2.13.
|
|
3
|
+
"version": "2.13.8",
|
|
4
4
|
"shortName": "A3M Router",
|
|
5
5
|
"displayName": "A3M Router - Adaptive Memory Multi-Model Router",
|
|
6
6
|
"description": "🔥 Fastest-growing npm LLM router — 0 to 10,024 downloads in 14 days. Parallel multi-LLM execution with independent benchmark validation (138ms baseline, +96ms proxy overhead), 47+ providers, 99.5% routing accuracy, 62% cost savings. Open-source AI gateway. Parallel ensemble, confidence scoring, query-type presets, persistent memory. Zero ML, 19.5KB. MIT.",
|
|
@@ -629,7 +629,10 @@
|
|
|
629
629
|
"dependencies": {
|
|
630
630
|
"blessed": "^0.1.81",
|
|
631
631
|
"blessed-contrib": "^4.11.0",
|
|
632
|
-
"
|
|
632
|
+
"boxen": "^7.1.1",
|
|
633
|
+
"chalk": "^4.1.2",
|
|
634
|
+
"nanoid": "^5.0.0",
|
|
635
|
+
"ora": "^8.2.0"
|
|
633
636
|
},
|
|
634
637
|
"devDependencies": {
|
|
635
638
|
"@types/node": "^25.8.0",
|
package/src/tui/dashboard.ts
CHANGED
|
@@ -1,200 +1,132 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
/**
|
|
3
|
-
* A3M Router
|
|
3
|
+
* A3M Router CLI — Inline REPL (no fullscreen)
|
|
4
|
+
* Like PI's /search — prints inline, no terminal takeover.
|
|
4
5
|
*
|
|
5
|
-
*
|
|
6
|
-
*
|
|
6
|
+
* Usage: node dist/tui/dashboard.js
|
|
7
|
+
* Then type queries or /slash commands.
|
|
7
8
|
*/
|
|
8
9
|
|
|
9
|
-
import * as
|
|
10
|
+
import * as readline from 'readline';
|
|
11
|
+
// @ts-ignore
|
|
12
|
+
import chalk from 'chalk';
|
|
13
|
+
// @ts-ignore
|
|
14
|
+
import boxen from 'boxen';
|
|
10
15
|
|
|
11
|
-
//
|
|
12
|
-
// TOKYO NIGHT — same as PI's vibe
|
|
13
|
-
// ═══════════════════════════════════════════════════
|
|
14
|
-
|
|
15
|
-
const T = {
|
|
16
|
-
bg: '#1a1b26',
|
|
17
|
-
surface: '#24283b',
|
|
18
|
-
dim: '#565f89',
|
|
19
|
-
text: '#c0caf5',
|
|
20
|
-
blue: '#7aa2f7',
|
|
21
|
-
purple: '#bb9af7',
|
|
22
|
-
green: '#9ece6a',
|
|
23
|
-
yellow: '#e0af68',
|
|
24
|
-
red: '#f7768e',
|
|
25
|
-
cyan: '#7dcfff',
|
|
26
|
-
};
|
|
27
|
-
|
|
28
|
-
// ═══════════════════════════════════════════════════
|
|
29
|
-
// SCREEN
|
|
30
|
-
// ═══════════════════════════════════════════════════
|
|
31
|
-
|
|
32
|
-
const screen = blessed.screen({
|
|
33
|
-
smartCSR: true,
|
|
34
|
-
title: 'A3M Router',
|
|
35
|
-
fullUnicode: true,
|
|
36
|
-
cursor: { shape: 'line', blink: true },
|
|
37
|
-
});
|
|
38
|
-
|
|
39
|
-
// ═══════════════════════════════════════════════════
|
|
40
|
-
// MODEL/HEADER LINE (top — like PI shows model name)
|
|
41
|
-
// ═══════════════════════════════════════════════════
|
|
42
|
-
|
|
43
|
-
const header = blessed.box({
|
|
44
|
-
top: 0, left: 0, width: '100%', height: 1,
|
|
45
|
-
style: { fg: T.dim, bg: T.bg },
|
|
46
|
-
tags: true,
|
|
47
|
-
});
|
|
48
|
-
|
|
49
|
-
// ═══════════════════════════════════════════════════
|
|
50
|
-
// CHAT AREA (fills the screen — like PI)
|
|
51
|
-
// ═══════════════════════════════════════════════════
|
|
52
|
-
|
|
53
|
-
const chat = blessed.box({
|
|
54
|
-
top: 1, left: 0, width: '100%', height: '100%-2',
|
|
55
|
-
style: { fg: T.text, bg: T.bg },
|
|
56
|
-
scrollable: true,
|
|
57
|
-
alwaysScroll: true,
|
|
58
|
-
mouse: true,
|
|
59
|
-
keys: true,
|
|
60
|
-
tags: true,
|
|
61
|
-
padding: { left: 2, right: 2, top: 1, bottom: 0 },
|
|
62
|
-
});
|
|
63
|
-
|
|
64
|
-
// ═══════════════════════════════════════════════════
|
|
65
|
-
// PROMPT LINE (bottom — like PI's > prompt)
|
|
66
|
-
// ═══════════════════════════════════════════════════
|
|
67
|
-
|
|
68
|
-
const prompt = blessed.textbox({
|
|
69
|
-
bottom: 0, left: 0, width: '100%', height: 1,
|
|
70
|
-
style: { fg: T.text, bg: T.surface },
|
|
71
|
-
inputOnFocus: true,
|
|
72
|
-
keys: true,
|
|
73
|
-
tags: true,
|
|
74
|
-
});
|
|
75
|
-
|
|
76
|
-
// ═══════════════════════════════════════════════════
|
|
16
|
+
// ═══════════════════════════════════════════════
|
|
77
17
|
// STATE
|
|
78
|
-
//
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
let
|
|
83
|
-
let
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
function
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
18
|
+
// ═══════════════════════════════════════════════
|
|
19
|
+
|
|
20
|
+
let activeModel = 'nvidia/llama-3.1-8b';
|
|
21
|
+
let totalCost = 0.000087;
|
|
22
|
+
let reqCount = 4;
|
|
23
|
+
let showStats = true;
|
|
24
|
+
|
|
25
|
+
// ═══════════════════════════════════════════════
|
|
26
|
+
// HELPERS
|
|
27
|
+
// ═══════════════════════════════════════════════
|
|
28
|
+
|
|
29
|
+
function dim(s: string) { return chalk.dim(s); }
|
|
30
|
+
function badge(t: string) { return chalk.dim(`[${t}]`); }
|
|
31
|
+
|
|
32
|
+
function headerLine(): string {
|
|
33
|
+
return [
|
|
34
|
+
chalk.bold.hex('#bb9af7')('⚡ A3M Router'),
|
|
35
|
+
dim('·'),
|
|
36
|
+
chalk.hex('#9ece6a')(activeModel),
|
|
37
|
+
dim('·'),
|
|
38
|
+
dim(`${reqCount} req`),
|
|
39
|
+
dim('·'),
|
|
40
|
+
dim(`$${totalCost.toFixed(6)}`),
|
|
41
|
+
].join(' ');
|
|
42
|
+
}
|
|
101
43
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
if (l.role === 'system') {
|
|
106
|
-
out += ` ${D(l.text)}\n`;
|
|
107
|
-
} else if (l.role === 'user') {
|
|
108
|
-
out += `\n {bold}{#7dcfff-fg}▸{/} ${l.text}\n`;
|
|
109
|
-
} else {
|
|
110
|
-
// A3M response — with badges
|
|
111
|
-
const parts: string[] = [];
|
|
112
|
-
if (l.model) parts.push(`{#9ece6a-fg}${l.model}{/}`);
|
|
113
|
-
if (l.ms) parts.push(`{#e0af68-fg}${l.ms}ms{/}`);
|
|
114
|
-
if (l.cost !== undefined) parts.push(`{#ff9e64-fg}$${l.cost.toFixed(6)}{/}`);
|
|
115
|
-
out += `\n {bold}{#bb9af7-fg}A3M{/} ${parts.join(` ${D('·')} `)}\n`;
|
|
116
|
-
out += ` ${l.text}\n`;
|
|
117
|
-
}
|
|
118
|
-
}
|
|
44
|
+
function printSystem(text: string) {
|
|
45
|
+
console.log(' ' + dim(text));
|
|
46
|
+
}
|
|
119
47
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
``,
|
|
125
|
-
` ${D('Type anything — auto-routed to cheapest capable model.')}`,
|
|
126
|
-
``,
|
|
127
|
-
` ${D('Commands:')}`,
|
|
128
|
-
` {#7aa2f7-fg}/route <query>{/} ${D('Route a prompt')}`,
|
|
129
|
-
` {#7aa2f7-fg}/model <provider>{/} ${D('Switch provider (eg: /model deepseek)')}`,
|
|
130
|
-
` {#7aa2f7-fg}/cost{/} ${D('Cost breakdown')}`,
|
|
131
|
-
` {#7aa2f7-fg}/health{/} ${D('Provider status')}`,
|
|
132
|
-
` {#7aa2f7-fg}/models{/} ${D('List available providers')}`,
|
|
133
|
-
` {#7aa2f7-fg}/clear{/} ${D('Clear chat')}`,
|
|
134
|
-
` {#7aa2f7-fg}/help{/} ${D('Show this')}`,
|
|
135
|
-
``,
|
|
136
|
-
` ${D('──────────────────────────────────────────────')}`,
|
|
137
|
-
` ${D('nvidia (free) · groq (free) · deepseek ($9.46) · cerebras (free)')}`,
|
|
138
|
-
`\n`,
|
|
139
|
-
].join('\n');
|
|
140
|
-
}
|
|
48
|
+
function printUser(text: string) {
|
|
49
|
+
console.log('');
|
|
50
|
+
console.log(' ' + chalk.bold.hex('#7dcfff')('▸ ') + text);
|
|
51
|
+
}
|
|
141
52
|
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
53
|
+
function printA3M(text: string, model?: string, ms?: number, cost?: number) {
|
|
54
|
+
const parts: string[] = [];
|
|
55
|
+
if (model) parts.push(chalk.hex('#9ece6a')(model));
|
|
56
|
+
if (ms) parts.push(chalk.hex('#e0af68')(`${ms}ms`));
|
|
57
|
+
if (cost !== undefined) parts.push(chalk.hex('#ff9e64')(`$${cost.toFixed(6)}`));
|
|
58
|
+
console.log('');
|
|
59
|
+
console.log(' ' + chalk.bold.hex('#bb9af7')('A3M') + ' ' + parts.join(' ' + dim('·') + ' '));
|
|
60
|
+
console.log(' ' + text);
|
|
145
61
|
}
|
|
146
62
|
|
|
147
|
-
//
|
|
148
|
-
//
|
|
149
|
-
//
|
|
63
|
+
// ═══════════════════════════════════════════════
|
|
64
|
+
// COMMANDS
|
|
65
|
+
// ═══════════════════════════════════════════════
|
|
150
66
|
|
|
151
67
|
function handle(input: string) {
|
|
152
68
|
const cmd = input.trim();
|
|
153
69
|
if (!cmd) return;
|
|
154
70
|
|
|
155
|
-
|
|
71
|
+
printUser(cmd);
|
|
156
72
|
|
|
157
73
|
if (cmd === '/help' || cmd === '/h') {
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
74
|
+
printA3M([
|
|
75
|
+
chalk.bold('Commands:'),
|
|
76
|
+
'',
|
|
77
|
+
` ${chalk.hex('#7aa2f7')('/route <query>')} ${dim('Route a prompt')}`,
|
|
78
|
+
` ${chalk.hex('#7aa2f7')('/model <provider>')} ${dim('Switch provider (nvidia, deepseek, groq, etc)')}`,
|
|
79
|
+
` ${chalk.hex('#7aa2f7')('/cost')} ${dim('Cost breakdown')}`,
|
|
80
|
+
` ${chalk.hex('#7aa2f7')('/health')} ${dim('Provider status')}`,
|
|
81
|
+
` ${chalk.hex('#7aa2f7')('/models')} ${dim('List available providers')}`,
|
|
82
|
+
` ${chalk.hex('#7aa2f7')('/stats')} ${dim('Toggle stats header')}`,
|
|
83
|
+
` ${chalk.hex('#7aa2f7')('/clear')} ${dim('Clear screen')}`,
|
|
84
|
+
` ${chalk.hex('#7aa2f7')('/exit, /q')} ${dim('Quit')}`,
|
|
85
|
+
'',
|
|
86
|
+
dim('Or just type anything — auto-routed to cheapest model.'),
|
|
87
|
+
].join('\n'));
|
|
88
|
+
} else if (cmd === '/exit' || cmd === '/q' || cmd === ':q') {
|
|
89
|
+
console.log(dim('\n Goodbye.\n'));
|
|
162
90
|
process.exit(0);
|
|
91
|
+
} else if (cmd === '/clear' || cmd === '/cls') {
|
|
92
|
+
console.clear();
|
|
93
|
+
console.log(headerLine());
|
|
94
|
+
console.log('');
|
|
95
|
+
} else if (cmd === '/stats') {
|
|
96
|
+
showStats = !showStats;
|
|
97
|
+
printSystem(showStats ? 'Stats header: ON' : 'Stats header: OFF');
|
|
163
98
|
} else if (cmd === '/cost') {
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
99
|
+
printA3M('Cost breakdown:', '—', 0, 0);
|
|
100
|
+
printSystem(` nvidia $0.000000 (free)`);
|
|
101
|
+
printSystem(` deepseek $0.000009 ($9.46 remaining)`);
|
|
102
|
+
printSystem(` groq $0.000000 (free)`);
|
|
103
|
+
printSystem(` cerebras $0.000000 (free)`);
|
|
104
|
+
printSystem(` ───────────────────────`);
|
|
105
|
+
printSystem(` TOTAL $${totalCost.toFixed(6)} (${reqCount} requests)`);
|
|
106
|
+
printSystem(` Savings 99.97% vs all-premium`);
|
|
170
107
|
} else if (cmd === '/health') {
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
lines.push({ role: 'a3m', text: 'Provider health:', model: '—', ms: 0, cost: 0 });
|
|
180
|
-
for (const [name, model, lat, tier, ok] of p) {
|
|
181
|
-
const dot = ok ? `{#9ece6a-fg}●{/}` : `{#f7768e-fg}✕{/}`;
|
|
182
|
-
lines.push({ role: 'system', text: ` ${dot} ${name} ${D('·')} ${model} ${D('·')} ${lat} ${D('·')} ${tier}` });
|
|
183
|
-
}
|
|
108
|
+
printA3M('Provider health:', '—', 0, 0);
|
|
109
|
+
printSystem(` ${chalk.hex('#9ece6a')('●')} nvidia llama-3.1-8b 85ms free`);
|
|
110
|
+
printSystem(` ${chalk.hex('#9ece6a')('●')} deepseek v4-flash 210ms mid`);
|
|
111
|
+
printSystem(` ${chalk.hex('#9ece6a')('●')} groq 8b-instant 150ms cheap`);
|
|
112
|
+
printSystem(` ${chalk.hex('#9ece6a')('●')} cerebras 3.3-70b 320ms cheap`);
|
|
113
|
+
printSystem(` ${chalk.hex('#f7768e')('✕')} mistral small OFFLINE`);
|
|
114
|
+
printSystem(` ${chalk.hex('#9ece6a')('●')} ollama llama3 50ms local`);
|
|
115
|
+
printSystem(` ${dim('4/6 healthy · 45ms avg')}`);
|
|
184
116
|
} else if (cmd === '/models') {
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
117
|
+
printA3M('Available providers (47+):', '—', 0, 0);
|
|
118
|
+
printSystem(` ${chalk.hex('#9ece6a')('● nvidia')} (free, default) ${chalk.hex('#7dcfff')('● groq')} (free) ${chalk.hex('#e0af68')('● deepseek')} (cheap)`);
|
|
119
|
+
printSystem(` ${chalk.hex('#bb9af7')('● cerebras')} (free) ${chalk.hex('#7aa2f7')('● mistral')} (mid) ${chalk.hex('#f7768e')('● openai')} (premium)`);
|
|
120
|
+
printSystem(` ${chalk.hex('#9ece6a')('● ollama')} (local) ${chalk.hex('#7dcfff')('● google')} (free)`);
|
|
121
|
+
printSystem(` ${dim('Use /model <name> to switch')}`);
|
|
190
122
|
} else if (cmd.startsWith('/model ')) {
|
|
191
123
|
const wanted = cmd.replace('/model ', '').trim();
|
|
192
124
|
const valid = ['nvidia', 'deepseek', 'groq', 'cerebras', 'mistral', 'openai', 'ollama', 'google'];
|
|
193
125
|
if (valid.includes(wanted)) {
|
|
194
126
|
activeModel = `${wanted}/auto`;
|
|
195
|
-
|
|
127
|
+
printSystem(`Switched to ${chalk.hex('#9ece6a')(activeModel)}`);
|
|
196
128
|
} else {
|
|
197
|
-
|
|
129
|
+
printSystem(`Unknown: ${wanted}. Options: ${valid.join(', ')}`);
|
|
198
130
|
}
|
|
199
131
|
} else if (cmd.startsWith('/route ') || cmd.startsWith('/r ')) {
|
|
200
132
|
const query = cmd.replace(/^\/r(oute)?\s*/, '');
|
|
@@ -202,56 +134,76 @@ function handle(input: string) {
|
|
|
202
134
|
const cost = Math.random() * 0.00008;
|
|
203
135
|
totalCost += cost;
|
|
204
136
|
reqCount++;
|
|
205
|
-
|
|
206
|
-
role: 'a3m',
|
|
207
|
-
text: query,
|
|
208
|
-
model: activeModel,
|
|
209
|
-
ms,
|
|
210
|
-
cost,
|
|
211
|
-
});
|
|
137
|
+
printA3M(query, activeModel, ms, cost);
|
|
212
138
|
} else {
|
|
213
139
|
// Plain text = auto-route
|
|
214
140
|
const ms = Math.floor(Math.random() * 100) + 30;
|
|
215
141
|
const cost = Math.random() * 0.00005;
|
|
216
142
|
totalCost += cost;
|
|
217
143
|
reqCount++;
|
|
218
|
-
|
|
219
|
-
role: 'a3m',
|
|
220
|
-
text: cmd,
|
|
221
|
-
model: activeModel,
|
|
222
|
-
ms,
|
|
223
|
-
cost,
|
|
224
|
-
});
|
|
144
|
+
printA3M(cmd, activeModel, ms, cost);
|
|
225
145
|
}
|
|
226
146
|
|
|
227
|
-
|
|
147
|
+
// Reprint prompt
|
|
148
|
+
if (showStats) {
|
|
149
|
+
process.stdout.write('\n' + dim(headerLine()) + '\n');
|
|
150
|
+
}
|
|
228
151
|
}
|
|
229
152
|
|
|
230
|
-
//
|
|
231
|
-
//
|
|
232
|
-
//
|
|
153
|
+
// ═══════════════════════════════════════════════
|
|
154
|
+
// STARTUP
|
|
155
|
+
// ═══════════════════════════════════════════════
|
|
156
|
+
|
|
157
|
+
console.clear();
|
|
158
|
+
|
|
159
|
+
// Welcome banner
|
|
160
|
+
console.log(boxen(
|
|
161
|
+
[
|
|
162
|
+
chalk.bold.hex('#bb9af7')('⚡ A3M Router'),
|
|
163
|
+
'',
|
|
164
|
+
dim('One prompt in. The right model out.'),
|
|
165
|
+
'',
|
|
166
|
+
dim('Type anything — auto-routed to cheapest model.'),
|
|
167
|
+
dim('Commands: /route /cost /health /models /help /exit'),
|
|
168
|
+
'',
|
|
169
|
+
chalk.hex('#9ece6a')('nvidia (free)') + dim(' · ') +
|
|
170
|
+
chalk.hex('#7dcfff')('groq (free)') + dim(' · ') +
|
|
171
|
+
chalk.hex('#e0af68')('deepseek ($9.46)'),
|
|
172
|
+
].join('\n'),
|
|
173
|
+
{
|
|
174
|
+
padding: 1,
|
|
175
|
+
margin: { top: 1, bottom: 1 },
|
|
176
|
+
borderStyle: 'round',
|
|
177
|
+
borderColor: 'magenta',
|
|
178
|
+
dimBorder: true,
|
|
179
|
+
}
|
|
180
|
+
));
|
|
233
181
|
|
|
234
|
-
|
|
182
|
+
console.log(headerLine());
|
|
183
|
+
console.log('');
|
|
235
184
|
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
185
|
+
// REPL
|
|
186
|
+
const rl = readline.createInterface({
|
|
187
|
+
input: process.stdin,
|
|
188
|
+
output: process.stdout,
|
|
189
|
+
prompt: chalk.hex('#7dcfff')('▸ '),
|
|
190
|
+
terminal: true,
|
|
239
191
|
});
|
|
240
192
|
|
|
241
|
-
prompt
|
|
242
|
-
const val = prompt.getValue().trim();
|
|
243
|
-
prompt.clearValue();
|
|
244
|
-
handle(val);
|
|
245
|
-
});
|
|
193
|
+
rl.prompt();
|
|
246
194
|
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
195
|
+
rl.on('line', (line: string) => {
|
|
196
|
+
handle(line);
|
|
197
|
+
rl.prompt();
|
|
198
|
+
});
|
|
250
199
|
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
200
|
+
rl.on('close', () => {
|
|
201
|
+
console.log(dim('\n Goodbye.\n'));
|
|
202
|
+
process.exit(0);
|
|
203
|
+
});
|
|
254
204
|
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
205
|
+
// Handle Ctrl+C gracefully
|
|
206
|
+
process.on('SIGINT', () => {
|
|
207
|
+
console.log(dim('\n Use /exit to quit.\n'));
|
|
208
|
+
rl.prompt();
|
|
209
|
+
});
|