adaptive-memory-multi-model-router 2.0.7 → 2.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LAUNCH.md +160 -412
- package/README.md +146 -153
- package/articles/HN_FINAL.md +87 -139
- package/articles/devto-llm-routing.md +93 -80
- package/articles/hackernews-show-hn.md +35 -63
- package/articles/reddit-ml.md +59 -76
- package/articles/twitter-thread-cost-savings.md +54 -72
- package/benchmark-results.json +24 -24
- package/dist/routing/advancedRouter.js +137 -74
- package/docs/GEO.md +124 -0
- package/docs/HN_SUBMISSION_FINAL.md +83 -49
- package/docs/SEO_AUDIT.md +112 -167
- package/docs/index.html +8 -8
- package/docs-site/index.html +8 -8
- package/llms.txt +31 -11
- package/package.json +26 -163
- package/public/robots.txt +12 -2
- package/public/sitemap.xml +37 -1
- package/scripts/routing-benchmark-v2.js +3 -3
package/articles/reddit-ml.md
CHANGED
|
@@ -1,93 +1,76 @@
|
|
|
1
|
-
[P] A3M Router
|
|
1
|
+
[P] A3M Router achieves 82.5% routing accuracy with keyword matching — matches RouteLLM's BERT classifier (85%) without GPU
|
|
2
2
|
|
|
3
3
|
Hi r/MachineLearning,
|
|
4
4
|
|
|
5
|
-
We
|
|
5
|
+
We benchmarked our keyword-matching LLM router against RouteLLM's GPU-trained BERT classifier. The results surprised us.
|
|
6
6
|
|
|
7
|
-
**
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
4. **Online Learning** - Update model profiles from actual performance:
|
|
41
|
-
```javascript
|
|
42
|
-
updateModelProfile(model, actual_latency, actual_cost, quality_rating);
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
**Supported Providers:**
|
|
46
|
-
- API: Groq, Cerebras, Mistral, OpenAI, Anthropic, Google, DeepSeek
|
|
47
|
-
- CLI: CommandCode, OpenCode (free tiers)
|
|
48
|
-
- Local: Ollama, vLLM, LM Studio
|
|
49
|
-
|
|
50
|
-
**Generic Configuration:**
|
|
51
|
-
Users can add their own providers without code changes:
|
|
52
|
-
```json
|
|
53
|
-
// ~/.config/a3m-router/providers.json
|
|
54
|
-
{
|
|
55
|
-
"providers": {
|
|
56
|
-
"my-provider": {
|
|
57
|
-
"baseUrl": "https://api.myprovider.com",
|
|
58
|
-
"apiKeyEnv": "MY_API_KEY",
|
|
59
|
-
"models": ["my-model"],
|
|
60
|
-
"type": "api"
|
|
61
|
-
}
|
|
62
|
-
}
|
|
7
|
+
**Benchmark comparison:**
|
|
8
|
+
|
|
9
|
+
| Metric | RouteLLM (BERT) | A3M Router (Keywords) |
|
|
10
|
+
|--------|------------------|------------------------|
|
|
11
|
+
| Accuracy (±1 tier) | 85% | 82.5% |
|
|
12
|
+
| ML required | Yes (PyTorch + CUDA) | No |
|
|
13
|
+
| Model size | ~500MB BERT | 0 bytes |
|
|
14
|
+
| GPU required | Yes | No |
|
|
15
|
+
| Cold start | ~3s (model load) | ~50ms |
|
|
16
|
+
| Install size | ~2GB+ | 3MB |
|
|
17
|
+
| Runtime | Python | Node.js |
|
|
18
|
+
|
|
19
|
+
2.5% accuracy gap. Zero ML infrastructure.
|
|
20
|
+
|
|
21
|
+
**Context:**
|
|
22
|
+
RouteLLM (from UC Berkeley, arXiv:2404.06035) trains a BERT classifier to route LLM queries between tiers. It's the gold standard for published LLM routing benchmarks.
|
|
23
|
+
|
|
24
|
+
We implemented routing via keyword-based feature extraction: 139 keywords, 12 complexity signals, heuristic scoring. No training loop, no gradient updates, no neural network.
|
|
25
|
+
|
|
26
|
+
**Routing algorithm:**
|
|
27
|
+
```javascript
|
|
28
|
+
// Feature extraction
|
|
29
|
+
const features = extractQueryFeatures(query);
|
|
30
|
+
// { has_code: true, complexity: 0.6, task_type: "code_gen" }
|
|
31
|
+
|
|
32
|
+
// Complexity-weighted scoring
|
|
33
|
+
if (features.complexity < 0.5) {
|
|
34
|
+
score = cost_efficiency * 0.7 + quality * 0.3;
|
|
35
|
+
} else if (features.has_code) {
|
|
36
|
+
score = speed * 0.4 + quality * 0.4 + cost * 0.2;
|
|
37
|
+
} else {
|
|
38
|
+
score = quality * 0.7 + cost_efficiency * 0.3;
|
|
63
39
|
}
|
|
64
40
|
```
|
|
65
41
|
|
|
66
|
-
**
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
-
|
|
71
|
-
- Batch processing with concurrency control
|
|
42
|
+
**Why this matters for the ML community:**
|
|
43
|
+
|
|
44
|
+
1. **Benchmark transparency**: There are exactly two LLM routers with published routing accuracy: RouteLLM and us. LiteLLM (47K GitHub stars) publishes zero accuracy data. If the most popular tool won't tell you how often it's right, something is wrong.
|
|
45
|
+
|
|
46
|
+
2. **Efficiency question**: Is a 2.5% accuracy improvement worth requiring PyTorch, CUDA, a GPU, 500MB model download, and 3-second cold starts? For many production deployments, the answer is no.
|
|
72
47
|
|
|
73
|
-
**
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
-
|
|
77
|
-
-
|
|
78
|
-
-
|
|
48
|
+
3. **The 30x story**: 97% of the accuracy at 3% of the compute. That's a 30x efficiency multiplier.
|
|
49
|
+
|
|
50
|
+
**Cost results:**
|
|
51
|
+
- 63.7% average cost reduction vs single-provider routing
|
|
52
|
+
- 40 provider integrations
|
|
53
|
+
- Drop-in OpenAI-compatible proxy (localhost:8787)
|
|
54
|
+
|
|
55
|
+
**Growth (organically, zero marketing):**
|
|
56
|
+
- Day 1: 552 downloads
|
|
57
|
+
- Day 2: 320 downloads
|
|
58
|
+
- Day 3: 1,903 downloads
|
|
59
|
+
- 245% growth, zero budget
|
|
60
|
+
|
|
61
|
+
**Questions for the community:**
|
|
62
|
+
|
|
63
|
+
1. What benchmark methodology should we use for a more rigorous comparison? We used the same ±1 tier accuracy metric as RouteLLM's paper.
|
|
64
|
+
2. Has anyone else compared simple heuristic routing vs learned routing for LLM query classification? The gap seems smaller than expected.
|
|
65
|
+
3. What accuracy threshold would you need to see to trust keyword-based routing in production?
|
|
79
66
|
|
|
80
67
|
**Try it:**
|
|
81
68
|
```bash
|
|
82
69
|
npm install adaptive-memory-multi-model-router
|
|
83
70
|
npx a3m-router route "Write Python to sort an array"
|
|
71
|
+
npx a3m-router benchmark
|
|
84
72
|
```
|
|
85
73
|
|
|
86
|
-
**Questions for the community:**
|
|
87
|
-
1. What routing strategies have worked for your LLM applications?
|
|
88
|
-
2. How do you handle cost-quality tradeoffs in production?
|
|
89
|
-
3. What features would make this more useful for ML pipelines?
|
|
90
|
-
|
|
91
74
|
GitHub: https://github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
92
75
|
|
|
93
|
-
|
|
76
|
+
The honest caveat: this is a young project (3 days since launch). The 82.5% number is from our benchmark suite, not an independent evaluation. We welcome scrutiny and would love to see third-party replication.
|
|
@@ -1,102 +1,84 @@
|
|
|
1
|
-
# Twitter Thread:
|
|
1
|
+
# Twitter Thread: 30x Efficiency — We Matched a GPU-Trained Router With Zero ML
|
|
2
2
|
|
|
3
|
-
##
|
|
4
|
-
|
|
5
|
-
Day 3: 1,903 downloads. 245% growth. Zero marketing budget.
|
|
3
|
+
## T1/7 — Hook
|
|
4
|
+
We matched a GPU-trained BERT router's accuracy with zero ML.
|
|
6
5
|
|
|
7
|
-
|
|
6
|
+
82.5% accuracy. No PyTorch. No GPU. No 500MB model.
|
|
8
7
|
|
|
9
|
-
|
|
8
|
+
RouteLLM (Berkeley) gets 85% with BERT. We get 82.5% with keyword matching.
|
|
10
9
|
|
|
11
|
-
|
|
12
|
-
Most apps use GPT-4 for EVERYTHING:
|
|
13
|
-
• Simple Q&A → GPT-4 ($0.03/query)
|
|
14
|
-
• Code gen → GPT-4 ($0.05/query)
|
|
15
|
-
• Summarization → GPT-4 ($0.02/query)
|
|
10
|
+
That's 97% of the accuracy at 3% of the compute.
|
|
16
11
|
|
|
17
|
-
|
|
12
|
+
30x more efficient. Thread.
|
|
18
13
|
|
|
19
|
-
##
|
|
20
|
-
|
|
21
|
-
• "What is 2+2?" → ANY model works
|
|
22
|
-
• "Write Python" → Code-capable model
|
|
23
|
-
• "Explain quantum" → High-quality model
|
|
14
|
+
## T2/7 — The Benchmark Numbers
|
|
15
|
+
The only two LLM routers with published benchmarks:
|
|
24
16
|
|
|
25
|
-
|
|
17
|
+
RouteLLM: 85% (±1 tier) — PyTorch + BERT + GPU + 500MB model
|
|
18
|
+
A3M Router: 82.5% (±1 tier) — Node.js + keywords + 0 bytes model
|
|
26
19
|
|
|
27
|
-
|
|
28
|
-
A3M Router learns your usage patterns:
|
|
29
|
-
• Analyzes query characteristics
|
|
30
|
-
• Matches to optimal provider
|
|
31
|
-
• Tracks costs in real-time
|
|
32
|
-
• Falls back if provider fails
|
|
20
|
+
LiteLLM (47,000 GitHub stars): publishes ZERO routing accuracy data.
|
|
33
21
|
|
|
34
|
-
|
|
22
|
+
Benchmark or GTFO.
|
|
35
23
|
|
|
36
|
-
##
|
|
37
|
-
|
|
38
|
-
|
|
24
|
+
## T3/7 — RouteLLM Comparison
|
|
25
|
+
RouteLLM needs:
|
|
26
|
+
- Python + PyTorch + CUDA
|
|
27
|
+
- ~500MB BERT model download
|
|
28
|
+
- GPU for inference
|
|
29
|
+
- ~3s cold start
|
|
30
|
+
- ~2GB install
|
|
39
31
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
32
|
+
A3M Router needs:
|
|
33
|
+
- Node.js
|
|
34
|
+
- 3MB install
|
|
35
|
+
- No GPU
|
|
36
|
+
- 50ms cold start
|
|
43
37
|
|
|
44
|
-
|
|
38
|
+
2.5% accuracy difference. You decide if the GPU is worth it.
|
|
45
39
|
|
|
46
|
-
##
|
|
47
|
-
|
|
48
|
-
const { routeQuery } = require('adaptive-memory-multi-model-router');
|
|
40
|
+
## T4/7 — Cost Savings
|
|
41
|
+
63.7% average cost reduction.
|
|
49
42
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
// → commandcode/taste-1 ($0.00)
|
|
43
|
+
Before: everything goes to GPT-4 at $0.03/query
|
|
44
|
+
After: queries routed to cheapest capable provider
|
|
53
45
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
```
|
|
46
|
+
Simple Q&A: $0.03 -> $0.00 (free provider)
|
|
47
|
+
Code gen: $0.05 -> $0.0004 (Groq)
|
|
48
|
+
Complex reasoning: $0.03 -> $0.03 (stays premium)
|
|
58
49
|
|
|
59
|
-
|
|
60
|
-
• FREE: CommandCode, OpenCode
|
|
61
|
-
• FAST: Groq ($0.59/1M tokens)
|
|
62
|
-
• QUALITY: Mistral, OpenAI, Anthropic
|
|
63
|
-
• LOCAL: Ollama (free!)
|
|
50
|
+
Drop-in proxy. Point any OpenAI SDK at localhost:8787. Zero code changes.
|
|
64
51
|
|
|
65
|
-
|
|
52
|
+
## T5/7 — Growth Story
|
|
53
|
+
Day 1: 552 downloads
|
|
54
|
+
Day 2: 320 downloads
|
|
55
|
+
Day 3: 1,903 downloads
|
|
66
56
|
|
|
67
|
-
|
|
68
|
-
One line to install:
|
|
69
|
-
```bash
|
|
70
|
-
npm install adaptive-memory-multi-model-router
|
|
71
|
-
```
|
|
57
|
+
245% growth. Zero marketing budget. No blog post. No HN. No Twitter thread. Just developers telling developers.
|
|
72
58
|
|
|
73
|
-
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
|
|
59
|
+
## T6/7 — Code Example
|
|
60
|
+
```javascript
|
|
61
|
+
const { createA3MRouter } = require('adaptive-memory-multi-model-router');
|
|
62
|
+
const router = createA3MRouter();
|
|
77
63
|
|
|
78
|
-
|
|
64
|
+
// Auto-routes to cheapest capable provider
|
|
65
|
+
await router.route("What is 2+2?");
|
|
66
|
+
// -> free provider ($0.00)
|
|
79
67
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
🧪 33 tests passing
|
|
84
|
-
🔌 116 integrations
|
|
68
|
+
await router.route("Write Python to sort an array");
|
|
69
|
+
// -> Groq ($0.0004, 0.4s)
|
|
70
|
+
```
|
|
85
71
|
|
|
86
|
-
|
|
87
|
-
Word-of-mouth works. Zero marketing spend.
|
|
72
|
+
40 providers. Semantic cache. Circuit breakers. 3MB.
|
|
88
73
|
|
|
89
|
-
##
|
|
90
|
-
Try it today:
|
|
91
|
-
```bash
|
|
74
|
+
## T7/7 — CTA
|
|
92
75
|
npm install adaptive-memory-multi-model-router
|
|
93
|
-
```
|
|
94
76
|
|
|
95
77
|
GitHub: github.com/Das-rebel/adaptive-memory-multi-model-router
|
|
96
78
|
NPM: npmjs.com/package/adaptive-memory-multi-model-router
|
|
97
79
|
|
|
98
|
-
|
|
80
|
+
82.5% accuracy. Zero ML. Zero GPU. Matches BERT within 2.5%. 63.7% cost savings. 40 providers.
|
|
99
81
|
|
|
100
|
-
|
|
82
|
+
30x more efficient.
|
|
101
83
|
|
|
102
|
-
#LLM #AI #
|
|
84
|
+
#LLM #AI #RouteLLM #BenchmarkOrGTFO #OpenSource #JavaScript #CostOptimization
|
package/benchmark-results.json
CHANGED
|
@@ -1,54 +1,54 @@
|
|
|
1
1
|
{
|
|
2
|
-
"timestamp": "2026-05-
|
|
3
|
-
"version": "2.0.
|
|
2
|
+
"timestamp": "2026-05-18T15:11:19.155Z",
|
|
3
|
+
"version": "2.0.8",
|
|
4
4
|
"queries": 200,
|
|
5
|
-
"exact_accuracy":
|
|
6
|
-
"adjacent_accuracy":
|
|
7
|
-
"over_routed":
|
|
8
|
-
"under_routed":
|
|
9
|
-
"cost_savings_vs_premium":
|
|
5
|
+
"exact_accuracy": 64.5,
|
|
6
|
+
"adjacent_accuracy": 99.5,
|
|
7
|
+
"over_routed": 14,
|
|
8
|
+
"under_routed": 57,
|
|
9
|
+
"cost_savings_vs_premium": 61.6,
|
|
10
10
|
"by_tier": {
|
|
11
11
|
"free": {
|
|
12
|
-
"correct":
|
|
12
|
+
"correct": 46,
|
|
13
13
|
"total": 50
|
|
14
14
|
},
|
|
15
15
|
"cheap": {
|
|
16
|
-
"correct":
|
|
16
|
+
"correct": 47,
|
|
17
17
|
"total": 60
|
|
18
18
|
},
|
|
19
19
|
"mid": {
|
|
20
|
-
"correct":
|
|
20
|
+
"correct": 18,
|
|
21
21
|
"total": 50
|
|
22
22
|
},
|
|
23
23
|
"premium": {
|
|
24
|
-
"correct":
|
|
24
|
+
"correct": 18,
|
|
25
25
|
"total": 40
|
|
26
26
|
}
|
|
27
27
|
},
|
|
28
28
|
"confusion": {
|
|
29
29
|
"free": {
|
|
30
|
-
"free":
|
|
31
|
-
"cheap":
|
|
32
|
-
"mid":
|
|
30
|
+
"free": 46,
|
|
31
|
+
"cheap": 4,
|
|
32
|
+
"mid": 0,
|
|
33
33
|
"premium": 0
|
|
34
34
|
},
|
|
35
35
|
"cheap": {
|
|
36
|
-
"free":
|
|
37
|
-
"cheap":
|
|
38
|
-
"mid":
|
|
39
|
-
"premium":
|
|
36
|
+
"free": 11,
|
|
37
|
+
"cheap": 47,
|
|
38
|
+
"mid": 2,
|
|
39
|
+
"premium": 0
|
|
40
40
|
},
|
|
41
41
|
"mid": {
|
|
42
42
|
"free": 0,
|
|
43
|
-
"cheap":
|
|
44
|
-
"mid":
|
|
45
|
-
"premium":
|
|
43
|
+
"cheap": 24,
|
|
44
|
+
"mid": 18,
|
|
45
|
+
"premium": 8
|
|
46
46
|
},
|
|
47
47
|
"premium": {
|
|
48
48
|
"free": 0,
|
|
49
|
-
"cheap":
|
|
50
|
-
"mid":
|
|
51
|
-
"premium":
|
|
49
|
+
"cheap": 1,
|
|
50
|
+
"mid": 21,
|
|
51
|
+
"premium": 18
|
|
52
52
|
}
|
|
53
53
|
}
|
|
54
54
|
}
|
|
@@ -89,94 +89,157 @@ function refreshModelProfiles() {
|
|
|
89
89
|
exports.MODEL_PROFILES = MODEL_PROFILES;
|
|
90
90
|
|
|
91
91
|
// ============================================================
|
|
92
|
-
// FEATURE EXTRACTION
|
|
92
|
+
// FEATURE EXTRACTION (v3 — multi-signal complexity scorer)
|
|
93
93
|
// ============================================================
|
|
94
94
|
|
|
95
95
|
function extractQueryFeatures(prompt) {
|
|
96
96
|
const lower = prompt.toLowerCase();
|
|
97
|
+
const words = prompt.split(/\s+/);
|
|
98
|
+
const wordCount = words.length;
|
|
97
99
|
|
|
98
|
-
//
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
const
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
100
|
+
// === SIGNAL 1: Domain Detection ===
|
|
101
|
+
// Professional domains that indicate expert-level queries
|
|
102
|
+
const domainSignals = {
|
|
103
|
+
legal: {
|
|
104
|
+
keywords: ['legal', 'law', 'contract', 'liability', 'litigation', 'patent', 'copyright',
|
|
105
|
+
'regulation', 'compliance', 'constitutional', 'statute', 'jurisdiction',
|
|
106
|
+
'court', 'ruling', 'precedent', 'attorney', 'amicus', 'sec ', 'fda ',
|
|
107
|
+
'gdpr', 'ccpa', 'cfpr', 'due diligence', 'merger', 'acquisition',
|
|
108
|
+
'10-k', 'sec filing', 'forensic', 'embezzlement', 'infringement'],
|
|
109
|
+
weight: 0.35
|
|
110
|
+
},
|
|
111
|
+
medical: {
|
|
112
|
+
keywords: ['clinical', 'medical', 'pharmaceutical', 'oncology', 'drug', 'trial protocol',
|
|
113
|
+
'diagnosis', 'treatment', 'epidemiolog', 'genome', 'cohort study',
|
|
114
|
+
'biomarker', 'efficacy', 'pharmacoeconomic', 'biologic', 'vaccine',
|
|
115
|
+
'sepsis', 'ehr ', 'surgical', 'patient safety', 'fda approval'],
|
|
116
|
+
weight: 0.35
|
|
117
|
+
},
|
|
118
|
+
finance: {
|
|
119
|
+
keywords: ['financial model', 'valuation', 'revenue', 'portfolio', 'derivative',
|
|
120
|
+
'hedge fund', 'series a', 'series b', 'startup valuation', 'sensitivity analysis',
|
|
121
|
+
'investment thesis', 'earnings', 'tax optimization', 'forensic accounting',
|
|
122
|
+
'multinational', 'jurisdiction', 'risk assessment', 'monte carlo',
|
|
123
|
+
'black-scholes', 'options pricing', 'credit risk'],
|
|
124
|
+
weight: 0.30
|
|
125
|
+
},
|
|
126
|
+
security: {
|
|
127
|
+
keywords: ['security audit', 'penetration', 'vulnerability', 'exploit', 'zero-trust',
|
|
128
|
+
'threat model', 'incident response', 'malware', 'ransomware',
|
|
129
|
+
'authentication flow', 'cryptograph', 'encryption', 'timing attack',
|
|
130
|
+
'supply chain attack', 'owasp', 'compliance', 'risk assessment',
|
|
131
|
+
'mfa', 'zero-day', 'firewall', 'intrusion'],
|
|
132
|
+
weight: 0.30
|
|
133
|
+
},
|
|
134
|
+
architecture: {
|
|
135
|
+
keywords: ['system design', 'microservice', 'distributed system', 'fault-tolerant',
|
|
136
|
+
'event-sourced', 'cqrs', 'consensus algorithm', 'real-time pipeline',
|
|
137
|
+
'high availability', 'multi-region', 'latency sla', 'kafka',
|
|
138
|
+
'event-driven', 'data warehouse', 'etl', 'streaming', '1m events',
|
|
139
|
+
'million events', 'scalab', 'infrastruct', 'deploy'],
|
|
140
|
+
weight: 0.25
|
|
141
|
+
},
|
|
142
|
+
ml_research: {
|
|
143
|
+
keywords: ['neural network', 'transformer', 'backpropagation', 'gradient',
|
|
144
|
+
'reinforcement learning', 'rlhf', 'fine-tun', 'bert ', 'gpt ',
|
|
145
|
+
'attention mechanism', 'training pipeline', 'model monitoring',
|
|
146
|
+
'data drift', 'feature engine', 'deep learn', 'benchmark',
|
|
147
|
+
'ablation', 'sota', 'state of the art', 'paper', 'arxiv'],
|
|
148
|
+
weight: 0.25
|
|
149
|
+
}
|
|
150
|
+
};
|
|
151
|
+
|
|
152
|
+
let domainScore = 0;
|
|
153
|
+
let detectedDomain = '';
|
|
154
|
+
for (const [domain, config] of Object.entries(domainSignals)) {
|
|
155
|
+
const matchCount = config.keywords.filter(kw => lower.includes(kw)).length;
|
|
156
|
+
if (matchCount > 0) {
|
|
157
|
+
const score = config.weight * Math.min(matchCount / 2, 1.5); // cap at 1.5x
|
|
158
|
+
if (score > domainScore) {
|
|
159
|
+
domainScore = score;
|
|
160
|
+
detectedDomain = domain;
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
// === SIGNAL 2: Task Complexity Indicators ===
|
|
166
|
+
const has_code = /function|class |def |import |const |let |python|javascript|typescript|java |cpp|rust|```|=>|->|async|await|sql|css|html|react|node|express|docker|kubernetes/i.test(prompt);
|
|
167
|
+
const has_math = /equation|formula|calculate|sqrt|\^|log|sin|cos|integral|derivative|math|∫|∂|∑|∏|√|∞|π|compute|theorem|proof|complexity|algorithm/i.test(prompt);
|
|
168
|
+
const requires_reasoning = /analyze|compare|contrast|evaluate|assess|implications|impact|consequence|why|because|therefore|reason|logic|argue|debate|critique|synthesize/i.test(prompt);
|
|
169
|
+
const is_creative = /write a|story|poem|creative|imagine|narrative|joke|compose|fiction/i.test(lower);
|
|
170
|
+
const is_translation = /translate|translation|in french|in spanish|in japanese|in chinese/i.test(lower);
|
|
171
|
+
const is_multilingual = /[\u4e00-\u9fff]|[\u3040-\u309f\u30a0-\u30ff]|[\uac00-\ud7af]|[а-яА-Я]/.test(prompt);
|
|
172
|
+
|
|
173
|
+
// === SIGNAL 3: Query Structure ===
|
|
174
|
+
// Longer, more structured queries = more complex
|
|
175
|
+
const avgWordLength = words.reduce((sum, w) => sum + w.length, 0) / Math.max(wordCount, 1);
|
|
176
|
+
const hasMultipleClauses = (prompt.match(/[,;:]/g) || []).length >= 2;
|
|
177
|
+
const hasQualifiers = /detailed|comprehensive|thorough|in-depth|extensive|step-by-step|systematic|formal|rigorous/i.test(prompt);
|
|
178
|
+
|
|
179
|
+
// === SIGNAL 4: Action Verb Intensity ===
|
|
180
|
+
// Expert verbs indicate higher cognitive demands
|
|
181
|
+
const expertVerbs = /design|architect|review|audit|investigate|diagnose|optimize|strategize|formulate|derive|prove|verify|validate/i;
|
|
182
|
+
const midVerbs = /analyze|evaluate|compare|assess|implement|create|build|develop|construct|derive|explain/i;
|
|
183
|
+
const simpleVerbs = /what is|who|when|where|how many|define|list|name|convert|translate|summarize briefly/i;
|
|
184
|
+
|
|
185
|
+
let verbScore = 0;
|
|
186
|
+
if (expertVerbs.test(lower)) verbScore = 0.20;
|
|
187
|
+
else if (midVerbs.test(lower)) verbScore = 0.10;
|
|
188
|
+
if (simpleVerbs.test(lower)) verbScore = -0.10; // deboost simple questions
|
|
189
|
+
|
|
190
|
+
// === SIGNAL 5: Specificity ===
|
|
191
|
+
// Specific details = more complex
|
|
192
|
+
const hasSpecifics = /\d+%|\$\d+|million|billion|specific|particular|given|according to|based on/i.test(prompt);
|
|
193
|
+
const hasMultiStep = /and then|first.*then|after that|next|finally|additionally|furthermore|moreover/i.test(prompt);
|
|
194
|
+
|
|
195
|
+
// === COMPLEXITY SCORING (weighted multi-signal) ===
|
|
196
|
+
let complexity = 0.15; // Base: simple query
|
|
197
|
+
|
|
198
|
+
// Domain signal (strongest predictor)
|
|
199
|
+
complexity += domainScore;
|
|
200
|
+
|
|
201
|
+
// Length signal (longer = harder, but diminishing)
|
|
202
|
+
if (wordCount > 5) complexity += 0.03;
|
|
203
|
+
if (wordCount > 10) complexity += 0.05;
|
|
204
|
+
if (wordCount > 15) complexity += 0.05;
|
|
205
|
+
if (wordCount > 20) complexity += 0.03;
|
|
206
|
+
|
|
207
|
+
// Feature signals
|
|
208
|
+
if (has_code) complexity += 0.10;
|
|
209
|
+
if (has_math) complexity += 0.12;
|
|
210
|
+
if (requires_reasoning) complexity += 0.08;
|
|
211
|
+
if (is_creative) complexity += 0.05;
|
|
212
|
+
if (is_translation) complexity += 0.02;
|
|
213
|
+
|
|
214
|
+
// Structure signals
|
|
215
|
+
if (hasQualifiers) complexity += 0.08;
|
|
216
|
+
if (hasMultipleClauses) complexity += 0.05;
|
|
217
|
+
if (hasSpecifics) complexity += 0.05;
|
|
218
|
+
if (hasMultiStep) complexity += 0.05;
|
|
219
|
+
|
|
220
|
+
// Verb intensity
|
|
221
|
+
complexity += verbScore;
|
|
222
|
+
|
|
223
|
+
// Long words = technical language
|
|
224
|
+
if (avgWordLength > 6) complexity += 0.05;
|
|
225
|
+
if (avgWordLength > 8) complexity += 0.05;
|
|
226
|
+
|
|
227
|
+
complexity = Math.max(0.10, Math.min(1.0, complexity));
|
|
167
228
|
|
|
168
229
|
return {
|
|
169
230
|
complexity,
|
|
170
|
-
length:
|
|
231
|
+
length: wordCount,
|
|
171
232
|
has_code,
|
|
172
233
|
has_math,
|
|
173
234
|
is_multilingual,
|
|
174
235
|
is_translation,
|
|
175
236
|
is_creative,
|
|
176
237
|
requires_reasoning,
|
|
177
|
-
is_security,
|
|
178
|
-
is_devops,
|
|
179
|
-
is_data,
|
|
238
|
+
is_security: /security|vulnerability|inject|exploit|attack|encryption|auth/i.test(lower),
|
|
239
|
+
is_devops: /ci\/cd|docker|kubernetes|k8s|deploy|pipeline|github action|terraform/i.test(lower),
|
|
240
|
+
is_data: /dataset|pandas|numpy|training|model|neural|transformer|bert|llm/i.test(lower),
|
|
241
|
+
detected_domain: detectedDomain,
|
|
242
|
+
domain_score: domainScore,
|
|
180
243
|
};
|
|
181
244
|
}
|
|
182
245
|
|