adaptive-memory-multi-model-router 2.16.1 → 2.16.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -0
- package/data/jev-distill.jsonl +320 -0
- package/dist/routing/jev/jevRouter.d.ts +49 -0
- package/dist/routing/jev/jevRouter.js +231 -0
- package/dist/routing/jev/jevRouter.js.map +1 -0
- package/dist/routing/jev/optionAttention.d.ts +63 -0
- package/dist/routing/jev/optionAttention.js +158 -0
- package/dist/routing/jev/optionAttention.js.map +1 -0
- package/dist/routing/jev/remote.d.ts +14 -0
- package/dist/routing/jev/remote.js +55 -0
- package/dist/routing/jev/remote.js.map +1 -0
- package/dist/routing/jev/types.d.ts +71 -0
- package/dist/routing/jev/types.js +19 -0
- package/dist/routing/jev/types.js.map +1 -0
- package/dist/routing/jev/weights/jev-router-weights.json +1 -0
- package/dist/server/modelMapper.js +18 -0
- package/dist/server/modelMapper.js.map +1 -1
- package/package.json +4 -2
- package/python/pyproject.toml +1 -1
- package/src/cli/setupWizard.ts +4 -2
- package/src/routing/jev/jevRouter.ts +251 -0
- package/src/routing/jev/optionAttention.ts +186 -0
- package/src/routing/jev/remote.ts +50 -0
- package/src/routing/jev/types.ts +79 -0
- package/src/routing/jev/weights/jev-router-weights.json +1 -0
- package/src/server/modelMapper.ts +17 -0
- package/tests/routing/jev.test.ts +110 -0
- package/tools/calibrate_temp.py +81 -0
- package/tools/distill.mjs +143 -0
- package/tools/train_jev.py +213 -0
package/README.md
CHANGED
|
@@ -100,6 +100,11 @@ response = client.chat.completions.create(
|
|
|
100
100
|
|
|
101
101
|
**Result:** Simple questions cost 300x less. Complex queries still go to premium models when needed.
|
|
102
102
|
|
|
103
|
+
**New — System One routing (`model="jev-auto"`):** a single-pass, calibrated
|
|
104
|
+
decision head distilled from the heuristic router — ~2ms, 97.8% agreement,
|
|
105
|
+
per-choice probabilities, and it scores **unseen providers** through their
|
|
106
|
+
text (dynamic option sets). Falls back to the full router below p<0.22.
|
|
107
|
+
|
|
103
108
|
---
|
|
104
109
|
|
|
105
110
|
## 🚀 Performance Benchmarks
|
|
@@ -175,6 +180,18 @@ Then routes to the right tier:
|
|
|
175
180
|
| **Mid** | GPT-4o-mini, Claude-haiku | Standard tasks |
|
|
176
181
|
| **Premium** | GPT-4o, Claude-sonnet, Gemini | Complex reasoning |
|
|
177
182
|
|
|
183
|
+
### Two routing engines
|
|
184
|
+
|
|
185
|
+
| Mode | Engine | Latency | Notes |
|
|
186
|
+
|------|--------|---------|-------|
|
|
187
|
+
| `model="auto"` | Heuristic System 2 (features + EXP3 diversity) | ~0.4ms | Default |
|
|
188
|
+
| `model="jev-auto"` | **System One** option-attention head | ~2ms warm | Calibrated probs, dynamic option sets, confidence-guarded fallback to `auto` |
|
|
189
|
+
|
|
190
|
+
The Jev head ([open System One interface pattern](https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD))
|
|
191
|
+
is distilled from the heuristic router via `npm run jev:distill && npm run jev:train`.
|
|
192
|
+
Optionally point it at a Jev-compatible server ([openjev-sglang](https://github.com/ekzhang/openjev-sglang)
|
|
193
|
+
or api.typesafe.ai) with `A3M_JEV_URL`.
|
|
194
|
+
|
|
178
195
|
---
|
|
179
196
|
|
|
180
197
|
## Provider Coverage
|