auto-model-router 0.1.4 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.omp-plugin/marketplace.json +2 -2
- package/README.md +43 -55
- package/bun.lock +606 -0
- package/package.json +2 -1
- package/research/analyze-ledger.ts +173 -0
- package/research/apply-cost-tuning.ts +73 -0
- package/research/cost-analysis.ts +150 -0
- package/research/feed-check.ts +64 -0
- package/research/model-recommendations.ts +86 -0
- package/research/project-yield.ts +96 -0
- package/research/run-eval.ts +133 -0
- package/research/status.ts +55 -0
- package/research/tier-fill.ts +109 -0
- package/research/tier-map.ts +123 -0
- package/src/catalog/benchmark-feeds.ts +397 -0
- package/src/catalog/openrouter-catalog.ts +30 -0
- package/src/config/defaults.ts +30 -0
- package/src/config/load.ts +2 -0
- package/src/config/schema.ts +34 -0
- package/src/config/types.ts +106 -0
- package/src/cost/ledger.ts +23 -2
- package/src/cost/types.ts +24 -0
- package/src/eval/calibrate.ts +131 -0
- package/src/eval/grade.ts +115 -0
- package/src/eval/judge.ts +71 -0
- package/src/eval/run.ts +126 -0
- package/src/eval/tasks.ts +272 -0
- package/src/router/candidates.ts +13 -6
- package/src/router/classify.ts +16 -3
- package/src/router/explore.ts +59 -0
- package/src/router/select.ts +54 -4
- package/src/router/tier-plan.ts +57 -1
- package/src/router/types.ts +13 -0
- package/src/server/turn.ts +9 -2
- package/src/util/sqlite.ts +68 -1
- package/test/benchmark-feeds.test.ts +222 -0
- package/test/classify.test.ts +59 -0
- package/test/eval.test.ts +184 -0
- package/test/exploration.test.ts +251 -0
- package/test/failover.test.ts +4 -0
- package/test/hold-exploration.test.ts +124 -0
- package/test/tier-plan.test.ts +55 -1
- package/test/tokens.test.ts +7 -0
- package/test/trust-attribution.test.ts +96 -2
- package/test/turn.test.ts +45 -0
- package/tools/smoke.ts +2 -0
- package/tools/sync-marketplace-version.ts +60 -0
|
@@ -7,14 +7,14 @@
|
|
|
7
7
|
},
|
|
8
8
|
"metadata": {
|
|
9
9
|
"description": "auto-model-router: a local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter",
|
|
10
|
-
"version": "0.1
|
|
10
|
+
"version": "0.2.1",
|
|
11
11
|
"pluginRoot": "."
|
|
12
12
|
},
|
|
13
13
|
"plugins": [
|
|
14
14
|
{
|
|
15
15
|
"name": "auto-model-router",
|
|
16
16
|
"description": "Local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter. Runs in-process, routes per turn by price and task complexity, with budget caps, mid-stream escalation, and cache-aware hysteresis.",
|
|
17
|
-
"version": "0.1
|
|
17
|
+
"version": "0.2.1",
|
|
18
18
|
"author": {
|
|
19
19
|
"name": "drewappling",
|
|
20
20
|
"email": "drewappling@gmail.com"
|
package/README.md
CHANGED
|
@@ -74,9 +74,10 @@ without touching routing.
|
|
|
74
74
|
- **Reported** — `usage.cost` from OpenRouter, authoritative after the fact.
|
|
75
75
|
Drives the ledger, `stats`, and prediction-error calibration.
|
|
76
76
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
77
|
+
## Installing
|
|
78
|
+
|
|
79
|
+
No separate Bun install is needed for the embedded path. The standalone
|
|
80
|
+
`serve` binary (`npm install -g auto-model-router`) bundles Bun.
|
|
80
81
|
|
|
81
82
|
Two ways to get the router into omp. The **npm package** is the modern path —
|
|
82
83
|
it installs the `auto-model-router` binary and wires the omp extensions; the
|
|
@@ -137,45 +138,6 @@ or install it from the marketplace (see below). The plugin declares all three
|
|
|
137
138
|
extensions (`router-embed`, `router-toast`, `router-configure`), so installing
|
|
138
139
|
it wires the router in without editing `config.yml` by hand.
|
|
139
140
|
|
|
140
|
-
There is nothing to install system-wide. Run the cross-platform installer
|
|
141
|
-
(Windows, macOS, Linux) from the repo:
|
|
142
|
-
|
|
143
|
-
```bash
|
|
144
|
-
bun tools/install.ts
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
It wires the auto-model-router extensions into omp's `~/.omp/agent/config.yml`
|
|
148
|
-
(`$PI_CODING_AGENT_DIR/config.yml` when that env var relocates the agent dir),
|
|
149
|
-
backing up the previous file first. It is idempotent — re-running is a no-op.
|
|
150
|
-
|
|
151
|
-
Options:
|
|
152
|
-
|
|
153
|
-
```bash
|
|
154
|
-
bun tools/install.ts --no-toast --no-configure # only the required embed extension
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
The installer adds:
|
|
158
|
-
|
|
159
|
-
- `router-embed.ts` — **required**; runs the router in-process.
|
|
160
|
-
- `router-toast.ts` — optional; chosen-model toasts.
|
|
161
|
-
- `router-configure.ts` — optional; the `/router` command.
|
|
162
|
-
|
|
163
|
-
Or add the paths by hand to omp's `~/.omp/agent/config.yml`:
|
|
164
|
-
|
|
165
|
-
```yaml
|
|
166
|
-
# ~/.omp/agent/config.yml
|
|
167
|
-
extensions:
|
|
168
|
-
- /path/to/auto-model-router/omp-extension/router-embed.ts
|
|
169
|
-
- /path/to/auto-model-router/omp-extension/router-toast.ts # optional: chosen-model toasts
|
|
170
|
-
- /path/to/auto-model-router/omp-extension/router-configure.ts # optional: /router command
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
Then restart the omp session (extensions load at session start).
|
|
174
|
-
|
|
175
|
-
or install it from the marketplace (see below). The plugin declares all three
|
|
176
|
-
extensions (`router-embed`, `router-toast`, `router-configure`), so installing
|
|
177
|
-
it wires the router in without editing `config.yml` by hand.
|
|
178
|
-
|
|
179
141
|
### Install from the marketplace
|
|
180
142
|
|
|
181
143
|
This repo doubles as its own marketplace: it ships a catalog at
|
|
@@ -194,6 +156,9 @@ or in the TUI:
|
|
|
194
156
|
/marketplace install auto-model-router@auto-model-router
|
|
195
157
|
```
|
|
196
158
|
|
|
159
|
+
After installing, restart the omp session (extensions load at session start),
|
|
160
|
+
then `/model` and pick `auto-model-router/auto`.
|
|
161
|
+
|
|
197
162
|
### Install from the Pi package marketplace
|
|
198
163
|
|
|
199
164
|
The repo is also a Pi package (see the `pi` manifest and `pi-package` keyword
|
|
@@ -210,12 +175,23 @@ or from git:
|
|
|
210
175
|
pi install git:github.com/drewappling/auto-model-router
|
|
211
176
|
```
|
|
212
177
|
|
|
213
|
-
|
|
178
|
+
#### Releasing
|
|
179
|
+
|
|
180
|
+
Cut releases with `npm version` (or `bun run release <patch|minor|major>`), not a
|
|
181
|
+
bare `npm publish`:
|
|
214
182
|
|
|
215
183
|
```bash
|
|
216
|
-
npm
|
|
184
|
+
npm version patch && git push --follow-tags # or: bun run release patch
|
|
217
185
|
```
|
|
218
186
|
|
|
187
|
+
`npm version` runs the `version` lifecycle script
|
|
188
|
+
(`tools/sync-marketplace-version.ts`), which rewrites the Git-marketplace
|
|
189
|
+
catalog (`.omp-plugin/marketplace.json`) to the new version and stages it into
|
|
190
|
+
the version commit — so the npm package and the marketplace catalog can never
|
|
191
|
+
drift. Pushing the `vX.Y.Z` tag triggers the release workflow (npm publish,
|
|
192
|
+
which auto-indexes on pi.dev/packages, plus a GitHub Release). A bare
|
|
193
|
+
`npm publish` skips both the catalog sync and the tag, so avoid it.
|
|
194
|
+
|
|
219
195
|
### Hermes
|
|
220
196
|
|
|
221
197
|
Install the router globally (puts the `serve` binary on PATH) and
|
|
@@ -279,17 +255,29 @@ providers:
|
|
|
279
255
|
|
|
280
256
|
### The OpenRouter key
|
|
281
257
|
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
258
|
+
**omp does not need to be authenticated to OpenRouter.** On a routed turn omp
|
|
259
|
+
never calls OpenRouter directly: the embed extension registers the
|
|
260
|
+
`auto-model-router` provider with a placeholder bearer (`embedded`) pointing at
|
|
261
|
+
the in-process router, and the router holds the real OpenRouter key and makes
|
|
262
|
+
the upstream call. omp only needs to see that the provider "has credentials",
|
|
263
|
+
which the placeholder satisfies.
|
|
264
|
+
|
|
265
|
+
There should be exactly one OpenRouter key on the machine. The router resolves
|
|
266
|
+
it in this order:
|
|
267
|
+
|
|
268
|
+
1. `openrouter.apiKey` in `$AUTO_MODEL_ROUTER_HOME/config.yml` — router-owned,
|
|
269
|
+
never enters omp's environment. Set it with `auto-model-router config` or by
|
|
270
|
+
hand.
|
|
271
|
+
2. `OPENROUTER_API_KEY` in the environment omp launches from (including any
|
|
272
|
+
`.env` omp loaded).
|
|
273
|
+
3. **omp's own auth store** — `~/.omp/agent/agent.db`, provider `openrouter`, so
|
|
274
|
+
`/login openrouter` inside omp is sufficient and nothing needs copying.
|
|
275
|
+
|
|
276
|
+
Options 1–2 give the router its own key with omp left unauthenticated; option 3
|
|
277
|
+
is a zero-config convenience for when you *have* logged omp in. The store is
|
|
278
|
+
opened read-only and never written: omp owns it, including OAuth refresh. An
|
|
279
|
+
expired OAuth access token is rejected rather than sent, because refreshing is
|
|
280
|
+
omp's job and a stale bearer just burns a turn on a 401. Under
|
|
293
281
|
`OMP_AUTH_BROKER_URL` the local store is not consulted at all, since a broker
|
|
294
282
|
replaces it.
|
|
295
283
|
|