auto-model-router 0.1.3 → 0.2.0

package/.omp-plugin/marketplace.json

@@ -7,14 +7,14 @@
   },
   "metadata": {
     "description": "auto-model-router: a local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter",
-    "version": "0.1.0",
+    "version": "0.2.0",
     "pluginRoot": "."
   },
   "plugins": [
     {
       "name": "auto-model-router",
       "description": "Local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter. Runs in-process, routes per turn by price and task complexity, with budget caps, mid-stream escalation, and cache-aware hysteresis.",
-      "version": "0.1.0",
+      "version": "0.2.0",
       "author": {
         "name": "drewappling",
         "email": "drewappling@gmail.com"

package/README.md

@@ -6,13 +6,14 @@ OpenRouter model **per turn** based on measured price and estimated task
 complexity — including mid-conversation, when a session shifts from mechanical
 tool-loop churn to genuine reasoning work.
 
-All LLM inference is offloaded to OpenRouter. Nothing runs on-device except
-routing arithmetic.
-
 auto-model-router runs **embedded inside the omp process** (as an omp extension) — no
 separate server, no orphaned process. It binds a free OS-assigned port and
 lives and dies with the omp session.
 
+For non-omp harnesses (Hermes, Claude, any OpenAI-compatible client), run it as
+a standalone process with `auto-model-router serve --port <n>` — the same core,
+on a fixed port, owned by you. See [Hermes](#hermes) below.
+
 ## Why this exists when OpenRouter already ships routers
 
 OpenRouter has `openrouter/auto` (market-spend classifier) and
@@ -62,7 +63,7 @@ without touching routing.
 | `src/router/` | Feature extraction, complexity classification, candidate filtering and scoring, hysteresis, cache-breakpoint placement, budget guard, probe planning. |
 | `src/upstream/` | OpenRouter transport: streaming dispatch, `session_id` stickiness, error classification, fallback arrays. |
 | `src/config/` | Configuration loading, schema validation, and the built-in defaults. |
-| `src/cli/` | `stats`, `models`, `explain`, `config` commands. |
+| `src/cli/` | `serve`, `stats`, `models`, `explain`, `config` commands. |
 | `omp-extension/` | The omp extensions: `router-embed.ts`, `router-toast.ts`, `router-configure.ts`. |
 
 ### Two cost numbers, never conflated
@@ -73,16 +74,33 @@ without touching routing.
 - **Reported** — `usage.cost` from OpenRouter, authoritative after the fact.
   Drives the ledger, `stats`, and prediction-error calibration.
 
-## Requirements
+## Installing
+
+No separate Bun install is needed for the embedded path. The standalone
+`serve` binary (`npm install -g auto-model-router`) bundles Bun.
+
+Two ways to get the router into omp. The **npm package** is the modern path —
+it installs the `auto-model-router` binary and wires the omp extensions; the
+**repo-local installer** is for developing against the source.
+
+### Via npm (installs the `auto-model-router` binary)
 
-- **omp** (the Oh My Pi harness) — the router runs as an omp extension.
-- **Bun** `>= 1.2.0` — omp itself is a Bun process; the router code runs inside
-  it. No separate Bun install is needed for the embedded path.
+```bash
+npm install -g auto-model-router
+```
+
+Then add the shipped extensions to omp's `~/.omp/agent/config.yml`
+(`$PI_CODING_AGENT_DIR/config.yml` when that env var relocates the agent dir):
 
-## Installation
+```yaml
+# ~/.omp/agent/config.yml
+extensions:
+  - auto-model-router/omp-extension/router-embed.ts
+  - auto-model-router/omp-extension/router-toast.ts      # optional: chosen-model toasts
+  - auto-model-router/omp-extension/router-configure.ts # optional: /router command
+```
 
-There is nothing to install system-wide. Run the cross-platform installer
-(Windows, macOS, Linux) from the repo:
+### From the repo (cross-platform installer)
 
 ```bash
 bun tools/install.ts
@@ -138,6 +156,9 @@ or in the TUI:
 /marketplace install auto-model-router@auto-model-router
 ```
 
+After installing, restart the omp session (extensions load at session start),
+then `/model` and pick `auto-model-router/auto`.
+
 ### Install from the Pi package marketplace
 
 The repo is also a Pi package (see the `pi` manifest and `pi-package` keyword
@@ -154,58 +175,109 @@ or from git:
 pi install git:github.com/drewappling/auto-model-router
 ```
 
-To publish to npm (which auto-indexes on pi.dev/packages):
+#### Releasing
+
+Cut releases with `npm version` (or `bun run release <patch|minor|major>`), not a
+bare `npm publish`:
 
 ```bash
-npm publish
+npm version patch && git push --follow-tags   # or: bun run release patch
 ```
 
+`npm version` runs the `version` lifecycle script
+(`tools/sync-marketplace-version.ts`), which rewrites the Git-marketplace
+catalog (`.omp-plugin/marketplace.json`) to the new version and stages it into
+the version commit — so the npm package and the marketplace catalog can never
+drift. Pushing the `vX.Y.Z` tag triggers the release workflow (npm publish,
+which auto-indexes on pi.dev/packages, plus a GitHub Release). A bare
+`npm publish` skips both the catalog sync and the tag, so avoid it.
+
 ### Hermes
 
-Hermes speaks the OpenAI-compatible wire, so it connects to the router with no
-code change. Two ways to run the router for Hermes:
+Install the router globally (puts the `serve` binary on PATH) and
+install the native plugin, then point Hermes at it:
 
-**Standalone server (recommended for Hermes):** run the router as its own
-process on a fixed port, then point Hermes at it:
+**1. Install the router binary:**
+
+```bash
+npm install -g auto-model-router
+```
+
+**2. Install the Hermes plugin.** Copy `hermes-plugin/` to
+`$HERMES_HOME/plugins/model-providers/auto-model-router/` (where
+`HERMES_HOME` is `C:\Users\<you>\AppData\Local\hermes` on Windows,
+`~/.hermes` on macOS/Linux):
+
+```bash
+mkdir -p "$HERMES_HOME/plugins/model-providers"
+cp -r hermes-plugin/ "$HERMES_HOME/plugins/model-providers/auto-model-router/"
+```
+
+**3. Surface the provider in Hermes's picker.** Hermes only lists providers
+that have a credential. The router itself is keyless (it resolves its own
+OpenRouter key), but to make Hermes show it as selectable, add a marker value
+to `$HERMES_HOME/.env`:
+
+```bash
+echo "AUTO_MODEL_ROUTER_API_KEY=local" >> "$HERMES_HOME/.env"
+```
+
+**4. Restart Hermes.** On load, the plugin spawns the router (`auto-model-router
+serve`) as a subprocess on port 8788 and registers the provider profile. Select
+`auto-model-router/auto` as the model.
+
+The plugin runs the router against its **own** config home
+(`$HERMES_HOME/auto-model-router/`), separate from omp's
+`~/.auto-model-router/`, so the two harnesses never share a ledger or
+conversation state and don't leak routing toasts into each other's UIs.
+
+The router serves `GET /v1/models` (returning the `auto`, `auto-cheap`,
+`auto-max` profiles) and `POST /v1/chat/completions`, which Hermes's custom
+endpoint discovery verifies. The router's own OpenRouter key resolution
+(config → env → omp auth store) applies — Hermes does not need its own
+OpenRouter key.
+
+**Standalone alternative (no plugin):** run the router yourself, then add a
+custom provider:
 
 ```bash
 auto-model-router serve --port 8788
 ```
 
 ```yaml
-# ~/.hermes/config.yaml
+# $HERMES_HOME/config.yaml
 providers:
   auto-model-router:
     base_url: http://127.0.0.1:8788/v1
-    api_key: no-key-required
+    api_key: local
     default_model: auto
 ```
 
-**Hermes plugin (native):** copy `hermes-plugin/` to
-`$HERMES_HOME/plugins/model-providers/auto-model-router/` and restart Hermes.
-The plugin spawns the router as a subprocess on load and registers the provider
-profile, so Hermes routes each turn through the router automatically.
-
-The router serves `GET /v1/models` (returning the `auto`, `auto-cheap`,
-`auto-max` profiles) and `POST /v1/chat/completions`, which Hermes's custom
-endpoint discovery verifies. Select `auto-model-router/auto` as the model and
-the router routes each turn by price and complexity. The router's own OpenRouter
-key resolution (config → env → omp auth store) applies — Hermes does not need
-its own OpenRouter key.
-
 ### The OpenRouter key
 
-There should be exactly one OpenRouter key on the machine, and omp already owns
-a credential store. Resolution order:
-
-1. `openrouter.apiKey` in `$AUTO_MODEL_ROUTER_HOME/config.yml`
-2. `OPENROUTER_API_KEY` (including any `.env` omp loaded into the environment)
-3. **omp's own auth store** — `~/.omp/agent/agent.db`, provider `openrouter`
-
-So `/login openrouter` inside omp is sufficient setup; nothing needs copying.
-The store is opened read-only and never written: omp owns it, including OAuth
-refresh. An expired OAuth access token is rejected rather than sent, because
-refreshing is omp's job and a stale bearer just burns a turn on a 401. Under
+**omp does not need to be authenticated to OpenRouter.** On a routed turn omp
+never calls OpenRouter directly: the embed extension registers the
+`auto-model-router` provider with a placeholder bearer (`embedded`) pointing at
+the in-process router, and the router holds the real OpenRouter key and makes
+the upstream call. omp only needs to see that the provider "has credentials",
+which the placeholder satisfies.
+
+There should be exactly one OpenRouter key on the machine. The router resolves
+it in this order:
+
+1. `openrouter.apiKey` in `$AUTO_MODEL_ROUTER_HOME/config.yml` — router-owned,
+   never enters omp's environment. Set it with `auto-model-router config` or by
+   hand.
+2. `OPENROUTER_API_KEY` in the environment omp launches from (including any
+   `.env` omp loaded).
+3. **omp's own auth store** — `~/.omp/agent/agent.db`, provider `openrouter`, so
+   `/login openrouter` inside omp is sufficient and nothing needs copying.
+
+Options 1–2 give the router its own key with omp left unauthenticated; option 3
+is a zero-config convenience for when you *have* logged omp in. The store is
+opened read-only and never written: omp owns it, including OAuth refresh. An
+expired OAuth access token is rejected rather than sent, because refreshing is
+omp's job and a stale bearer just burns a turn on a 401. Under
 `OMP_AUTH_BROKER_URL` the local store is not consulted at all, since a broker
 replaces it.
 
@@ -307,7 +379,8 @@ disk and back up the previous file to a timestamped `.bak`.
 | --- | --- | --- |
 | `OPENROUTER_API_KEY` | OpenRouter key (overrides the auth store). | — |
 | `AUTO_MODEL_ROUTER_HOME` | Config + database directory. | `~/.auto-model-router` |
-| `AUTO_MODEL_ROUTER_PORT` | Pin a specific bind port (rarely needed; the embedded router picks a free one otherwise). | OS-assigned |
+| `AUTO_MODEL_ROUTER_HOST` | Bind address override. | `127.0.0.1` |
+| `AUTO_MODEL_ROUTER_LOG` | Log level: `silent`/`error`/`warn`/`info`/`debug`. | `info` |
 | `AUTO_MODEL_ROUTER_LOG` | Log level: `silent`/`error`/`warn`/`info`/`debug`. | `info` |
 | `AUTO_MODEL_ROUTER_DB` | Override the ledger path. | `$AUTO_MODEL_ROUTER_HOME/router.db` |
 | `AUTO_MODEL_ROUTER_URL` | Toast/base URL override (the toast reads the shared port file first). | — |
@@ -472,8 +545,15 @@ on each other:
 - **Per-harness daily budget** — each harness sends an `X-Omp-Harness` header
   (from the provider block's `headers:`), and the router scopes the rolling
   24h `perDayUsd` ceiling to it. One harness can't exhaust the day for another.
-- **Per-harness toasts** — set `OMP_HARNESS_ID` to the same value so the
-  extension only toasts that harness's model choices.
+- **Per-session toasts** — the toast surfaces only the decisions made by *its
+  own* omp session. The embed extension tags every request with an
+  `X-Omp-Session` header (`ctx.sessionManager.getSessionId()`), the router
+  records it on each ledger row, and the toast filters on it. Two concurrent
+  interactive sessions — even of the same harness — never surface each other's
+  model choices. This needs no configuration.
+- **Per-harness toasts** — additionally set `OMP_HARNESS_ID` to the same value
+  so the extension only toasts that harness's model choices. Session scoping is
+  finer-grained; harness scoping still applies on top when set.
 
 Configure a harness by setting `server.harnessId`; set the same id in that
 harness's `OMP_HARNESS_ID` env var.
@@ -509,7 +589,8 @@ bound, even though it changes every session.
 The toast logic is a pure, unit-tested module
 (`omp-extension/toast-logic.ts`, covered by `test/toast-logic.test.ts`): it
 toasts only decisions newer than the last seen one, skips `wasted` escalation
-attempts, and prefers the actual serving slug over the requested one.
+attempts, prefers the actual serving slug over the requested one, and filters
+to the toast's own omp session id (and harness id, when set).
 
 ---