auto-model-router 0.3.3 → 0.4.0

package/.omp-plugin/marketplace.json

@@ -7,14 +7,14 @@
   },
   "metadata": {
     "description": "auto-model-router: a local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter",
-    "version": "0.3.3",
+    "version": "0.4.0",
     "pluginRoot": "."
   },
   "plugins": [
     {
       "name": "auto-model-router",
       "description": "Local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter. Runs in-process, routes per turn by price and task complexity, with budget caps, mid-stream escalation, and cache-aware hysteresis.",
-      "version": "0.3.3",
+      "version": "0.4.0",
       "author": {
         "name": "drewappling",
         "email": "drewappling@gmail.com"

package/README.md

@@ -500,10 +500,12 @@ What it shows, for the window:
 | Block | Columns |
 | --- | --- |
 | totals | spend, dispatches, conversations, $/dispatch, prompt and completion tokens, cache hit rate, model switches, escalations, failovers, errors (aborted separately) |
+| prompt anatomy | mean share of prompt bytes by role (tool results, assistant, user, system), tool schemas beside them, the older half of the conversation, and tool results older than the newest 20 messages — what compaction can reach. Recorded per turn from v0.3.5. |
 | providers | per upstream (`openrouter`, `ollama`): dispatches, spend, share, cache hit, mean TTFT, tokens/s, escalations, errors |
-| models | per served slug (top 12 by spend): the same plus the tier mix it was routed for |
+| models | per served slug (top 12 by spend): the same plus user feedback (`+good/-bad` from `/router good\|bad`) and the tier mix it was routed for |
 | tiers | per tier: dispatches, spend, share, cache hit, mean prompt tokens, escalations |
 | by day | UTC calendar days: dispatches, spend, cache hit |
+| same traffic on one model | the window's tokens priced on each `report.baselines` model at list price with the window's cache hit rate, and what share the router saved against it |
 
 Spend follows the ledger's rule — the provider's reported cost when it gave
 one, else the usage-priced figure the router computed, else the forecast.
@@ -530,6 +532,10 @@ Status); the subcommands go straight there:
 | `/router config` | Section picker over **every** config key: Server, OpenRouter, Ollama Cloud, Benchmarks, Tiers, Tasks, Filters, Classifier, Escalation, Hysteresis, Exploration, Cache, Compaction, Context (agentdox), Budget, Ledger, Logging, Profiles. Only `ollama.prices` and `ollama.twins` (maps) stay YAML-only. |
 | `/router report` | Usage analytics in a fullscreen hub styled like `/models`: pick a view in the sidebar, set the window (24h / 7d / 30d / 90d) and the harness scope there too. `/router report 30d --all` presets them. See [Usage reports](#usage-reports). |
 | `/router status` | The router's `/health`: key sources, catalog size and age, Ollama availability, plan usage and cost bias, agentdox bridge. |
+| `/router why` | Explain this session's last routed turn: model and provider, tier, classification source and confidence, cost, cache hit, latency, the full decision trail and classifier reasons, any feedback already given. |
+| `/router good` / `/router bad [note]` | Judge that turn. Recorded against the model that served it (`POST /v1/router/feedback`), shown per model in the report's `feedback` column, and the label the de-escalation work needs. `/router feedback good\|bad` is the same. |
+| `/router pin <model\|off>` | Route this session to one model until cleared (admitted past price, quality and trust filters; tool support and context window still apply). Escalations and failovers after the first attempt still run. |
+| `/router tier <tier\|off> [turns]` | Force a tier for N committed turns (default 10; 0 = until cleared). Shown with no argument. Overrides are per omp session, live in the router process only, and lapse after 12 idle hours. |
 
 Picking a section lists its fields with their current values (pending edits
 marked), so you see the settings before choosing one to change. Each field
@@ -663,6 +669,8 @@ Each task (`coding`, `vision`, `documentation`, `data`, `chat`) is a
 | `contextHeadroom` | `1.25` | Fraction of context kept free (a model must fit prompt × this). |
 | `latencyWeight` | `0` | How hard to penalise slow models in scoring (soft multiplier on effective cost). `0` disables it. |
 | `latencyMinSamples` | `20` | Streamed samples before latency is judged against a model. |
+| `cacheReliabilityMinSamples` | `10` | Warm-expected samples before a model's observed cache hit rate discounts its "stay warm" price in the stay/switch comparison. A model whose cache misses when it should be warm (measured: 5-6% on glm/gemini, 11% on ling, 50% on nex) is kept less eagerly. `0` assumes every cache is reliable. |
+| `latencyWeightContinuation` | unset | Latency weight on tool-result continuations (the agent loop's own follow-ups). Unset ⇒ `latencyWeight` everywhere; lower it to spend speed only where a person waits on first token. |
 | `maxExpectedWaitMs` | unset | Absolute expected-wait ceiling (ms): a hard drop for models *proven* slower (≥ `latencyMinSamples`), regardless of price. The soft penalty is multiplicative and capped, so it cannot demote a slow-but-cheap model — this can. New models keep their cold-start turns; relaxed with trust in tier rescue. Undefined ⇒ off. |
 | `escalationCostWeight` | `0` | Price a model's measured escalation rate at what an escalated retry actually bills (the ledger's $/prompt-token of `attempt > 0` rows), 0–1. The trust divisor reads a 4% escalation rate as a 4% surcharge; the real cost is a whole re-dispatch on the next tier's model. `0` disables the term. |
 
@@ -671,6 +679,7 @@ Each task (`coding`, `vision`, `documentation`, `data`, `chat`) is a
 | Key | Default | Meaning |
 | --- | --- | --- |
 | `ambiguityThreshold` | `0.6` | Below this heuristic confidence, the adjudicator model decides the tier. |
+| `learnedModelPath` | unset | A model written by `bun tools/train-classifier.ts` (logistic regression over the ledger's recorded features, label = the turn escalated). When set, every decision records `learned: p(escalate)=…`. Advisory only: it never moves a tier until replay shows it should. |
 | `model` | `qwen/qwen3.7-flash` | Adjudicator model slug. |
 | `maxCostFraction` | `0.02` | Adjudicator cost cap as a fraction of the turn's budget. |
 | `maxCostUsd` | `0.002` | Absolute adjudicator cost cap, USD. |
@@ -709,6 +718,7 @@ event has already arrived is treated as a completed turn, not an error.
 | `holdTurnsAfterEscalation` | `4` | Hold longer after an escalation. |
 | `switchMargin` | `1.3` | Switching must beat the warm-cache discount by this factor. Lower = switch away from a warm model more readily. |
 | `switchHorizonTurns` | `1` | Turns the stay/switch comparison is amortised over: `H × stayWarm` vs `switchCold + (H − 1) × newWarm`. `1` is the one-turn comparison, which can keep a dear model warm indefinitely when the cheaper winner is itself dear cold; a small `H` lets a switch that pays for itself within a few turns go ahead. |
+| `confirmUpgradesBelowConfidence` | `0.6` | A heuristic tier upgrade classified below this confidence waits one turn while the current model's cache is warm; a second consecutive upgrade classification confirms it. Escalations, explicit high reasoning and failing tool loops bypass the wait. `0` disables. Measured: 65 of 67 moderate→hard upgrades in a week bounced back within 3 turns, each paying a cold hard-tier read of a ~120k prompt. |
 | `cacheWarmTtlMs` | `300000` (5 min) | How long a model's prompt cache is considered warm. |
 | `maxDowngradePerTurn` | `1` | Max tiers a turn may drop in one step (avoids quality cliffs). |
 | `breakHoldOnMechanical` | `false` | Let a tool-result continuation that classifies *below* the held tier escape the hold (still bounded by `maxDowngradePerTurn`). Worth enabling when the held tier is expensive. |
@@ -748,6 +758,7 @@ shrunk results stay shrunk (rewriting them would break the prompt cache).
 | `perTurnUsd` | unset | Per-turn cap (checked against the cold forecast). |
 | `perConversationUsd` | unset | Per-conversation cap. |
 | `perDayUsd` | unset | Rolling 24h cap, scoped per harness when `harnessId` is set. |
+| `perMonthUsd` | unset | Calendar-month (UTC) target. Paced: the daily cap becomes min(`perDayUsd`, remaining ÷ days left), so a month running ahead tightens automatically. The breach reason names the pace. |
 | `onExceeded` | `downgrade` | `downgrade` = pick the cheapest viable model; `reject` = fail the turn. |
 
 ### `profiles` — the virtual models omp sees
@@ -763,6 +774,12 @@ Each profile is a complete entry (arrays replace wholesale):
 | `maxTokens` | `32000` | Advertised max output tokens. |
 | `budget` | unset | Per-profile budget overrides. |
 
+### `report` — usage-report options
+
+| Key | Default | Meaning |
+| --- | --- | --- |
+| `baselines` | `anthropic/claude-opus-5`, `anthropic/claude-sonnet-5` | Models the report prices the window's traffic on as a single-model counterfactual. Unknown slugs are skipped. |
+
 ### `ledger` — cost measurement
 
 | Key | Default | Meaning |
@@ -826,7 +843,13 @@ What happens once it is on:
   flags these rows (`usage.cachedEstimated`) and reports show their cache
   rate as `~N%`. `/router status` shows ollama.com's own dollar reading as
   the cross-check: the plan is read from `POST /api/me` and its published
-  allowance applied (`planCreditsUsd` overrides it).
+  allowance applied (`planCreditsUsd` overrides it). **The estimate is
+  calibrated against the meter:** every usage poll records the meter beside
+  the ledger's Ollama total, and once the span carries ~$0.50 of metered
+  spend the ratio (clamped to 0.5–2×) scales every new Ollama cost the
+  router records, so the ledger tracks the bill rather than the list price.
+  Status shows the factor and, at the last week's burn, how many days of
+  credits remain.
 - **Same economics, same failover.** Candidates from both providers are ranked
   together; `costBias` tilts the comparison while a plan's included credits
   would otherwise go unused. **Credit-aware by default:** the router reads the

package/omp-extension/report-logic.ts

@@ -76,6 +76,8 @@ export interface HealthSnapshot {
 		lastTrip?: { kind?: string; atMs?: number; message?: string } | null;
 		usage?: { monthlyUsedFraction?: number | null; activityCostUsd?: number | null; plan?: string | null; fetchedAtMs?: number | null } | null;
 		meter?: { usedUsd?: number; creditsUsd?: number; plan?: string | null } | null;
+		calibration?: { factor?: number; meterDeltaUsd?: number; ledgerDeltaUsd?: number; spanHours?: number } | null;
+		runway?: { dailyBurnUsd?: number; creditsLeftUsd?: number; days?: number | null } | null;
 		costBias?: { configured?: number; effective?: number; biasUntilUsage?: number };
 	} | null;
 	catalog?: {
@@ -111,8 +113,99 @@ export function renderStatus(baseUrl: string, h: HealthSnapshot, nowMs = Date.no
 		const bias = o.costBias === undefined ? "" : ` · cost bias ×${o.costBias.effective ?? o.costBias.configured ?? 1} (until ${((o.costBias.biasUntilUsage ?? 1) * 100).toFixed(0)}%)`;
 		const trip = o.lastTrip !== undefined && o.lastTrip !== null ? ` · last trip ${o.lastTrip.kind ?? "?"}${o.lastTrip.atMs ? ` ${mins(nowMs - o.lastTrip.atMs)} ago` : ""}` : "";
 		out.push(`ollama cloud: ${o.models ?? 0} models · ${avail} · key ${o.apiKeySource ?? "?"} · ${usage}${bias}${trip}`);
+		const c = o.calibration;
+		const rw = o.runway;
+		const calText = c !== undefined && c !== null && c.factor !== undefined ? `ledger estimate ×${c.factor.toFixed(2)} to match the meter (${(c.spanHours ?? 0).toFixed(0)}h span)` : "ledger estimate uncalibrated (needs ~$0.50 of metered spend)";
+		const rwText = rw !== undefined && rw !== null ? ` · burn $${(rw.dailyBurnUsd ?? 0).toFixed(2)}/day · ${rw.days === null || rw.days === undefined ? "credits left: unknown burn" : `~${Math.round(rw.days)} days of credits left`}` : "";
+		if (o.meter !== undefined && o.meter !== null) out.push(`ollama billing: ${calText}${rwText}`);
 	}
 	const a = h.agentdox;
 	out.push(a === undefined || a === null ? "agentdox: off" : `agentdox: ${a.url ?? "?"} scope ${a.defaultScope ?? "?"}${a.recordTurns === true ? " · recording turns" : ""}`);
 	return out.join("\n");
 }
+
+// ---------------------------------------------------------------------------
+// /router why, feedback, pin, tier
+// ---------------------------------------------------------------------------
+
+/** The decision fields `/router why` renders, as `/v1/router/decisions` returns them. */
+export interface WhyEntry {
+	id: string;
+	createdAtMs: number;
+	turn: number;
+	slug: string;
+	servedSlug: string | null;
+	tier: string;
+	classificationSource: string;
+	confidence: number | null;
+	task: string | null;
+	reasons: string[];
+	classifierReasons: string[] | null;
+	reportedUsd: number | null;
+	predictedUsd: number;
+	usage: { promptTokens: number; cachedTokens: number; completionTokens: number; cachedEstimated?: boolean };
+	latencyMs: number;
+	ttftMs: number | null;
+	attempt: number;
+	escalationSignal: string | null;
+	feedback?: Array<{ verdict: string; note: string; createdAtMs: number }>;
+}
+
+const money = (v: number): string => (v >= 1 ? `$${v.toFixed(2)}` : `$${v.toFixed(4)}`);
+
+/** Plain-text explanation of one routed turn. */
+export function renderWhy(e: WhyEntry, nowMs = Date.now()): string {
+	const served = e.servedSlug ?? e.slug;
+	const provider = served.startsWith("ollama/") ? "ollama" : "openrouter";
+	const cache = e.usage.promptTokens > 0 ? `${((100 * e.usage.cachedTokens) / e.usage.promptTokens).toFixed(0)}%${e.usage.cachedEstimated === true ? " est." : ""}` : "–";
+	const age = Math.max(0, nowMs - e.createdAtMs);
+	const ago = age < 60_000 ? `${Math.round(age / 1000)}s ago` : age < 3_600_000 ? `${Math.round(age / 60_000)}m ago` : `${(age / 3_600_000).toFixed(1)}h ago`;
+	const out: string[] = [];
+	out.push(`turn ${e.turn} · ${ago} · ${provider} · ${served}${e.servedSlug !== null && e.servedSlug !== e.slug ? ` (asked ${e.slug})` : ""} [${e.tier}${e.attempt > 0 ? `, attempt ${e.attempt + 1}` : ""}]`);
+	out.push(
+		`classified ${e.tier} by ${e.classificationSource}${e.confidence !== null ? ` at ${(e.confidence * 100).toFixed(0)}% confidence` : ""}${e.task !== null ? ` · task ${e.task}` : ""}`,
+	);
+	out.push(
+		`cost ${money(e.reportedUsd ?? e.predictedUsd)}${e.reportedUsd === null ? " (forecast)" : ""} · prompt ${e.usage.promptTokens.toLocaleString("en-US")} tok (cache ${cache}) · completion ${e.usage.completionTokens.toLocaleString("en-US")} tok · ttft ${e.ttftMs === null ? "–" : `${(e.ttftMs / 1000).toFixed(1)}s`} · total ${(e.latencyMs / 1000).toFixed(1)}s`,
+	);
+	if (e.escalationSignal !== null) out.push(`escalation signal: ${e.escalationSignal}`);
+	out.push("", "decision trail:");
+	for (const r of e.reasons) out.push(`  - ${r}`);
+	if (e.classifierReasons !== null && e.classifierReasons.length > 0) {
+		out.push("", "classifier:");
+		for (const r of e.classifierReasons) out.push(`  - ${r}`);
+	}
+	if (e.feedback !== undefined && e.feedback.length > 0) {
+		out.push("", "feedback:");
+		for (const f of e.feedback) out.push(`  - ${f.verdict}${f.note !== "" ? `: ${f.note}` : ""}`);
+	}
+	out.push("", `judge it: /router good  or  /router bad [note]`);
+	return out.join("\n");
+}
+
+export type OverrideRequest = { kind: "pin"; slug: string | null } | { kind: "tier"; tier: string | null; turns: number } | { kind: "show" } | { kind: "clear" } | { kind: "error"; message: string };
+
+const TIERS = ["trivial", "simple", "moderate", "hard"];
+
+/** Parses `/router pin <slug|off>` and `/router tier <tier|off> [turns]`. */
+export function parseOverrideArgs(verb: "pin" | "tier", text: string): OverrideRequest {
+	const parts = text.trim().split(/\s+/).filter((t) => t !== "");
+	const first = parts[0] ?? "";
+	if (first === "") return { kind: "show" };
+	if (["off", "clear", "none"].includes(first.toLowerCase())) return verb === "pin" ? { kind: "pin", slug: null } : { kind: "tier", tier: null, turns: 0 };
+	if (verb === "pin") return { kind: "pin", slug: first };
+	const tier = first.toLowerCase();
+	if (!TIERS.includes(tier)) return { kind: "error", message: `unknown tier "${first}" (${TIERS.join(" | ")} | off)` };
+	const turns = parts[1] === undefined ? 10 : Number.parseInt(parts[1], 10);
+	if (!Number.isInteger(turns) || turns < 0) return { kind: "error", message: `turns must be a whole number (0 = until cleared), got "${parts[1]}"` };
+	return { kind: "tier", tier, turns };
+}
+
+/** One-line description of a live override, for notifications. */
+export function describeOverride(o: { slug: string | null; tier: string | null; turnsLeft: number } | null): string {
+	if (o === null || (o.slug === null && o.tier === null)) return "no override on this session";
+	const parts: string[] = [];
+	if (o.slug !== null) parts.push(`pinned to ${o.slug}`);
+	if (o.tier !== null) parts.push(`tier forced to ${o.tier}`);
+	return `${parts.join(", ")} · ${o.turnsLeft === 0 ? "until cleared" : `${o.turnsLeft} turn${o.turnsLeft === 1 ? "" : "s"} left`}`;
+}

package/omp-extension/router-configure.ts

@@ -12,6 +12,15 @@
  *                           them. Headless sessions get the text instead.
  *   /router status          the router's /health: keys, catalog, Ollama
  *                           availability and plan usage, agentdox.
+ *   /router why             explain this session's last routed turn: model,
+ *                           tier, confidence, cost, cache, the decision trail.
+ *   /router good | bad [note]
+ *                           judge that turn; recorded against the model that
+ *                           served it (/router feedback good|bad is the same).
+ *   /router pin <model|off> route this session to one model until cleared.
+ *   /router tier <tier|off> [turns]
+ *                           force a tier for N turns (default 10; 0 = until
+ *                           cleared). Escalations and failovers still apply.
  *
  * Configuration walks the same sections and fields as `auto-model-router
  * config` (reusing `WIZARD_SECTIONS` / `PROFILE_FIELDS` from the router's
@@ -49,7 +58,17 @@ import { matchesKey, truncateToWidth, visibleWidth } from "@oh-my-pi/pi-tui";
 
 import { editProfile, editSectionMenu, type ConfigUi, type SelectOption } from "./configure-logic.ts";
 import { ReportHub } from "./report-hub.ts";
-import { fetchReport, parseReportArgs, renderStatus, type HealthSnapshot, type ReportRequest } from "./report-logic.ts";
+import {
+	describeOverride,
+	fetchReport,
+	parseOverrideArgs,
+	parseReportArgs,
+	renderStatus,
+	renderWhy,
+	type HealthSnapshot,
+	type ReportRequest,
+	type WhyEntry,
+} from "./report-logic.ts";
 import { routerAuthHeaders, routerBaseUrl } from "./router-url.ts";
 
 // This harness's id, matching the X-Omp-Harness header the router records.
@@ -76,10 +95,28 @@ export default function (pi: ExtensionAPI): void {
 				case "status":
 				case "health":
 					return status(pi, ctx);
+				case "why":
+				case "explain":
+					return why(pi, ctx);
+				case "good":
+				case "bad":
+					return feedback(ctx, verb.toLowerCase() as "good" | "bad", tail);
+				case "feedback": {
+					const [v = "", ...note] = rest;
+					if (v !== "good" && v !== "bad") {
+						ctx.ui.notify("usage: /router feedback good|bad [note]", "warn");
+						return;
+					}
+					return feedback(ctx, v, note.join(" "));
+				}
+				case "pin":
+					return override(ctx, "pin", tail);
+				case "tier":
+					return override(ctx, "tier", tail);
 				case "":
 					break;
 				default:
-					ctx.ui.notify(`unknown /router subcommand "${verb}" (config | report [7d] [--all] | status)`, "warn");
+					ctx.ui.notify(`unknown /router subcommand "${verb}" (config | report | status | why | good | bad | pin | tier)`, "warn");
 					return;
 			}
 
@@ -87,11 +124,15 @@ export default function (pi: ExtensionAPI): void {
 				{ label: "Configure", description: "edit any router setting" },
 				{ label: "Report", description: "usage analytics; window and scope adjustable inside" },
 				{ label: "Status", description: "keys, catalog, Ollama, agentdox" },
+				{ label: "Why", description: "explain this session's last routed turn" },
+				{ label: "Override", description: "pin a model or force a tier for this session" },
 			]);
 			if (chosen === undefined) return;
 			if (chosen === "Configure") return configure(ctx);
 			if (chosen === "Status") return status(pi, ctx);
 			if (chosen === "Report") return report(pi, ctx, "");
+			if (chosen === "Why") return why(pi, ctx);
+			if (chosen === "Override") return override(ctx, "tier", "");
 		},
 	});
 }
@@ -173,6 +214,91 @@ async function report(pi: ExtensionAPI, ctx: ExtensionContext, argText: string):
 	post(pi, renderUsageReport(data));
 }
 
+/** POST JSON to the router; throws on a non-2xx with the router's message. */
+async function routerPost<T>(path: string, body: unknown): Promise<T> {
+	const res = await fetch(`${routerBaseUrl()}${path}`, {
+		method: "POST",
+		headers: { ...routerAuthHeaders(), "content-type": "application/json" },
+		body: JSON.stringify(body),
+		signal: AbortSignal.timeout(5_000),
+	});
+	const json = (await res.json().catch(() => null)) as Record<string, unknown> | null;
+	if (!res.ok) {
+		const err = json?.error as { message?: string } | undefined;
+		throw new Error(err?.message ?? `router returned ${res.status}`);
+	}
+	return json as T;
+}
+
+async function why(pi: ExtensionAPI, ctx: ExtensionContext): Promise<void> {
+	const session = ctx.sessionManager.getSessionId();
+	try {
+		const res = await fetch(`${routerBaseUrl()}/v1/router/decisions?limit=1&session=${encodeURIComponent(session)}`, {
+			headers: routerAuthHeaders(),
+			signal: AbortSignal.timeout(5_000),
+		});
+		if (!res.ok) throw new Error(`router returned ${res.status}`);
+		const body = (await res.json()) as { entries: WhyEntry[] };
+		const entry = body.entries[0];
+		if (entry === undefined) {
+			ctx.ui.notify("no routed turn in this session yet", "info");
+			return;
+		}
+		post(pi, renderWhy(entry));
+	} catch (err) {
+		ctx.ui.notify(`router unreachable: ${err instanceof Error ? err.message : String(err)}`, "error");
+	}
+}
+
+async function feedback(ctx: ExtensionContext, verdict: "good" | "bad", note: string): Promise<void> {
+	try {
+		const r = await routerPost<{ slug: string; tier: string }>("/v1/router/feedback", { ompSessionId: ctx.sessionManager.getSessionId(), verdict, note });
+		ctx.ui.notify(`recorded ${verdict} for ${r.slug} [${r.tier}]${note !== "" ? `: ${note}` : ""}`, "info");
+	} catch (err) {
+		ctx.ui.notify(`feedback not recorded: ${err instanceof Error ? err.message : String(err)}`, "error");
+	}
+}
+
+async function override(ctx: ExtensionContext, verb: "pin" | "tier", text: string): Promise<void> {
+	const session = ctx.sessionManager.getSessionId();
+	let req = parseOverrideArgs(verb, text);
+	if (req.kind === "show" && verb === "tier" && text === "") {
+		// Menu path: pick a tier interactively.
+		const chosen = await ctx.ui.select("Force a tier for this session", [
+			{ label: "trivial", description: "cheapest models" },
+			{ label: "simple", description: "" },
+			{ label: "moderate", description: "" },
+			{ label: "hard", description: "strongest models" },
+			{ label: "off", description: "clear any override" },
+		]);
+		if (chosen === undefined) return;
+		req = parseOverrideArgs("tier", chosen);
+	}
+	if (req.kind === "error") {
+		ctx.ui.notify(req.message, "warn");
+		return;
+	}
+	try {
+		if (req.kind === "show") {
+			const res = await fetch(`${routerBaseUrl()}/v1/router/override?session=${encodeURIComponent(session)}`, { headers: routerAuthHeaders(), signal: AbortSignal.timeout(5_000) });
+			const body = (await res.json()) as { override: { slug: string | null; tier: string | null; turnsLeft: number } | null };
+			ctx.ui.notify(describeOverride(body.override), "info");
+			return;
+		}
+		const payload: Record<string, unknown> = { ompSessionId: session };
+		if (req.kind === "clear") payload.clear = true;
+		else if (req.kind === "pin") payload.slug = req.slug;
+		else {
+			payload.tier = req.tier;
+			payload.turns = req.turns;
+		}
+		const r = await routerPost<{ override: { slug: string | null; tier: string | null; turnsLeft: number } | null }>("/v1/router/override", payload);
+		ctx.ui.notify(describeOverride(r.override), "info");
+	} catch (err) {
+		ctx.ui.notify(`override not applied: ${err instanceof Error ? err.message : String(err)}`, "error");
+	}
+}
+
 async function status(pi: ExtensionAPI, ctx: ExtensionContext): Promise<void> {
 	try {
 		post(pi, await loadStatus());

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "auto-model-router",
-  "version": "0.3.3",
+  "version": "0.4.0",
   "private": false,
   "description": "Local cost/complexity-aware model router for Oh My Pi, backed by OpenRouter",
   "type": "module",

package/src/catalog/ollama-catalog.ts

@@ -22,6 +22,7 @@
  * price and twin; the dispatch client strips the `ollama/` prefix.
  */
 
+import type { Database } from "bun:sqlite";
 import type { OllamaConfig } from "../config/types.ts";
 import type { Logger } from "../util/log.ts";
 import { normalizeModelKey } from "./benchmark-feeds.ts";
@@ -202,13 +203,35 @@ export function ollamaApiRoot(baseUrl: string): string {
 	return baseUrl.replace(/\/+$/, "").replace(/\/v1$/, "");
 }
 
-export function createOllamaCatalog(cfg: OllamaConfig, log: Logger, fetchImpl: FetchLike = fetch): OllamaCatalogSource {
+/** The persisted Ollama model set, or [] when none has been built yet. */
+export function loadOllamaCatalogCache(db: Database): { models: CatalogModel[]; fetchedAtMs: number } {
+	const row = db.query("SELECT payload, fetched_at_ms FROM ollama_catalog_cache WHERE id = 1").get() as
+		| { payload: string; fetched_at_ms: number }
+		| null;
+	if (row === null) return { models: [], fetchedAtMs: 0 };
+	try {
+		const parsed = JSON.parse(row.payload) as unknown;
+		return { models: Array.isArray(parsed) ? (parsed as CatalogModel[]) : [], fetchedAtMs: row.fetched_at_ms };
+	} catch {
+		return { models: [], fetchedAtMs: 0 };
+	}
+}
+
+export function createOllamaCatalog(cfg: OllamaConfig, log: Logger, fetchImpl: FetchLike = fetch, db?: Database): OllamaCatalogSource {
 	const root = ollamaApiRoot(cfg.baseUrl);
 	const direct = isOllamaDotCom(cfg.baseUrl);
 	const headers: Record<string, string> = {};
 	if (cfg.apiKey !== "") headers.authorization = `Bearer ${cfg.apiKey}`;
-	let models: CatalogModel[] = [];
+	// Hydrate from disk so a restart peeks a real set before the first listing;
+	// listedAtMs stays 0 so the first get() still refreshes.
+	let models: CatalogModel[] = db === undefined ? [] : loadOllamaCatalogCache(db).models;
 	let listedAtMs = 0;
+	const persist = db === undefined
+		? null
+		: db.query(
+				`INSERT INTO ollama_catalog_cache (id, payload, fetched_at_ms) VALUES (1, ?, ?)
+				 ON CONFLICT(id) DO UPDATE SET payload = excluded.payload, fetched_at_ms = excluded.fetched_at_ms`,
+			);
 	let inflight: Promise<CatalogModel[]> | null = null;
 	// `/api/show` results are stable per id; fetched once per process.
 	const shown = new Map<string, { contextLength: number | null; capabilities: string[] }>();
@@ -257,6 +280,11 @@ export function createOllamaCatalog(cfg: OllamaConfig, log: Logger, fetchImpl: F
 				log.warn("ollama listing yielded no priced cloud models; keeping the previous set", { listed: listings.length });
 			} else {
 				models = built;
+				try {
+					persist?.run(JSON.stringify(models), Date.now());
+				} catch (err) {
+					log.debug("ollama catalog persist failed", { error: err instanceof Error ? err.message : String(err) });
+				}
 			}
 			listedAtMs = Date.now();
 		} catch (err) {

package/src/cli/config-wizard.ts

@@ -190,7 +190,9 @@ export const WIZARD_SECTIONS: readonly SectionSpec[] = [
 			{ path: "filters.latencyReferenceMs", label: "Latency reference TTFT", kind: "number", min: 1, hint: "ms" },
 			{ path: "filters.latencyReferenceTokensPerSec", label: "Latency reference speed", kind: "number", min: 1, hint: "tok/s" },
 			{ path: "filters.latencyMinSamples", label: "Latency min samples", kind: "number", min: 0 },
+			{ path: "filters.cacheReliabilityMinSamples", label: "Cache reliability min samples", kind: "number", min: 0, hint: "0=assume caches reliable" },
 			{ path: "filters.maxExpectedWaitMs", label: "Max expected wait", kind: "number", min: 1, optional: true, hint: "ms, hard ceiling" },
+			{ path: "filters.latencyWeightContinuation", label: "Latency weight on tool continuations", kind: "number", min: 0, optional: true, hint: "unset = latencyWeight" },
 			{ path: "filters.escalationCostWeight", label: "Escalation cost weight", kind: "number", min: 0, max: 1 },
 		],
 	},
@@ -199,6 +201,7 @@ export const WIZARD_SECTIONS: readonly SectionSpec[] = [
 		fields: [
 			{ path: "classifier.ambiguityThreshold", label: "Ambiguity threshold", kind: "number", min: 0, max: 1 },
 			{ path: "classifier.model", label: "Adjudicator model", kind: "string", optional: true },
+			{ path: "classifier.learnedModelPath", label: "Learned risk model file", kind: "string", optional: true, hint: "from tools/train-classifier.ts; advisory" },
 			{ path: "classifier.maxCostFraction", label: "Max cost fraction", kind: "number", min: 0, max: 1 },
 			{ path: "classifier.maxCostUsd", label: "Max adjudication cost $", kind: "number", min: 0 },
 			{ path: "classifier.timeoutMs", label: "Adjudicator timeout", kind: "number", min: 1, hint: "ms" },
@@ -232,6 +235,7 @@ export const WIZARD_SECTIONS: readonly SectionSpec[] = [
 			{ path: "hysteresis.holdTurnsAfterEscalation", label: "Hold turns after escalation", kind: "number", min: 0 },
 			{ path: "hysteresis.switchMargin", label: "Switch margin", kind: "number", min: 0 },
 			{ path: "hysteresis.switchHorizonTurns", label: "Switch horizon", kind: "number", min: 1, hint: "turns amortised" },
+			{ path: "hysteresis.confirmUpgradesBelowConfidence", label: "Confirm upgrades below confidence", kind: "number", min: 0, max: 1, hint: "0=off; low-confidence tier-ups wait a turn" },
 			{ path: "hysteresis.cacheWarmTtlMs", label: "Cache-warm TTL", kind: "number", min: 0, hint: "ms" },
 			{ path: "hysteresis.maxDowngradePerTurn", label: "Max downgrade per turn", kind: "number", min: 0, hint: "tiers" },
 			{ path: "hysteresis.breakHoldOnMechanical", label: "Break hold on mechanical turns", kind: "boolean" },
@@ -299,9 +303,14 @@ export const WIZARD_SECTIONS: readonly SectionSpec[] = [
 			{ path: "budget.perTurnUsd", label: "Per-turn cap $", kind: "number", min: 0, optional: true },
 			{ path: "budget.perConversationUsd", label: "Per-conversation cap $", kind: "number", min: 0, optional: true },
 			{ path: "budget.perDayUsd", label: "Per-day cap $", kind: "number", min: 0, optional: true },
+			{ path: "budget.perMonthUsd", label: "Per-month target $", kind: "number", min: 0, optional: true, hint: "paced into a daily cap" },
 			{ path: "budget.onExceeded", label: "On exceeded", kind: "enum", options: ["downgrade", "reject"] },
 		],
 	},
+	{
+		title: "Report",
+		fields: [{ path: "report.baselines", label: "Counterfactual baseline models", kind: "stringArray", hint: "comma-separated slugs" }],
+	},
 	{
 		title: "Ledger",
 		fields: [

package/src/cli/report.ts

@@ -7,7 +7,8 @@
 
 import { existsSync } from "node:fs";
 import { loadConfig } from "../config/load.ts";
-import { buildUsageReport, renderUsageReport } from "../cost/report.ts";
+import { createCatalog } from "../catalog/openrouter-catalog.ts";
+import { baselinePrices, buildUsageReport, renderUsageReport } from "../cost/report.ts";
 import { openDb } from "../util/sqlite.ts";
 import { configOpts, flagInt, flagString, type CliArgs } from "./args.ts";
 
@@ -28,7 +29,11 @@ export async function reportCommand(args: CliArgs): Promise<void> {
 
 	const db = openDb(cfg.ledger.path);
 	try {
-		const report = buildUsageReport(db, { windowDays: days, harnessId });
+		// Baseline prices from the cached catalog: no network for a report.
+		const dead = { dispatch: () => Promise.reject(new Error("offline")), complete: () => Promise.reject(new Error("offline")), fetchModels: () => Promise.reject(new Error("offline")), fetchModelsForUser: () => Promise.reject(new Error("offline")) };
+		const snapshot = createCatalog(cfg, dead, db).peek();
+		const baselines = baselinePrices(cfg.report.baselines, (s) => snapshot?.models.find((m) => m.slug === s));
+		const report = buildUsageReport(db, { windowDays: days, harnessId, baselines });
 		if (args.flags.has("json")) console.log(JSON.stringify(report, null, 2));
 		else console.log(renderUsageReport(report));
 	} finally {

package/src/config/defaults.ts

@@ -118,6 +118,9 @@ export const DEFAULT_CONFIG: RouterConfig = {
 		// slow models (e.g. deepseek-v4-flash ~20 tok/s) fall under it.
 		latencyReferenceTokensPerSec: 30,
 		latencyMinSamples: 20,
+		// Discount a warm model's stay price by its observed hit rate once this
+		// many warm-expected samples exist. See FilterConfig.cacheReliabilityMinSamples.
+		cacheReliabilityMinSamples: 10,
 		// Off: pricing a model's measured escalation rate at what an escalated
 		// retry actually costs changes rankings, so it is opt-in after a replay
 		// run prices it. See FilterConfig.escalationCostWeight.
@@ -127,6 +130,8 @@ export const DEFAULT_CONFIG: RouterConfig = {
 		ambiguityThreshold: 0.6,
 		// Cheapest competent slug in the catalog; adjudication prompts are tiny.
 		model: "qwen/qwen3.7-flash",
+		// Off until a model is trained; see tools/train-classifier.ts.
+		learnedModelPath: "",
 		maxCostFraction: 0.02,
 		maxCostUsd: 0.002,
 		timeoutMs: 4_000,
@@ -174,6 +179,9 @@ export const DEFAULT_CONFIG: RouterConfig = {
 		// 1 = the shipped one-turn comparison. Raise to amortise a switch over the
 		// turns that follow it; see HysteresisConfig.switchHorizonTurns.
 		switchHorizonTurns: 1,
+		// Low-confidence heuristic upgrades from a warm model wait one turn; see
+		// HysteresisConfig.confirmUpgradesBelowConfidence for the measurement.
+		confirmUpgradesBelowConfidence: 0.6,
 		// OpenRouter sticky sessions expire in 5-10 minutes.
 		cacheWarmTtlMs: 300_000,
 		maxDowngradePerTurn: 1,
@@ -278,6 +286,10 @@ export const DEFAULT_CONFIG: RouterConfig = {
 		elideSupersededReads: true,
 		collapseDuplicateResults: true,
 	},
+	report: {
+		// The frontier pair most omp users would otherwise run on.
+		baselines: ["anthropic/claude-opus-5", "anthropic/claude-sonnet-5"],
+	},
 	budget: {
 		// No caps by default; at a configured ceiling, downgrade rather than fail.
 		onExceeded: "downgrade",

package/src/config/schema.ts

@@ -96,13 +96,16 @@ const filters = z.strictObject({
 	latencyReferenceMs: z.number().positive().optional(),
 	latencyReferenceTokensPerSec: z.number().positive().optional(),
 	latencyMinSamples: z.number().int().nonnegative().optional(),
+	cacheReliabilityMinSamples: z.number().int().nonnegative().optional(),
 	maxExpectedWaitMs: z.number().positive().optional(),
+	latencyWeightContinuation: z.number().nonnegative().optional(),
 	escalationCostWeight: z.number().min(0).max(1).optional(),
 });
 
 const classifier = z.strictObject({
 	ambiguityThreshold: z.number().min(0).max(1).optional(),
 	model: z.string().min(1).optional(),
+	learnedModelPath: z.string().optional(),
 	maxCostFraction: z.number().min(0).max(1).optional(),
 	maxCostUsd: z.number().nonnegative().optional(),
 	timeoutMs: z.number().positive().optional(),
@@ -136,6 +139,7 @@ const hysteresis = z.strictObject({
 	holdTurnsAfterEscalation: z.number().int().nonnegative().optional(),
 	switchMargin: z.number().positive().optional(),
 	switchHorizonTurns: z.number().int().positive().optional(),
+	confirmUpgradesBelowConfidence: z.number().min(0).max(1).optional(),
 	cacheWarmTtlMs: z.number().nonnegative().optional(),
 	maxDowngradePerTurn: z.number().int().nonnegative().optional(),
 	breakHoldOnMechanical: z.boolean().optional(),
@@ -204,6 +208,7 @@ const budget = z.strictObject({
 	perTurnUsd: z.number().nonnegative().optional(),
 	perConversationUsd: z.number().nonnegative().optional(),
 	perDayUsd: z.number().nonnegative().optional(),
+	perMonthUsd: z.number().nonnegative().optional(),
 	onExceeded: z.enum(["downgrade", "reject"]).optional(),
 });
 
@@ -264,6 +269,7 @@ export const configInputSchema = z.strictObject({
 	compaction: compaction.optional(),
 	budget: budget.optional(),
 	profiles: z.array(profile).optional(),
+	report: z.strictObject({ baselines: z.array(z.string()).optional() }).optional(),
 	ledger: ledger.optional(),
 	adaptiveTierFloors: z.boolean().optional(),
 	adaptivePriceCeilings: z.boolean().optional(),