openmerit 0.1.1 → 0.1.2

package/README.md

@@ -32,10 +32,10 @@ pi task A → saved pi trace → extension trial job → pi trials B, C → scor
    ([`src/frontier.ts`](src/frontier.ts), 3 objectives: quality ↑, price ↓,
    latency ↓). The aggregate across all of an agent's tasks is the union of
    its task frontiers (`openmerit frontier`).
-4. **Model discovery** — the OpenRouter catalog is snapshotted and diffed;
-   newly appeared models are queued for future release experiments. The
-   per-task loop shortlists currently available pi/OpenRouter models with
-   OpenRouter benchmark signals and the existing strategist
+4. **Model discovery** — the active Pi session's scoped models (or Pi's
+   authenticated available-model registry when unscoped) are the candidate
+   pool. OpenRouter is an optional route and enrichment source: its catalog can
+   be snapshotted and its benchmark signals can improve the shortlist
    ([`src/catalog.ts`](src/catalog.ts) + [`src/daemon.ts`](src/daemon.ts)).
 5. **Informing the main harness** — recommendations land in
    `~/.openmerit/recommendations.jsonl`. The pi extension
@@ -47,8 +47,9 @@ pi task A → saved pi trace → extension trial job → pi trials B, C → scor
 
 ## Install from npm
 
-You need Node 22.18+, pi 0.85.1+, and an OpenRouter API key with available
-credits. Install the package into pi, then initialize its policy and state:
+You need Node 22.18+, pi 0.85.1+, and at least two eligible model routes
+authenticated in Pi. OpenRouter is supported but not required. Install the
+package into pi, then initialize its policy and state:
 
 ```bash
 pi install npm:openmerit
@@ -59,7 +60,7 @@ pi list
 `pi install` makes the extension and its trial engine available to pi. The
 one-off `npx` command creates `~/.openmerit/policy.json`; install OpenMerit
 globally with `npm install --global openmerit` only if you also want persistent
-shell access to `openmerit status`, `frontier`, or the optional watcher. Install
+shell access to `openmerit status`, `doctor`, `frontier`, or the optional watcher. Install
 the extension from only one source—remove any older copied `openmerit.ts` first
 so pi does not load it twice.
 
@@ -85,40 +86,44 @@ Keep that checkout available because pi loads a local package from its path.
    policy**. The shipped policy uses `"mode": "recommend"` and does not change
    the active model without approval.
 
-2. Give **Pi and the extension trial job** OpenRouter access. The job reads
-   `OPENROUTER_API_KEY` from its environment or `~/.openmerit/.env`. If using
-   the file, edit it to contain `OPENROUTER_API_KEY=your-key` and protect it:
+2. Authenticate the providers you want to compare in Pi, using `/login` or
+   Pi's normal environment/model configuration. OpenMerit takes its eligible
+   model routes, capabilities, prices, and credentials from Pi; judge and
+   strategist calls use the same routes and do not require duplicate keys.
+
+   For example, verify one provider without printing its credential:
 
    ```bash
-   nano ~/.openmerit/.env
-   chmod 600 ~/.openmerit/.env
+   pi auth check --provider openai --json
    ```
 
-   Pi does **not** read OpenMerit's `.env` file. Log in to OpenRouter with
-   `/login openrouter` inside pi, or export `OPENROUTER_API_KEY` in the shell
-   that starts pi. Verify pi's side without printing the key:
+   Add an OpenRouter route the same way if you want its routed catalog:
 
    ```bash
    pi auth check --provider openrouter --json
    ```
 
-   A ready configuration prints `{"status":"ready","provider":"openrouter",...}`.
-   Older `model_search/.env` files are not read by the trial engine.
+   An `OPENROUTER_API_KEY` in the process environment or
+   `~/.openmerit/.env` additionally enables OpenRouter catalog and public-
+   benchmark enrichment. It is optional for native-provider comparisons. If
+   using the file, protect it:
+
+   ```bash
+   nano ~/.openmerit/.env
+   chmod 600 ~/.openmerit/.env
+   ```
 
-   OpenRouter is currently required for candidate runs and catalog discovery.
-   Judge and strategist calls can use OpenAI or Gemini directly, but require
-   `OPENMERIT_DIRECT_PROVIDER=openai` or
-   `OPENMERIT_DIRECT_PROVIDER=google` plus the corresponding provider key;
-   OpenRouter model IDs remain on OpenRouter by default. The harness and
-   provider interfaces are extension seams for future integrations, not a
-   claim that other harnesses already work.
+   Pi does not read OpenMerit's `.env` file for its ordinary sessions, so an
+   OpenRouter route still needs Pi authentication. Older `model_search/.env`
+   files are not read.
 
 3. Verify the extension appears in `pi list`. The instruction file
    [`instructions/OPENMERIT.md`](instructions/OPENMERIT.md) can be added to a
    pi project's AGENTS.md for agent context, but the extension does
-   not require it.
+   not require it. Run `npx openmerit doctor` after starting Pi once to inspect
+   the eligible route snapshot and configuration without printing secrets.
 
-4. Start pi in one terminal with an OpenRouter model:
+4. Start pi in one terminal with any configured model. For example:
 
    ```bash
    pi --provider openrouter --model openai/gpt-4o-mini
@@ -127,7 +132,8 @@ Keep that checkout available because pi loads a local package from its path.
    For a first text task, ask: “Give the shortest valid word ladder from cat
    to dog. Each step changes one letter and must be a common English word.
    Return only the path.” Wait for A to finish and leave the pi session open.
-   The extension automatically starts B and C **sequentially through pi**,
+   The extension automatically starts B and C **sequentially through their
+   selected Pi routes**,
    reports each score in Pi, and writes a recommendation for this exact
    session. With the shipped supervised policy, use
    `/openmerit` inside pi to inspect the evidence and `/openmerit apply` to
@@ -173,7 +179,8 @@ schema, so its published scores are not directly comparable to these pi runs.
 
 ### Inspect or troubleshoot a run
 
-`npx openmerit status` shows the latest pi model and pending recommendations;
+`npx openmerit doctor` checks Pi, policy, eligible routes, and optional
+OpenRouter enrichment. `npx openmerit status` shows the latest pi model and pending recommendations;
 `npx openmerit frontier` shows measured quality, blended price, latency, and
 the chosen frontier per task. `/openmerit` inside pi shows the current model,
 fallback, the model currently being compared, completed models with quality,
@@ -189,23 +196,26 @@ candidate-spend threshold.
 
 If no comparison starts after Pi settles, confirm that pi loaded the extension
 (`pi list`), the task finished, and the pi session is saved (do not use
-`--no-session`). Image candidates must advertise
-image input in OpenRouter's current model catalog. The extension queues
+`--no-session`). Image candidates must advertise image input in Pi's model
+registry. The extension queues
 completed tasks from its current session and runs one comparison at a time.
 Closing or switching the session cancels the active job.
 Candidate runs use the same text and uploaded image bytes, but they do not
 replay earlier answers or file changes. An exact task in another session
 (including identical image bytes) appears as **advice**, not a pending swap;
 the new session still gets its own comparison. The optional `trial` command
-remains for controlled text-task runs from a task JSON file.
+remains for controlled text-task runs from a task JSON file; that legacy
+standalone command is still OpenRouter-specific in 0.1.2. The automatic Pi
+session path is provider-neutral.
 
 ## Alpha boundaries
 
-- With the extension installed and an OpenRouter key configured, each supported
-  settled task can start comparison calls automatically. Candidate, judge, and
-  strategist requests send the task text, attached files or images, and
-  candidate output to OpenRouter and can incur charges. Use non-sensitive test
-  tasks and conservative account limits while evaluating this alpha.
+- With the extension installed and at least two eligible Pi routes, each
+  supported settled task can start comparison calls automatically. Candidate,
+  judge, and strategist requests send the task text, attached files or images,
+  and candidate output to the configured providers and can incur charges. Use
+  non-sensitive test tasks and conservative account limits while evaluating
+  this alpha.
 - The extension queues tasks from its current pi session and compares one at a
   time. Candidate runs use isolated temporary copies of the original working
   directory and have no Pi tools by default. Their temporary changes are
@@ -223,25 +233,35 @@ remains for controlled text-task runs from a task JSON file.
   candidate sandbox and passed back to Pi as `@` file inputs. This covers PDFs,
   CSVs, spreadsheets, and other files that Pi can open; OpenMerit does not
   implement a separate parser for them.
-- `ledger.json` counts reported candidate-trial spend and the daily number of
-  trials. It does not yet include rubric, judge, or strategist request cost,
-  and `max_usd_per_trial` is not yet a hard provider-side cap.
+- `ledger.json` counts reported candidate, rubric, judge, and strategist spend
+  for automatic session comparisons plus the daily candidate count.
+  `max_usd_per_trial` is a conservative admission estimate based on known Pi
+  prices and a 4K answer; it is not a provider-side hard cap. Routes without
+  known pricing are excluded from automatic comparisons and cannot auto-apply.
 - Each comparison has one observed baseline plus a small candidate slate and
   one quality score per answer. Treat recommendations as experimental evidence,
   not a universal model ranking.
-- Candidate execution and catalog discovery currently use Pi through
-  OpenRouter. `HarnessAdapter`, `ModelProvider`, and `CatalogProvider` are
-  implementation boundaries for future harness/provider integrations.
+- Candidate execution, judging, and strategy use the provider/model routes
+  exposed by Pi. OpenRouter remains an optional route plus catalog/benchmark
+  enrichment source. `HarnessAdapter`, `ModelProviderAdapter`,
+  `ObservationSource`, and `EventSink` remain separate integration boundaries;
+  Pi and local JSONL are the implementations shipped in this release.
+- Candidate subprocesses can use Pi built-ins and custom/local routes available
+  without loading extensions (for example routes from Pi's model
+  configuration). A provider registered only at runtime by another extension
+  is visible in the route snapshot but cannot yet be executed by the isolated
+  subprocess.
 
 ## State layout (`~/.openmerit/`)
 
 | file | contents |
 |---|---|
 | `policy.json` | the policy file (gate thresholds, budgets, intervals) |
-| `harness-state.json` | written by the pi extension: current model, fallback, latest session and settled task |
-| `recommendations.jsonl` | shared append-only channel: session-bound swap recommendations + status updates |
-| `trials.jsonl` | every model trial point (source, score, price, latency) |
-| `observations.jsonl` | task observations extracted from session traces |
+| `harness-state.json` | current/fallback routes, Pi's eligible route snapshot, latest session and settled task |
+| `recommendations.jsonl` | append-only session-bound recommendations, routes, evidence, gate reasons, and status updates |
+| `trials.jsonl` | every model trial point, including its provider route when known |
+| `traces/observations.jsonl` | task observations extracted from session traces |
+| `events.jsonl` | versioned provider-neutral observation, trial, and recommendation events for future sinks |
 | `traces/trials/*.jsonl` | raw Pi JSON event streams for candidate trials |
 | `catalog/snapshot.json` + `candidates.json` | catalog snapshot (including input modalities) + new-model queue |
 | `benchmarks/digest.json` | public-benchmark scores per model (seed + refresh) |

package/dist/cli.js

@@ -1,5 +1,5 @@
 #!/usr/bin/env node
-/** openmerit CLI: init / watch / trial / frontier / recommend / status. */
+/** openmerit CLI: init / watch / trial / frontier / recommend / status / doctor. */
 import { copyFileSync, existsSync, mkdirSync, readFileSync } from "node:fs";
 import { dirname, join } from "node:path";
 import { fileURLToPath } from "node:url";
@@ -11,10 +11,12 @@ import { buildRecommendation } from "./recommend.js";
 import { appendJsonl, paths, readJson, readJsonl, taskKey } from "./store.js";
 import { autoTaskTick, recommendTick, runDaemon, tickOnce } from "./daemon.js";
 import { budgetOk, recordTrialSpend } from "./trials.js";
-import { availablePiModels, recordedActiveTask, runPiTrial, scorePiRun, settledActiveTask } from "./pi-trials.js";
+import { availablePiModels, availablePiRoutes, piExecutable, recordedActiveTask, runPiTrial, scorePiRun, settledActiveTask } from "./pi-trials.js";
 import { pickNext, STRAT_PREFS } from "./strategist.js";
 import { JUDGE_PREFS } from "./judge.js";
 import { openRouterBenchmarkCandidates, relevantBenchmarks } from "./benchmarks.js";
+import { modelRoute, routeLabel } from "./routes.js";
+import { spawnSync } from "node:child_process";
 const ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
 function resolvePref(cat, prefs, label) {
     for (const p of prefs)
@@ -43,7 +45,48 @@ function cmdInit() {
     console.log(`ext.    -> ${join(ROOT, "extension", "openmerit.ts")}`);
     console.log(`pi npm  -> pi install npm:openmerit`);
     console.log(`pi local-> pi install "${ROOT}"`);
-    console.log("\nNext: set OPENROUTER_API_KEY (env or ~/.openmerit/.env), install one extension source into pi, then start pi; the extension launches comparisons automatically with candidate tools disabled by default.");
+    console.log("\nNext: authenticate at least two models in pi, install one extension source, then start pi. OpenRouter is optional enrichment; candidate tools are disabled by default.");
+}
+function optionalOpenRouterKey() {
+    try {
+        return loadKey();
+    }
+    catch {
+        return null;
+    }
+}
+function cmdDoctor() {
+    const executable = piExecutable();
+    const version = spawnSync(executable, ["--version"], { encoding: "utf8" });
+    const state = readJson(paths.harnessState(), {});
+    console.log(`pi executable : ${executable}`);
+    console.log(`pi detected   : ${version.status === 0 ? `yes (${version.stdout.trim()})` : "no"}`);
+    try {
+        const policy = loadPolicy(paths.policy());
+        console.log(`policy        : valid (v${policy.version}, ${policy.mode})`);
+    }
+    catch (error) {
+        console.log(`policy        : invalid (${error.message})`);
+    }
+    let routes = state.routes ?? [];
+    const hasLiveSnapshot = routes.length > 0;
+    if (routes.length === 0 && version.status === 0) {
+        try {
+            routes = availablePiRoutes();
+        }
+        catch { /* reported below */ }
+    }
+    console.log(`eligible routes: ${routes.length}`);
+    for (const route of routes.slice(0, 8))
+        console.log(`  - ${routeLabel(route)}${route.cost ? "" : " (price unknown)"}`);
+    if (routes.length > 8)
+        console.log(`  … ${routes.length - 8} more`);
+    console.log(`active route  : ${state.currentRoute ? routeLabel(state.currentRoute) : "not reported; start pi once"}`);
+    const pricedRoutes = routes.filter((route) => !!route.cost).length;
+    console.log(`alternates    : ${hasLiveSnapshot
+        ? pricedRoutes >= 2 ? "ready" : "need at least two priced eligible pi routes"
+        : "start Pi once to capture route prices and capabilities"}`);
+    console.log(`OpenRouter enrichment: ${optionalOpenRouterKey() ? "available" : "not configured (optional)"}`);
 }
 async function cmdTrial(taskFile, rounds) {
     const cfg = JSON.parse(readFileSync(taskFile, "utf8"));
@@ -189,7 +232,8 @@ function cmdStatus() {
         latestById.set(r.id, r);
     const recs = [...latestById.values()].filter((r) => r.status === "pending");
     const frontiers = loadFrontiers();
-    console.log(`harness model : ${harness.currentModel ?? "unknown"} (as of ${harness.updatedAt ?? "n/a"})`);
+    console.log(`harness model : ${harness.currentRoute ? routeLabel(harness.currentRoute) : harness.currentModel ?? "unknown"} ` +
+        `(as of ${harness.updatedAt ?? "n/a"})`);
     console.log(`tasks tracked : ${frontiers.length}`);
     console.log(`new models queued for trial: ${queue.newModels.length}${queue.newModels.length ? " — " + queue.newModels.slice(0, 5).map((m) => m.id).join(", ") + (queue.newModels.length > 5 ? "…" : "") : ""}`);
     console.log(`pending recommendations: ${recs.length}`);
@@ -205,16 +249,22 @@ async function main() {
             cmdInit();
             break;
         case "session-trial": {
-            const [sessionFile, settledAt, currentModel, settledTaskKey, cwd, bytesText] = args;
+            const [sessionFile, settledAt, currentModel, settledTaskKey, cwd, bytesText, provider, providerModelId] = args;
             if (!sessionFile || !settledAt || !currentModel || !settledTaskKey || !cwd || !bytesText)
                 throw new Error("usage: openmerit session-trial <session-file> <settled-at> <model> <task-key> <cwd> <session-bytes>");
             const sessionBytes = Number(bytesText);
             if (!Number.isSafeInteger(sessionBytes) || sessionBytes <= 0)
                 throw new Error("invalid session byte limit");
-            const task = settledActiveTask({ sessionFile, settledAt, currentModel, settledTaskKey, cwd, sessionBytes });
+            const state = readJson(paths.harnessState(), {});
+            const currentRoute = provider && providerModelId
+                ? state.routes?.find((route) => route.provider === provider && route.modelId === providerModelId) ??
+                    modelRoute(provider, providerModelId)
+                : state.currentRoute;
+            const task = settledActiveTask({ sessionFile, settledAt, currentModel, currentRoute,
+                routes: state.routes, settledTaskKey, cwd, sessionBytes });
             if (!task)
                 throw new Error("the specified pi session has no completed, supported task matching this marker");
-            await autoTaskTick(loadKey(), policy, task);
+            await autoTaskTick(optionalOpenRouterKey(), policy, task);
             break;
         }
         case "watch":
@@ -240,6 +290,9 @@ async function main() {
         case "status":
             cmdStatus();
             break;
+        case "doctor":
+            cmdDoctor();
+            break;
         default:
             console.log(`openmerit — external model-merit harness
 
@@ -250,7 +303,8 @@ usage: openmerit <command>
   trial <task.json>    iterative model search on one task spec [--rounds N]
   frontier [taskKey]   print pareto frontier(s)
   recommend            emit recommendations now
-  status               harness model, queued models, pending recommendations`);
+  status               harness model, queued models, pending recommendations
+  doctor               verify pi, policy, routes, and optional enrichment`);
             if (cmd && cmd !== "help" && cmd !== "--help")
                 process.exitCode = 1;
     }

package/dist/daemon.js

@@ -9,15 +9,46 @@ import { loadKey } from "./llm.js";
 import { providerAllowed } from "./policy.js";
 import { buildRecommendation } from "./recommend.js";
 import { appendJsonl, paths, readJson, readJsonl, writeJson, } from "./store.js";
-import { ingestNewTraces } from "./traces.js";
-import { budgetOk, ensureRubric, recordTrialSpend, runTrial } from "./trials.js";
+import { budgetOk, ensureRubric, recordMeritSpend, recordTrialSpend, runTrial, trialBudgetOk } from "./trials.js";
 import { JUDGE_PREFS } from "./judge.js";
-import { availablePiModels, runPiTrial, scorePiRun, settledActiveTask } from "./pi-trials.js";
+import { runPiTrial, scorePiRun, settledActiveTask } from "./pi-trials.js";
 import { taskInputKey } from "./task-input.js";
 import { pickNext, STRAT_PREFS } from "./strategist.js";
+import { enrichRouteEntry, routeCatalog, routeKey, routeLabel } from "./routes.js";
+import { PiCliChatClient } from "./llm.js";
+import { LocalJsonlEventSink, PiTraceObservationSource, meritEvent } from "./integrations.js";
+function resolveRoutePref(cat, prefs, label, requireImages = false) {
+    const available = [...cat.values()].filter((entry) => !!entry.route && entry.priceKnown !== false &&
+        (!requireImages || entry.inputModalities?.includes("image")));
+    for (const pref of prefs) {
+        if (!pref)
+            continue;
+        const routed = cat.get(pref);
+        if (routed && available.includes(routed))
+            return routed;
+        const exact = available.find((entry) => entry.id === pref);
+        if (exact)
+            return exact;
+    }
+    for (const pref of prefs) {
+        if (!pref)
+            continue;
+        const fuzzy = available.find((entry) => entry.id.includes(pref));
+        if (fuzzy)
+            return fuzzy;
+    }
+    const fallback = available.sort((a, b) => (a.priceKnown === false ? 1 : 0) - (b.priceKnown === false ? 1 : 0) || a.price - b.price)[0];
+    if (fallback)
+        return fallback;
+    throw new Error(`could not resolve ${label} route from Pi's eligible models`);
+}
 function emitTrialProgress(progress) {
     console.log(`[openmerit/progress] ${JSON.stringify(progress)}`);
 }
+function persistTrial(point, sink) {
+    appendJsonl(paths.trials(), point);
+    sink.emit(meritEvent("trial.completed", point));
+}
 function resolvePref(cat, prefs, label) {
     for (const p of prefs) {
         if (p && cat.has(p))
@@ -34,10 +65,12 @@ function resolvePref(cat, prefs, label) {
     throw new Error(`could not resolve ${label} model from preferences`);
 }
 /** Ingest new session-trace bytes into the observation log. */
-export function tracesTick() {
-    const fresh = ingestNewTraces();
-    for (const o of fresh)
+export function tracesTick(source = new PiTraceObservationSource(), sink = new LocalJsonlEventSink()) {
+    const fresh = source.read();
+    for (const o of fresh) {
         appendJsonl(paths.observations(), o);
+        sink.emit(meritEvent("observation.recorded", o, source.id));
+    }
     if (fresh.length > 0)
         console.log(`[openmerit] traces: ${fresh.length} new observation(s)`);
     return fresh;
@@ -68,6 +101,7 @@ export async function catalogTick(key, policy) {
  * thin). Budget-capped per policy.
  */
 export async function trialTick(key, policy) {
+    const sink = new LocalJsonlEventSink();
     const budget = budgetOk(policy);
     if (!budget.ok) {
         console.log(`[openmerit] trials paused: ${budget.reason}`);
@@ -106,7 +140,7 @@ export async function trialTick(key, policy) {
         const rubric = await ensureRubric(key, judge, obs.taskLabel, rubricCache);
         const { point, costUsd, error } = await runTrial(key, judge, obs.taskLabel, rubric, candidate.id, cat.get(candidate.id));
         recordTrialSpend(costUsd);
-        appendJsonl(paths.trials(), { ...point, taskKey: tKey });
+        persistTrial({ ...point, taskKey: tKey }, sink);
         if (error)
             console.log(`[openmerit] trial failed: ${error.slice(0, 120)}`);
         else
@@ -115,6 +149,7 @@ export async function trialTick(key, policy) {
 }
 /** Compare the one completed active pi task, sequentially, then target its session. */
 export async function autoTaskTick(key, policy, settledTask) {
+    const sink = new LocalJsonlEventSink();
     const task = settledTask === undefined ? settledActiveTask() : settledTask;
     if (!task)
         return null;
@@ -130,74 +165,94 @@ export async function autoTaskTick(key, policy, settledTask) {
     // Claim the settled turn before provider calls so the next poll cannot duplicate it.
     writeJson(processedPath, { marker, status: "running", updatedAt: new Date().toISOString() });
     try {
-        const cat = await fetchCatalog(key);
-        const piModels = availablePiModels();
-        for (const id of [...cat.keys()]) {
-            const e = cat.get(id);
-            if (!piModels.has(id) || e.price > policy.max_usd_per_m || !providerAllowed(policy, id) ||
-                (task.images.length > 0 && !e.inputModalities?.includes("image")))
+        let cat = routeCatalog(task.routes);
+        if (!cat.has(routeKey(task.route)))
+            cat.set(routeKey(task.route), routeCatalog([task.route]).values().next().value);
+        if (key) {
+            try {
+                const live = await fetchCatalog(key);
+                saveSnapshot(live);
+                cat = new Map([...cat].map(([id, entry]) => [id,
+                    entry.route?.provider === "openrouter" ? enrichRouteEntry(entry, live.get(entry.id)) : entry]));
+            }
+            catch (error) {
+                console.log(`[openmerit] OpenRouter enrichment unavailable: ${error.message}`);
+            }
+        }
+        for (const [id, entry] of [...cat]) {
+            if ((entry.priceKnown !== false && entry.price > policy.max_usd_per_m) ||
+                !providerAllowed(policy, entry.id, entry.route?.provider) ||
+                (task.images.length > 0 && !entry.inputModalities?.includes("image")) ||
+                (id !== routeKey(task.route) && !trialBudgetOk(policy, entry, task.task.length).ok))
                 cat.delete(id);
         }
-        if (!cat.has(task.model))
-            throw new Error(`active model ${task.model} is not trialable in pi/OpenRouter`);
-        const judge = resolvePref(cat, [policy.judge_model, ...JUDGE_PREFS], "judge");
-        const strategist = resolvePref(cat, [policy.strategist_model, ...STRAT_PREFS], "strategist");
+        const activeEntry = cat.get(routeKey(task.route));
+        if (!activeEntry)
+            throw new Error(`active route ${task.route.provider}/${task.route.modelId} is excluded by policy or task capabilities`);
+        if (cat.size < 2)
+            throw new Error("Pi exposes no eligible alternate model route for this task");
+        const judge = resolveRoutePref(cat, [policy.judge_model, ...JUDGE_PREFS, task.model], "judge", task.images.length > 0);
+        const strategist = resolveRoutePref(cat, [policy.strategist_model, ...STRAT_PREFS, task.model], "strategist");
+        const client = new PiCliChatClient(undefined, recordMeritSpend);
         const { category, benchmarks } = relevantBenchmarks(task.task);
         let benchmarkCandidates = [];
         try {
-            benchmarkCandidates = await openRouterBenchmarkCandidates(key, category, cat);
+            if (key)
+                benchmarkCandidates = await openRouterBenchmarkCandidates(key, category, new Map([...cat.values()].map((entry) => [entry.id, entry])));
         }
         catch (e) {
             console.log(`[openmerit] benchmark shortlist unavailable: ${e.message}`);
         }
-        const rubric = await ensureRubric(key, judge, task.task, new Map());
-        const tKey = taskInputKey(task.task, task.images);
+        const rubric = await ensureRubric(key ?? "", judge.route, task.task, new Map(), client);
+        const tKey = taskInputKey(task.task, task.images, task.files);
         const points = [];
         const tried = new Set();
         const failedVendors = new Set();
         const total = Math.max(1, Math.floor(policy.watch.models_per_task));
         if (settledTask !== undefined)
             emitTrialProgress({ phase: "start", taskKey: tKey,
-                model: task.model, index: 1, total });
-        const baseline = await scorePiRun(key, judge, task.task, rubric, task.model, cat.get(task.model), task.run, "trace", task.images);
+                model: routeLabel(task.route), index: 1, total });
+        const baseline = await scorePiRun(key ?? "", judge.route, task.task, rubric, task.model, activeEntry, task.run, "trace", task.images, client);
         points.push(baseline.point);
-        tried.add(task.model);
-        appendJsonl(paths.trials(), { ...baseline.point, taskKey: tKey, sessionFile: task.sessionFile });
+        tried.add(routeKey(task.route));
+        persistTrial({ ...baseline.point, taskKey: tKey, sessionFile: task.sessionFile }, sink);
         console.log(`[openmerit] task ${tKey}: A=${task.model} score=${baseline.point.score.toFixed(2)} from pi trace`);
         if (settledTask !== undefined)
             emitTrialProgress({ phase: "complete", taskKey: tKey,
-                model: task.model, index: 1, total, score: baseline.point.score,
+                model: routeLabel(task.route), index: 1, total, score: baseline.point.score,
                 price: baseline.point.price, latencyMs: baseline.point.latencyMs,
                 costUsd: baseline.costUsd, toolCalls: baseline.point.toolCalls,
                 toolErrors: baseline.point.toolErrors, changedFiles: baseline.point.changedFiles });
         for (let i = 1; i < total; i++) {
             if (!budgetOk(policy).ok)
                 break;
-            const pick = await pickNext(key, strategist, task.task, rubric, benchmarks, points, cat, tried, policy.max_usd_per_m, failedVendors, benchmarkCandidates);
+            const pick = await pickNext(key ?? "", strategist.route, task.task, rubric, benchmarks, points, cat, tried, policy.max_usd_per_m, failedVendors, benchmarkCandidates, client);
             if (!pick)
                 break;
             if (settledTask !== undefined)
                 emitTrialProgress({ phase: "start", taskKey: tKey,
-                    model: pick.model, index: i + 1, total });
-            const { point, costUsd, error } = await runPiTrial(key, judge, task.task, rubric, pick.model, cat.get(pick.model), task.cwd, task.images, task.files);
-            tried.add(pick.model);
+                    model: pick.route ? routeLabel(pick.route) : pick.model, index: i + 1, total });
+            const entry = pick.route ? cat.get(routeKey(pick.route)) : cat.get(pick.model);
+            const { point, costUsd, error } = await runPiTrial(key ?? "", judge.route, task.task, rubric, pick.model, entry, task.cwd, task.images, task.files, undefined, client);
+            tried.add(pick.route ? routeKey(pick.route) : pick.model);
             points.push(point);
             recordTrialSpend(costUsd);
-            appendJsonl(paths.trials(), { ...point, taskKey: tKey, sessionFile: task.sessionFile });
+            persistTrial({ ...point, taskKey: tKey, sessionFile: task.sessionFile }, sink);
             if (error)
-                failedVendors.add(pick.model.split("/")[0]);
+                failedVendors.add(pick.route?.provider ?? pick.model.split("/")[0]);
             console.log(`[openmerit] task ${tKey}: ${pick.model} score=${point.score.toFixed(2)} cost=$${costUsd.toFixed(4)} ` +
                 `tools=${point.toolCalls ?? 0} toolErrors=${point.toolErrors ?? 0} changedFiles=${point.changedFiles ?? 0}` +
                 `${error ? ` error=${error.slice(0, 80)}` : ""}`);
             if (settledTask !== undefined)
                 emitTrialProgress({ phase: "complete", taskKey: tKey,
-                    model: pick.model, index: i + 1, total, score: point.score, price: point.price,
+                    model: pick.route ? routeLabel(pick.route) : pick.model, index: i + 1, total, score: point.score, price: point.price,
                     latencyMs: point.latencyMs, costUsd, error: error?.slice(0, 120),
                     toolCalls: point.toolCalls, toolErrors: point.toolErrors, changedFiles: point.changedFiles });
         }
-        const rec = buildRecommendation(tKey, task.task.slice(0, 120), task.model, points, policy, task.sessionFile);
+        const rec = buildRecommendation(tKey, task.task.slice(0, 120), task.model, points, policy, task.sessionFile, task.route);
         if (rec) {
             appendJsonl(paths.recommendations(), rec);
+            sink.emit(meritEvent("recommendation.created", rec));
             console.log(`[openmerit] task ${tKey}: selected ${rec.recommended.model}; auto=${rec.policy.autoApply}`);
         }
         else
@@ -214,8 +269,10 @@ export async function autoTaskTick(key, policy, settledTask) {
 }
 /** Rebuild frontiers from trials and emit recommendations for the harness's current model. */
 export function recommendTick(policy) {
+    const sink = new LocalJsonlEventSink();
     const harness = readJson(paths.harnessState(), {});
     const currentModel = harness.currentModel ?? null;
+    const currentRoute = harness.currentRoute ?? null;
     const trials = readJsonl(paths.trials());
     const observations = readJsonl(paths.observations());
     const labelByTask = new Map();
@@ -237,16 +294,17 @@ export function recommendTick(policy) {
         latestById.set(r.id, r);
     const existing = new Set([...latestById.values()]
         .filter((r) => r.status === "pending" || r.status === "dismissed")
-        .map((r) => `${r.taskKey}:${r.recommended.model}`));
+        .map((r) => `${r.taskKey}:${r.recommended.route ? routeKey(r.recommended.route) : `openrouter:${r.recommended.model}`}`));
     const emitted = [];
     for (const [tKey, points] of byTask) {
-        const rec = buildRecommendation(tKey, labelByTask.get(tKey) ?? tKey, currentModel, points, policy);
+        const rec = buildRecommendation(tKey, labelByTask.get(tKey) ?? tKey, currentModel, points, policy, null, currentRoute);
         if (!rec)
             continue;
-        const dedupeKey = `${rec.taskKey}:${rec.recommended.model}`;
+        const dedupeKey = `${rec.taskKey}:${rec.recommended.route ? routeKey(rec.recommended.route) : `openrouter:${rec.recommended.model}`}`;
         if (existing.has(dedupeKey))
             continue;
         appendJsonl(paths.recommendations(), rec);
+        sink.emit(meritEvent("recommendation.created", rec));
         emitted.push(rec);
         console.log(`[openmerit] recommendation: ${rec.recommended.model} <- ${currentModel ?? "unknown"} ` +
             `(gain ${rec.evidence.scoreGain}, auto=${rec.policy.autoApply})`);
@@ -261,12 +319,12 @@ export async function tickOnce(policy) {
         key = loadKey();
     }
     catch (e) {
-        console.log(`[openmerit] ${e.message}; skipping provider ticks`);
+        console.log(`[openmerit] ${e.message}; OpenRouter enrichment disabled`);
     }
     if (key) {
         await catalogTick(key, policy);
-        await autoTaskTick(key, policy);
     }
+    await autoTaskTick(key, policy);
 }
 /** Run forever, honoring policy watch intervals. */
 export async function runDaemon(policy) {
@@ -291,17 +349,15 @@ export async function runDaemon(policy) {
             catch {
                 /* no key; skip provider ticks */
             }
-            if (key) {
-                try {
-                    if (wantCatalog) {
-                        await catalogTick(key, policy);
-                        lastCatalog = now;
-                    }
-                    await autoTaskTick(key, policy);
-                }
-                catch (e) {
-                    console.log(`[openmerit] provider tick failed: ${e.message}`);
+            try {
+                if (key && wantCatalog) {
+                    await catalogTick(key, policy);
+                    lastCatalog = now;
                 }
+                await autoTaskTick(key, policy);
+            }
+            catch (e) {
+                console.log(`[openmerit] provider tick failed: ${e.message}`);
             }
         }
         await new Promise((r) => setTimeout(r, tracesMs));