llm-relay 0.4.1 → 0.6.0

package/README.md

@@ -311,26 +311,48 @@ Precedence for a subagent request: `@relay:` directive → `subagents[<tier>]`
 ### Choosing where to offload (`llm-relay candidates`)
 
 ```
-target                            pools / tiers        live  SWE  LCB  BFCL   arena  rank  verdict  p95    up%  quota  breaker  ctx
-nim/z-ai/glm-5.2                  coding,@opus         yes   42   53   -      1469~  #31   Perfect  310ms  100  84%    closed   128k
-nim/meta/llama-3.1-8b-instruct    fast,@haiku,@fable   yes   -    -    25.83  1211   #308  Perfect  120ms  100  84%    closed   128k
+target                          pools / tiers      str      agentic coding BFCL   arena  $/Mout  verdict  p95    quota  breaker  ctx
+nim/z-ai/glm-5.2                coding,@opus       83.3/4   43.1    68.8   -      -      $2.402~ Perfect  310ms  84%    closed   1049k~
+nim/moonshotai/kimi-k2.6        coding,@sonnet     77.4/5   30.3    61.8   -      1461   $2.72~  Perfect  280ms  84%    closed   262k~
+nim/meta/llama-3.1-8b-instruct  fast,@haiku,@fable  9.7/6    0.5     5.4   25.83  1211   $0.08~  Perfect  120ms  84%    closed   131k~
 ```
 
-Every offload target with its dimensions side by side: capability (SWE-bench, HumanEval,
-LiveCodeBench, BFCL tool-use, Arena rating/rank, context window), live behaviour (verdict, avg/p95
-latency, jitter, uptime), availability right now (provider quota, circuit-breaker state, whether the
-provider still lists the model) and traffic actually observed through the proxy.
-
-A blank cell means **not measured**, not bad. The two capability sources have different coverage:
-SWE-bench/HumanEval/LiveCodeBench come from a hand-maintained table in `benchmarks.ts` that lags the
-roster, while BFCL and Arena come from the synced leaderboard (`npm run sync:tiers`). `~` marks a
-score **borrowed from a similarly-named model** — the leaderboard has no `glm-5.2` row, so those
-numbers are `glm-5.2-max`'s. The JSON carries `capability.matched_name` and `capability.match`
-(`exact` | `fuzzy`) so the substitution is never invisible.
-
-**Nothing is ranked or averaged** — capability, latency and remaining quota trade off differently
-per task, and one blended number answers neither "cheapest that can do it" nor "best available". The
-composites the proxy itself sorts by are reported under `sortInputs`, labelled as what they drive.
+Every offload target with its dimensions side by side: capability from each leaderboard separately,
+live behaviour (verdict, avg/p95 latency, jitter, uptime), availability and cost right now (quota,
+circuit-breaker state, price per million tokens, whether the provider still lists the model), and
+traffic actually observed through the proxy.
+
+**Capability comes from `npm run sync:tiers`**, which merges four sources into
+`docs/tier-data.json` (~770 models) — see [docs/capability-sources.md](docs/capability-sources.md):
+
+| Source | Contributes |
+|---|---|
+| OpenRouter | Artificial Analysis intelligence / coding / **agentic** indices, Design Arena Elo, context length, pricing, tool support — and the only source whose ids match routing specs exactly |
+| BFCL | tool-call accuracy, multi-turn, irrelevance detection |
+| LMArena | general preference rating + rank |
+| Aider | polyglot edit benchmark + edit-format compliance |
+
+They **disagree** — the agentic index puts deepseek above kimi while the coding index puts kimi
+above deepseek — which is exactly why each keeps its own column, and why a blank cell means *not
+measured*, never *bad*.
+
+**Nothing is ranked or averaged across dimensions** — capability, latency and remaining quota trade
+off differently per task, and one blended number answers neither "cheapest that can do it" nor "best
+available".
+
+`str` is the single exception, and it exists only because ordering a pool requires an order. It is a
+weighted mean of whatever rank-normalized signals a model actually has (tool-use and agentic ability
+weighted highest — this proxy drives tool loops), and it never appears without its provenance:
+`83.3/4` means four published signals backed it, while `obs` (ranked on this proxy's own traffic,
+≥5 calls) and `neut` (nothing known) mark the fallbacks.
+
+**Limits and prices are per-(provider, model), and labelled.** The same model id on two providers is
+two deployments — different context ceilings, different output caps, and possibly free on one and
+metered on the other. Where a provider publishes its own figures (Groq, Mistral, OpenRouter) those
+are used; where it publishes none (NIM returns only `id`/`object`/`created`/`owned_by`) the table
+falls back to another provider's figure for the same id and marks it `~`, or to the hardcoded
+table's guess and marks it `?`. Unmarked means the serving provider published it. An unknown price
+stays blank rather than being guessed.
 
 `GET /candidates` returns the full JSON (the table shows a subset). The CLI prefers a running proxy
 so the live columns come from warm ping history rather than a cold start.

package/dist/benchmarks.d.ts

@@ -1,13 +1,39 @@
 import type { ResolvedTarget } from "./config.js";
-export interface BenchmarkScores {
-    sweBench?: number;
-    humanEval?: number;
-    liveCodeBench?: number;
-    arenaElo?: number;
+/** Where a strength score came from. Ranking is only as trustworthy as its basis. */
+export type StrengthBasis = "snapshot" | "telemetry" | "neutral";
+export interface Strength {
+    /** 0-100, comparable across bases. Higher is better. */
+    score: number;
+    basis: StrengthBasis;
+    /** snapshot only: which signals backed it, and how many. 1 signal is a guess, 5 is a consensus. */
+    signals?: string[];
+    signalCount?: number;
+    /** snapshot only: `fuzzy` means the row belongs to a similarly-named, DIFFERENT model. */
+    match?: "exact" | "fuzzy";
+    /** snapshot only: the row actually used. */
+    matchedName?: string;
 }
-/** Find benchmark scores for a model ID or provider/model spec. */
-export declare function getBenchmarkScores(modelId: string): BenchmarkScores;
-/** Calculate a single composite quality score (0-100) for ranking targets. */
-export declare function calculateQualityScore(scores: BenchmarkScores): number;
-/** Rank an array of resolved targets by coding benchmark quality (descending). */
+/**
+ * Strength of one target, from the best evidence available, in this order:
+ *
+ *  1. the synced multi-source snapshot — real published capability, refreshed by `sync:tiers`;
+ *  2. observed runtime telemetry — not capability, but this deployment's own evidence that the
+ *     model answers successfully and quickly. Needs ≥5 real calls, so a brand-new model does not
+ *     get ranked off one lucky request;
+ *  3. neutral — nothing is known, so claim nothing.
+ *
+ * The basis travels with the score precisely so a telemetry-derived number is never mistaken for
+ * a benchmark one.
+ *
+ * There is deliberately no hardcoded-table rung. `BENCHMARK_DB` used to sit here; every pattern it
+ * carried is present in the snapshot, so it contributed nothing but a stale, provenance-free number
+ * that outranked the synced data for any model it happened to substring-match.
+ */
+export declare function getStrength(spec: string, opts?: {
+    telemetryPath?: string;
+}): Strength;
+/**
+ * Rank targets strongest-first. Ties are left in config order (`sort` is stable), which is what
+ * makes a pool's declared order the tie-breaker when nothing distinguishes two candidates.
+ */
 export declare function rankTargetsByBenchmark(targets: ResolvedTarget[]): ResolvedTarget[];

package/dist/benchmarks.js

@@ -1,79 +1,60 @@
-/** Built-in coding benchmark database for popular coding models. */
-const BENCHMARK_DB = [
-    // Anthropic / Claude models
-    { pattern: "claude-3-7-sonnet", scores: { sweBench: 70.3, humanEval: 92.0, liveCodeBench: 64.5, arenaElo: 1320 } },
-    { pattern: "claude-3-5-sonnet", scores: { sweBench: 49.0, humanEval: 93.7, liveCodeBench: 58.2, arenaElo: 1280 } },
-    { pattern: "claude-3-opus", scores: { sweBench: 38.0, humanEval: 84.9, liveCodeBench: 45.0, arenaElo: 1250 } },
-    { pattern: "claude-3-5-haiku", scores: { sweBench: 40.6, humanEval: 88.1, liveCodeBench: 48.0, arenaElo: 1220 } },
-    // OpenAI / GPT models
-    { pattern: "o3-mini", scores: { sweBench: 71.0, humanEval: 94.5, liveCodeBench: 66.0, arenaElo: 1330 } },
-    { pattern: "o1", scores: { sweBench: 48.9, humanEval: 92.4, liveCodeBench: 61.5, arenaElo: 1310 } },
-    { pattern: "gpt-4o", scores: { sweBench: 38.8, humanEval: 90.2, liveCodeBench: 52.1, arenaElo: 1286 } },
-    { pattern: "gpt-4o-mini", scores: { sweBench: 29.0, humanEval: 87.2, liveCodeBench: 41.5, arenaElo: 1200 } },
-    // DeepSeek models
-    { pattern: "deepseek-r1", scores: { sweBench: 49.2, humanEval: 96.1, liveCodeBench: 65.9, arenaElo: 1350 } },
-    { pattern: "deepseek-v3", scores: { sweBench: 42.0, humanEval: 90.0, liveCodeBench: 58.0, arenaElo: 1310 } },
-    { pattern: "deepseek-coder", scores: { sweBench: 38.5, humanEval: 90.2, liveCodeBench: 53.0, arenaElo: 1240 } },
-    // Qwen models
-    { pattern: "qwen-2.5-coder-32b", scores: { sweBench: 41.2, humanEval: 92.7, liveCodeBench: 56.4, arenaElo: 1250 } },
-    { pattern: "qwen-2.5-72b", scores: { sweBench: 37.0, humanEval: 86.6, liveCodeBench: 51.0, arenaElo: 1240 } },
-    { pattern: "qwq-32b", scores: { sweBench: 44.0, humanEval: 93.0, liveCodeBench: 59.0, arenaElo: 1270 } },
-    // GLM / Z-AI / Nemotron / Llama / Mistral models
-    { pattern: "glm-4", scores: { sweBench: 35.0, humanEval: 85.0, liveCodeBench: 47.0, arenaElo: 1200 } },
-    { pattern: "glm-5", scores: { sweBench: 42.0, humanEval: 89.0, liveCodeBench: 53.0, arenaElo: 1240 } },
-    { pattern: "nemotron", scores: { sweBench: 36.5, humanEval: 86.0, liveCodeBench: 48.0, arenaElo: 1210 } },
-    { pattern: "llama-3.3-70b", scores: { sweBench: 36.0, humanEval: 88.6, liveCodeBench: 49.2, arenaElo: 1230 } },
-    { pattern: "mistral-large", scores: { sweBench: 34.0, humanEval: 84.0, liveCodeBench: 45.0, arenaElo: 1210 } },
-    { pattern: "codestral", scores: { sweBench: 33.5, humanEval: 81.1, liveCodeBench: 46.0, arenaElo: 1200 } },
-];
-/** Find benchmark scores for a model ID or provider/model spec. */
-export function getBenchmarkScores(modelId) {
-    const norm = modelId.toLowerCase();
-    for (const entry of BENCHMARK_DB) {
-        if (typeof entry.pattern === "string") {
-            if (norm.includes(entry.pattern))
-                return entry.scores;
-        }
-        else if (entry.pattern.test(norm)) {
-            return entry.scores;
-        }
-    }
-    return {};
-}
-/** Calculate a single composite quality score (0-100) for ranking targets. */
-export function calculateQualityScore(scores) {
-    let totalWeight = 0;
-    let weightedSum = 0;
-    if (scores.sweBench !== undefined) {
-        weightedSum += scores.sweBench * 3.0; // Primary coding agent benchmark
-        totalWeight += 3.0;
+import { loadTierData, findTierModel } from "./tier-data.js";
+import { getRealWorldScore } from "./ping/runtime-telemetry.js";
+const NEUTRAL = 50;
+/**
+ * Strength of one target, from the best evidence available, in this order:
+ *
+ *  1. the synced multi-source snapshot — real published capability, refreshed by `sync:tiers`;
+ *  2. observed runtime telemetry — not capability, but this deployment's own evidence that the
+ *     model answers successfully and quickly. Needs ≥5 real calls, so a brand-new model does not
+ *     get ranked off one lucky request;
+ *  3. neutral — nothing is known, so claim nothing.
+ *
+ * The basis travels with the score precisely so a telemetry-derived number is never mistaken for
+ * a benchmark one.
+ *
+ * There is deliberately no hardcoded-table rung. `BENCHMARK_DB` used to sit here; every pattern it
+ * carried is present in the snapshot, so it contributed nothing but a stale, provenance-free number
+ * that outranked the synced data for any model it happened to substring-match.
+ */
+export function getStrength(spec, opts = {}) {
+    const hit = findTierModel(spec, loadTierData()?.byNorm ?? []);
+    if (hit && typeof hit.rec.strength === "number") {
+        return {
+            score: Math.round(hit.rec.strength * 1000) / 10, // 0-1 → 0-100
+            basis: "snapshot",
+            signals: hit.rec.signals ?? [],
+            signalCount: hit.rec.signal_count ?? 0,
+            match: hit.match,
+            matchedName: hit.rec.norm,
+        };
     }
-    if (scores.liveCodeBench !== undefined) {
-        weightedSum += scores.liveCodeBench * 2.0;
-        totalWeight += 2.0;
+    const i = spec.indexOf("/");
+    if (i !== -1) {
+        const observed = getRealWorldScore(spec.slice(0, i), spec.slice(i + 1), opts.telemetryPath ? { path: opts.telemetryPath } : {});
+        if (observed !== null)
+            return { score: observed, basis: "telemetry" };
     }
-    if (scores.humanEval !== undefined) {
-        weightedSum += scores.humanEval * 1.0;
-        totalWeight += 1.0;
-    }
-    if (totalWeight > 0) {
-        return Math.round((weightedSum / totalWeight) * 10) / 10;
-    }
-    if (scores.arenaElo !== undefined) {
-        return Math.max(0, Math.min(100, Math.round((scores.arenaElo - 1000) / 4)));
-    }
-    return 50.0; // Default baseline score for unlisted models
+    return { score: NEUTRAL, basis: "neutral" };
 }
-/** Rank an array of resolved targets by coding benchmark quality (descending). */
+/**
+ * Rank targets strongest-first. Ties are left in config order (`sort` is stable), which is what
+ * makes a pool's declared order the tie-breaker when nothing distinguishes two candidates.
+ */
 export function rankTargetsByBenchmark(targets) {
     if (targets.length <= 1)
         return [...targets];
-    return [...targets].sort((a, b) => {
-        const specA = a.model ? `${a.provider}/${a.model}` : a.provider;
-        const specB = b.model ? `${b.provider}/${b.model}` : b.provider;
-        const scoreA = calculateQualityScore(getBenchmarkScores(specA));
-        const scoreB = calculateQualityScore(getBenchmarkScores(specB));
-        return scoreB - scoreA; // Highest score first
-    });
+    const specOf = (t) => (t.model ? `${t.provider}/${t.model}` : t.provider);
+    const cache = new Map();
+    const score = (t) => {
+        const spec = specOf(t);
+        let v = cache.get(spec);
+        if (v === undefined) {
+            v = getStrength(spec).score;
+            cache.set(spec, v);
+        }
+        return v;
+    };
+    return [...targets].sort((a, b) => score(b) - score(a));
 }
 //# sourceMappingURL=benchmarks.js.map

package/dist/benchmarks.js.map

@@ -1 +1 @@
-{"version":3,"file":"benchmarks.js","sourceRoot":"","sources":["../src/benchmarks.ts"],"names":[],"mappings":"AASA,oEAAoE;AACpE,MAAM,YAAY,GAAiE;IACjF,4BAA4B;IAC5B,EAAE,OAAO,EAAE,mBAAmB,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAClH,EAAE,OAAO,EAAE,mBAAmB,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAClH,EAAE,OAAO,EAAE,eAAe,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAC9G,EAAE,OAAO,EAAE,kBAAkB,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAEjH,sBAAsB;IACtB,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACxG,EAAE,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACnG,EAAE,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACvG,EAAE,OAAO,EAAE,aAAa,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAE5G,kBAAkB;IAClB,EAAE,OAAO,EAAE,aAAa,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAC5G,EAAE,OAAO,EAAE,aAAa,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAC5G,EAAE,OAAO,EAAE,gBAAgB,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAE/G,cAAc;IACd,EAAE,OAAO,EAAE,oBAAoB,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACnH,EAAE,OAAO,EAAE,cAAc,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAC7G,EAAE,OAAO,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAExG,iDAAiD;IACjD,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACtG,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACtG,EAAE,OAAO,EAAE,UAAU,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IACzG,EAAE,OAAO,EAAE,eAAe,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAC9G,EAAE,OAAO,EAAE,eAAe,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;IAC9G,EAAE,OAAO,EAAE,WAAW,EAAE,MAAM,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,aAAa,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,EAAE;CAC3G,CAAC;AAEF,mEAAmE;AACnE,MAAM,UAAU,kBAAkB,CAAC,OAAe;IAChD,MAAM,IAAI,GAAG,OAAO,CAAC,WAAW,EAAE,CAAC;IACnC,KAAK,MAAM,KAAK,IAAI,YAAY,EAAE,CAAC;QACjC,IAAI,OAAO,KAAK,CAAC,OAAO,KAAK,QAAQ,EAAE,CAAC;YACtC,IAAI,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,OAAO,CAAC;gBAAE,OAAO,KAAK,CAAC,MAAM,CAAC;QACxD,CAAC;aAAM,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YACpC,OAAO,KAAK,CAAC,MAAM,CAAC;QACtB,CAAC;IACH,CAAC;IACD,OAAO,EAAE,CAAC;AACZ,CAAC;AAED,8EAA8E;AAC9E,MAAM,UAAU,qBAAqB,CAAC,MAAuB;IAC3D,IAAI,WAAW,GAAG,CAAC,CAAC;IACpB,IAAI,WAAW,GAAG,CAAC,CAAC;IAEpB,IAAI,MAAM,CAAC,QAAQ,KAAK,SAAS,EAAE,CAAC;QAClC,WAAW,IAAI,MAAM,CAAC,QAAQ,GAAG,GAAG,CAAC,CAAC,iCAAiC;QACvE,WAAW,IAAI,GAAG,CAAC;IACrB,CAAC;IACD,IAAI,MAAM,CAAC,aAAa,KAAK,SAAS,EAAE,CAAC;QACvC,WAAW,IAAI,MAAM,CAAC,aAAa,GAAG,GAAG,CAAC;QAC1C,WAAW,IAAI,GAAG,CAAC;IACrB,CAAC;IACD,IAAI,MAAM,CAAC,SAAS,KAAK,SAAS,EAAE,CAAC;QACnC,WAAW,IAAI,MAAM,CAAC,SAAS,GAAG,GAAG,CAAC;QACtC,WAAW,IAAI,GAAG,CAAC;IACrB,CAAC;IAED,IAAI,WAAW,GAAG,CAAC,EAAE,CAAC;QACpB,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC;IAC3D,CAAC;IACD,IAAI,MAAM,CAAC,QAAQ,KAAK,SAAS,EAAE,CAAC;QAClC,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC;IAC9E,CAAC;IAED,OAAO,IAAI,CAAC,CAAC,6CAA6C;AAC5D,CAAC;AAED,kFAAkF;AAClF,MAAM,UAAU,sBAAsB,CAAC,OAAyB;IAC9D,IAAI,OAAO,CAAC,MAAM,IAAI,CAAC;QAAE,OAAO,CAAC,GAAG,OAAO,CAAC,CAAC;IAE7C,OAAO,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;QAChC,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC;QAChE,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC;QAEhE,MAAM,MAAM,GAAG,qBAAqB,CAAC,kBAAkB,CAAC,KAAK,CAAC,CAAC,CAAC;QAChE,MAAM,MAAM,GAAG,qBAAqB,CAAC,kBAAkB,CAAC,KAAK,CAAC,CAAC,CAAC;QAEhE,OAAO,MAAM,GAAG,MAAM,CAAC,CAAC,sBAAsB;IAChD,CAAC,CAAC,CAAC;AACL,CAAC"}
+{"version":3,"file":"benchmarks.js","sourceRoot":"","sources":["../src/benchmarks.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,gBAAgB,CAAC;AAC7D,OAAO,EAAE,iBAAiB,EAAE,MAAM,6BAA6B,CAAC;AAkBhE,MAAM,OAAO,GAAG,EAAE,CAAC;AAEnB;;;;;;;;;;;;;;;GAeG;AACH,MAAM,UAAU,WAAW,CAAC,IAAY,EAAE,OAAmC,EAAE;IAC7E,MAAM,GAAG,GAAG,aAAa,CAAC,IAAI,EAAE,YAAY,EAAE,EAAE,MAAM,IAAI,EAAE,CAAC,CAAC;IAC9D,IAAI,GAAG,IAAI,OAAO,GAAG,CAAC,GAAG,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;QAChD,OAAO;YACL,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,EAAE,cAAc;YAC/D,KAAK,EAAE,UAAU;YACjB,OAAO,EAAE,GAAG,CAAC,GAAG,CAAC,OAAO,IAAI,EAAE;YAC9B,WAAW,EAAE,GAAG,CAAC,GAAG,CAAC,YAAY,IAAI,CAAC;YACtC,KAAK,EAAE,GAAG,CAAC,KAAK;YAChB,WAAW,EAAE,GAAG,CAAC,GAAG,CAAC,IAAI;SAC1B,CAAC;IACJ,CAAC;IAED,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC;QACb,MAAM,QAAQ,GAAG,iBAAiB,CAChC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,EAChB,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,EACjB,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CACvD,CAAC;QACF,IAAI,QAAQ,KAAK,IAAI;YAAE,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,KAAK,EAAE,WAAW,EAAE,CAAC;IACxE,CAAC;IAED,OAAO,EAAE,KAAK,EAAE,OAAO,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC;AAC9C,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,sBAAsB,CAAC,OAAyB;IAC9D,IAAI,OAAO,CAAC,MAAM,IAAI,CAAC;QAAE,OAAO,CAAC,GAAG,OAAO,CAAC,CAAC;IAE7C,MAAM,MAAM,GAAG,CAAC,CAAiB,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC;IAC1F,MAAM,KAAK,GAAG,IAAI,GAAG,EAAkB,CAAC;IACxC,MAAM,KAAK,GAAG,CAAC,CAAiB,EAAE,EAAE;QAClC,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;QACvB,IAAI,CAAC,GAAG,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACxB,IAAI,CAAC,KAAK,SAAS,EAAE,CAAC;YACpB,CAAC,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC;YAC5B,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACrB,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,OAAO,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;AAC1D,CAAC"}

package/dist/candidates.d.ts

@@ -1,17 +1,19 @@
 import type { Config } from "./config.js";
 import type { ModelCatalog } from "./catalog.js";
 import type { PingLoop } from "./ping/cadence.js";
-import { type BenchmarkScores } from "./benchmarks.js";
+import { type StrengthBasis } from "./benchmarks.js";
+import { type MetadataSource } from "./metadata.js";
 import { type CircuitBreaker } from "./circuit-breaker.js";
 import { type CapabilityScore } from "./registry.js";
 /**
  * Everything known about one offload destination, kept as SEPARATE raw dimensions.
  *
- * Deliberately un-blended: benchmark quality, live stability, quota and observed traffic measure
- * different things and trade off against each other differently per task ("cheapest that can do
- * it" vs "best available"). Averaging them into one number would bury exactly the judgement the
- * reader is here to make. The two composites that already exist in the product are reported under
- * `sortInputs`, labelled as what they drive — not as a recommendation.
+ * Deliberately un-blended: each leaderboard, live stability, cost, quota and observed traffic
+ * measure different things and trade off against each other differently per task ("cheapest that
+ * can do it" vs "best available"). Averaging them into one number would bury exactly the judgement
+ * the reader is here to make. The one scalar that does exist lives under `sortInputs` with the
+ * basis and signals that produced it — because ordering a pool requires an order, not because it
+ * is a recommendation.
  */
 export interface Candidate {
     spec: string;
@@ -24,7 +26,6 @@ export interface Candidate {
     hasKey: boolean;
     /** In the provider's live /models catalog. null = not checkable (anthropic kind, or catalog down). */
     listed: boolean | null;
-    benchmarks: BenchmarkScores;
     /** BFCL tool-use + LMArena scores, best-effort id→leaderboard match (null when no confident match). */
     capability: CapabilityScore | null;
     health: {
@@ -50,12 +51,53 @@ export interface Candidate {
         avgLatencyMs: number | null;
         lastCalledAt: string | null;
     } | null;
+    /**
+     * Limits, each with its own provenance. `provider` = this provider published it about its own
+     * deployment; `reference` = borrowed from another provider serving the same model id (different
+     * deployment, so indicative only — `metadataReferenceFrom` names it); `static-table` = the
+     * hardcoded table, including its blanket 128k/4096 guess.
+     */
     contextLength: number | null;
+    contextLengthSource: MetadataSource | null;
     maxOutputTokens: number | null;
-    /** Existing composites, reported for transparency about what the proxy itself sorts by. */
+    maxOutputTokensSource: MetadataSource | null;
+    metadataReferenceFrom?: string;
+    /** Per-million-token price, with the same provenance rules — a model free on one host and
+     *  metered on another must not report the other's rate. */
+    pricePerMTokIn: number | null;
+    pricePerMTokOut: number | null;
+    priceSource: MetadataSource | null;
+    supportsTools: boolean | null;
+    /** Which leaderboards published anything about this model. */
+    capabilitySources: string[];
+    /** Raw per-source capability values — kept separate, never collapsed into one another. */
+    scores: {
+        /** Berkeley Function-Calling: tool-call accuracy, multi-turn, irrelevance detection. */
+        bfclOverall: number | null;
+        bfclMultiTurn: number | null;
+        bfclIrrelevance: number | null;
+        /** Artificial Analysis indices, via OpenRouter. */
+        aaIntelligence: number | null;
+        aaCoding: number | null;
+        aaAgentic: number | null;
+        /** Aider polyglot edit benchmark + edit-format compliance. */
+        aiderPassRate: number | null;
+        aiderWellFormed: number | null;
+        /** Design Arena Elo, averaged per arena (means, not measurements — see the `_mean` naming). */
+        designArenaAgentsEloMean: number | null;
+        designArenaModelsEloMean: number | null;
+        /** LMArena. */
+        arenaRating: number | null;
+        arenaRank: number | null;
+    };
+    /** What the proxy itself sorts by — the ONE place a scalar exists, with its provenance. */
     sortInputs: {
         /** Drives `routing.benchmarkSort` ordering within a pool. */
-        benchmarkQuality: number;
+        strength: number;
+        /** snapshot | telemetry | neutral. A telemetry score is not a capability score. */
+        strengthBasis: StrengthBasis;
+        /** How many published signals backed it. 1 is a guess; 5 is a consensus. */
+        strengthSignals: string[];
         /** Drives circuit-breaker candidate ordering. 100 when untracked. */
         breakerStability: number;
     };

package/dist/candidates.js

@@ -1,12 +1,14 @@
 import { POOL_PREFIX } from "./config.js";
-import { getBenchmarkScores, calculateQualityScore } from "./benchmarks.js";
-import { getModelMetadata } from "./metadata.js";
+import { getStrength } from "./benchmarks.js";
+import { findTierModel } from "./tier-data.js";
+import { resolveMetadata } from "./metadata.js";
 import { globalCircuitBreaker } from "./circuit-breaker.js";
 import { loadRuntimeTelemetry } from "./ping/runtime-telemetry.js";
 import { loadTierData, joinCapability } from "./registry.js";
 const NOTE = "Raw per-dimension data for choosing an offload target. Nothing here is ranked or averaged — " +
-    "order is config order (pools, then subagent targets). `sortInputs` reports the composites the " +
-    "proxy already sorts by; they are not a recommendation.";
+    "order is config order (pools, then subagent targets), and every source's score is kept " +
+    "separately under `scores`. `sortInputs` reports the ONE scalar the proxy needs for pool " +
+    "ordering, with the basis and signal list that produced it; it is not a recommendation.";
 /** Expand a spec that may itself be `pool/<name>` into concrete "provider/model" specs. */
 function expandSpec(spec, cfg) {
     const i = spec.indexOf("/");
@@ -69,8 +71,25 @@ export async function buildCandidates(cfg, opts = {}) {
         const summary = opts.pingLoop && model ? opts.pingLoop.getModelSummary(provider, model) : null;
         const state = breaker.getState(spec);
         const obs = telemetry.models[`${provider}/${model}`];
-        const meta = getModelMetadata(model ?? provider);
-        const benchmarks = getBenchmarkScores(spec);
+        const strength = getStrength(spec);
+        const tier = findTierModel(model ?? spec, byNorm)?.rec;
+        const num = (k) => (typeof tier?.[k] === "number" ? tier[k] : null);
+        // Limits THIS provider publishes about its own deployment, if any. NIM publishes none;
+        // Groq and Mistral publish real ones. The snapshot's numbers come from OpenRouter, so for a
+        // NIM target they are a different deployment's figures and are labelled `reference`, never
+        // presented as this provider's own.
+        const providerLimits = p && p.kind === "openai" && model && opts.catalog
+            ? await opts.catalog.limits(provider, p, model).catch(() => null)
+            : null;
+        const meta = resolveMetadata(model ?? provider, {
+            providerLimits,
+            reference: {
+                contextLength: num("context_length"),
+                pricePromptPerToken: num("price_prompt"),
+                priceCompletionPerToken: num("price_completion"),
+                from: "openrouter",
+            },
+        });
         candidates.push({
             spec,
             provider,
@@ -79,7 +98,6 @@ export async function buildCandidates(cfg, opts = {}) {
             subagentTiers: membership.subagentTiers,
             hasKey: p?.authEnv ? !!process.env[p.authEnv]?.trim() : true,
             listed,
-            benchmarks,
             capability: model ? joinCapability(model, byNorm) : null,
             health: summary
                 ? {
@@ -107,10 +125,34 @@ export async function buildCandidates(cfg, opts = {}) {
                     lastCalledAt: obs.lastCalledAt ? new Date(obs.lastCalledAt).toISOString() : null,
                 }
                 : null,
-            contextLength: meta.contextLength ?? null,
-            maxOutputTokens: meta.maxOutputTokens ?? null,
+            contextLength: meta.contextLength,
+            contextLengthSource: meta.contextLengthSource,
+            maxOutputTokens: meta.maxOutputTokens,
+            maxOutputTokensSource: meta.maxOutputTokensSource,
+            ...(meta.referenceFrom ? { metadataReferenceFrom: meta.referenceFrom } : {}),
+            pricePerMTokIn: meta.pricePerMTokIn,
+            pricePerMTokOut: meta.pricePerMTokOut,
+            priceSource: meta.priceSource,
+            supportsTools: typeof tier?.supports_tools === "boolean" ? tier.supports_tools : null,
+            capabilitySources: Array.isArray(tier?.sources) ? tier.sources : [],
+            scores: {
+                bfclOverall: num("bfcl_overall"),
+                bfclMultiTurn: num("bfcl_multi_turn"),
+                bfclIrrelevance: num("bfcl_irrelevance"),
+                aaIntelligence: num("aa_intelligence"),
+                aaCoding: num("aa_coding"),
+                aaAgentic: num("aa_agentic"),
+                aiderPassRate: num("aider_pass_rate"),
+                aiderWellFormed: num("aider_well_formed"),
+                designArenaAgentsEloMean: num("design_arena_agents_elo_mean"),
+                designArenaModelsEloMean: num("design_arena_models_elo_mean"),
+                arenaRating: num("arena_rating"),
+                arenaRank: num("arena_rank"),
+            },
             sortInputs: {
-                benchmarkQuality: calculateQualityScore(benchmarks),
+                strength: strength.score,
+                strengthBasis: strength.basis,
+                strengthSignals: strength.signals ?? [],
                 breakerStability: breaker.getStabilityScore(spec),
             },
         });

package/dist/candidates.js.map

@@ -1 +1 @@
-{"version":3,"file":"candidates.js","sourceRoot":"","sources":["../src/candidates.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAG1C,OAAO,EAAE,kBAAkB,EAAE,qBAAqB,EAAwB,MAAM,iBAAiB,CAAC;AAClG,OAAO,EAAE,gBAAgB,EAAE,MAAM,eAAe,CAAC;AACjD,OAAO,EAAE,oBAAoB,EAAuB,MAAM,sBAAsB,CAAC;AACjF,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,YAAY,EAAE,cAAc,EAAwB,MAAM,eAAe,CAAC;AAkEnF,MAAM,IAAI,GACR,8FAA8F;IAC9F,gGAAgG;IAChG,wDAAwD,CAAC;AAE3D,2FAA2F;AAC3F,SAAS,UAAU,CAAC,IAAY,EAAE,GAAW;IAC3C,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAChD,IAAI,IAAI,KAAK,WAAW;QAAE,OAAO,CAAC,IAAI,CAAC,CAAC;IACxC,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/C,OAAO,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;AACzC,CAAC;AAED;;;;GAIG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,GAAG,GAAG,IAAI,GAAG,EAAwD,CAAC;IAC5E,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAE;QAC7B,IAAI,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACtB,IAAI,CAAC,CAAC,EAAE,CAAC;YACP,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;YACrC,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACnB,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,IAAI,EAAE,CAAC,EAAE,CAAC;QACpE,KAAK,MAAM,CAAC,IAAI,KAAK;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACnD,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,SAAS,IAAI,EAAE,CAAC,EAAE,CAAC;QACvE,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,IAAI,EAAE,GAAG,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC3E,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;AAClG,CAAC;AAED,+DAA+D;AAC/D,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,GAAW,EACX,OAOI,EAAE;IAEN,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,oBAAoB,CAAC;IACrD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;IACvC,MAAM,MAAM,GAAG,YAAY,EAAE,EAAE,MAAM,IAAI,EAAE,CAAC;IAC5C,MAAM,SAAS,GAAG,oBAAoB,EAAE,CAAC;IAEzC,MAAM,UAAU,GAAgB,EAAE,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,EAAE,UAAU,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,EAAE,QAAQ,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,IAAI,CAAC,QAAQ;YAAE,SAAS;QAC1D,MAAM,CAAC,GAA+B,GAAG,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QAE9D,IAAI,MAAM,GAAmB,IAAI,CAAC;QAClC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;YACtD,IAAI,CAAC;gBACH,MAAM,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC;YACtD,CAAC;YAAC,MAAM,CAAC;gBACP,MAAM,GAAG,IAAI,CAAC;YAChB,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,QAAQ,IAAI,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,eAAe,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAC/F,MAAM,KAAK,GAAG,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrC,MAAM,GAAG,GAAG,SAAS,CAAC,MAAM,CAAC,GAAG,QAAQ,IAAI,KAAK,EAAE,CAAC,CAAC;QACrD,MAAM,IAAI,GAAG,gBAAgB,CAAC,KAAK,IAAI,QAAQ,CAAC,CAAC;QACjD,MAAM,UAAU,GAAG,kBAAkB,CAAC,IAAI,CAAC,CAAC;QAE5C,UAAU,CAAC,IAAI,CAAC;YACd,IAAI;YACJ,QAAQ;YACR,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC3B,KAAK,EAAE,UAAU,CAAC,KAAK;YACvB,aAAa,EAAE,UAAU,CAAC,aAAa;YACvC,MAAM,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,IAAI;YAC5D,MAAM;YACN,UAAU;YACV,UAAU,EAAE,KAAK,CAAC,CAAC,CAAC,cAAc,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI;YACxD,MAAM,EAAE,OAAO;gBACb,CAAC,CAAC;oBACE,OAAO,EAAE,OAAO,CAAC,OAAO;oBACxB,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,QAAQ,EAAE,OAAO,CAAC,QAAQ;oBAC1B,SAAS,EAAE,OAAO,CAAC,SAAS;oBAC5B,YAAY,EAAE,OAAO,CAAC,YAAY;oBAClC,UAAU,EAAE,OAAO,CAAC,UAAU;iBAC/B;gBACH,CAAC,CAAC,IAAI;YACR,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;YAC7E,OAAO,EAAE;gBACP,IAAI,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,EAAE,KAAK,CAAC;gBACrC,mBAAmB,EAAE,KAAK,EAAE,mBAAmB,IAAI,CAAC;gBACpD,UAAU,EAAE,KAAK,EAAE,UAAU,IAAI,IAAI;gBACrC,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,IAAI,CAAC,CAAC,GAAG,KAAK,CAAC;aACtE;YACD,QAAQ,EAAE,GAAG;gBACX,CAAC,CAAC;oBACE,UAAU,EAAE,GAAG,CAAC,UAAU;oBAC1B,YAAY,EAAE,GAAG,CAAC,YAAY;oBAC9B,YAAY,EAAE,GAAG,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,cAAc,GAAG,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI;oBACzF,YAAY,EAAE,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,IAAI;iBACjF;gBACH,CAAC,CAAC,IAAI;YACR,aAAa,EAAE,IAAI,CAAC,aAAa,IAAI,IAAI;YACzC,eAAe,EAAE,IAAI,CAAC,eAAe,IAAI,IAAI;YAC7C,UAAU,EAAE;gBACV,gBAAgB,EAAE,qBAAqB,CAAC,UAAU,CAAC;gBACnD,gBAAgB,EAAE,OAAO,CAAC,iBAAiB,CAAC,IAAI,CAAC;aAClD;SACF,CAAC,CAAC;IACL,CAAC;IAED,OAAO;QACL,YAAY,EAAE,IAAI,CAAC,GAAG,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,EAAE;QACvD,eAAe,EAAE,GAAG,CAAC,OAAO,CAAC,OAAO,KAAK,IAAI;QAC7C,IAAI,EAAE,IAAI;QACV,UAAU;KACX,CAAC;AACJ,CAAC"}
+{"version":3,"file":"candidates.js","sourceRoot":"","sources":["../src/candidates.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAG1C,OAAO,EAAE,WAAW,EAAsB,MAAM,iBAAiB,CAAC;AAClE,OAAO,EAAE,aAAa,EAAE,MAAM,gBAAgB,CAAC;AAC/C,OAAO,EAAE,eAAe,EAAuB,MAAM,eAAe,CAAC;AACrE,OAAO,EAAE,oBAAoB,EAAuB,MAAM,sBAAsB,CAAC;AACjF,OAAO,EAAE,oBAAoB,EAAE,MAAM,6BAA6B,CAAC;AACnE,OAAO,EAAE,YAAY,EAAE,cAAc,EAAwB,MAAM,eAAe,CAAC;AA2GnF,MAAM,IAAI,GACR,8FAA8F;IAC9F,yFAAyF;IACzF,0FAA0F;IAC1F,wFAAwF,CAAC;AAE3F,2FAA2F;AAC3F,SAAS,UAAU,CAAC,IAAY,EAAE,GAAW;IAC3C,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAChD,IAAI,IAAI,KAAK,WAAW;QAAE,OAAO,CAAC,IAAI,CAAC,CAAC;IACxC,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IAC/C,OAAO,GAAG,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;AACzC,CAAC;AAED;;;;GAIG;AACH,SAAS,YAAY,CAAC,GAAW;IAC/B,MAAM,GAAG,GAAG,IAAI,GAAG,EAAwD,CAAC;IAC5E,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,EAAE;QAC7B,IAAI,CAAC,GAAG,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACtB,IAAI,CAAC,CAAC,EAAE,CAAC;YACP,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,EAAE,aAAa,EAAE,EAAE,EAAE,CAAC;YACrC,GAAG,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;QACnB,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,IAAI,EAAE,CAAC,EAAE,CAAC;QACpE,KAAK,MAAM,CAAC,IAAI,KAAK;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACnD,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,SAAS,IAAI,EAAE,CAAC,EAAE,CAAC;QACvE,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,IAAI,EAAE,GAAG,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC3E,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,MAAM,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAC5B,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;AAClG,CAAC;AAED,+DAA+D;AAC/D,MAAM,CAAC,KAAK,UAAU,eAAe,CACnC,GAAW,EACX,OAOI,EAAE;IAEN,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,oBAAoB,CAAC;IACrD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC;IACvC,MAAM,MAAM,GAAG,YAAY,EAAE,EAAE,MAAM,IAAI,EAAE,CAAC;IAC5C,MAAM,SAAS,GAAG,oBAAoB,EAAE,CAAC;IAEzC,MAAM,UAAU,GAAgB,EAAE,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,EAAE,UAAU,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC;QACnD,MAAM,EAAE,QAAQ,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,QAAQ,IAAI,QAAQ,KAAK,IAAI,CAAC,QAAQ;YAAE,SAAS;QAC1D,MAAM,CAAC,GAA+B,GAAG,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC;QAE9D,IAAI,MAAM,GAAmB,IAAI,CAAC;QAClC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;YACtD,IAAI,CAAC;gBACH,MAAM,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC;YACtD,CAAC;YAAC,MAAM,CAAC;gBACP,MAAM,GAAG,IAAI,CAAC;YAChB,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,QAAQ,IAAI,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,eAAe,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAC/F,MAAM,KAAK,GAAG,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrC,MAAM,GAAG,GAAG,SAAS,CAAC,MAAM,CAAC,GAAG,QAAQ,IAAI,KAAK,EAAE,CAAC,CAAC;QACrD,MAAM,QAAQ,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,IAAI,GAAG,aAAa,CAAC,KAAK,IAAI,IAAI,EAAE,MAAM,CAAC,EAAE,GAAG,CAAC;QACvD,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAE,IAAI,CAAC,CAAC,CAAY,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAExF,uFAAuF;QACvF,4FAA4F;QAC5F,2FAA2F;QAC3F,oCAAoC;QACpC,MAAM,cAAc,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ,IAAI,KAAK,IAAI,IAAI,CAAC,OAAO;YACtE,CAAC,CAAC,MAAM,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,EAAE,CAAC,EAAE,KAAK,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC;YACjE,CAAC,CAAC,IAAI,CAAC;QACT,MAAM,IAAI,GAAG,eAAe,CAAC,KAAK,IAAI,QAAQ,EAAE;YAC9C,cAAc;YACd,SAAS,EAAE;gBACT,aAAa,EAAE,GAAG,CAAC,gBAAgB,CAAC;gBACpC,mBAAmB,EAAE,GAAG,CAAC,cAAc,CAAC;gBACxC,uBAAuB,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBAChD,IAAI,EAAE,YAAY;aACnB;SACF,CAAC,CAAC;QAEH,UAAU,CAAC,IAAI,CAAC;YACd,IAAI;YACJ,QAAQ;YACR,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC3B,KAAK,EAAE,UAAU,CAAC,KAAK;YACvB,aAAa,EAAE,UAAU,CAAC,aAAa;YACvC,MAAM,EAAE,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,IAAI;YAC5D,MAAM;YACN,UAAU,EAAE,KAAK,CAAC,CAAC,CAAC,cAAc,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI;YACxD,MAAM,EAAE,OAAO;gBACb,CAAC,CAAC;oBACE,OAAO,EAAE,OAAO,CAAC,OAAO;oBACxB,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,KAAK,EAAE,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI;oBAChD,QAAQ,EAAE,OAAO,CAAC,QAAQ;oBAC1B,SAAS,EAAE,OAAO,CAAC,SAAS;oBAC5B,YAAY,EAAE,OAAO,CAAC,YAAY;oBAClC,UAAU,EAAE,OAAO,CAAC,UAAU;iBAC/B;gBACH,CAAC,CAAC,IAAI;YACR,YAAY,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;YAC7E,OAAO,EAAE;gBACP,IAAI,EAAE,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,EAAE,KAAK,CAAC;gBACrC,mBAAmB,EAAE,KAAK,EAAE,mBAAmB,IAAI,CAAC;gBACpD,UAAU,EAAE,KAAK,EAAE,UAAU,IAAI,IAAI;gBACrC,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,IAAI,CAAC,CAAC,GAAG,KAAK,CAAC;aACtE;YACD,QAAQ,EAAE,GAAG;gBACX,CAAC,CAAC;oBACE,UAAU,EAAE,GAAG,CAAC,UAAU;oBAC1B,YAAY,EAAE,GAAG,CAAC,YAAY;oBAC9B,YAAY,EAAE,GAAG,CAAC,UAAU,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,cAAc,GAAG,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,IAAI;oBACzF,YAAY,EAAE,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,IAAI;iBACjF;gBACH,CAAC,CAAC,IAAI;YACR,aAAa,EAAE,IAAI,CAAC,aAAa;YACjC,mBAAmB,EAAE,IAAI,CAAC,mBAAmB;YAC7C,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,qBAAqB,EAAE,IAAI,CAAC,qBAAqB;YACjD,GAAG,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,qBAAqB,EAAE,IAAI,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC5E,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,eAAe,EAAE,IAAI,CAAC,eAAe;YACrC,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,aAAa,EAAE,OAAO,IAAI,EAAE,cAAc,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC,IAAI;YACrF,iBAAiB,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC,CAAC,CAAE,IAAI,CAAC,OAAoB,CAAC,CAAC,CAAC,EAAE;YACjF,MAAM,EAAE;gBACN,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,kBAAkB,CAAC;gBACxC,cAAc,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACtC,QAAQ,EAAE,GAAG,CAAC,WAAW,CAAC;gBAC1B,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;gBAC5B,aAAa,EAAE,GAAG,CAAC,iBAAiB,CAAC;gBACrC,eAAe,EAAE,GAAG,CAAC,mBAAmB,CAAC;gBACzC,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,wBAAwB,EAAE,GAAG,CAAC,8BAA8B,CAAC;gBAC7D,WAAW,EAAE,GAAG,CAAC,cAAc,CAAC;gBAChC,SAAS,EAAE,GAAG,CAAC,YAAY,CAAC;aAC7B;YACD,UAAU,EAAE;gBACV,QAAQ,EAAE,QAAQ,CAAC,KAAK;gBACxB,aAAa,EAAE,QAAQ,CAAC,KAAK;gBAC7B,eAAe,EAAE,QAAQ,CAAC,OAAO,IAAI,EAAE;gBACvC,gBAAgB,EAAE,OAAO,CAAC,iBAAiB,CAAC,IAAI,CAAC;aAClD;SACF,CAAC,CAAC;IACL,CAAC;IAED,OAAO;QACL,YAAY,EAAE,IAAI,CAAC,GAAG,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,CAAC,WAAW,EAAE;QACvD,eAAe,EAAE,GAAG,CAAC,OAAO,CAAC,OAAO,KAAK,IAAI;QAC7C,IAAI,EAAE,IAAI;QACV,UAAU;KACX,CAAC;AACJ,CAAC"}

package/dist/catalog.d.ts

@@ -1,4 +1,21 @@
 import type { ProviderConfig } from "./config.js";
+/**
+ * Limits a provider publishes about its OWN deployment of a model.
+ *
+ * Deliberately per-(provider, model): the same model id served by two providers is two different
+ * deployments with different ceilings, so one provider's numbers must never be presented as
+ * another's. Null means "this provider does not publish it" — NIM's /models returns only
+ * id/object/created/owned_by, while Groq and Mistral publish real limits.
+ */
+export interface ModelLimits {
+    contextLength: number | null;
+    maxOutputTokens: number | null;
+    /** Per-TOKEN price, as published. Per-provider for the same reason limits are. */
+    pricePromptPerToken: number | null;
+    priceCompletionPerToken: number | null;
+}
+/** Read limits + pricing out of one `/models` record, including a nested `top_provider` (OpenRouter). */
+export declare function limitsFromRecord(rec: Record<string, unknown>): ModelLimits;
 /**
  * Live per-provider model catalog — model ids are DISCOVERED from each provider's
  * OpenAI-compatible `/models` endpoint, never hand-maintained. In-memory TTL cache
@@ -14,6 +31,8 @@ export declare class ModelCatalog {
     private refreshing;
     /** In-flight blocking fetches (cold start / forced) — dedups concurrent requests. */
     private pending;
+    /** Limits harvested by the most recent `fetch()`, handed to the Entry by its caller. */
+    private lastLimits;
     constructor(opts?: {
         ttlMs?: number;
         cachePath?: string | null;
@@ -53,5 +72,16 @@ export declare class ModelCatalog {
         now?: number;
         fetchFn?: typeof fetch;
     }): Promise<boolean | null>;
+    /**
+     * Limits this provider publishes for one of its own models, or null when it publishes none.
+     *
+     * Null is meaningful and must not be papered over with another provider's number — see
+     * `resolveMetadata()` in metadata.ts, which decides what to fall back to and labels it.
+     */
+    limits(name: string, cfg: ProviderConfig, model: string, opts?: {
+        force?: boolean;
+        now?: number;
+        fetchFn?: typeof fetch;
+    }): Promise<ModelLimits | null>;
     private fetch;
 }