pi-provider-litellm 3.0.1 → 3.1.0

package/README.md

@@ -176,6 +176,7 @@ Native Messages authenticates with `x-api-key`; every transport carries the `x-l
 | `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Background and explicit discovery fetch timeout in ms; `0` disables automatic discovery |
 | `LITELLM_CLI_JWT_EXPIRATION_HOURS` | `24` | CLI SSO token lifetime fallback for older proxies whose poll response omits `expires_in`; mirror a non-default proxy setting locally |
 | `LITELLM_VERBOSE_DISCOVERY` | unset | If `1`, enable progress messages during model and MCP discovery (login, refresh, startup), including MCP prepared/registered/dropped counts. Progress messages are off by default; MCP safety diagnostics (see below) are always reported regardless of this setting |
+| `LITELLM_MODELS_DEV` | unset | If `1`, enrich discovered metadata (limits, prices, effort lists) from models.dev, cached for 28 days in `litellm-models-dev.json`. Off by default because LiteLLM's `/model/info` is authoritative; use it when LiteLLM's model map lacks a model's metadata |
 
 Only use a trusted `LITELLM_API_KEY_HELPER` or `!command`. Prefer an absolute executable path, keep secrets out of command arguments, and print only the token to stdout without logging it to stderr.
 
@@ -293,7 +294,7 @@ Before tagging a release, keep `package.json` and `package-lock.json` versions i
 
 Dynamic catalogs are persisted by Pi in `~/.pi/agent/models-store.json`. Credentials remain in `~/.pi/agent/auth.json`. Legacy `litellm-models.json` model caches are ignored and never deleted. `litellm-models-dev.json` is the models.dev cache and is refreshed in place.
 
-For every genuine `/model/info` row, including deployment details fetched through `/health`, the extension requests `https://models.dev/api.json` for enrichment and caches the result for 28 days in `~/.pi/agent/litellm-models-dev.json`. Health-only entries without deployment details use the bounded Pi catalog lookup without models.dev enrichment. `PI_OFFLINE` suppresses activation-time discovery and the models.dev request. `LITELLM_OFFLINE=1` also disables LiteLLM discovery; direct discovery callers use only an existing models.dev cache and do not refresh it.
+LiteLLM's `/model/info` is the metadata authority, so models.dev enrichment is an opt-in escape hatch. With `LITELLM_MODELS_DEV=1`, for every genuine `/model/info` row, including deployment details fetched through `/health`, the extension requests `https://models.dev/api.json` for enrichment and caches the result for 28 days in `~/.pi/agent/litellm-models-dev.json`. Without it, discovery neither requests models.dev nor reads that cache; Pi's own catalog still applies. Health-only entries without deployment details use the bounded Pi catalog lookup without models.dev enrichment. `PI_OFFLINE` suppresses activation-time discovery and the models.dev request. `LITELLM_OFFLINE=1` also disables LiteLLM discovery; direct discovery callers use only an existing models.dev cache and do not refresh it.
 
 Opening `/model` refreshes configured provider catalogs in the background using Pi's native model lifecycle.
 
@@ -329,7 +330,7 @@ Discovery policy is versioned in Pi's persisted model catalog. After upgrading,
 LiteLLM may load-balance one public `model_name` across deployments with different backends or model versions. The extension reduces `/model/info` rows conservatively before publishing one Pi model. A route group that mixes a chat-style deployment with an explicitly incompatible mode such as embedding is withheld entirely and reported with a bounded diagnostic.
 
 - Responses is selected only when every deployment supports it under the protocol rules above; a deployment that requires Chat keeps the group on Chat.
-- Vision and reasoning are advertised only when every routable deployment resolves them as supported. An explicit `supports_reasoning: false` suppresses all thinking controls, even when catalog or router effort metadata exists. Catalog thinking controls are intersected per level: a level is exposed only when every deployment maps it to the same value, while disagreement or absence becomes an explicit denial. Reasoning selectors additionally require an accepted wire carrier on every deployment. Public effort lists are intersected, explicit LiteLLM denials win, and `xhigh`/`max` need unanimous explicit support. Null or absent standard-effort flags have no opinion.
+- Vision and reasoning are advertised only when every routable deployment resolves them as supported. An explicit `supports_reasoning: false` suppresses all thinking controls, even when catalog or router effort metadata exists. Catalog thinking controls are intersected per level: a level is exposed only when every deployment maps it to the same value, while disagreement or absence becomes an explicit denial. Catalog maps are not complete effort lists, so absence has one exception: an omitted `minimal`, `low`, `medium` or `high` counts as Pi's default spelling of that level. An omitted `off`, `xhigh` or `max` still differs from an explicit mapping. Reasoning selectors additionally require an accepted wire carrier on every deployment. Public effort lists are intersected, explicit LiteLLM denials win, and `xhigh`/`max` need unanimous explicit support. Null or absent standard-effort flags have no opinion. A deployment that declares `model_info.reasoning_effort_levels` answers every level from that list, ahead of its per-level flags, as LiteLLM itself reads it.
 - Context and output limits use the minimum resolved value across deployments.
 - Each displayed base-price field uses the maximum only when every deployment resolves that field; unresolved fields remain zero and the model name is suffixed with ` (incomplete metadata)`.
 - When complete base pricing comes from the catalog under unanimous catalog authority, tiered pricing is the conservative worst-case envelope. Its usable finite non-negative thresholds are the sorted union of every deployment's thresholds, and each price field at each interval is the maximum applicable rate across all deployments, floored by each deployment's base rate. Differing tier breakpoints are therefore preserved rather than treated as incompatible. An explicit `/model/info` base-price field replaces catalog pricing for that field at every threshold; catalog tiers remain for unaffected fields and are omitted only when every base-price field is explicit.
@@ -345,7 +346,9 @@ The ` (incomplete metadata)` suffix marks reduced `/model/info` groups, incomple
 
 ### Reasoning controls
 
-Reasoning levels require deployment evidence for the parameter Pi sends: `reasoning_effort` for effort levels or `thinking` for a native thinking switch. Public route names never authorize these controls. Evidence-free fallback models retain catalog presentation metadata but expose no speculative selector.
+Reasoning levels require deployment evidence for the parameter Pi sends: `reasoning_effort` for effort levels or `thinking` for a native thinking switch. Public route names never authorize these controls.
+
+LiteLLM omits `supported_openai_params` for a deployment its model map does not describe. Such a deployment keeps the `reasoning_effort` carrier when its `model_info` explicitly sets `supports_reasoning: true`, which is the operator's opt-in. A `supported_openai_params` list that omits `reasoning_effort` still denies effort levels unless `litellm_params.allowed_openai_params` adds it. Kimi and DeepSeek generations are excluded because their contracts name their own carriers. Evidence-free fallback models retain catalog presentation metadata but expose no speculative selector.
 
 Kimi K2.5/K2.6 use an on/off thinking switch; K2.7 Code/Highspeed is always thinking. Kimi K3 and DeepSeek V4 use public effort evidence with LiteLLM overrides. Responses levels are translated to valid `reasoning.effort` values; Chat-only compatibility fields never reach Responses models. Legacy cached level maps remain usable while discovery refreshes, and stored Moonshot compatibility can restore safe display normalization.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-provider-litellm",
-  "version": "3.0.1",
+  "version": "3.1.0",
   "description": "LiteLLM proxy provider extension for Pi",
   "type": "module",
   "license": "MIT",

package/src/discover.ts

@@ -399,7 +399,7 @@ function adaptPublicCatalogRecord(
   catalogModelId: string,
   record: PublicCatalogRecord | undefined,
 ): CatalogResolution {
-  const piProvider = adapterCatalogProvider(record?.provider ?? provider);
+  const piProvider = adapterCatalogProvider(record?.piProvider ?? record?.provider ?? provider);
   const resolved = piProvider
     ? (resolveCatalogModel(record?.modelId ?? catalogModelId, piProvider) ??
       resolveCatalogModel(catalogModelId, piProvider))

package/src/index.ts

@@ -31,7 +31,14 @@ import {
 } from "./mcp-tools.js";
 import { createLiteLLMProvider, DEFAULT_LITELLM_BASE_URL, isPlaceholderHost, toNativeModels } from "./provider.js";
 import { createSkillsPromptSection, createSkillToolDefinitions, listSkills } from "./skills.js";
-import type { LiteLLMApi, LiteLLMModel, LiteLLMModelPolicy, LiteLLMRuntimeAuth, ResolvedCredentials } from "./types.js";
+import type {
+  DiscoveryOptions,
+  LiteLLMApi,
+  LiteLLMModel,
+  LiteLLMModelPolicy,
+  LiteLLMRuntimeAuth,
+  ResolvedCredentials,
+} from "./types.js";
 
 const PROVIDER_NAME = "litellm";
 const SETTINGS_KEY = "litellm";
@@ -44,6 +51,7 @@ const ENV_TIMEOUT = "LITELLM_DISCOVERY_TIMEOUT_MS";
 const ENV_CLI_JWT_EXPIRATION_HOURS = "LITELLM_CLI_JWT_EXPIRATION_HOURS";
 const ENV_OFFLINE = "LITELLM_OFFLINE";
 const ENV_VERBOSE_DISCOVERY = "LITELLM_VERBOSE_DISCOVERY";
+const ENV_MODELS_DEV = "LITELLM_MODELS_DEV";
 const MODELS_DEV_CACHE_FILENAME = "litellm-models-dev.json";
 const DEFAULT_TIMEOUT_MS = 5000;
 const SEED_TIMEOUT_MS = 3000;
@@ -444,6 +452,16 @@ function isHostOffline(): boolean {
   return process.env.PI_OFFLINE !== undefined;
 }
 
+/**
+ * LiteLLM's /model/info is the metadata authority, so models.dev is an opt-in escape hatch.
+ * Without a cache path and with refresh suppressed, discovery reads neither the models.dev
+ * network nor a cache left by an earlier release; Pi's own catalog still applies.
+ */
+function modelsDevDiscoveryOptions(): Pick<DiscoveryOptions, "modelsDev" | "modelsDevCachePath"> {
+  if (process.env[ENV_MODELS_DEV] !== "1") return { modelsDev: false };
+  return { modelsDev: !isHostOffline(), modelsDevCachePath: join(getAgentDir(), MODELS_DEV_CACHE_FILENAME) };
+}
+
 /**
  * Absolute budget for activation-time discovery. Pi cannot paint its UI until every extension has
  * activated, so this is deliberately capped rather than following LITELLM_DISCOVERY_TIMEOUT_MS,
@@ -1866,8 +1884,7 @@ export default async function (pi: ExtensionAPI): Promise<void> {
       signal,
       headers: auth.headers,
       allowInsecureHttp: auth.allowInsecureHttp,
-      modelsDev: !isHostOffline(),
-      modelsDevCachePath: join(getAgentDir(), MODELS_DEV_CACHE_FILENAME),
+      ...modelsDevDiscoveryOptions(),
       silent: !isVerboseDiscovery(),
       onProgress: isVerboseDiscovery() ? (message) => process.stderr.write(`LiteLLM: ${message}\n`) : undefined,
     });

package/src/model-groups.ts

@@ -181,6 +181,17 @@ function normalizeEffort(level: string): (typeof EXTENDED_LEVELS)[number] | unde
   return EXTENDED_LEVELS.find((candidate) => candidate === normalized);
 }
 
+// LiteLLM reads a declared `reasoning_effort_levels` list whole, ahead of the
+// per-level flags, because flags cannot state a set such as low/high/max: medium
+// has no opt-out. A declared list therefore answers every level for its deployment.
+function reportedLevel(entry: ModelInfoEntry, level: (typeof EXTENDED_LEVELS)[number]): boolean | undefined {
+  const declared: unknown = entry.model_info?.reasoning_effort_levels;
+  if (Array.isArray(declared)) {
+    return declared.some((effort) => typeof effort === "string" && normalizeEffort(effort) === level);
+  }
+  return wireBoolean(entry.model_info?.[LITELLM_LEVEL_FLAGS[level]]);
+}
+
 // A public effort list is complete: listed standard levels are enabled and omitted
 // ones are denied. Without a public list, standard levels stay absent so Pi keeps
 // its defaults. LiteLLM flags may add or remove levels, while xhigh/max always
@@ -200,23 +211,25 @@ function reasoningLevelMap(
       map[level] = publicSets.every((set) => set.has(level)) ? (level === "off" ? "none" : level) : null;
     }
   }
-  // A catalog level map is direct evidence even when its selectable projection
-  // is empty. Preserve its denials independently from public effort lists, whose
-  // unrecognized values intentionally have no opinion.
-  for (const catalogMap of catalogMaps) Object.assign(map, deniedLevels(catalogMap));
-
-  for (const [level, flag] of Object.entries(LITELLM_LEVEL_FLAGS) as Array<
-    [keyof typeof LITELLM_LEVEL_FLAGS, (typeof LITELLM_LEVEL_FLAGS)[keyof typeof LITELLM_LEVEL_FLAGS]]
-  >) {
-    const reported = entries.map((entry) => wireBoolean(entry.model_info?.[flag]));
+  // A catalog level map is not a complete list: a null denies, a value supplies the
+  // wire spelling, and an omitted standard level keeps Pi's default. Flattening it
+  // into a list denied every standard level a map left implicit, such as Codex
+  // `gpt-5.6-sol`, which only states `xhigh`, `max`, and `minimal`.
+  for (const catalogMap of catalogMaps) {
+    for (const [level, value] of Object.entries(catalogMap ?? {}) as [keyof typeof map, string | null][]) {
+      if (value === null || (value !== undefined && map[level] !== null)) map[level] = value;
+    }
+  }
+
+  for (const level of EXTENDED_LEVELS) {
+    const reported = entries.map((entry) => reportedLevel(entry, level));
     if (reported.some((value) => value === false)) map[level] = null;
     else if (reported.length > 0 && reported.every((value) => value === true)) {
       map[level] = level === "off" ? "none" : level;
     }
   }
   for (const level of ["xhigh", "max"] as const) {
-    const flag = LITELLM_LEVEL_FLAGS[level];
-    if (!entries.every((entry) => wireBoolean(entry.model_info?.[flag]) === true)) map[level] = null;
+    if (!entries.every((entry) => reportedLevel(entry, level) === true)) map[level] = null;
   }
   return map;
 }
@@ -241,6 +254,15 @@ function acceptedParams(entry: ModelInfoEntry): Set<string> {
   return params;
 }
 
+// LiteLLM omits supported_openai_params (or returns null) for a deployment its
+// model map does not describe, so that absence says nothing about the carrier. An
+// explicit `supports_reasoning: true` is the operator's opt-in there, as in
+// LiteLLM's own effort resolution. Present but malformed data is not an absence.
+function optsIntoEffortCarrier(entry: ModelInfoEntry): boolean {
+  const supported: unknown = entry.model_info?.supported_openai_params;
+  return (supported === undefined || supported === null) && wireBoolean(entry.model_info?.supports_reasoning) === true;
+}
+
 function intersectParams(entries: readonly ModelInfoEntry[]): string[] {
   const [first, ...rest] = entries.map(acceptedParams);
   if (!first) return [];
@@ -830,16 +852,17 @@ export function reduceModelGroup(
     if (tiers) cost.tiers = tiers;
   }
   const acceptedOpenAIParams = intersectParams(deployments);
-  const acceptsResponsesReasoningControl = acceptedOpenAIParams.includes("reasoning_effort");
-  const publicEfforts = catalogAuthority.map(
-    (catalog) =>
-      catalog?.effortLevels ??
-      (catalog?.thinkingLevelMap
-        ? Object.entries(catalog.thinkingLevelMap)
-            .filter(([, value]) => value !== null)
-            .map(([level]) => level)
-        : undefined),
+  // Kimi and DeepSeek generations name their own carriers, so an operator opt-in
+  // cannot substitute for the parameter evidence their contracts require.
+  const namedCarrierFamily = catalogs.some((catalog) =>
+    ["kimi", "deepseek", "conflicting"].includes(catalog?.semanticFamily ?? ""),
   );
+  const acceptsResponsesReasoningControl =
+    acceptedOpenAIParams.includes("reasoning_effort") ||
+    (!namedCarrierFamily &&
+      deployments.length > 0 &&
+      deployments.every((entry) => acceptedParams(entry).has("reasoning_effort") || optsIntoEffortCarrier(entry)));
+  const publicEfforts = catalogAuthority.map((catalog) => catalog?.effortLevels);
   const evidenceLevelMap = reasoningLevelMap(deployments, publicEfforts, [
     intersectThinkingLevelMaps(catalogAuthority.map((catalog) => catalog?.thinkingLevelMap)),
   ]);
@@ -848,10 +871,8 @@ export function reduceModelGroup(
     thinkingLevelMap = intersectThinkingLevelMaps(catalogs.map((catalog) => catalog?.messagesThinkingLevelMap));
     // Native serializer restrictions apply even when catalog pricing is withheld.
     // Router flags can also deny Pi's implicit default levels, never add a level.
-    for (const [level, flag] of Object.entries(LITELLM_LEVEL_FLAGS) as Array<
-      [keyof typeof LITELLM_LEVEL_FLAGS, (typeof LITELLM_LEVEL_FLAGS)[keyof typeof LITELLM_LEVEL_FLAGS]]
-    >) {
-      const reported = deployments.map((entry) => wireBoolean(entry.model_info?.[flag]));
+    for (const level of EXTENDED_LEVELS) {
+      const reported = deployments.map((entry) => reportedLevel(entry, level));
       if (
         reported.some((value) => value === false) ||
         ((level === "xhigh" || level === "max") &&
@@ -886,9 +907,7 @@ export function reduceModelGroup(
   const semanticMap = semanticReasoningPolicy.thinkingLevelMap;
   const hasEvidenceLevels = Object.values(evidenceLevelMap ?? {}).some((level) => level !== null);
   const semanticBinary = semanticReasoningPolicy.compat?.supportsReasoningEffort === false && !hasEvidenceLevels;
-  const explicitOffDenial = deployments.some(
-    (entry) => wireBoolean(entry.model_info?.supports_none_reasoning_effort) === false,
-  );
+  const explicitOffDenial = deployments.some((entry) => reportedLevel(entry, "off") === false);
   const reasoningPolicy = semanticMap
     ? {
         ...semanticReasoningPolicy,
@@ -933,11 +952,6 @@ export function catalogResolution(provider: string, model: Model<Api>): CatalogR
     provider,
     catalogModelId: model.id,
     reasoning: model.reasoning,
-    effortLevels: model.thinkingLevelMap
-      ? Object.entries(model.thinkingLevelMap)
-          .filter(([, value]) => value !== null)
-          .map(([level]) => level)
-      : undefined,
     thinkingLevelMap: model.thinkingLevelMap,
     vision: model.input.includes("image"),
     contextWindow: model.contextWindow,

package/src/public-catalog.ts

@@ -12,6 +12,9 @@ const KNOWN_PROVIDERS = new Set<string>(getProviders());
 export interface PublicCatalogRecord {
   source: "models.dev" | "pi-vendor" | "pi-adapter";
   provider: string;
+  // The Pi catalog for fields a models.dev record omits, when the record was found
+  // under another vendor's key (ChatGPT routes read OpenAI's but bill as Codex).
+  piProvider?: string;
   modelId: string;
   limits?: { context?: number; output?: number };
   cost?: { input?: number; output?: number; cacheRead?: number; cacheWrite?: number };
@@ -54,6 +57,8 @@ const PROVIDER_ALIASES: Readonly<Record<string, readonly string[]>> = {
   azure_ai: ["azure", "openai"],
   bedrock: ["amazon-bedrock"],
   bedrock_converse: ["amazon-bedrock"],
+  // models.dev has no ChatGPT subscription provider; its routes serve OpenAI models.
+  chatgpt: ["chatgpt", "openai"],
   deepseek: ["deepseek"],
   fireworks: ["fireworks-ai"],
   fireworks_ai: ["fireworks-ai"],
@@ -199,11 +204,6 @@ function mapModelsDev(provider: string, modelId: string, model: ModelsDevModel):
 
 function toPiRecord(source: "pi-vendor" | "pi-adapter", provider: string, model: Model<Api>): PublicCatalogRecord {
   const thinkingLevelMap = model.thinkingLevelMap as Record<string, unknown> | undefined;
-  const effortLevels = thinkingLevelMap
-    ? Object.entries(thinkingLevelMap)
-        .filter(([, value]) => value !== null)
-        .map(([level]) => level)
-    : [];
   return {
     source,
     provider,
@@ -216,7 +216,6 @@ function toPiRecord(source: "pi-vendor" | "pi-adapter", provider: string, model:
       cacheWrite: model.cost.cacheWrite,
     },
     modalities: model.input.includes("image") ? ["text", "image"] : ["text"],
-    ...(effortLevels.length > 0 ? { effortLevels } : {}),
     ...(thinkingLevelMap ? { thinkingLevelMap } : {}),
   };
 }
@@ -245,14 +244,17 @@ export async function loadPublicCatalog(options: LoadPublicCatalogOptions = {}):
     lookup(provider, id) {
       const providers = providerCandidates(provider);
       const ids = lookupIds(id);
+      const vendor = providers.find((candidate) => candidate !== "azure") ?? providers[0];
       for (const candidate of providers) {
         const models = catalog?.[candidate]?.models;
         for (const modelId of ids) {
           const model = models?.[modelId];
-          if (model) return mapModelsDev(candidate, modelId, model);
+          if (!model) continue;
+          const record = mapModelsDev(candidate, modelId, model);
+          if (candidate === providers[0] || !vendor) return record;
+          return { ...record, piProvider: (PI_PROVIDER_ALIASES[vendor] ?? [vendor])[0] };
         }
       }
-      const vendor = providers.find((candidate) => candidate !== "azure") ?? providers[0];
       if (vendor) {
         for (const piProvider of PI_PROVIDER_ALIASES[vendor] ?? [vendor]) {
           const model = findPiModel(piProvider, ids);

package/src/thinking-levels.ts

@@ -19,6 +19,10 @@ type ThinkingLevelDefinitionsAreExhaustive = AssertNever<
 
 type ThinkingLevelMap = DiscoveredModel["thinkingLevelMap"];
 
+// Pi sends an omitted standard level as its own name. An omitted `off` sends no
+// disable value and omitted `xhigh`/`max` are unsupported, so those stay distinct.
+const IMPLICIT_LEVELS = new Set<string>(["minimal", "low", "medium", "high"]);
+
 // Intersects independently authoritative maps. A level mentioned by any source
 // is denied unless every source supplies the same mapping for that level.
 export function intersectThinkingLevelMaps(maps: readonly ThinkingLevelMap[]): ThinkingLevelMap {
@@ -26,8 +30,9 @@ export function intersectThinkingLevelMaps(maps: readonly ThinkingLevelMap[]): T
 
   const intersection: NonNullable<ThinkingLevelMap> = {};
   for (const [level] of THINKING_LEVEL_DEFINITIONS) {
-    const values = maps.map((map) => map?.[level]);
-    if (values.every((value) => value === undefined)) continue;
+    const stated = maps.map((map) => map?.[level]);
+    if (stated.every((value) => value === undefined)) continue;
+    const values = stated.map((value) => (value === undefined && IMPLICIT_LEVELS.has(level) ? level : value));
     const first = values[0];
     intersection[level] = first !== undefined && values.every((value) => value === first) ? first : null;
   }

package/src/types.ts

@@ -63,6 +63,8 @@ export interface DiscoveryOptions {
   signal?: AbortSignal;
   headers?: Record<string, string>;
   allowInsecureHttp?: boolean;
+  // `false` reads only an existing cache at `modelsDevCachePath`; with no cache
+  // path, models.dev is not consulted at all.
   modelsDev?: boolean;
   modelsDevCachePath?: string;
 }
@@ -89,6 +91,7 @@ export interface ModelInfoEntry {
     max_input_tokens?: number;
     max_output_tokens?: number;
     supports_reasoning?: boolean;
+    reasoning_effort_levels?: string[];
     supports_none_reasoning_effort?: boolean;
     supports_minimal_reasoning_effort?: boolean;
     supports_low_reasoning_effort?: boolean;