pi-provider-litellm 1.2.9 → 2.0.0

package/README.md

@@ -1,8 +1,8 @@
 # pi-provider-litellm
 
-LiteLLM proxy provider extension for [Pi](https://pi.dev).
+LiteLLM proxy native Provider extension for [Pi](https://pi.dev). Pi 0.81.0+ is required.
 
-Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, `/litellm-refresh`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
+Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
 
 ## Install
 
@@ -40,7 +40,7 @@ Inside pi:
 /login litellm
 ```
 
-You can also run `/login`, select `Use a subscription`, then select `LiteLLM`.
+To configure an API key, run `/login`, choose `Sign in with an API key`, then choose `LiteLLM API key`. With `/login litellm`, choose `Sign in with an API key` directly.
 
 You'll be prompted for the base URL and API key. Credentials are persisted to `~/.pi/agent/auth.json`.
 
@@ -48,12 +48,11 @@ You'll be prompted for the base URL and API key. Credentials are persisted to `~
 
 If your LiteLLM proxy requires SSO/OAuth authentication (enterprise deployments), you can authenticate via a browser SSO flow and optionally pair the resulting JWT with a stable virtual key:
 
-1. Run `/login litellm` inside pi
+1. Run `/login litellm` inside pi and select `Sign in with LiteLLM SSO`
 2. Enter the proxy URL
-3. At the login method prompt, enter `2` for SSO / Enterprise JWT
-4. Your default browser opens the LiteLLM SSO login URL (e.g. `https://litellm.your-domain.com/sso/key/generate`) automatically — the URL is also displayed in case it can't be opened. Authenticate via SSO
-5. Copy your token from the LiteLLM UI and paste it at the prompt (copying a full `Bearer ...` header value is fine — the prefix is stripped automatically)
-6. When prompted to generate a virtual key, press Enter to accept (recommended) or enter `n` to use the JWT directly
+3. Your default browser opens the LiteLLM SSO login URL (e.g. `https://litellm.your-domain.com/sso/key/generate`) automatically — the URL is also displayed in case it can't be opened. Authenticate via SSO
+4. Copy your token from the LiteLLM UI and paste it at the prompt (copying a full `Bearer ...` header value is fine — the prefix is stripped automatically)
+5. When prompted to generate a virtual key, press Enter to accept (recommended) or enter `n` to use the JWT directly
 
 When you generate a virtual key, the resulting `sk-...` key is stored as your credential and used for all API requests. If the proxy's key policy attaches an expiry to the generated key, Pi will prompt you to re-authenticate when it nears expiry; otherwise the key is treated as permanent until revoked in LiteLLM.
 
@@ -119,6 +118,25 @@ Provider fields:
 
 `/login litellm` and Google ADC token auth remain scoped to the default `litellm` provider. Aliases use their configured `apiKey` or manually stored auth entries matching the alias name.
 
+### Optional LiteLLM features
+
+LiteLLM Skills and MCP integration are enabled by default. Disable either feature globally in `~/.pi/agent/settings.json`:
+
+```json
+{
+  "litellm": {
+    "skills": {
+      "enabled": false
+    },
+    "mcp": {
+      "enabled": false
+    }
+  }
+}
+```
+
+Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` disables LiteLLM MCP discovery and tool registration. Restart Pi after changing these settings so previously registered tools are removed.
+
 ## Use
 
 ```
@@ -133,10 +151,14 @@ Provider fields:
 | `LITELLM_HEADERS` | unset | JSON object of extra headers sent to LiteLLM provider, discovery, MCP, and Skills Gateway requests. Provider aliases can use it with `"headers": "$LITELLM_HEADERS"`. |
 | `LITELLM_GCLOUD_TOKEN_AUTH` | unset | If set to a non-empty value other than `0`, use Google Application Default Credentials as the LiteLLM bearer token source. This takes precedence over `LITELLM_API_KEY_HELPER` and `LITELLM_API_KEY` when no stored `/login litellm` credential exists. |
 | `GOOGLE_APPLICATION_CREDENTIALS` | Google default ADC path | Optional path to an ADC JSON file used by `LITELLM_GCLOUD_TOKEN_AUTH`. If unset, the extension checks the default gcloud ADC locations. |
-| `LITELLM_OFFLINE` | unset | If `1`, skip discovery on this start; use cache only |
-| `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Discovery fetch timeout in ms; `0` to skip discovery |
+| `LITELLM_OFFLINE` | unset | If `1`, disable all model and MCP discovery, including post-login discovery; use cached models only |
+| `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Background and explicit discovery fetch timeout in ms; `0` disables automatic discovery |
+| `LITELLM_VERBOSE_DISCOVERY` | unset | If `1`, enable progress messages during model and MCP discovery (login, refresh, startup); discovery is silent by default |
+| `LITELLM_MODELS_DEV` | enabled | Set to `0` to disable models.dev metadata enrichment, including its cache and network request; `/v1/models` still uses Pi catalog metadata and defaults |
 
-`LITELLM_DISCOVERY_TIMEOUT_MS=0` only disables startup and refresh model discovery. It does not replace the base URL or API key settings required to send requests when you are not using `/login litellm`.
+`LITELLM_DISCOVERY_TIMEOUT_MS=0` disables automatic and explicit refresh model discovery. It does not replace the base URL or API key settings required to send requests when you are not using `/login litellm`.
+
+Models.dev metadata is cached in `litellm-models-dev.json` under the Pi agent directory for 28 days. Fresh data avoids the public request; stale data is used immediately while one background refresh updates the cache. Set `LITELLM_MODELS_DEV=0` when your LiteLLM metadata is authoritative and no external enrichment is wanted.
 
 ### Google ADC token auth
 
@@ -157,7 +179,7 @@ If your LiteLLM proxy exposes MCP REST endpoints, this extension discovers tools
 - `GET /mcp-rest/tools/list`
 - `POST /mcp-rest/tools/call`
 
-Each discovered tool is registered as a native Pi tool named `mcp_<server>_<tool>`, with simple JSON Schema parameters mapped to Pi/TypeBox parameters. Complex schemas fall back to a single `args` object. MCP discovery runs after successful live model discovery, `/login litellm`, or `/litellm-refresh`; it does not force network or helper-token access when a fresh model cache is used. MCP tools run in Pi's parallel tool mode and retry transient failures once.
+Each discovered tool is registered as a native Pi tool named `mcp_<server>_<tool>`, with simple JSON Schema parameters mapped to Pi/TypeBox parameters. Complex schemas fall back to a single `args` object. MCP discovery runs after Pi refreshes LiteLLM models or after `/login litellm`; extension activation never waits for it. MCP tools run in Pi's parallel tool mode and retry transient failures once.
 
 ## LiteLLM Skill Hub
 
@@ -171,11 +193,11 @@ If your LiteLLM proxy exposes `/claude-code/marketplace.json`, enabled skills ar
 
 The `LiteLLM Smoke` GitHub Actions workflow starts VidaiMock and a real LiteLLM proxy on the runner. LiteLLM exposes OpenAI-compatible and Anthropic routes whose upstreams are served by VidaiMock, then this extension's smoke runner discovers those models through LiteLLM and sends `/v1/chat/completions` requests through the proxy.
 
-This keeps the LiteLLM integration path under test but does not call real LLM APIs. No provider API keys or GitHub Models permission are required. The smoke runner also asserts that discovery came from `/model/info` (`LITELLM_SMOKE_EXPECT_SOURCE`) so a silent fallback to `/v1/models` fails the run. The workflow also runs auth checks plus optional Postgres-backed auth checks when `LITELLM_LICENSE` is configured for virtual-key and admin-route behavior, then runs a non-interactive Pi CLI smoke with `--list-models` and `-p` against both the OpenAI-compatible and Anthropic-backed routes, so extension loading, model discovery, and real completion paths are covered without opening the TUI.
+This keeps the LiteLLM integration path under test but does not call real LLM APIs. No provider API keys or GitHub Models permission are required. The smoke runner also asserts that discovery came from `/model/info` (`LITELLM_SMOKE_EXPECT_SOURCE`) so a silent fallback to `/v1/models` fails the run. The workflow also runs auth checks plus optional Postgres-backed auth checks when `LITELLM_LICENSE` is configured for virtual-key and admin-route behavior, then runs a non-interactive Pi CLI smoke with `--list-models` and `-p` against both the OpenAI-compatible and Anthropic-backed routes, so extension loading, model discovery, and real completion paths are covered without opening the TUI. It also runs an interactive Pi TUI smoke covering `/login litellm` and Pi's native `/model` refresh.
 
 ## Development
 
-This package requires Node.js `>=22.19.0`. CI currently uses Node `24.16.0`.
+This package requires Node.js `>=22.19.0`. CI currently uses Node `26.5.0`.
 
 ```bash
 npm ci
@@ -200,16 +222,11 @@ Releases are driven by semver tags named `v*.*.*`. The GitHub release workflow i
 
 Before tagging a release, keep `package.json` and `package-lock.json` versions in sync and verify the dry-run package contents.
 
-## Slash commands
-
-- `/litellm-refresh` — force re-fetch the model list for all configured LiteLLM providers, ignoring cache
-- `/litellm-refresh <provider>` — refresh one configured provider alias, for example `/litellm-refresh litellm-anthropic`
-
-## Cache
+## Model catalog
 
-The default provider model list is cached at `~/.pi/agent/litellm-models.json` with a keyed fingerprint of the base URL + API key. Alias provider caches use `~/.pi/agent/litellm-models-<provider>.json`. Changing the base URL or key invalidates that provider's cache automatically.
+Dynamic catalogs are persisted by Pi in `~/.pi/agent/models-store.json`. Credentials remain in `~/.pi/agent/auth.json`. Legacy `litellm-models*.json` files are ignored and are not deleted.
 
-If the cache is older than 24 hours, the extension refreshes it in the background on session start (non-blocking). Failures are silent — the cached models remain in use. Run `/litellm-refresh` to force an immediate refresh.
+Opening `/model` refreshes configured provider catalogs in the background using Pi's native model lifecycle.
 
 ## Troubleshooting
 
@@ -223,7 +240,7 @@ If the cache is older than 24 hours, the extension refreshes it in the backgroun
 | No models with gcloud auth | Verify `gcloud auth application-default login` has been run or set `GOOGLE_APPLICATION_CREDENTIALS` to an `authorized_user` ADC file |
 | Enterprise SSO login shows "virtual key generation failed" | The LiteLLM instance may lack a database (`/key/generate` requires one), your user account may lack key-generation permission, or the request timed out; the JWT is used directly as a fallback |
 | Enterprise SSO token prompt fails with "SSO token is required" | The token field was left empty — paste the token copied from the LiteLLM UI |
-| MCP tools not showing | Verify the proxy exposes `/mcp-rest/tools/list` and run `/litellm-refresh` after fixing the proxy |
+| MCP tools not showing | Verify the proxy exposes `/mcp-rest/tools/list` and open `/model` after fixing the proxy |
 | Skills not affecting prompts | Verify the proxy exposes `/claude-code/marketplace.json` or `/v1/skills` and returns enabled skills |
 
 ## License

package/dist/cache.d.ts

@@ -1,5 +1 @@
-import type { CacheFile } from "./types.js";
-export declare function fingerprint(apiKey: string): string;
-export declare function isCacheValid(cache: CacheFile | null, baseUrl: string, apiKey: string): boolean;
-export declare function readCache(path: string): Promise<CacheFile | null>;
-export declare function writeCache(path: string, cache: CacheFile): Promise<void>;
+export declare function writeJsonAtomic(path: string, value: unknown): Promise<void>;

package/dist/cache.js

@@ -1,47 +1,8 @@
-import { scryptSync } from "node:crypto";
-import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
+import { mkdir, rename, writeFile } from "node:fs/promises";
 import { dirname } from "node:path";
-const FINGERPRINT_SALT = "pi-provider-litellm-cache-fingerprint-v1";
-export function fingerprint(apiKey) {
-    return scryptSync(apiKey, FINGERPRINT_SALT, 32).toString("hex");
-}
-export function isCacheValid(cache, baseUrl, apiKey) {
-    if (!cache)
-        return false;
-    return cache.baseUrl === baseUrl && cache.apiKeyFingerprint === fingerprint(apiKey);
-}
-export async function readCache(path) {
-    let raw;
-    try {
-        raw = await readFile(path, "utf8");
-    }
-    catch {
-        return null;
-    }
-    let parsed;
-    try {
-        parsed = JSON.parse(raw);
-    }
-    catch {
-        return null;
-    }
-    if (!isCacheFileShape(parsed))
-        return null;
-    return parsed;
-}
-function isCacheFileShape(value) {
-    if (typeof value !== "object" || value === null)
-        return false;
-    const v = value;
-    return (typeof v.baseUrl === "string" &&
-        typeof v.apiKeyFingerprint === "string" &&
-        typeof v.fetchedAt === "number" &&
-        (v.source === "model_info" || v.source === "models_list" || v.source === "health") &&
-        Array.isArray(v.models));
-}
-export async function writeCache(path, cache) {
+export async function writeJsonAtomic(path, value) {
     await mkdir(dirname(path), { recursive: true });
     const tmp = `${path}.${process.pid}.${Date.now()}.tmp`;
-    await writeFile(tmp, JSON.stringify(cache, null, 2), "utf8");
+    await writeFile(tmp, JSON.stringify(value, null, 2), "utf8");
     await rename(tmp, path);
 }

package/dist/cost.d.ts

@@ -1,12 +1,2 @@
-import type { ExtensionAPI, ProviderModelConfig } from "@earendil-works/pi-coding-agent";
-export interface ModelCostInfo {
-    inputCostPerToken: number;
-    outputCostPerToken: number;
-    cacheReadCostPerToken: number;
-    cacheWriteCostPerToken: number;
-}
-export interface ProviderModels {
-    provider: string;
-    models: ProviderModelConfig[];
-}
-export declare function setupLiteLLMCostTracking(pi: ExtensionAPI, providerModels: ProviderModels[]): (providerModels: ProviderModels[]) => void;
+import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+export declare function setupLiteLLMCostTracking(pi: ExtensionAPI, providerNames: string[]): void;

package/dist/discover.d.ts

@@ -1,12 +1,10 @@
-import type { ProviderModelConfig } from "@earendil-works/pi-coding-agent";
-import type { DiscoveryOptions, DiscoveryResult } from "./types.js";
+import type { DiscoveredModel, DiscoveryOptions, DiscoveryResult } from "./types.js";
 export declare function normalizeBaseUrl(input: string): string;
 export declare function isMoonshotModel(modelId: string): boolean;
 export declare function isGpt55Model(modelId: string): boolean;
 export declare function shouldSuppressReasoningContent(modelId: string): boolean;
-export declare function buildCompat(modelId: string): ProviderModelConfig["compat"];
-export declare function withTimeout(timeoutMs: number, signal?: AbortSignal): {
-    signal: AbortSignal;
-    cancel: () => void;
-};
-export declare function discoverModels(baseUrl: string, apiKey: string, options?: DiscoveryOptions): Promise<DiscoveryResult>;
+export declare function buildCompat(modelId: string): DiscoveredModel["compat"];
+export declare function discoverModels(baseUrl: string, apiKey: string, options?: DiscoveryOptions & {
+    onProgress?: (message: string) => void;
+    silent?: boolean;
+}): Promise<DiscoveryResult>;

package/dist/discover.js

@@ -1,19 +1,23 @@
+import { readFile } from "node:fs/promises";
 import { getModels, getProviders } from "@earendil-works/pi-ai/compat";
+import { writeJsonAtomic } from "./cache.js";
 const DEFAULT_TIMEOUT_MS = 5000;
 const DEFAULT_CONTEXT_WINDOW = 128_000;
 const DEFAULT_MAX_TOKENS = 16_384;
 const KNOWN_PROVIDER_SET = new Set(getProviders());
 const MODELS_DEV_URL = "https://models.dev/api.json";
-let modelsDevCatalog;
+const MODELS_DEV_CACHE_TTL_MS = 28 * 24 * 60 * 60 * 1000;
+const modelsDevCaches = new Map();
+const modelsDevRefreshes = new Map();
 export function normalizeBaseUrl(input) {
     return input.replace(/\/+$/, "").replace(/\/v1\/?$/i, "");
 }
 const RESPONSES_MODE_PATTERN = /^responses?$/i;
 function isResponsesMode(mode) {
-    return mode !== undefined && RESPONSES_MODE_PATTERN.test(mode);
+    return mode != null && RESPONSES_MODE_PATTERN.test(mode);
 }
 function isChatStyleMode(mode) {
-    return mode === undefined || mode === "chat" || isResponsesMode(mode);
+    return mode == null || mode === "chat" || isResponsesMode(mode);
 }
 // Matches both the conventional `anthropic/...` prefix and aliases that
 // LiteLLM deployments commonly assign to Anthropic-backed routes (e.g.
@@ -104,6 +108,7 @@ function mapModelInfoCost(info, fallback) {
         cacheWrite: info.cache_creation_input_token_cost !== undefined
             ? info.cache_creation_input_token_cost * 1_000_000
             : (fallback?.cacheWrite ?? 0),
+        ...(fallback?.tiers ? { tiers: fallback.tiers } : {}),
     };
 }
 function getFallbackProviderAndModel(id, ownedBy) {
@@ -118,70 +123,180 @@ function findModelsDevModel(catalog, id, ownedBy) {
     const { provider, modelId } = getFallbackProviderAndModel(id, ownedBy);
     if (!provider)
         return undefined;
-    return catalog?.[provider]?.models?.[modelId];
+    const models = catalog?.[provider]?.models;
+    return models && Object.hasOwn(models, modelId) ? models[modelId] : undefined;
 }
-export function withTimeout(timeoutMs, signal) {
-    const controller = new AbortController();
-    const onAbort = () => controller.abort(signal?.reason);
-    if (signal) {
-        if (signal.aborted)
-            controller.abort(signal.reason);
-        else
-            signal.addEventListener("abort", onAbort, { once: true });
-    }
-    const timer = setTimeout(() => controller.abort(new Error(`Timed out after ${timeoutMs}ms`)), timeoutMs);
-    return {
-        signal: controller.signal,
-        cancel: () => {
-            clearTimeout(timer);
-            signal?.removeEventListener("abort", onAbort);
-        },
-    };
+function awaitWithSignal(promise, signal) {
+    if (!signal)
+        return promise;
+    if (signal.aborted)
+        return Promise.reject(signal.reason);
+    return new Promise((resolve, reject) => {
+        const cleanup = () => signal.removeEventListener("abort", onAbort);
+        const onAbort = () => {
+            cleanup();
+            reject(signal.reason);
+        };
+        signal.addEventListener("abort", onAbort, { once: true });
+        promise.then((value) => {
+            cleanup();
+            resolve(value);
+        }, (error) => {
+            cleanup();
+            reject(error);
+        });
+    });
 }
 async function fetchJson(url, apiKey, options) {
     const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
-    const { signal, cancel } = withTimeout(timeoutMs, options.signal);
-    try {
-        const response = await fetch(url, {
-            headers: { ...options.headers, Authorization: `Bearer ${apiKey}`, Accept: "application/json" },
-            signal,
-        });
-        if (!response.ok)
-            return { ok: false, status: response.status };
-        const data = (await response.json());
-        return { ok: true, data };
-    }
-    finally {
-        cancel();
-    }
+    const response = await fetch(url, {
+        headers: { ...options.headers, Authorization: `Bearer ${apiKey}`, Accept: "application/json" },
+        signal: options.signal
+            ? AbortSignal.any([options.signal, AbortSignal.timeout(timeoutMs)])
+            : AbortSignal.timeout(timeoutMs),
+    });
+    if (!response.ok)
+        return { ok: false, status: response.status };
+    const data = (await response.json());
+    return { ok: true, data };
 }
 async function fetchPublicJson(url, options) {
     const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
-    const { signal, cancel } = withTimeout(timeoutMs, options.signal);
-    try {
-        const response = await fetch(url, {
-            headers: { Accept: "application/json" },
-            signal,
-        });
-        if (!response.ok)
-            throw new Error(`${url} returned ${response.status}`);
-        return (await response.json());
-    }
-    finally {
-        cancel();
+    const response = await fetch(url, {
+        headers: { Accept: "application/json" },
+        signal: options.signal
+            ? AbortSignal.any([options.signal, AbortSignal.timeout(timeoutMs)])
+            : AbortSignal.timeout(timeoutMs),
+    });
+    if (!response.ok)
+        throw new Error(`${url} returned ${response.status}`);
+    return (await response.json());
+}
+function isRecord(value) {
+    return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function finiteNumber(value) {
+    return typeof value === "number" && Number.isFinite(value) ? value : undefined;
+}
+function normalizeModelsDevCatalog(value) {
+    if (!isRecord(value))
+        return undefined;
+    const catalog = {};
+    for (const [providerId, providerValue] of Object.entries(value)) {
+        if (!isRecord(providerValue) || !isRecord(providerValue.models))
+            continue;
+        const models = {};
+        for (const [modelId, modelValue] of Object.entries(providerValue.models)) {
+            if (!isRecord(modelValue))
+                continue;
+            const model = {};
+            if (typeof modelValue.name === "string")
+                model.name = modelValue.name;
+            if (typeof modelValue.reasoning === "boolean")
+                model.reasoning = modelValue.reasoning;
+            if (isRecord(modelValue.modalities) && Array.isArray(modelValue.modalities.input)) {
+                const input = modelValue.modalities.input.filter((entry) => typeof entry === "string");
+                if (input.length > 0)
+                    model.modalities = { input };
+            }
+            if (isRecord(modelValue.limit)) {
+                const limit = {};
+                const context = finiteNumber(modelValue.limit.context);
+                const input = finiteNumber(modelValue.limit.input);
+                const output = finiteNumber(modelValue.limit.output);
+                if (context !== undefined)
+                    limit.context = context;
+                if (input !== undefined)
+                    limit.input = input;
+                if (output !== undefined)
+                    limit.output = output;
+                if (Object.keys(limit).length > 0)
+                    model.limit = limit;
+            }
+            if (isRecord(modelValue.cost)) {
+                const cost = {};
+                const input = finiteNumber(modelValue.cost.input);
+                const output = finiteNumber(modelValue.cost.output);
+                const cacheRead = finiteNumber(modelValue.cost.cache_read);
+                const cacheWrite = finiteNumber(modelValue.cost.cache_write);
+                if (input !== undefined)
+                    cost.input = input;
+                if (output !== undefined)
+                    cost.output = output;
+                if (cacheRead !== undefined)
+                    cost.cache_read = cacheRead;
+                if (cacheWrite !== undefined)
+                    cost.cache_write = cacheWrite;
+                if (Object.keys(cost).length > 0)
+                    model.cost = cost;
+            }
+            models[modelId] = model;
+        }
+        catalog[providerId] = { models };
     }
+    return catalog;
 }
-async function getModelsDevCatalog(options) {
-    if (modelsDevCatalog)
-        return modelsDevCatalog;
+async function readModelsDevCache(path) {
     try {
-        modelsDevCatalog = await fetchPublicJson(MODELS_DEV_URL, options);
-        return modelsDevCatalog;
+        const parsed = JSON.parse(await readFile(path, "utf8"));
+        if (!isRecord(parsed) ||
+            typeof parsed.fetchedAt !== "number" ||
+            !Number.isFinite(parsed.fetchedAt) ||
+            parsed.fetchedAt < 0 ||
+            parsed.fetchedAt > Date.now()) {
+            return undefined;
+        }
+        const catalog = normalizeModelsDevCatalog(parsed.catalog);
+        return catalog ? { fetchedAt: parsed.fetchedAt, catalog } : undefined;
     }
     catch {
         return undefined;
     }
 }
+function refreshModelsDevCatalog(key, options) {
+    const active = modelsDevRefreshes.get(key);
+    if (active)
+        return active;
+    const refresh = (async () => {
+        try {
+            const catalog = normalizeModelsDevCatalog(await fetchPublicJson(MODELS_DEV_URL, options));
+            if (!catalog)
+                return undefined;
+            const cache = { fetchedAt: Date.now(), catalog };
+            modelsDevCaches.set(key, cache);
+            if (options.modelsDevCachePath) {
+                await writeJsonAtomic(options.modelsDevCachePath, cache).catch(() => undefined);
+            }
+            return catalog;
+        }
+        catch {
+            return undefined;
+        }
+        finally {
+            modelsDevRefreshes.delete(key);
+        }
+    })();
+    modelsDevRefreshes.set(key, refresh);
+    return refresh;
+}
+async function getModelsDevCatalog(options) {
+    const key = options.modelsDevCachePath ?? MODELS_DEV_URL;
+    const refreshOptions = { ...options, timeoutMs: DEFAULT_TIMEOUT_MS, signal: undefined };
+    let cache = modelsDevCaches.get(key);
+    if (!cache && options.modelsDevCachePath) {
+        cache = await readModelsDevCache(options.modelsDevCachePath);
+        if (cache)
+            modelsDevCaches.set(key, cache);
+    }
+    if (!cache) {
+        const timeout = AbortSignal.timeout(options.timeoutMs ?? DEFAULT_TIMEOUT_MS);
+        return awaitWithSignal(refreshModelsDevCatalog(key, refreshOptions), options.signal ? AbortSignal.any([options.signal, timeout]) : timeout);
+    }
+    if (Date.now() - cache.fetchedAt < MODELS_DEV_CACHE_TTL_MS)
+        return cache.catalog;
+    void refreshModelsDevCatalog(key, refreshOptions);
+    return cache.catalog;
+}
 function mapModelsDevMetadata(model) {
     if (!model)
         return {};
@@ -221,6 +336,7 @@ function mapFromModelInfo(entry) {
         id,
         name: id,
         reasoning: info.supports_reasoning ?? false,
+        ...(catalogModel?.thinkingLevelMap ? { thinkingLevelMap: catalogModel.thinkingLevelMap } : {}),
         input: info.supports_vision ? ["text", "image"] : ["text"],
         cost: mapModelInfoCost(info, catalogModel?.cost),
         contextWindow: info.max_input_tokens ?? DEFAULT_CONTEXT_WINDOW,
@@ -230,25 +346,12 @@ function mapFromModelInfo(entry) {
     };
 }
 function mapFromHealthModelInfo(entry, fallbackId) {
-    const model = mapFromModelInfo(entry);
-    if (model || !fallbackId)
-        return model;
-    if (!isChatStyleMode(entry.model_info?.mode))
-        return undefined;
-    const info = entry.model_info ?? {};
-    const responsesMode = isResponsesMode(info.mode);
-    const catalogModel = findCatalogModel(fallbackId);
-    return {
-        id: fallbackId,
-        name: fallbackId,
-        reasoning: info.supports_reasoning ?? false,
-        input: info.supports_vision ? ["text", "image"] : ["text"],
-        cost: mapModelInfoCost(info, catalogModel?.cost),
-        contextWindow: info.max_input_tokens ?? DEFAULT_CONTEXT_WINDOW,
-        maxTokens: info.max_output_tokens ?? DEFAULT_MAX_TOKENS,
-        compat: buildCompat(fallbackId),
-        ...(responsesMode ? { api: "openai-responses" } : {}),
-    };
+    if (entry.model_name || !fallbackId)
+        return mapFromModelInfo(entry);
+    const model = mapFromModelInfo({ ...entry, model_name: fallbackId });
+    if (model)
+        delete model.thinkingLevelMap;
+    return model;
 }
 function mapFromHealthEndpoint(entry) {
     const id = entry.model;
@@ -286,45 +389,80 @@ function mapFromModelsList(entry, modelsDev) {
     };
 }
 async function discoverFromHealth(base, apiKey, options) {
+    const progress = options.silent ? undefined : options.onProgress;
+    progress?.("Querying /health endpoint...");
     const healthResult = await fetchJson(`${base}/health`, apiKey, options);
     if (!healthResult.ok)
         return [];
     const endpoints = (healthResult.data.healthy_endpoints ?? []).filter((entry) => entry.model || entry.model_id);
+    progress?.(`Discovered ${endpoints.length} model endpoints, fetching details...`);
+    let completed = 0;
     const models = await Promise.all(endpoints.map(async (endpoint) => {
-        if (!endpoint.model_id)
-            return mapFromHealthEndpoint(endpoint);
-        const infoResult = await fetchJson(`${base}/model/info?litellm_model_id=${encodeURIComponent(endpoint.model_id)}`, apiKey, options);
-        if (!infoResult.ok)
-            return mapFromHealthEndpoint(endpoint);
-        const entry = infoResult.data.data?.[0];
-        return entry ? mapFromHealthModelInfo(entry, endpoint.model) : mapFromHealthEndpoint(endpoint);
+        let model = mapFromHealthEndpoint(endpoint);
+        if (endpoint.model_id) {
+            const infoResult = await fetchJson(`${base}/model/info?litellm_model_id=${encodeURIComponent(endpoint.model_id)}`, apiKey, options);
+            const entry = infoResult.ok ? infoResult.data.data?.[0] : undefined;
+            if (entry)
+                model = mapFromHealthModelInfo(entry, endpoint.model);
+        }
+        completed++;
+        if (completed % 10 === 0 || completed === endpoints.length) {
+            progress?.(`Fetched ${completed}/${endpoints.length} models...`);
+        }
+        return model;
     }));
     return models.filter((model) => model !== undefined);
 }
+function deduplicateModels(models) {
+    const seen = new Set();
+    return models.filter((m) => {
+        if (seen.has(m.id))
+            return false;
+        seen.add(m.id);
+        return true;
+    });
+}
 export async function discoverModels(baseUrl, apiKey, options = {}) {
     const base = normalizeBaseUrl(baseUrl);
+    const progress = options.silent ? undefined : options.onProgress;
+    progress?.("Querying /model/info endpoint...");
     const infoResult = await fetchJson(`${base}/model/info`, apiKey, options);
     if (infoResult.ok) {
-        const models = (infoResult.data.data ?? [])
-            .map(mapFromModelInfo)
-            .filter((m) => m !== undefined);
+        const entries = new Map();
+        for (const entry of infoResult.data.data ?? []) {
+            if (!entry.model_name)
+                continue;
+            const previous = entries.get(entry.model_name);
+            entries.set(entry.model_name, {
+                ...previous,
+                ...entry,
+                model_info: { ...previous?.model_info, ...entry.model_info },
+            });
+        }
+        const models = [...entries.values()].map(mapFromModelInfo).filter((m) => m !== undefined);
         return { source: "model_info", models };
     }
     if (![401, 403, 404].includes(infoResult.status)) {
         throw new Error(`/model/info returned ${infoResult.status}`);
     }
+    progress?.("/model/info unavailable, trying /v1/models...");
     const listResult = await fetchJson(`${base}/v1/models`, apiKey, options);
     if (!listResult.ok) {
         if ([401, 403, 404].includes(listResult.status)) {
+            progress?.("/v1/models unavailable, falling back to /health endpoint...");
             const models = await discoverFromHealth(base, apiKey, options);
             if (models.length > 0)
-                return { source: "health", models };
+                return { source: "health", models: deduplicateModels(models) };
         }
         throw new Error(`/v1/models returned ${listResult.status}`);
     }
-    const modelsDev = await getModelsDevCatalog(options);
+    let modelsDev;
+    if (options.modelsDev !== false) {
+        progress?.("Loading models.dev catalog for metadata enrichment...");
+        modelsDev = await getModelsDevCatalog(options);
+    }
     const models = (listResult.data.data ?? [])
         .map((entry) => mapFromModelsList(entry, modelsDev))
         .filter((m) => m !== undefined);
-    return { source: "models_list", models };
+    return { source: "models_list", models: deduplicateModels(models) };
 }