pi-provider-litellm 1.2.8 → 1.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 LiteLLM proxy provider extension for [Pi](https://pi.dev).
 
-Discovers models from a self-hosted LiteLLM proxy and registers them under the `litellm` provider. Supports `/login litellm`, `/litellm-refresh`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
+Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, `/litellm-refresh`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
 
 ## Install
 
@@ -68,6 +68,76 @@ export LITELLM_API_KEY="sk-..."
 
 Stored pi credentials for `litellm` take precedence over `LITELLM_API_KEY`; the environment key is used when no saved credential exists. `LITELLM_BASE_URL` is used when no saved login base URL exists.
 
+### Multiple LiteLLM provider aliases
+
+Add alias providers in `~/.pi/agent/settings.json` under `litellm.providers`. Each alias is registered as a separate Pi provider name, so models appear as `litellm/model-id` and `litellm-anthropic/model-id`.
+
+```json
+{
+  "litellm": {
+    "providers": {
+      "litellm-anthropic": {
+        "baseUrl": "https://litellm.your-domain.com",
+        "apiKey": "$LITELLM_CLAUDE_KEY",
+        "headers": "$LITELLM_HEADERS"
+      }
+    }
+  }
+}
+```
+
+You can also override the default provider through the same shape:
+
+```json
+{
+  "litellm": {
+    "providers": {
+      "litellm": {
+        "baseUrl": "https://litellm.your-domain.com",
+        "apiKey": "$LITELLM_API_KEY",
+        "headers": "$LITELLM_HEADERS"
+      },
+      "litellm-anthropic": {
+        "baseUrl": "https://litellm.your-domain.com",
+        "apiKey": "$LITELLM_CLAUDE_KEY",
+        "headers": "$LITELLM_HEADERS"
+      }
+    }
+  }
+}
+```
+
+Provider fields:
+
+| Field | Default | Effect |
+|---|---|---|
+| `baseUrl` | `LITELLM_BASE_URL` for `litellm`; required for aliases | LiteLLM proxy URL, with or without `/v1` |
+| `apiKey` | `LITELLM_API_KEY_HELPER`/`LITELLM_API_KEY` for `litellm`; required for aliases | Pi config value for this provider's key. Use `$ENV_VAR`, `${ENV_VAR}`, `!command`, or a literal key. Escape a literal `$` as `$$`. |
+| `headers` | `$LITELLM_HEADERS` for `litellm`; unset for aliases | JSON string env reference or inline object of request headers |
+| `displayName` | provider name | Label shown in Pi UI |
+| `enabled` | `true` | Set `false` to skip an alias |
+
+`/login litellm` and Google ADC token auth remain scoped to the default `litellm` provider. Aliases use their configured `apiKey` or manually stored auth entries matching the alias name.
+
+### Optional LiteLLM features
+
+LiteLLM Skills and MCP integration are enabled by default. Disable either feature globally in `~/.pi/agent/settings.json`:
+
+```json
+{
+  "litellm": {
+    "skills": {
+      "enabled": false
+    },
+    "mcp": {
+      "enabled": false
+    }
+  }
+}
+```
+
+Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` disables LiteLLM MCP discovery and tool registration. Restart Pi after changing these settings so previously registered tools are removed.
+
 ## Use
 
 ```
@@ -79,12 +149,13 @@ Stored pi credentials for `litellm` take precedence over `LITELLM_API_KEY`; the
 | Variable | Default | Effect |
 |---|---|---|
 | `LITELLM_API_KEY_HELPER` | unset | Command that prints a fresh LiteLLM bearer token. Takes precedence over `LITELLM_API_KEY`. Registered as a `!command` provider key; Pi re-runs it on every request (the per-request auth path is uncached), so rotating/short-lived tokens stay fresh. |
-| `LITELLM_GCLOUD_TOKEN_AUTH` | unset | If set to a non-empty value other than `0`, use Google Application Default Credentials as the LiteLLM bearer token source when no stored `/login litellm` credential exists. If ADC is unavailable, the extension falls back to `LITELLM_API_KEY_HELPER` or `LITELLM_API_KEY`. |
+| `LITELLM_HEADERS` | unset | JSON object of extra headers sent to LiteLLM provider, discovery, MCP, and Skills Gateway requests. Provider aliases can use it with `"headers": "$LITELLM_HEADERS"`. |
+| `LITELLM_GCLOUD_TOKEN_AUTH` | unset | If set to a non-empty value other than `0`, use Google Application Default Credentials as the LiteLLM bearer token source. This takes precedence over `LITELLM_API_KEY_HELPER` and `LITELLM_API_KEY` when no stored `/login litellm` credential exists. |
 | `GOOGLE_APPLICATION_CREDENTIALS` | Google default ADC path | Optional path to an ADC JSON file used by `LITELLM_GCLOUD_TOKEN_AUTH`. If unset, the extension checks the default gcloud ADC locations. |
-| `LITELLM_OFFLINE` | unset | If `1`, skip discovery on this start; use cache only |
-| `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Discovery fetch timeout in ms; `0` to skip discovery |
+| `LITELLM_OFFLINE` | unset | If `1`, disable all model and MCP discovery, including `/litellm-refresh` and post-login MCP discovery; use cached models only |
+| `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Background and explicit discovery fetch timeout in ms; `0` disables automatic discovery |
 
-`LITELLM_DISCOVERY_TIMEOUT_MS=0` only disables startup and refresh model discovery. It does not replace the base URL or API key settings required to send requests when you are not using `/login litellm`.
+`LITELLM_DISCOVERY_TIMEOUT_MS=0` disables automatic and explicit refresh model discovery. It does not replace the base URL or API key settings required to send requests when you are not using `/login litellm`.
 
 ### Google ADC token auth
 
@@ -96,7 +167,7 @@ export LITELLM_BASE_URL="https://litellm.your-domain.com"
 export LITELLM_GCLOUD_TOKEN_AUTH=1
 ```
 
-Only `authorized_user` ADC files are supported. Service account JSON files are rejected with a warning. Tokens are cached in memory for 50 minutes and are keyed by the ADC identity, so switching `GOOGLE_APPLICATION_CREDENTIALS` or changing `gcloud auth application-default login` users refreshes both request auth and model discovery cache identity. When ADC is available, the registered provider key is a Pi `!command`, so request-time auth resolves a fresh token when Pi sends model requests. If ADC cannot be read, provider registration uses the same helper or environment API key that discovery used.
+Only `authorized_user` ADC files are supported. Service account JSON files are rejected with a warning. Tokens are cached in memory for 50 minutes and the registered provider key is a Pi `!command`, so request-time auth resolves a fresh token when Pi sends model requests.
 
 ## LiteLLM MCP tools
 
@@ -105,25 +176,59 @@ If your LiteLLM proxy exposes MCP REST endpoints, this extension discovers tools
 - `GET /mcp-rest/tools/list`
 - `POST /mcp-rest/tools/call`
 
-Each discovered tool is registered as a native Pi tool named `mcp_<server>_<tool>`, with simple JSON Schema parameters mapped to Pi/TypeBox parameters. Complex schemas fall back to a single `args` object. The extension accepts LiteLLM's `tools` response shape with `inputSchema` and `mcp_info`, and executes tools with LiteLLM's `server_id`, `name`, and `arguments` fields. MCP discovery runs after successful live model discovery, `/login litellm`, or `/litellm-refresh`; it does not force network or helper-token access when a fresh model cache is used.
+Each discovered tool is registered as a native Pi tool named `mcp_<server>_<tool>`, with simple JSON Schema parameters mapped to Pi/TypeBox parameters. Complex schemas fall back to a single `args` object. MCP discovery runs after background model discovery, `/login litellm`, or `/litellm-refresh`; extension activation never waits for it. MCP tools run in Pi's parallel tool mode and retry transient failures once.
 
-## LiteLLM Skills Gateway
+## LiteLLM Skill Hub
 
-If your LiteLLM proxy exposes `/v1/skills`, enabled skills are fetched before each agent turn and appended to the system prompt as a `litellm_skills` section. The extension also registers Pi tools for basic Skills Gateway management:
+If your LiteLLM proxy exposes `/claude-code/marketplace.json`, enabled skills are fetched before each agent turn and appended to the system prompt as a `litellm_skills` section. The extension falls back to the legacy `/v1/skills` Skills Gateway path when Skill Hub is unavailable. It also registers Pi tools for basic skill management:
 
 - `litellm_skill_list`
 - `litellm_skill_create`
 - `litellm_skill_delete`
 
+## Mocked LiteLLM smoke workflow
+
+The `LiteLLM Smoke` GitHub Actions workflow starts VidaiMock and a real LiteLLM proxy on the runner. LiteLLM exposes OpenAI-compatible and Anthropic routes whose upstreams are served by VidaiMock, then this extension's smoke runner discovers those models through LiteLLM and sends `/v1/chat/completions` requests through the proxy.
+
+This keeps the LiteLLM integration path under test but does not call real LLM APIs. No provider API keys or GitHub Models permission are required. The smoke runner also asserts that discovery came from `/model/info` (`LITELLM_SMOKE_EXPECT_SOURCE`) so a silent fallback to `/v1/models` fails the run. The workflow also runs auth checks plus optional Postgres-backed auth checks when `LITELLM_LICENSE` is configured for virtual-key and admin-route behavior, then runs a non-interactive Pi CLI smoke with `--list-models` and `-p` against both the OpenAI-compatible and Anthropic-backed routes, so extension loading, model discovery, and real completion paths are covered without opening the TUI. It also runs an interactive Pi TUI smoke covering `/login litellm`, `/litellm-refresh`, and `/model`.
+
+## Development
+
+This package requires Node.js `>=22.19.0`. CI currently uses Node `26.5.0`.
+
+```bash
+npm ci
+npm run check
+npm run clean && npm run build
+```
+
+`npm run check` runs Biome, type checking, and the Vitest suite. Runtime changes must be built before local Pi smoke checks because the extension entrypoint is `./dist/index.js`.
+
+Before changing package contents or dependency policy, also run:
+
+```bash
+npm run supply-chain:guard
+npm pack --dry-run
+```
+
+The published npm package should contain only `dist`, `README.md`, and `LICENSE`.
+
+## Release
+
+Releases are driven by semver tags named `v*.*.*`. The GitHub release workflow installs from the lockfile, runs the checks, builds `dist`, verifies the package tarball, publishes to npm with provenance, and creates a GitHub release.
+
+Before tagging a release, keep `package.json` and `package-lock.json` versions in sync and verify the dry-run package contents.
+
 ## Slash commands
 
-- `/litellm-refresh` — force re-fetch the model list, ignoring cache
+- `/litellm-refresh` — force re-fetch the model list for all configured LiteLLM providers, ignoring cache
+- `/litellm-refresh <provider>` — refresh one configured provider alias, for example `/litellm-refresh litellm-anthropic`
 
 ## Cache
 
-The model list is cached at `~/.pi/agent/litellm-models.json` with the base URL and a keyed fingerprint of the selected auth source. Changing the base URL, API key, helper command, or Google ADC identity invalidates the cache automatically.
+The default provider model list is cached at `~/.pi/agent/litellm-models.json` with a keyed fingerprint of the base URL + API key. Alias provider caches use `~/.pi/agent/litellm-models-<provider>.json`. Changing the base URL or key invalidates that provider's cache automatically.
 
-If the cache is older than 24 hours, the extension refreshes it in the background on session start (non-blocking). Failures are silent — the cached models remain in use. Run `/litellm-refresh` to force an immediate refresh.
+When the cache is missing, invalid, or older than 24 hours, the extension starts without waiting for discovery and refreshes models in the background on session start. Existing cached models remain available while an invalid or stale cache refreshes. Failures are silent; run `/litellm-refresh` to force an immediate refresh and see its result.
 
 ## Troubleshooting
 
@@ -138,7 +243,7 @@ If the cache is older than 24 hours, the extension refreshes it in the backgroun
 | Enterprise SSO login shows "virtual key generation failed" | The LiteLLM instance may lack a database (`/key/generate` requires one), your user account may lack key-generation permission, or the request timed out; the JWT is used directly as a fallback |
 | Enterprise SSO token prompt fails with "SSO token is required" | The token field was left empty — paste the token copied from the LiteLLM UI |
 | MCP tools not showing | Verify the proxy exposes `/mcp-rest/tools/list` and run `/litellm-refresh` after fixing the proxy |
-| Skills not affecting prompts | Verify the proxy exposes `/v1/skills` and returns enabled skills |
+| Skills not affecting prompts | Verify the proxy exposes `/claude-code/marketplace.json` or `/v1/skills` and returns enabled skills |
 
 ## License
 

package/dist/cost.d.ts

@@ -1,8 +1,6 @@
 import type { ExtensionAPI, ProviderModelConfig } from "@earendil-works/pi-coding-agent";
-export interface ModelCostInfo {
-    inputCostPerToken: number;
-    outputCostPerToken: number;
-    cacheReadCostPerToken: number;
-    cacheWriteCostPerToken: number;
+export interface ProviderModels {
+    provider: string;
+    models: ProviderModelConfig[];
 }
-export declare function setupLiteLLMCostTracking(pi: ExtensionAPI, models: ProviderModelConfig[]): (models: ProviderModelConfig[]) => void;
+export declare function setupLiteLLMCostTracking(pi: ExtensionAPI, providerModels: ProviderModels[]): (providerModels: ProviderModels[]) => void;

package/dist/discover.d.ts

@@ -2,10 +2,13 @@ import type { ProviderModelConfig } from "@earendil-works/pi-coding-agent";
 import type { DiscoveryOptions, DiscoveryResult } from "./types.js";
 export declare function normalizeBaseUrl(input: string): string;
 export declare function isMoonshotModel(modelId: string): boolean;
+export declare function isGpt55Model(modelId: string): boolean;
 export declare function shouldSuppressReasoningContent(modelId: string): boolean;
 export declare function buildCompat(modelId: string): ProviderModelConfig["compat"];
 export declare function withTimeout(timeoutMs: number, signal?: AbortSignal): {
     signal: AbortSignal;
     cancel: () => void;
 };
-export declare function discoverModels(baseUrl: string, apiKey: string, options?: DiscoveryOptions): Promise<DiscoveryResult>;
+export declare function discoverModels(baseUrl: string, apiKey: string, options?: DiscoveryOptions & {
+    onProgress?: (message: string) => void;
+}): Promise<DiscoveryResult>;

package/dist/discover.js

@@ -1,4 +1,4 @@
-import { getModels, getProviders } from "@earendil-works/pi-ai";
+import { getModels, getProviders } from "@earendil-works/pi-ai/compat";
 const DEFAULT_TIMEOUT_MS = 5000;
 const DEFAULT_CONTEXT_WINDOW = 128_000;
 const DEFAULT_MAX_TOKENS = 16_384;
@@ -8,6 +8,13 @@ let modelsDevCatalog;
 export function normalizeBaseUrl(input) {
     return input.replace(/\/+$/, "").replace(/\/v1\/?$/i, "");
 }
+const RESPONSES_MODE_PATTERN = /^responses?$/i;
+function isResponsesMode(mode) {
+    return mode != null && RESPONSES_MODE_PATTERN.test(mode);
+}
+function isChatStyleMode(mode) {
+    return mode == null || mode === "chat" || isResponsesMode(mode);
+}
 // Matches both the conventional `anthropic/...` prefix and aliases that
 // LiteLLM deployments commonly assign to Anthropic-backed routes (e.g.
 // `google/claude-sonnet-4-6`, `opus-4.7`, `sonnet-4.6`, `haiku-4.5`). Without
@@ -16,9 +23,16 @@ export function normalizeBaseUrl(input) {
 const ANTHROPIC_MODEL_PATTERN = /(?:^|[-_/.:])(?:anthropic\/|(?:claude|opus|sonnet|haiku)(?=$|[-_/.:]))/i;
 const MOONSHOT_MODEL_PATTERN = /^(moonshotai\/|moonshot\/|kimi[-/])/i;
 const FORCED_THINKING_MODEL_PATTERN = /(?:^|[-/])thinking(?:[-/]|$)/i;
+// Deployments expose the gpt-5.5 route under varying names (`llm-gateway/gpt-5.5`,
+// bare `gpt-5.5`, dated ids like `gpt-5.5-20260504143601`); match them all so the
+// tool+reasoning workaround survives route renames.
+const GPT55_MODEL_PATTERN = /(?:^|\/)gpt-5\.5(?:$|[-.])/i;
 export function isMoonshotModel(modelId) {
     return MOONSHOT_MODEL_PATTERN.test(modelId);
 }
+export function isGpt55Model(modelId) {
+    return GPT55_MODEL_PATTERN.test(modelId);
+}
 export function shouldSuppressReasoningContent(modelId) {
     return isMoonshotModel(modelId) && !FORCED_THINKING_MODEL_PATTERN.test(modelId);
 }
@@ -90,6 +104,7 @@ function mapModelInfoCost(info, fallback) {
         cacheWrite: info.cache_creation_input_token_cost !== undefined
             ? info.cache_creation_input_token_cost * 1_000_000
             : (fallback?.cacheWrite ?? 0),
+        ...(fallback?.tiers ? { tiers: fallback.tiers } : {}),
     };
 }
 function getFallbackProviderAndModel(id, ownedBy) {
@@ -129,7 +144,7 @@ async function fetchJson(url, apiKey, options) {
     const { signal, cancel } = withTimeout(timeoutMs, options.signal);
     try {
         const response = await fetch(url, {
-            headers: { Authorization: `Bearer ${apiKey}`, Accept: "application/json" },
+            headers: { ...options.headers, Authorization: `Bearer ${apiKey}`, Accept: "application/json" },
             signal,
         });
         if (!response.ok)
@@ -199,27 +214,31 @@ function mapFromModelInfo(entry) {
     if (!id)
         return undefined;
     const info = entry.model_info ?? {};
-    if (info.mode && info.mode !== "chat")
+    if (!isChatStyleMode(info.mode))
         return undefined;
+    const responsesMode = isResponsesMode(info.mode);
     const catalogModel = findCatalogModel(id);
     return {
         id,
         name: id,
         reasoning: info.supports_reasoning ?? false,
+        ...(catalogModel?.thinkingLevelMap ? { thinkingLevelMap: catalogModel.thinkingLevelMap } : {}),
         input: info.supports_vision ? ["text", "image"] : ["text"],
         cost: mapModelInfoCost(info, catalogModel?.cost),
         contextWindow: info.max_input_tokens ?? DEFAULT_CONTEXT_WINDOW,
         maxTokens: info.max_output_tokens ?? DEFAULT_MAX_TOKENS,
         compat: buildCompat(id),
+        ...(responsesMode ? { api: "openai-responses" } : {}),
     };
 }
 function mapFromHealthModelInfo(entry, fallbackId) {
     const model = mapFromModelInfo(entry);
     if (model || !fallbackId)
         return model;
-    if (entry.model_info?.mode && entry.model_info.mode !== "chat")
+    if (!isChatStyleMode(entry.model_info?.mode))
         return undefined;
     const info = entry.model_info ?? {};
+    const responsesMode = isResponsesMode(info.mode);
     const catalogModel = findCatalogModel(fallbackId);
     return {
         id: fallbackId,
@@ -230,6 +249,7 @@ function mapFromHealthModelInfo(entry, fallbackId) {
         contextWindow: info.max_input_tokens ?? DEFAULT_CONTEXT_WINDOW,
         maxTokens: info.max_output_tokens ?? DEFAULT_MAX_TOKENS,
         compat: buildCompat(fallbackId),
+        ...(responsesMode ? { api: "openai-responses" } : {}),
     };
 }
 function mapFromHealthEndpoint(entry) {
@@ -268,45 +288,75 @@ function mapFromModelsList(entry, modelsDev) {
     };
 }
 async function discoverFromHealth(base, apiKey, options) {
+    options.onProgress?.("Querying /health endpoint...");
     const healthResult = await fetchJson(`${base}/health`, apiKey, options);
     if (!healthResult.ok)
         return [];
     const endpoints = (healthResult.data.healthy_endpoints ?? []).filter((entry) => entry.model || entry.model_id);
+    options.onProgress?.(`Discovered ${endpoints.length} model endpoints, fetching details...`);
+    let completed = 0;
     const models = await Promise.all(endpoints.map(async (endpoint) => {
-        if (!endpoint.model_id)
-            return mapFromHealthEndpoint(endpoint);
-        const infoResult = await fetchJson(`${base}/model/info?litellm_model_id=${encodeURIComponent(endpoint.model_id)}`, apiKey, options);
-        if (!infoResult.ok)
-            return mapFromHealthEndpoint(endpoint);
-        const entry = infoResult.data.data?.[0];
-        return entry ? mapFromHealthModelInfo(entry, endpoint.model) : mapFromHealthEndpoint(endpoint);
+        let model = mapFromHealthEndpoint(endpoint);
+        if (endpoint.model_id) {
+            const infoResult = await fetchJson(`${base}/model/info?litellm_model_id=${encodeURIComponent(endpoint.model_id)}`, apiKey, options);
+            const entry = infoResult.ok ? infoResult.data.data?.[0] : undefined;
+            if (entry)
+                model = mapFromHealthModelInfo(entry, endpoint.model);
+        }
+        completed++;
+        if (completed % 10 === 0 || completed === endpoints.length) {
+            options.onProgress?.(`Fetched ${completed}/${endpoints.length} models...`);
+        }
+        return model;
     }));
     return models.filter((model) => model !== undefined);
 }
+function deduplicateModels(models) {
+    const seen = new Set();
+    return models.filter((m) => {
+        if (seen.has(m.id))
+            return false;
+        seen.add(m.id);
+        return true;
+    });
+}
 export async function discoverModels(baseUrl, apiKey, options = {}) {
     const base = normalizeBaseUrl(baseUrl);
+    options.onProgress?.("Querying /model/info endpoint...");
     const infoResult = await fetchJson(`${base}/model/info`, apiKey, options);
     if (infoResult.ok) {
-        const models = (infoResult.data.data ?? [])
-            .map(mapFromModelInfo)
-            .filter((m) => m !== undefined);
+        const entries = new Map();
+        for (const entry of infoResult.data.data ?? []) {
+            if (!entry.model_name)
+                continue;
+            const previous = entries.get(entry.model_name);
+            entries.set(entry.model_name, {
+                ...previous,
+                ...entry,
+                model_info: { ...previous?.model_info, ...entry.model_info },
+            });
+        }
+        const models = [...entries.values()].map(mapFromModelInfo).filter((m) => m !== undefined);
         return { source: "model_info", models };
     }
     if (![401, 403, 404].includes(infoResult.status)) {
         throw new Error(`/model/info returned ${infoResult.status}`);
     }
+    options.onProgress?.("/model/info unavailable, trying /v1/models...");
     const listResult = await fetchJson(`${base}/v1/models`, apiKey, options);
     if (!listResult.ok) {
         if ([401, 403, 404].includes(listResult.status)) {
+            options.onProgress?.("/v1/models unavailable, falling back to /health endpoint...");
             const models = await discoverFromHealth(base, apiKey, options);
             if (models.length > 0)
-                return { source: "health", models };
+                return { source: "health", models: deduplicateModels(models) };
         }
         throw new Error(`/v1/models returned ${listResult.status}`);
     }
+    options.onProgress?.("Fetching models.dev catalog for metadata enrichment...");
     const modelsDev = await getModelsDevCatalog(options);
     const models = (listResult.data.data ?? [])
         .map((entry) => mapFromModelsList(entry, modelsDev))
         .filter((m) => m !== undefined);
-    return { source: "models_list", models };
+    return { source: "models_list", models: deduplicateModels(models) };
 }