pi-provider-litellm 4.2.0 → 4.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 LiteLLM proxy native Provider extension for [Pi](https://pi.dev). Pi 0.99.2+ is required; on older Pi, install `npm:pi-provider-litellm@3.4.0`.
 
-Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
+Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404 or an empty list, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
 
 ## Install
 
@@ -42,11 +42,17 @@ Inside pi:
 /login litellm
 ```
 
-To configure an API key, run `/login`, choose `Sign in with an API key`, then choose `LiteLLM API key`. With `/login litellm`, choose `Sign in with an API key` directly.
+Or for a configured alias provider (e.g. `team-a`):
+
+```
+/login team-a
+```
+
+To configure an API key, run `/login`, choose `Sign in with an API key`, then choose `LiteLLM API key` (or `<Provider Name> API key`). With `/login litellm`, choose `Sign in with an API key` directly; similarly, run `/login <alias>` to sign in to any configured alias directly.
 
 You'll be prompted for the base URL and API key. Credentials are persisted to `~/.pi/agent/auth.json`.
 
-If a proxy URL is already known — from `providers.<name>.baseUrl`, `LITELLM_BASE_URL`, or a previous login — pi offers it as the first option instead of asking you to retype it. Pick `Enter a different URL…` to point at another proxy.
+If a proxy URL is already known — from `providers.<name>.baseUrl`, `LITELLM_PROVIDER_<NAME>_BASE_URL`, `LITELLM_BASE_URL`, or a previous login — pi offers it as the first option instead of asking you to retype it. Pick `Enter a different URL…` to point at another proxy.
 
 #### Enterprise SSO login
 
@@ -54,9 +60,10 @@ If your LiteLLM proxy supports SSO/OAuth authentication, Pi selects its supporte
 
 1. Run `/login litellm` inside pi and select `Sign in with LiteLLM SSO`
 2. Confirm the offered proxy URL, or enter one if this is the first login
-3. Complete sign-in in the browser Pi opens, then return to Pi
+3. Without an `oidc` setting, choose `LiteLLM SSO` or your own identity provider (see [Direct OIDC login](#direct-oidc-login))
+4. Complete sign-in in the browser Pi opens, then return to Pi
 
-Without an `oidc` setting (see [Direct OIDC login](#direct-oidc-login)), Pi first checks `/.well-known/litellm-cli-auth`. A proxy exposing the supported CLI-auth contract uses authorization code + PKCE (`S256`) with a temporary `127.0.0.1` callback; the browser must run on the same machine as Pi. Advertised endpoints must stay on the proxy's origin, and discovery, registration, and token requests do not follow HTTP redirects. Invalid discovery stops login.
+With `LiteLLM SSO`, Pi first checks `/.well-known/litellm-cli-auth`. A proxy exposing the supported CLI-auth contract uses authorization code + PKCE (`S256`) with a temporary `127.0.0.1` callback; the browser must run on the same machine as Pi. Advertised endpoints must stay on the proxy's origin, and discovery, registration, and token requests do not follow HTTP redirects. Invalid discovery stops login.
 
 Pi stores PKCE access and refresh tokens in `~/.pi/agent/auth.json` with file mode `0600`, refreshes automatically, and saves each rotated token pair. Temporary network failures or HTTP 429/5xx responses allow reuse of the existing access token only until its exact expiry. Rejected or revoked refresh credentials require `/login litellm` again.
 
@@ -76,7 +83,9 @@ Prerequisites:
 - The IdP has a public client (no client secret) registered for authorization code + PKCE with the loopback redirect `http://127.0.0.1:<port>/callback`.
 - The browser runs on the same machine as Pi.
 
-Configure the default `litellm` provider in the global `~/.pi/agent/settings.json`. Project settings are never read for this, so a cloned repository cannot point login at its own IdP.
+To sign in without configuring anything, choose `Your identity provider (OIDC client ID)…` after confirming the proxy URL, then enter the issuer URL, the client ID, and the scopes (blank for `openid`). Pi checks them like the settings below and stores them with the credential, so the next login to the same proxy offers the same identity provider first. The login prompt does not ask for `redirectPorts`; an IdP that only accepts exact pre-registered redirect URIs needs the setting.
+
+To skip that prompt, configure the default `litellm` provider in the global `~/.pi/agent/settings.json`. Project settings are never read for this, so a cloned repository cannot point login at its own IdP.
 
 ```json
 { "litellm": { "providers": { "litellm": {
@@ -95,7 +104,7 @@ Configure the default `litellm` provider in the global `~/.pi/agent/settings.jso
 - `scope` (default `"openid"`): must include `openid`.
 - `redirectPorts` (optional): ports tried in order, for IdPs that only accept exact pre-registered redirect URIs. Without it the OS assigns a free port.
 
-With `oidc` set, `Sign in with LiteLLM SSO` confirms the proxy URL and then runs only this flow: Pi reads `<issuer>/.well-known/openid-configuration`, opens the IdP's authorization page, and exchanges the code for tokens. Login sends nothing to the proxy, and IdP requests never carry `LITELLM_HEADERS` or provider `headers`. An invalid `oidc` setting fails login with a message naming the field; it never falls back to the LiteLLM flows.
+With `oidc` set, `Sign in with LiteLLM SSO` confirms the proxy URL and then runs only this flow. Either way, Pi reads `<issuer>/.well-known/openid-configuration`, opens the IdP's authorization page, and exchanges the code for tokens. Login sends nothing to the proxy, and IdP requests never carry `LITELLM_HEADERS` or provider `headers`. An invalid `oidc` setting or entered value fails login with a message naming the field; it never falls back to the LiteLLM flows.
 
 The `id_token` is the bearer token. Pi checks its issuer, audience, authorized party (when present), subject, expiry, and nonce but not its signature; the proxy verifies the signature against the IdP's keys. Pi refreshes it with the refresh token, so the IdP must return a new `id_token` from the refresh grant. Some IdPs only issue a refresh token when the scope includes `offline_access`; without one, run `/login litellm` again when the `id_token` expires.
 
@@ -103,6 +112,8 @@ Pi never derives the IdP from the proxy: LiteLLM's own `/.well-known/openid-conf
 
 ### Option B — environment variables
 
+#### Single provider
+
 ```bash
 export LITELLM_BASE_URL="https://litellm.your-domain.com"
 export LITELLM_API_KEY="sk-..."
@@ -110,9 +121,74 @@ export LITELLM_API_KEY="sk-..."
 
 Stored pi credentials for `litellm` take precedence over `LITELLM_API_KEY`; the environment key is used when no saved credential exists. `LITELLM_BASE_URL` is used when no saved login base URL exists. Chat Completions and Responses models use the proxy root plus `/v1`; native Messages uses the proxy root directly.
 
+#### Multi-provider environment variables
+
+You can configure additional LiteLLM provider aliases directly through environment variables without editing `settings.json`.
+
+##### Per-provider environment variables (`LITELLM_PROVIDER_<NAME>_*`)
+
+Use the prefix `LITELLM_PROVIDER_<NAME>_` where `<NAME>` is an uppercase identifier (e.g., `TEAM_A`, `CORP_EAST`). `<NAME>` `LITELLM` is reserved for the default `litellm` provider, which reads only `LITELLM_PROVIDER_LITELLM_ENABLE_MCP` / `_MCP_ENABLED` and otherwise uses the [single-provider variables](#single-provider):
+
+| Variable | Effect |
+|---|---|
+| `LITELLM_PROVIDER_<NAME>_BASE_URL` | LiteLLM proxy base URL for this provider (e.g. `https://team-a.example.com`) |
+| `LITELLM_PROVIDER_<NAME>_API_KEY` | API key / bearer token for this provider |
+| `LITELLM_PROVIDER_<NAME>_API_KEY_HELPER` | Shell command that prints a fresh token for this provider |
+| `LITELLM_PROVIDER_<NAME>_HEADERS` | JSON string of extra request headers for this provider. Unlike `LITELLM_HEADERS`, `$VAR` and `${VAR}` inside the value are expanded, so write a literal `$` as `$$`; a reference to an unset variable drops all of this provider's headers |
+| `LITELLM_PROVIDER_<NAME>_DISPLAY_NAME` | Custom display name shown in Pi UI |
+| `LITELLM_PROVIDER_<NAME>_NAME` | Explicit provider ID override (e.g. `custom-name`) |
+| `LITELLM_PROVIDER_<NAME>_ALLOW_INSECURE_HTTP` | Set `"1"` or `"true"` to permit plaintext HTTP for non-loopback hosts |
+| `LITELLM_PROVIDER_<NAME>_USE_GCLOUD_AUTH` | Set `"1"` or `"true"` to enable Google ADC token authentication for this provider |
+| `LITELLM_PROVIDER_<NAME>_ENABLE_OAUTH` | Set `"1"` or `"true"` / `"0"` or `"false"` to enable or disable LiteLLM SSO/OAuth browser login |
+| `LITELLM_PROVIDER_<NAME>_ENABLE_MCP` / `_MCP_ENABLED` | Set `"0"` or `"false"` to disable MCP tool integration for this specific provider (or `"1"` / `"true"` to enable) |
+
+##### Bulk provider configuration
+
+- **`LITELLM_PROVIDERS`**: Canonical comma- or whitespace-separated list of provider IDs to register (e.g., `export LITELLM_PROVIDERS="team-a, team-b, corp-east"`).
+- **`LITELLM_PROVIDERS_JSON`**: Structured JSON string matching the `litellm.providers` schema in `settings.json` (e.g., `export LITELLM_PROVIDERS_JSON='{"team-a": {"baseUrl": "https://team-a.example.com", "apiKey": "sk-team-a"}}'`).
+
+##### Provider ID naming rules
+
+- **Default naming:** The uppercase token `<NAME>` in `LITELLM_PROVIDER_<NAME>_*` is automatically converted to lowercase kebab-case (e.g., `TEAM_A` becomes `team-a`, `CORP_EAST` becomes `corp-east`, and `DEV` becomes `dev`).
+- **Explicit override:** Set `LITELLM_PROVIDER_<NAME>_NAME` to specify an explicit provider ID (e.g., `LITELLM_PROVIDER_TEAM_A_NAME="custom-team"`).
+- **Canonical IDs:** Provider IDs specified in `LITELLM_PROVIDERS`, keys in `LITELLM_PROVIDERS_JSON`, and `settings.json` aliases that follow these naming rules preserve their declared IDs and read the `LITELLM_PROVIDER_<NAME>_*` variables whose `<NAME>` is the ID uppercased with `-` read as `_`, so `team_a` and `team-a` both read `TEAM_A`. When two declared IDs share a `<NAME>`, the one listed first in `LITELLM_PROVIDERS`, then `LITELLM_PROVIDERS_JSON`, then `settings.json`, reads those variables and the other reads none.
+- **Validation:** Provider IDs must match `/^[a-z0-9][a-z0-9_-]{0,63}$/` (1–64 characters, lowercase alphanumeric with `-` or `_`, starting with an alphanumeric character).
+- **Reserved names:** Tool and command names (`mcp`, `skills`, `codemode`, `tool_search`, and `litellm` as an alias) cannot be used as secondary provider IDs.
+
+### Option C — `models.json`
+
+For a long-lived API key, put the provider block in Pi's own `~/.pi/agent/models.json` instead of running `/login litellm`. No login is needed, and Pi injects the discovered models, so the block carries only `baseUrl` and `apiKey`:
+
+```json
+{
+  "providers": {
+    "litellm": {
+      "baseUrl": "https://litellm.your-domain.com",
+      "apiKey": "$LITELLM_API_KEY"
+    }
+  }
+}
+```
+
+`apiKey` is a Pi config value, so `$ENV_VAR`, `${ENV_VAR}`, `!command`, and a literal key all work, with a literal `$` escaped as `$$`. Prefer `$ENV_VAR` over a literal key: a key written literally here is exactly as readable on disk as one in `auth.json`.
+
+Only the global `~/.pi/agent/models.json` is read. A project-level file is never consulted, so a cloned repository cannot point this provider at a proxy of its own — the same rule [Direct OIDC login](#direct-oidc-login) states for `settings.json`.
+
+This covers long-lived tokens. SSO and other short-lived OAuth credentials still need `/login litellm`, because they require a browser round trip and refresh.
+
+#### Precedence
+
+A provider block may live in `models.json` or `settings.json`. The two merge per field, with `models.json` winning, and `skills`, `mcp`, and `budget` are read from `settings.json` only. `models.json` also accepts the default provider as `litellm.providers.litellm`, the same shape `settings.json` uses.
+
+A stored `/login` credential still takes precedence over a configured `apiKey`. Pi reads the top-level `providers.litellm` block itself, though, which changes three things:
+
+- Pi applies its `baseUrl` to every model, so a stored `/login` credential for a different proxy hides them all. To move a provider that already has a stored credential over to `models.json`, run `/logout litellm` first.
+- Pi hands its `apiKey` to the provider as a credential, so that key also outranks a Google ADC token. A key in `litellm.providers.litellm` or `settings.json` ranks below ADC.
+- Pi validates the fields it knows there, so `headers` must be an object. A `$LITELLM_HEADERS` string belongs in `litellm.providers.litellm`.
+
 ### Multiple LiteLLM provider aliases
 
-Add alias providers in `~/.pi/agent/settings.json` under `litellm.providers`. Each alias is registered as a separate Pi provider name, so models appear as `litellm/model-id` and `litellm-anthropic/model-id`.
+Add alias providers in `~/.pi/agent/settings.json` under `litellm.providers`, or in `~/.pi/agent/models.json` under the same shape. Pi rejects a `models.json` without a top-level `providers` object, so a file that holds only aliases needs `"providers": {}` as well. Each alias is registered as a separate Pi provider name, so models appear as `litellm/model-id` and `litellm-anthropic/model-id`. In `models.json`, the default provider is usually written as the top-level `providers.litellm` shown in Option C instead.
 
 ```json
 {
@@ -157,11 +233,30 @@ Provider fields:
 | `apiKey` | `LITELLM_API_KEY_HELPER`/`LITELLM_API_KEY` for `litellm`; required for aliases | Pi config value for this provider's key. Use `$ENV_VAR`, `${ENV_VAR}`, `!command`, or a literal key. Escape a literal `$` as `$$`. |
 | `headers` | `$LITELLM_HEADERS` for `litellm`; unset for aliases | JSON string env reference or inline object of request headers |
 | `displayName` | `LITELLM_DISPLAY_NAME` / `"LiteLLM"` for `litellm`; alias name for aliases | Label shown in Pi UI |
-| `enabled` | `true` | Set `false` to skip an alias |
-| `oidc` | unset | `litellm` only. Sign in directly with an OpenID Connect identity provider instead of the LiteLLM-hosted flows; see [Direct OIDC login](#direct-oidc-login) |
+| `enabled` | `true` | Set `false` to skip or disable an alias |
+| `oidc` | unset | `litellm` only, in global settings. Sign in directly with an OpenID Connect identity provider instead of the LiteLLM-hosted flows; see [Direct OIDC login](#direct-oidc-login) |
 | `allowInsecureHttp` | `false` | Set `true` to permit plaintext HTTP for this provider, for example `http://host.docker.internal`. Credentials and request data will not be encrypted. Loopback HTTP works without this setting. |
+| `useGcloudTokenAuth` | `false` | Aliases only. Set `true` to use Google Application Default Credentials (ADC) as this alias's bearer token. `litellm` ignores this field and uses ADC only when `LITELLM_GCLOUD_TOKEN_AUTH` is enabled. |
+| `enableOAuth` | `false` | Aliases only. Set `true` to offer LiteLLM SSO browser login for this alias. `litellm` ignores this field and always offers it. |
+
+#### Configuration & credential precedence
+
+- **Provider definition merging precedence:**
+  When discovering and configuring provider aliases from multiple sources, definitions are merged in the following order:
+  1. Base provider definitions from `LITELLM_PROVIDERS_JSON`
+  2. Discovered environment variables (`LITELLM_PROVIDER_<NAME>_*` and canonical list in `LITELLM_PROVIDERS`)
+  3. Disk settings from `~/.pi/agent/settings.json` under `litellm.providers`, merged per field with the provider blocks in `~/.pi/agent/models.json`, which win (see Option C). Disk settings override environment definitions, and setting `"enabled": false` removes or skips the provider.
+
+  Each later source overrides `displayName`, `allowInsecureHttp`, `useGcloudTokenAuth`, `enableOAuth`, and the MCP toggle. `baseUrl`, `apiKey`, and `headers` work differently: a value configured in `settings.json`, `models.json`, or `LITELLM_PROVIDERS_JSON` always wins, and `LITELLM_PROVIDER_<NAME>_BASE_URL`, `_API_KEY_HELPER`, `_API_KEY`, and `_HEADERS` apply only when neither sets that field.
 
-`/login litellm` and Google ADC token auth remain scoped to the default `litellm` provider. Aliases use their configured `apiKey` or manually stored auth entries matching the alias name.
+- **Credential resolution precedence:**
+  When authenticating requests for any provider:
+  1. Saved interactive credentials in `~/.pi/agent/auth.json` (from `/login <provider>` or OAuth SSO), then the `apiKey` of `models.json`'s top-level `providers.litellm`, which Pi hands to the provider as a credential
+  2. Google Application Default Credentials (for an alias with `useGcloudTokenAuth` or `_USE_GCLOUD_AUTH`; for `litellm` when `LITELLM_GCLOUD_TOKEN_AUTH` is enabled)
+  3. Configured `apiKey` from settings (`settings.json`, `litellm.providers` in `models.json`, or `LITELLM_PROVIDERS_JSON`, including `$ENV_VAR` and `!command`)
+  4. Provider-scoped API key helper (`LITELLM_PROVIDER_<NAME>_API_KEY_HELPER`)
+  5. Provider-scoped API key (`LITELLM_PROVIDER_<NAME>_API_KEY`)
+  6. For the default `litellm` provider only: fallback to global `LITELLM_API_KEY_HELPER` then `LITELLM_API_KEY`.
 
 ### Optional LiteLLM features
 
@@ -191,6 +286,12 @@ LiteLLM Skills, MCP integration, and the budget footer are enabled by default. D
 
 Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` stops the extension from registering the proxy's MCP server. Setting `budget.enabled` to `false` registers no budget footer, polls, or `/litellm-budget` command; `budget.display` is described under [Budget status](#budget-status). Restart Pi after changing these settings.
 
+MCP can also be enabled or disabled per provider, though a provider setting cannot turn it on while `mcp.enabled` is `false`:
+- In `~/.pi/agent/settings.json`: set `providers.<name>.mcp.enabled` (or `providers.<name>.enableMcp`) to `false` (or `true`).
+- In environment variables: set `LITELLM_PROVIDER_<NAME>_ENABLE_MCP` or `LITELLM_PROVIDER_<NAME>_MCP_ENABLED` to `"0"` or `"false"` to disable MCP for a specific provider.
+
+Restart Pi after changing these settings.
+
 `mcp.exposure` and `mcp.toolExposure` are passed to Pi unchanged and mean what they mean in a Pi `mcp.json` server entry: `codemode` (Pi's default when `exposure` is unset; callable from Pi's `codemode` scripts), `deferred` (declared to the model once Pi's `tool_search` loads it), `direct` (declared on every request), or `hidden`. `toolExposure` keys are tool names as LiteLLM's `/mcp` endpoint offers them, which LiteLLM prefixes with the server name, for example `github-search_code`; `*` matches any characters. Pi validates both settings, and an unusable value is reported once.
 
 Treat the configured LiteLLM proxy as trusted: Skills can add instructions to the system prompt, and MCP can expose tools the agent may call. Disable these integrations when the proxy, its administrators, or its configured content are not fully trusted.
@@ -250,6 +351,30 @@ A `/model/info` route group uses native Anthropic `/v1/messages` only when every
 
 Native Messages authenticates with `x-api-key`; every transport carries the `x-litellm-session-id` header described above. Strict JSON-schema tools are enabled separately from Messages transport and thinking compatibility: every deployment must affirm support for the actual route/API. Generic `supports_response_schema` and `supports_native_structured_output` metadata does not grant strict tools. Only deployments that LiteLLM reports with the `anthropic` adapter carry that evidence today. Claude served through Bedrock or Vertex therefore uses ordinary tool serialization, and a tool that requires strict sampling fails before the request is sent.
 
+## Image generation
+
+Routes whose every deployment reports `mode: "image_generation"` in `/model/info` are published as Pi image models
+(for example `litellm/gemini-nano-banana-2.1`). Like Pi's other image models they do not appear in `/model`; on Pi 1.0
+and later, scripts reach them through Pi's `codemode` tool:
+
+```js
+const painter = await models.getModelOfType("image", "litellm", "gemini-nano-banana-2.1");
+const result = await models.generateImages(painter, {
+  input: [{ type: "text", text: "A red fox in the snow, watercolor" }],
+});
+if (result.stopReason !== "stop") return result.errorMessage;
+for (const block of result.output) if (block.type === "image") image(block);
+```
+
+Requests go to the proxy's OpenAI-compatible `/v1/images/generations` with the provider's own credential and
+headers, so they work for any LiteLLM image deployment (`gpt-image-*`, `dall-e-*`, Gemini image models and others)
+and are budgeted and logged by the proxy like chat requests. Usage cost comes from LiteLLM's `x-litellm-response-cost`
+header when present; otherwise it is estimated from `input_cost_per_token` and `output_cost_per_image_token` (or
+`output_cost_per_token`). Only routes found through `/model/info` qualify: `/v1/models` and `/health` carry no mode,
+and groups that mix image and other modes, or wildcard routes, are withheld. The prompt is text only; reference images
+need `/v1/images/edits`, which this provider does not call, and backends that answer with image URLs instead of
+base64 data are reported as errors rather than fetched.
+
 ## Optional environment variables
 
 | Variable | Default | Effect |
@@ -257,10 +382,13 @@ Native Messages authenticates with `x-api-key`; every transport carries the `x-l
 | `LITELLM_API_KEY_HELPER` | unset | Command that prints a fresh LiteLLM bearer token. Takes precedence over `LITELLM_API_KEY`. The extension runs it while resolving request auth, and Pi's per-request auth path is uncached, so rotating/short-lived tokens stay fresh. |
 | `LITELLM_DISPLAY_NAME` | `LiteLLM` | Custom display name for the default `litellm` provider in the Pi UI. |
 | `LITELLM_HEADERS` | unset | JSON object of extra headers sent to LiteLLM provider, discovery, MCP, and Skills Gateway requests. Provider aliases can use it with `"headers": "$LITELLM_HEADERS"`. |
+| `LITELLM_PROVIDERS` | unset | Canonical comma- or whitespace-separated list of provider IDs to register. |
+| `LITELLM_PROVIDERS_JSON` | unset | JSON string of provider configuration objects matching the `litellm.providers` settings schema. |
+| `LITELLM_PROVIDER_<NAME>_*` | unset | Per-provider environment variables for base URL, API key, headers, display name, OAuth, Google auth, and MCP enable/disable toggle (`_ENABLE_MCP` / `_MCP_ENABLED`). See [Multi-provider environment variables](#multi-provider-environment-variables). |
 | `LITELLM_GCLOUD_TOKEN_AUTH` | unset | If set to a non-empty value other than `0`, use Google Application Default Credentials as the LiteLLM bearer token source. This takes precedence over `LITELLM_API_KEY_HELPER` and `LITELLM_API_KEY` when no stored `/login litellm` credential exists. |
 | `GOOGLE_APPLICATION_CREDENTIALS` | Google default ADC path | Optional path to an ADC JSON file used by `LITELLM_GCLOUD_TOKEN_AUTH`. If unset, the extension checks the default gcloud ADC locations. |
 | `LITELLM_OFFLINE` | unset | If `1`, disable all model and MCP discovery, including post-login discovery; use cached models only when their stored canonical proxy root exactly matches the active credential root, including any path prefix. URL-standard host casing and default ports are canonicalized, but paths remain case-sensitive. |
-| `PI_OFFLINE` | unset | Pi documents it as disabling automatic network activity, including model catalog refreshes. Through Pi 1.0.3, any set value (including `0` or an empty string) disables startup model discovery and `/model` network refreshes. Run `/litellm-refresh`, or unset it, to discover; cached models can still be used offline. |
+| `PI_OFFLINE` | unset | Pi documents it as disabling automatic network activity, including model catalog refreshes. Through Pi 1.2.0, any set value (including `0` or an empty string) disables startup model discovery and `/model` network refreshes. Run `/litellm-refresh`, or unset it, to discover; cached models can still be used offline. |
 | `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Background and explicit discovery fetch timeout in ms; `0` disables automatic discovery |
 | `LITELLM_CLI_JWT_EXPIRATION_HOURS` | `24` | CLI SSO token lifetime fallback for older proxies whose poll response omits `expires_in`; mirror a non-default proxy setting locally |
 | `LITELLM_VERBOSE_DISCOVERY` | unset | If `1`, enable progress messages during model discovery (login, refresh, startup), including startup skip reasons and defaulted `/model/info` context windows. Progress messages are off by default |
@@ -291,7 +419,9 @@ LiteLLM serves the MCP servers it gateways over streamable HTTP at `/mcp`. For e
 
 The registration does not copy the provider's API key or token. It uses Pi's `auth: { provider }` setting, so Pi sends the provider's current credential as `Authorization: Bearer` on every request, and token refreshes, Google ADC, and key helpers apply without registering again. Custom headers (`LITELLM_HEADERS`, or an alias's `headers`) are registered as configured, after `$` and a leading `!` are escaped so Pi does not expand them a second time. Pi sends provider credentials only over https or to a loopback host, so a proxy reached over plain http elsewhere (`allowInsecureHttp`) gets no MCP server, and the refusal is reported once.
 
-The extension registers at startup and checks again before each agent turn. `/login litellm` disconnects the server as soon as it starts, before the new credential is stored, and the next turn connects again with the new credential and proxy root; a logout that leaves no root removes the server. `LITELLM_OFFLINE=1`, `LITELLM_DISCOVERY_TIMEOUT_MS=0`, and `PI_OFFLINE` register nothing. A server with the same name in your own `mcp.json` takes precedence over the extension's.
+The extension registers at startup and updates the registration before each agent turn. `/login litellm` disconnects the server as soon as it starts, before the new credential is stored, and the next turn connects again with the new credential and proxy root; a logout that leaves no root removes the server. `LITELLM_OFFLINE=1`, `LITELLM_DISCOVERY_TIMEOUT_MS=0`, and `PI_OFFLINE` register nothing. A server with the same name in your own `mcp.json` takes precedence over the extension's.
+
+Before registering, the extension sends the proxy one MCP `initialize` with the credential and headers Pi will send, and ends the session it opened. LiteLLM 1.102.0 and later refuse it with 403 when no MCP servers are granted to the key, its team, or its organization, or none of them is allowed from your IP address. The extension then registers the server disabled: Pi does not connect to it, warn about it at startup, or turn on `codemode` for it, and `/mcp` lists it as disabled. Once servers are granted, enable it in `/mcp` for the session, or restart Pi, which checks again, as does a login. When a key helper (a `!command` key) or Google ADC is configured, the check is skipped and the server is registered enabled.
 
 Pi reads the provider's credential when it sends each request, not when the server was registered. If another Pi process stores a login for a different proxy, this session's open connection can send that credential to the previous proxy until the next turn moves the server.
 
@@ -309,6 +439,11 @@ If your LiteLLM proxy exposes `/claude-code/marketplace.json`, enabled skills ar
 
 `litellm_skill_create` needs Skill Hub `sourceJson` metadata and a proxy that exposes `/claude-code/plugins`. It does not create skills from code: LiteLLM's `POST /v1/skills` is Anthropic's multipart Skills API.
 
+## PiG
+
+A Go port for [PiG](https://github.com/MichaelKinsy/PiG) lives in [`pig/`](pig/README.md). It reads the same
+settings, environment variables and `auth.json` entries; see [pig/README.md](pig/README.md) to install it.
+
 ## Mocked LiteLLM smoke workflow
 
 The `LiteLLM Smoke` GitHub Actions workflow starts VidaiMock and a real LiteLLM proxy on the runner. LiteLLM exposes route-distinct Chat, Responses, native Messages, and mixed-deployment models whose upstreams are served by VidaiMock. The smoke runner discovers those models through LiteLLM, asserts each model's expected API, exercises `/v1/chat/completions`, `/v1/responses`, and `/v1/messages`, verifies the expected `x-litellm-response-cost` behavior, and proves endpoint coverage from captured LiteLLM request logs rather than response text.
@@ -350,6 +485,8 @@ LiteLLM's `/model/info` is the metadata authority, so models.dev enrichment is a
 
 Opening `/model` refreshes configured provider catalogs in the background using Pi's native model lifecycle when network discovery is enabled.
 
+Startup reuses that stored catalog instead of discovering again while the provider's entry is under 4 hours old and comes from the active proxy and the current discovery version. Interactive and RPC sessions still refresh it in the background; `pi -p` does not. For up to 4 hours after the last refresh, print mode can therefore miss a model added on the proxy or granted to your key, and after a key change on the same proxy it can list the previous key's models. Run `/litellm-refresh` or start an interactive session to update it sooner.
+
 Pi documents `PI_OFFLINE` and `--offline` as disabling automatic network activity, including model catalog refreshes. Pi 0.87.1 adopted that wording to resolve [earendil-works/pi#8684](https://github.com/earendil-works/pi/issues/8684), without changing the behavior. Pi still checks only whether the variable is set, so `PI_OFFLINE=0` and an empty value also disable model refreshes. With `LITELLM_VERBOSE_DISCOVERY=1`, disabled startup discovery reports its reason, for example `startup discovery skipped (PI_OFFLINE)`. To discover models while keeping `PI_OFFLINE`, run `/litellm-refresh`. To allow all discovery for one invocation, use `env -u PI_OFFLINE pi` and do not pass `--offline`. If you only want to disable Pi's version check, use `PI_SKIP_VERSION_CHECK=1` instead; it does not apply offline mode's other network restrictions. See [#155](https://github.com/balcsida/pi-provider-litellm/issues/155).
 
 ### Model host enforcement
@@ -418,7 +555,7 @@ The development probe runs against minimized snapshots with `npm run probe:proxy
 | No models, or no new routes, while `PI_OFFLINE` is set | Pi skips every model network refresh, including the one `/model` starts. Run `/litellm-refresh`, or unset `PI_OFFLINE` (`PI_OFFLINE=0` is not enough) |
 | `No models available` at startup, gone after a restart | A Pi startup race, not discovery — see [`No models available` at startup](#no-models-available-at-startup) |
 | "discovered no models" | Proxy returned an empty list — check pi's startup log and verify `/model/info`, `/v1/models`, or `/health` responds |
-| `/model/info` returning 401/403/404 | Expected behavior with virtual keys — extension falls back to `/v1/models` |
+| `/model/info` returning 401/403/404 or an empty list | Expected behavior with virtual keys — extension falls back to `/v1/models` |
 | Discovery times out | Increase `LITELLM_DISCOVERY_TIMEOUT_MS` or set `LITELLM_OFFLINE=1` to fall back on cached models. Offline mode does not recover a root mismatch, including a different path prefix — see [Model host enforcement](#model-host-enforcement) |
 | A provider shows no models | The base URL is missing, invalid, still the placeholder, or its full root, including any path prefix, differs from the root in the cached catalog. Check stderr and see [Model host enforcement](#model-host-enforcement) |
 | A configured or restored model fails to complete but never leaks the credential | Its `api` is absent from the current provider catalog, so Pi used global API fallback with the credential root pinned. Open `/model` against the active proxy to repopulate the protocol — see [Model host enforcement](#model-host-enforcement) |
@@ -432,7 +569,7 @@ The development probe runs against minimized snapshots with `npm run probe:proxy
 | Enterprise CLI SSO start/poll fails | Check the proxy logs and verify `/sso/cli/start` and `/sso/cli/poll/{login_id}` are reachable; only 404/405 falls back to legacy login |
 | Enterprise SSO login shows "virtual key generation failed" | The LiteLLM instance may lack a database (`/key/generate` requires one), your user account may lack key-generation permission, or the request timed out; the JWT is used directly as a fallback |
 | Enterprise SSO token prompt fails with "SSO token is required" | The token field was left empty — paste the token copied from the LiteLLM UI |
-| MCP tools not showing | Run `/mcp` and check the provider's server: its state shows a connection or sign-in error. Verify the proxy serves `/mcp` and that the key has MCP access |
+| MCP tools not showing | Run `/mcp` and check the provider's server: its state shows a connection or sign-in error. `disabled` means the proxy refused the key at startup, usually because no MCP servers are granted to it, its team, or its organization; after a grant, enable it in `/mcp` or restart Pi. Verify the proxy serves `/mcp` and that the key has MCP access |
 | MCP tools listed but the model cannot call them | `codemode` and `deferred` tools are reached through Pi's `codemode` and `tool_search` tools. Check that neither is disabled; `--no-extensions` disables them and Pi's MCP client alike. Setting `litellm.mcp.exposure` to `direct` declares the tools on every request instead |
 | Skills not affecting prompts | Verify the proxy exposes `/claude-code/marketplace.json` or `/v1/skills` and returns enabled skills |
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-provider-litellm",
-  "version": "4.2.0",
+  "version": "4.3.0",
   "description": "LiteLLM proxy provider extension for Pi",
   "type": "module",
   "license": "MIT",
@@ -74,7 +74,7 @@
     "@typescript/native-preview": "7.0.0-dev.20260707.2",
     "es-module-lexer": "^3.0.2",
     "tsx": "^4.21.0",
-    "typebox": "1.3.34",
+    "typebox": "1.3.37",
     "vitest": "^5.0.0"
   }
 }

package/src/discover.ts

@@ -8,6 +8,7 @@ import {
   resolveCatalogProvider,
   routesOnlyThrough,
 } from "./backend-identity.js";
+import { type DiscoveredImageModel, mapFromImageModelInfoGroup } from "./images.js";
 import {
   type CatalogResolution,
   catalogResolution,
@@ -87,7 +88,14 @@ export function normalizeBaseUrl(input: string, allowInsecureHttp = false): stri
   if (url.protocol !== "https:" && !(url.protocol === "http:" && (loopback || allowInsecureHttp))) {
     throw new Error("LiteLLM base URL must use HTTPS except for loopback hosts");
   }
-  return input.replace(/\/+$/, "").replace(/\/v1\/?$/i, "");
+  return trimTrailingSlashes(input).replace(/\/v1\/?$/i, "");
+}
+
+// A loop, not /\/+$/: that regex backtracks quadratically on a long slash run followed by another character.
+export function trimTrailingSlashes(value: string): string {
+  let end = value.length;
+  while (end > 0 && value[end - 1] === "/") end--;
+  return value.slice(0, end);
 }
 
 // Matches both the conventional `anthropic/...` prefix and aliases that
@@ -1167,9 +1175,12 @@ export async function discoverModels(
   const progress = options.silent ? undefined : options.onProgress;
   progress?.("Querying /model/info endpoint...");
   const infoResult = await fetchJson<ModelInfoResponse>(`${base}/model/info`, apiKey, options);
-  if (infoResult.ok) {
+  // An empty `/model/info` list is treated like an unavailable endpoint: some proxies
+  // answer 200 with no rows for keys that can still list models through `/v1/models`.
+  const infoEntries = infoResult.ok ? (infoResult.data.data ?? []) : [];
+  if (infoEntries.length > 0) {
     const groups = new Map<string, ModelInfoEntry[]>();
-    for (const entry of infoResult.data.data ?? []) {
+    for (const entry of infoEntries) {
       // A route without a readable public name cannot be grouped or addressed.
       const route = wireString(entry.model_name);
       if (!route) continue;
@@ -1294,12 +1305,21 @@ export async function discoverModels(
           "set model_info.max_input_tokens on every deployment in each route",
       );
     }
-    return { source: "model_info", models, ...(proxyVersion ? { proxyVersion } : {}) };
+    // Image routes are a separate operation in Pi; the chat reducer above withholds them.
+    const imageModels = [...groups.entries()]
+      .map(([route, group]) => mapFromImageModelInfoGroup(route, group))
+      .filter((model): model is DiscoveredImageModel => model !== undefined);
+    return {
+      source: "model_info",
+      models,
+      ...(imageModels.length > 0 ? { imageModels } : {}),
+      ...(proxyVersion ? { proxyVersion } : {}),
+    };
   }
-  if (![401, 403, 404].includes(infoResult.status)) {
+  if (!infoResult.ok && ![401, 403, 404].includes(infoResult.status)) {
     throw new Error(`/model/info returned ${infoResult.status}`);
   }
-  progress?.("/model/info unavailable, trying /v1/models...");
+  progress?.("/model/info unavailable or empty, trying /v1/models...");
   const listResult = await fetchJson<ModelsListResponse>(`${base}/v1/models`, apiKey, options);
   if (!listResult.ok) {
     if ([401, 403, 404].includes(listResult.status)) {