pi-provider-litellm 4.1.0 → 4.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 LiteLLM proxy native Provider extension for [Pi](https://pi.dev). Pi 0.99.2+ is required; on older Pi, install `npm:pi-provider-litellm@3.4.0`.
 
-Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
+Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404 or an empty list, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
 
 ## Install
 
@@ -42,11 +42,17 @@ Inside pi:
 /login litellm
 ```
 
-To configure an API key, run `/login`, choose `Sign in with an API key`, then choose `LiteLLM API key`. With `/login litellm`, choose `Sign in with an API key` directly.
+Or for a configured alias provider (e.g. `team-a`):
+
+```
+/login team-a
+```
+
+To configure an API key, run `/login`, choose `Sign in with an API key`, then choose `LiteLLM API key` (or `<Provider Name> API key`). With `/login litellm`, choose `Sign in with an API key` directly; similarly, run `/login <alias>` to sign in to any configured alias directly.
 
 You'll be prompted for the base URL and API key. Credentials are persisted to `~/.pi/agent/auth.json`.
 
-If a proxy URL is already known — from `providers.<name>.baseUrl`, `LITELLM_BASE_URL`, or a previous login — pi offers it as the first option instead of asking you to retype it. Pick `Enter a different URL…` to point at another proxy.
+If a proxy URL is already known — from `providers.<name>.baseUrl`, `LITELLM_PROVIDER_<NAME>_BASE_URL`, `LITELLM_BASE_URL`, or a previous login — pi offers it as the first option instead of asking you to retype it. Pick `Enter a different URL…` to point at another proxy.
 
 #### Enterprise SSO login
 
@@ -54,9 +60,10 @@ If your LiteLLM proxy supports SSO/OAuth authentication, Pi selects its supporte
 
 1. Run `/login litellm` inside pi and select `Sign in with LiteLLM SSO`
 2. Confirm the offered proxy URL, or enter one if this is the first login
-3. Complete sign-in in the browser Pi opens, then return to Pi
+3. Without an `oidc` setting, choose `LiteLLM SSO` or your own identity provider (see [Direct OIDC login](#direct-oidc-login))
+4. Complete sign-in in the browser Pi opens, then return to Pi
 
-Without an `oidc` setting (see [Direct OIDC login](#direct-oidc-login)), Pi first checks `/.well-known/litellm-cli-auth`. A proxy exposing the supported CLI-auth contract uses authorization code + PKCE (`S256`) with a temporary `127.0.0.1` callback; the browser must run on the same machine as Pi. Advertised endpoints must stay on the proxy's origin, and discovery, registration, and token requests do not follow HTTP redirects. Invalid discovery stops login.
+With `LiteLLM SSO`, Pi first checks `/.well-known/litellm-cli-auth`. A proxy exposing the supported CLI-auth contract uses authorization code + PKCE (`S256`) with a temporary `127.0.0.1` callback; the browser must run on the same machine as Pi. Advertised endpoints must stay on the proxy's origin, and discovery, registration, and token requests do not follow HTTP redirects. Invalid discovery stops login.
 
 Pi stores PKCE access and refresh tokens in `~/.pi/agent/auth.json` with file mode `0600`, refreshes automatically, and saves each rotated token pair. Temporary network failures or HTTP 429/5xx responses allow reuse of the existing access token only until its exact expiry. Rejected or revoked refresh credentials require `/login litellm` again.
 
@@ -76,7 +83,9 @@ Prerequisites:
 - The IdP has a public client (no client secret) registered for authorization code + PKCE with the loopback redirect `http://127.0.0.1:<port>/callback`.
 - The browser runs on the same machine as Pi.
 
-Configure the default `litellm` provider in the global `~/.pi/agent/settings.json`. Project settings are never read for this, so a cloned repository cannot point login at its own IdP.
+To sign in without configuring anything, choose `Your identity provider (OIDC client ID)…` after confirming the proxy URL, then enter the issuer URL, the client ID, and the scopes (blank for `openid`). Pi checks them like the settings below and stores them with the credential, so the next login to the same proxy offers the same identity provider first. The login prompt does not ask for `redirectPorts`; an IdP that only accepts exact pre-registered redirect URIs needs the setting.
+
+To skip that prompt, configure the default `litellm` provider in the global `~/.pi/agent/settings.json`. Project settings are never read for this, so a cloned repository cannot point login at its own IdP.
 
 ```json
 { "litellm": { "providers": { "litellm": {
@@ -95,7 +104,7 @@ Configure the default `litellm` provider in the global `~/.pi/agent/settings.jso
 - `scope` (default `"openid"`): must include `openid`.
 - `redirectPorts` (optional): ports tried in order, for IdPs that only accept exact pre-registered redirect URIs. Without it the OS assigns a free port.
 
-With `oidc` set, `Sign in with LiteLLM SSO` confirms the proxy URL and then runs only this flow: Pi reads `<issuer>/.well-known/openid-configuration`, opens the IdP's authorization page, and exchanges the code for tokens. Login sends nothing to the proxy, and IdP requests never carry `LITELLM_HEADERS` or provider `headers`. An invalid `oidc` setting fails login with a message naming the field; it never falls back to the LiteLLM flows.
+With `oidc` set, `Sign in with LiteLLM SSO` confirms the proxy URL and then runs only this flow. Either way, Pi reads `<issuer>/.well-known/openid-configuration`, opens the IdP's authorization page, and exchanges the code for tokens. Login sends nothing to the proxy, and IdP requests never carry `LITELLM_HEADERS` or provider `headers`. An invalid `oidc` setting or entered value fails login with a message naming the field; it never falls back to the LiteLLM flows.
 
 The `id_token` is the bearer token. Pi checks its issuer, audience, authorized party (when present), subject, expiry, and nonce but not its signature; the proxy verifies the signature against the IdP's keys. Pi refreshes it with the refresh token, so the IdP must return a new `id_token` from the refresh grant. Some IdPs only issue a refresh token when the scope includes `offline_access`; without one, run `/login litellm` again when the `id_token` expires.
 
@@ -103,6 +112,8 @@ Pi never derives the IdP from the proxy: LiteLLM's own `/.well-known/openid-conf
 
 ### Option B — environment variables
 
+#### Single provider
+
 ```bash
 export LITELLM_BASE_URL="https://litellm.your-domain.com"
 export LITELLM_API_KEY="sk-..."
@@ -110,9 +121,74 @@ export LITELLM_API_KEY="sk-..."
 
 Stored pi credentials for `litellm` take precedence over `LITELLM_API_KEY`; the environment key is used when no saved credential exists. `LITELLM_BASE_URL` is used when no saved login base URL exists. Chat Completions and Responses models use the proxy root plus `/v1`; native Messages uses the proxy root directly.
 
+#### Multi-provider environment variables
+
+You can configure additional LiteLLM provider aliases directly through environment variables without editing `settings.json`.
+
+##### Per-provider environment variables (`LITELLM_PROVIDER_<NAME>_*`)
+
+Use the prefix `LITELLM_PROVIDER_<NAME>_` where `<NAME>` is an uppercase identifier (e.g., `TEAM_A`, `CORP_EAST`). `<NAME>` `LITELLM` is reserved for the default `litellm` provider, which reads only `LITELLM_PROVIDER_LITELLM_ENABLE_MCP` / `_MCP_ENABLED` and otherwise uses the [single-provider variables](#single-provider):
+
+| Variable | Effect |
+|---|---|
+| `LITELLM_PROVIDER_<NAME>_BASE_URL` | LiteLLM proxy base URL for this provider (e.g. `https://team-a.example.com`) |
+| `LITELLM_PROVIDER_<NAME>_API_KEY` | API key / bearer token for this provider |
+| `LITELLM_PROVIDER_<NAME>_API_KEY_HELPER` | Shell command that prints a fresh token for this provider |
+| `LITELLM_PROVIDER_<NAME>_HEADERS` | JSON string of extra request headers for this provider. Unlike `LITELLM_HEADERS`, `$VAR` and `${VAR}` inside the value are expanded, so write a literal `$` as `$$`; a reference to an unset variable drops all of this provider's headers |
+| `LITELLM_PROVIDER_<NAME>_DISPLAY_NAME` | Custom display name shown in Pi UI |
+| `LITELLM_PROVIDER_<NAME>_NAME` | Explicit provider ID override (e.g. `custom-name`) |
+| `LITELLM_PROVIDER_<NAME>_ALLOW_INSECURE_HTTP` | Set `"1"` or `"true"` to permit plaintext HTTP for non-loopback hosts |
+| `LITELLM_PROVIDER_<NAME>_USE_GCLOUD_AUTH` | Set `"1"` or `"true"` to enable Google ADC token authentication for this provider |
+| `LITELLM_PROVIDER_<NAME>_ENABLE_OAUTH` | Set `"1"` or `"true"` / `"0"` or `"false"` to enable or disable LiteLLM SSO/OAuth browser login |
+| `LITELLM_PROVIDER_<NAME>_ENABLE_MCP` / `_MCP_ENABLED` | Set `"0"` or `"false"` to disable MCP tool integration for this specific provider (or `"1"` / `"true"` to enable) |
+
+##### Bulk provider configuration
+
+- **`LITELLM_PROVIDERS`**: Canonical comma- or whitespace-separated list of provider IDs to register (e.g., `export LITELLM_PROVIDERS="team-a, team-b, corp-east"`).
+- **`LITELLM_PROVIDERS_JSON`**: Structured JSON string matching the `litellm.providers` schema in `settings.json` (e.g., `export LITELLM_PROVIDERS_JSON='{"team-a": {"baseUrl": "https://team-a.example.com", "apiKey": "sk-team-a"}}'`).
+
+##### Provider ID naming rules
+
+- **Default naming:** The uppercase token `<NAME>` in `LITELLM_PROVIDER_<NAME>_*` is automatically converted to lowercase kebab-case (e.g., `TEAM_A` becomes `team-a`, `CORP_EAST` becomes `corp-east`, and `DEV` becomes `dev`).
+- **Explicit override:** Set `LITELLM_PROVIDER_<NAME>_NAME` to specify an explicit provider ID (e.g., `LITELLM_PROVIDER_TEAM_A_NAME="custom-team"`).
+- **Canonical IDs:** Provider IDs specified in `LITELLM_PROVIDERS`, keys in `LITELLM_PROVIDERS_JSON`, and `settings.json` aliases that follow these naming rules preserve their declared IDs and read the `LITELLM_PROVIDER_<NAME>_*` variables whose `<NAME>` is the ID uppercased with `-` read as `_`, so `team_a` and `team-a` both read `TEAM_A`. When two declared IDs share a `<NAME>`, the one listed first in `LITELLM_PROVIDERS`, then `LITELLM_PROVIDERS_JSON`, then `settings.json`, reads those variables and the other reads none.
+- **Validation:** Provider IDs must match `/^[a-z0-9][a-z0-9_-]{0,63}$/` (1–64 characters, lowercase alphanumeric with `-` or `_`, starting with an alphanumeric character).
+- **Reserved names:** Tool and command names (`mcp`, `skills`, `codemode`, `tool_search`, and `litellm` as an alias) cannot be used as secondary provider IDs.
+
+### Option C — `models.json`
+
+For a long-lived API key, put the provider block in Pi's own `~/.pi/agent/models.json` instead of running `/login litellm`. No login is needed, and Pi injects the discovered models, so the block carries only `baseUrl` and `apiKey`:
+
+```json
+{
+  "providers": {
+    "litellm": {
+      "baseUrl": "https://litellm.your-domain.com",
+      "apiKey": "$LITELLM_API_KEY"
+    }
+  }
+}
+```
+
+`apiKey` is a Pi config value, so `$ENV_VAR`, `${ENV_VAR}`, `!command`, and a literal key all work, with a literal `$` escaped as `$$`. Prefer `$ENV_VAR` over a literal key: a key written literally here is exactly as readable on disk as one in `auth.json`.
+
+Only the global `~/.pi/agent/models.json` is read. A project-level file is never consulted, so a cloned repository cannot point this provider at a proxy of its own — the same rule [Direct OIDC login](#direct-oidc-login) states for `settings.json`.
+
+This covers long-lived tokens. SSO and other short-lived OAuth credentials still need `/login litellm`, because they require a browser round trip and refresh.
+
+#### Precedence
+
+A provider block may live in `models.json` or `settings.json`. The two merge per field, with `models.json` winning, and `skills`, `mcp`, and `budget` are read from `settings.json` only. `models.json` also accepts the default provider as `litellm.providers.litellm`, the same shape `settings.json` uses.
+
+A stored `/login` credential still takes precedence over a configured `apiKey`. Pi reads the top-level `providers.litellm` block itself, though, which changes three things:
+
+- Pi applies its `baseUrl` to every model, so a stored `/login` credential for a different proxy hides them all. To move a provider that already has a stored credential over to `models.json`, run `/logout litellm` first.
+- Pi hands its `apiKey` to the provider as a credential, so that key also outranks a Google ADC token. A key in `litellm.providers.litellm` or `settings.json` ranks below ADC.
+- Pi validates the fields it knows there, so `headers` must be an object. A `$LITELLM_HEADERS` string belongs in `litellm.providers.litellm`.
+
 ### Multiple LiteLLM provider aliases
 
-Add alias providers in `~/.pi/agent/settings.json` under `litellm.providers`. Each alias is registered as a separate Pi provider name, so models appear as `litellm/model-id` and `litellm-anthropic/model-id`.
+Add alias providers in `~/.pi/agent/settings.json` under `litellm.providers`, or in `~/.pi/agent/models.json` under the same shape. Pi rejects a `models.json` without a top-level `providers` object, so a file that holds only aliases needs `"providers": {}` as well. Each alias is registered as a separate Pi provider name, so models appear as `litellm/model-id` and `litellm-anthropic/model-id`. In `models.json`, the default provider is usually written as the top-level `providers.litellm` shown in Option C instead.
 
 ```json
 {
@@ -157,15 +233,34 @@ Provider fields:
 | `apiKey` | `LITELLM_API_KEY_HELPER`/`LITELLM_API_KEY` for `litellm`; required for aliases | Pi config value for this provider's key. Use `$ENV_VAR`, `${ENV_VAR}`, `!command`, or a literal key. Escape a literal `$` as `$$`. |
 | `headers` | `$LITELLM_HEADERS` for `litellm`; unset for aliases | JSON string env reference or inline object of request headers |
 | `displayName` | `LITELLM_DISPLAY_NAME` / `"LiteLLM"` for `litellm`; alias name for aliases | Label shown in Pi UI |
-| `enabled` | `true` | Set `false` to skip an alias |
-| `oidc` | unset | `litellm` only. Sign in directly with an OpenID Connect identity provider instead of the LiteLLM-hosted flows; see [Direct OIDC login](#direct-oidc-login) |
+| `enabled` | `true` | Set `false` to skip or disable an alias |
+| `oidc` | unset | `litellm` only, in global settings. Sign in directly with an OpenID Connect identity provider instead of the LiteLLM-hosted flows; see [Direct OIDC login](#direct-oidc-login) |
 | `allowInsecureHttp` | `false` | Set `true` to permit plaintext HTTP for this provider, for example `http://host.docker.internal`. Credentials and request data will not be encrypted. Loopback HTTP works without this setting. |
+| `useGcloudTokenAuth` | `false` | Aliases only. Set `true` to use Google Application Default Credentials (ADC) as this alias's bearer token. `litellm` ignores this field and uses ADC only when `LITELLM_GCLOUD_TOKEN_AUTH` is enabled. |
+| `enableOAuth` | `false` | Aliases only. Set `true` to offer LiteLLM SSO browser login for this alias. `litellm` ignores this field and always offers it. |
+
+#### Configuration & credential precedence
 
-`/login litellm` and Google ADC token auth remain scoped to the default `litellm` provider. Aliases use their configured `apiKey` or manually stored auth entries matching the alias name.
+- **Provider definition merging precedence:**
+  When discovering and configuring provider aliases from multiple sources, definitions are merged in the following order:
+  1. Base provider definitions from `LITELLM_PROVIDERS_JSON`
+  2. Discovered environment variables (`LITELLM_PROVIDER_<NAME>_*` and canonical list in `LITELLM_PROVIDERS`)
+  3. Disk settings from `~/.pi/agent/settings.json` under `litellm.providers`, merged per field with the provider blocks in `~/.pi/agent/models.json`, which win (see Option C). Disk settings override environment definitions, and setting `"enabled": false` removes or skips the provider.
+
+  Each later source overrides `displayName`, `allowInsecureHttp`, `useGcloudTokenAuth`, `enableOAuth`, and the MCP toggle. `baseUrl`, `apiKey`, and `headers` work differently: a value configured in `settings.json`, `models.json`, or `LITELLM_PROVIDERS_JSON` always wins, and `LITELLM_PROVIDER_<NAME>_BASE_URL`, `_API_KEY_HELPER`, `_API_KEY`, and `_HEADERS` apply only when neither sets that field.
+
+- **Credential resolution precedence:**
+  When authenticating requests for any provider:
+  1. Saved interactive credentials in `~/.pi/agent/auth.json` (from `/login <provider>` or OAuth SSO), then the `apiKey` of `models.json`'s top-level `providers.litellm`, which Pi hands to the provider as a credential
+  2. Google Application Default Credentials (for an alias with `useGcloudTokenAuth` or `_USE_GCLOUD_AUTH`; for `litellm` when `LITELLM_GCLOUD_TOKEN_AUTH` is enabled)
+  3. Configured `apiKey` from settings (`settings.json`, `litellm.providers` in `models.json`, or `LITELLM_PROVIDERS_JSON`, including `$ENV_VAR` and `!command`)
+  4. Provider-scoped API key helper (`LITELLM_PROVIDER_<NAME>_API_KEY_HELPER`)
+  5. Provider-scoped API key (`LITELLM_PROVIDER_<NAME>_API_KEY`)
+  6. For the default `litellm` provider only: fallback to global `LITELLM_API_KEY_HELPER` then `LITELLM_API_KEY`.
 
 ### Optional LiteLLM features
 
-LiteLLM Skills and MCP integration are enabled by default. Disable either feature, or choose how MCP tools reach the model, globally in `~/.pi/agent/settings.json`:
+LiteLLM Skills, MCP integration, and the budget footer are enabled by default. Disable any of them, choose how MCP tools reach the model, or choose how budgets are shown, globally in `~/.pi/agent/settings.json`:
 
 ```json
 {
@@ -180,12 +275,22 @@ LiteLLM Skills and MCP integration are enabled by default. Disable either featur
         "*delete*": "hidden",
         "github-search_code": "direct"
       }
+    },
+    "budget": {
+      "enabled": true,
+      "display": "all"
     }
   }
 }
 ```
 
-Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` stops the extension from registering the proxy's MCP server. Restart Pi after changing these settings.
+Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` stops the extension from registering the proxy's MCP server. Setting `budget.enabled` to `false` registers no budget footer, polls, or `/litellm-budget` command; `budget.display` is described under [Budget status](#budget-status). Restart Pi after changing these settings.
+
+MCP can also be enabled or disabled per provider, though a provider setting cannot turn it on while `mcp.enabled` is `false`:
+- In `~/.pi/agent/settings.json`: set `providers.<name>.mcp.enabled` (or `providers.<name>.enableMcp`) to `false` (or `true`).
+- In environment variables: set `LITELLM_PROVIDER_<NAME>_ENABLE_MCP` or `LITELLM_PROVIDER_<NAME>_MCP_ENABLED` to `"0"` or `"false"` to disable MCP for a specific provider.
+
+Restart Pi after changing these settings.
 
 `mcp.exposure` and `mcp.toolExposure` are passed to Pi unchanged and mean what they mean in a Pi `mcp.json` server entry: `codemode` (Pi's default when `exposure` is unset; callable from Pi's `codemode` scripts), `deferred` (declared to the model once Pi's `tool_search` loads it), `direct` (declared on every request), or `hidden`. `toolExposure` keys are tool names as LiteLLM's `/mcp` endpoint offers them, which LiteLLM prefixes with the server name, for example `github-search_code`; `*` matches any characters. Pi validates both settings, and an unusable value is reported once.
 
@@ -203,12 +308,73 @@ Every request routed through a configured LiteLLM provider includes Pi's canonic
 `x-litellm-session-id` header. This groups Chat Completions, Responses, and native Messages requests in LiteLLM without
 adding transport-specific fields to request bodies.
 
+## Budget status
+
+Pi's footer shows the LiteLLM budgets set for your credential, its user, its team, and its organization, for the active model's provider:
+
+```
+LiteLLM key $3.10/$10 · user $40/$100 · team $412/$1k · member $20/$50 · org $9.2k/$50k
+```
+
+Each segment is spend and limit for one level: `key`, `user`, `team`, `member` (your own budget inside the team), and `org`, in that order. These are the limits set on each level, and LiteLLM may not enforce every one of them on this key: since LiteLLM 1.95, a user's personal budget applies to a team key only when the operator sets `apply_user_budget_to_team_keys`. A level whose limit is unset, `null`, or `0` is not shown, so with no budgets the footer is empty. A segment is dim, turns to the warning colour from 80 % used, and to the error colour from 100 %. Set `"display": "tightest"` to show only the level with the least money left, with its percentage and reset time; any other `display` value falls back to `"all"` with a warning:
+
+```
+LiteLLM team $412/$1k (41%) · resets 12d
+```
+
+The footer carries only the provider's display name, these labels, and numbers; team names, key aliases, and proxy messages never reach it.
+
+The extension reads `/key/info` first, then `/v2/user/info` (`/user/info` on older proxies) and `/team/info`, then `/organization/list`, with the provider's own credential and custom headers, and only from that provider's proxy. LiteLLM's permissions decide what you can see:
+
+- A team key without a user shows no `user`, `member`, or `org` level.
+- A JWT credential, such as the one a [Direct OIDC login](#direct-oidc-login) presents, has no key row, so `/key/info` cannot be read. It shows the `user` level and, when that user belongs to exactly one team, that team's `team`, `member`, and `org` levels. With several teams, the one LiteLLM charges depends on the proxy's JWT settings, so only `user` shows. A virtual key whose `/key/info` cannot be read shows no team levels, because its user's team need not be the one the key is charged to.
+- Organization budgets need LiteLLM Enterprise.
+- A key whose `allowed_routes` exclude the info routes cannot be polled. It falls back to the `x-litellm-key-spend` and `x-litellm-key-max-budget` headers on every model response, which show only the `key` level, and only when the key has a limit.
+- An endpoint that answers 4xx (other than 429) or 500 is not asked again until the credential or the custom headers change, or you run `/litellm-budget`. LiteLLM answers 500 for errors that repeat on every call, such as a proxy without a database. Other errors keep the last values and are retried at the next refresh. Nothing is reported while polling.
+
+It refreshes when a session starts, when you select a LiteLLM model whose numbers are older than a minute, and after a turn: 15 seconds later, so LiteLLM can record the turn's spend, and never within 60 seconds of the previous poll. Idle sessions do not poll. Between polls the response headers can only raise the key's spend. LiteLLM writes spend to its database every 10 to 60 seconds, and spend resets are applied by a job that runs about every 10 minutes, so spend can lag by up to a minute and a reset by about ten minutes; a reset time already past is not shown.
+
+`/litellm-budget` forgets remembered 4xx and 500 answers, polls every configured LiteLLM provider now (or `/litellm-budget <provider>` for one), and prints a breakdown with two-decimal amounts, percentages, reset times, and the levels this credential cannot read, with their HTTP status:
+
+```
+LiteLLM ("litellm") budget
+  key     $3.10 of $10 (31%), resets in 12h
+  team    $412.30 of $1,000 (41%), resets in 24d
+  Not readable with this credential: org (401)
+```
+
+Automatic polls need Pi's interactive UI and do not run under `LITELLM_OFFLINE=1`, `LITELLM_DISCOVERY_TIMEOUT_MS=0`, or `PI_OFFLINE`; header updates still apply. `/litellm-budget` is stopped by the first two only. The credential and custom headers are kept in memory only as a keyed digest, and proxy responses are reduced to the numbers shown and never logged.
+
 ## Model transport
 
 A `/model/info` route group uses native Anthropic `/v1/messages` only when every deployment explicitly reports Chat mode, identifies a Claude backend through an Anthropic-, Bedrock-, or Vertex-capable adapter, and every declared backend identity resolves the same Anthropic compatibility policy. A `claude-<name>-<major>-<minor>` backend id resolves through its `claude-<name>-<major>` entry when the exact id is absent; an unknown generation stays on Chat Completions. When `supported_endpoints` is a list, it must include `/v1/messages`; an explicit endpoint list without it keeps the group on Chat Completions. A `null` value, which LiteLLM v1.100 and later report for models absent from their model map, counts as absent. An unresolved routing or base-model identity denies native Messages rather than being discarded from the unanimity check. Mixed-generation Claude groups can use Messages when their serializer policies match, with reasoning levels restricted to their common supported set. Groups with incompatible policies, mixed families, or unknown backend evidence remain on Chat Completions; unanimous explicit Responses mode takes precedence. Routes discovered through `/health` never select native Messages: Messages candidates are downgraded to Chat Completions with reasoning controls closed, while correlated Responses detail remains on Responses. Evidence-free `/v1/models` and wholly health-only `/health` groups take transport and presentation metadata from the bounded Pi catalog lookup: `openai-responses` when the catalog says so, otherwise Chat Completions. They expose no speculative reasoning selector and never select native Messages. Catalog identity such as Amazon Bedrock remains separate from the selected wire protocol.
 
 Native Messages authenticates with `x-api-key`; every transport carries the `x-litellm-session-id` header described above. Strict JSON-schema tools are enabled separately from Messages transport and thinking compatibility: every deployment must affirm support for the actual route/API. Generic `supports_response_schema` and `supports_native_structured_output` metadata does not grant strict tools. Only deployments that LiteLLM reports with the `anthropic` adapter carry that evidence today. Claude served through Bedrock or Vertex therefore uses ordinary tool serialization, and a tool that requires strict sampling fails before the request is sent.
 
+## Image generation
+
+Routes whose every deployment reports `mode: "image_generation"` in `/model/info` are published as Pi image models
+(for example `litellm/gemini-nano-banana-2.1`). Like Pi's other image models they do not appear in `/model`; on Pi 1.0
+and later, scripts reach them through Pi's `codemode` tool:
+
+```js
+const painter = await models.getModelOfType("image", "litellm", "gemini-nano-banana-2.1");
+const result = await models.generateImages(painter, {
+  input: [{ type: "text", text: "A red fox in the snow, watercolor" }],
+});
+if (result.stopReason !== "stop") return result.errorMessage;
+for (const block of result.output) if (block.type === "image") image(block);
+```
+
+Requests go to the proxy's OpenAI-compatible `/v1/images/generations` with the provider's own credential and
+headers, so they work for any LiteLLM image deployment (`gpt-image-*`, `dall-e-*`, Gemini image models and others)
+and are budgeted and logged by the proxy like chat requests. Usage cost comes from LiteLLM's `x-litellm-response-cost`
+header when present; otherwise it is estimated from `input_cost_per_token` and `output_cost_per_image_token` (or
+`output_cost_per_token`). Only routes found through `/model/info` qualify: `/v1/models` and `/health` carry no mode,
+and groups that mix image and other modes, or wildcard routes, are withheld. The prompt is text only; reference images
+need `/v1/images/edits`, which this provider does not call, and backends that answer with image URLs instead of
+base64 data are reported as errors rather than fetched.
+
 ## Optional environment variables
 
 | Variable | Default | Effect |
@@ -216,10 +382,13 @@ Native Messages authenticates with `x-api-key`; every transport carries the `x-l
 | `LITELLM_API_KEY_HELPER` | unset | Command that prints a fresh LiteLLM bearer token. Takes precedence over `LITELLM_API_KEY`. The extension runs it while resolving request auth, and Pi's per-request auth path is uncached, so rotating/short-lived tokens stay fresh. |
 | `LITELLM_DISPLAY_NAME` | `LiteLLM` | Custom display name for the default `litellm` provider in the Pi UI. |
 | `LITELLM_HEADERS` | unset | JSON object of extra headers sent to LiteLLM provider, discovery, MCP, and Skills Gateway requests. Provider aliases can use it with `"headers": "$LITELLM_HEADERS"`. |
+| `LITELLM_PROVIDERS` | unset | Canonical comma- or whitespace-separated list of provider IDs to register. |
+| `LITELLM_PROVIDERS_JSON` | unset | JSON string of provider configuration objects matching the `litellm.providers` settings schema. |
+| `LITELLM_PROVIDER_<NAME>_*` | unset | Per-provider environment variables for base URL, API key, headers, display name, OAuth, Google auth, and MCP enable/disable toggle (`_ENABLE_MCP` / `_MCP_ENABLED`). See [Multi-provider environment variables](#multi-provider-environment-variables). |
 | `LITELLM_GCLOUD_TOKEN_AUTH` | unset | If set to a non-empty value other than `0`, use Google Application Default Credentials as the LiteLLM bearer token source. This takes precedence over `LITELLM_API_KEY_HELPER` and `LITELLM_API_KEY` when no stored `/login litellm` credential exists. |
 | `GOOGLE_APPLICATION_CREDENTIALS` | Google default ADC path | Optional path to an ADC JSON file used by `LITELLM_GCLOUD_TOKEN_AUTH`. If unset, the extension checks the default gcloud ADC locations. |
 | `LITELLM_OFFLINE` | unset | If `1`, disable all model and MCP discovery, including post-login discovery; use cached models only when their stored canonical proxy root exactly matches the active credential root, including any path prefix. URL-standard host casing and default ports are canonicalized, but paths remain case-sensitive. |
-| `PI_OFFLINE` | unset | Pi documents it as disabling automatic network activity, including model catalog refreshes. Through Pi 1.0.3, any set value (including `0` or an empty string) disables startup model discovery and `/model` network refreshes. Run `/litellm-refresh`, or unset it, to discover; cached models can still be used offline. |
+| `PI_OFFLINE` | unset | Pi documents it as disabling automatic network activity, including model catalog refreshes. Through Pi 1.2.0, any set value (including `0` or an empty string) disables startup model discovery and `/model` network refreshes. Run `/litellm-refresh`, or unset it, to discover; cached models can still be used offline. |
 | `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Background and explicit discovery fetch timeout in ms; `0` disables automatic discovery |
 | `LITELLM_CLI_JWT_EXPIRATION_HOURS` | `24` | CLI SSO token lifetime fallback for older proxies whose poll response omits `expires_in`; mirror a non-default proxy setting locally |
 | `LITELLM_VERBOSE_DISCOVERY` | unset | If `1`, enable progress messages during model discovery (login, refresh, startup), including startup skip reasons and defaulted `/model/info` context windows. Progress messages are off by default |
@@ -250,7 +419,9 @@ LiteLLM serves the MCP servers it gateways over streamable HTTP at `/mcp`. For e
 
 The registration does not copy the provider's API key or token. It uses Pi's `auth: { provider }` setting, so Pi sends the provider's current credential as `Authorization: Bearer` on every request, and token refreshes, Google ADC, and key helpers apply without registering again. Custom headers (`LITELLM_HEADERS`, or an alias's `headers`) are registered as configured, after `$` and a leading `!` are escaped so Pi does not expand them a second time. Pi sends provider credentials only over https or to a loopback host, so a proxy reached over plain http elsewhere (`allowInsecureHttp`) gets no MCP server, and the refusal is reported once.
 
-The extension registers at startup and checks again before each agent turn. `/login litellm` disconnects the server as soon as it starts, before the new credential is stored, and the next turn connects again with the new credential and proxy root; a logout that leaves no root removes the server. `LITELLM_OFFLINE=1`, `LITELLM_DISCOVERY_TIMEOUT_MS=0`, and `PI_OFFLINE` register nothing. A server with the same name in your own `mcp.json` takes precedence over the extension's.
+The extension registers at startup and updates the registration before each agent turn. `/login litellm` disconnects the server as soon as it starts, before the new credential is stored, and the next turn connects again with the new credential and proxy root; a logout that leaves no root removes the server. `LITELLM_OFFLINE=1`, `LITELLM_DISCOVERY_TIMEOUT_MS=0`, and `PI_OFFLINE` register nothing. A server with the same name in your own `mcp.json` takes precedence over the extension's.
+
+Before registering, the extension sends the proxy one MCP `initialize` with the credential and headers Pi will send, and ends the session it opened. LiteLLM 1.102.0 and later refuse it with 403 when no MCP servers are granted to the key, its team, or its organization, or none of them is allowed from your IP address. The extension then registers the server disabled: Pi does not connect to it, warn about it at startup, or turn on `codemode` for it, and `/mcp` lists it as disabled. Once servers are granted, enable it in `/mcp` for the session, or restart Pi, which checks again, as does a login. When a key helper (a `!command` key) or Google ADC is configured, the check is skipped and the server is registered enabled.
 
 Pi reads the provider's credential when it sends each request, not when the server was registered. If another Pi process stores a login for a different proxy, this session's open connection can send that credential to the previous proxy until the next turn moves the server.
 
@@ -268,6 +439,11 @@ If your LiteLLM proxy exposes `/claude-code/marketplace.json`, enabled skills ar
 
 `litellm_skill_create` needs Skill Hub `sourceJson` metadata and a proxy that exposes `/claude-code/plugins`. It does not create skills from code: LiteLLM's `POST /v1/skills` is Anthropic's multipart Skills API.
 
+## PiG
+
+A Go port for [PiG](https://github.com/MichaelKinsy/PiG) lives in [`pig/`](pig/README.md). It reads the same
+settings, environment variables and `auth.json` entries; see [pig/README.md](pig/README.md) to install it.
+
 ## Mocked LiteLLM smoke workflow
 
 The `LiteLLM Smoke` GitHub Actions workflow starts VidaiMock and a real LiteLLM proxy on the runner. LiteLLM exposes route-distinct Chat, Responses, native Messages, and mixed-deployment models whose upstreams are served by VidaiMock. The smoke runner discovers those models through LiteLLM, asserts each model's expected API, exercises `/v1/chat/completions`, `/v1/responses`, and `/v1/messages`, verifies the expected `x-litellm-response-cost` behavior, and proves endpoint coverage from captured LiteLLM request logs rather than response text.
@@ -309,6 +485,8 @@ LiteLLM's `/model/info` is the metadata authority, so models.dev enrichment is a
 
 Opening `/model` refreshes configured provider catalogs in the background using Pi's native model lifecycle when network discovery is enabled.
 
+Startup reuses that stored catalog instead of discovering again while the provider's entry is under 4 hours old and comes from the active proxy and the current discovery version. Interactive and RPC sessions still refresh it in the background; `pi -p` does not. For up to 4 hours after the last refresh, print mode can therefore miss a model added on the proxy or granted to your key, and after a key change on the same proxy it can list the previous key's models. Run `/litellm-refresh` or start an interactive session to update it sooner.
+
 Pi documents `PI_OFFLINE` and `--offline` as disabling automatic network activity, including model catalog refreshes. Pi 0.87.1 adopted that wording to resolve [earendil-works/pi#8684](https://github.com/earendil-works/pi/issues/8684), without changing the behavior. Pi still checks only whether the variable is set, so `PI_OFFLINE=0` and an empty value also disable model refreshes. With `LITELLM_VERBOSE_DISCOVERY=1`, disabled startup discovery reports its reason, for example `startup discovery skipped (PI_OFFLINE)`. To discover models while keeping `PI_OFFLINE`, run `/litellm-refresh`. To allow all discovery for one invocation, use `env -u PI_OFFLINE pi` and do not pass `--offline`. If you only want to disable Pi's version check, use `PI_SKIP_VERSION_CHECK=1` instead; it does not apply offline mode's other network restrictions. See [#155](https://github.com/balcsida/pi-provider-litellm/issues/155).
 
 ### Model host enforcement
@@ -377,7 +555,7 @@ The development probe runs against minimized snapshots with `npm run probe:proxy
 | No models, or no new routes, while `PI_OFFLINE` is set | Pi skips every model network refresh, including the one `/model` starts. Run `/litellm-refresh`, or unset `PI_OFFLINE` (`PI_OFFLINE=0` is not enough) |
 | `No models available` at startup, gone after a restart | A Pi startup race, not discovery — see [`No models available` at startup](#no-models-available-at-startup) |
 | "discovered no models" | Proxy returned an empty list — check pi's startup log and verify `/model/info`, `/v1/models`, or `/health` responds |
-| `/model/info` returning 401/403/404 | Expected behavior with virtual keys — extension falls back to `/v1/models` |
+| `/model/info` returning 401/403/404 or an empty list | Expected behavior with virtual keys — extension falls back to `/v1/models` |
 | Discovery times out | Increase `LITELLM_DISCOVERY_TIMEOUT_MS` or set `LITELLM_OFFLINE=1` to fall back on cached models. Offline mode does not recover a root mismatch, including a different path prefix — see [Model host enforcement](#model-host-enforcement) |
 | A provider shows no models | The base URL is missing, invalid, still the placeholder, or its full root, including any path prefix, differs from the root in the cached catalog. Check stderr and see [Model host enforcement](#model-host-enforcement) |
 | A configured or restored model fails to complete but never leaks the credential | Its `api` is absent from the current provider catalog, so Pi used global API fallback with the credential root pinned. Open `/model` against the active proxy to repopulate the protocol — see [Model host enforcement](#model-host-enforcement) |
@@ -391,7 +569,7 @@ The development probe runs against minimized snapshots with `npm run probe:proxy
 | Enterprise CLI SSO start/poll fails | Check the proxy logs and verify `/sso/cli/start` and `/sso/cli/poll/{login_id}` are reachable; only 404/405 falls back to legacy login |
 | Enterprise SSO login shows "virtual key generation failed" | The LiteLLM instance may lack a database (`/key/generate` requires one), your user account may lack key-generation permission, or the request timed out; the JWT is used directly as a fallback |
 | Enterprise SSO token prompt fails with "SSO token is required" | The token field was left empty — paste the token copied from the LiteLLM UI |
-| MCP tools not showing | Run `/mcp` and check the provider's server: its state shows a connection or sign-in error. Verify the proxy serves `/mcp` and that the key has MCP access |
+| MCP tools not showing | Run `/mcp` and check the provider's server: its state shows a connection or sign-in error. `disabled` means the proxy refused the key at startup, usually because no MCP servers are granted to it, its team, or its organization; after a grant, enable it in `/mcp` or restart Pi. Verify the proxy serves `/mcp` and that the key has MCP access |
 | MCP tools listed but the model cannot call them | `codemode` and `deferred` tools are reached through Pi's `codemode` and `tool_search` tools. Check that neither is disabled; `--no-extensions` disables them and Pi's MCP client alike. Setting `litellm.mcp.exposure` to `direct` declares the tools on every request instead |
 | Skills not affecting prompts | Verify the proxy exposes `/claude-code/marketplace.json` or `/v1/skills` and returns enabled skills |
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-provider-litellm",
-  "version": "4.1.0",
+  "version": "4.3.0",
   "description": "LiteLLM proxy provider extension for Pi",
   "type": "module",
   "license": "MIT",
@@ -74,7 +74,7 @@
     "@typescript/native-preview": "7.0.0-dev.20260707.2",
     "es-module-lexer": "^3.0.2",
     "tsx": "^4.21.0",
-    "typebox": "1.3.34",
+    "typebox": "1.3.37",
     "vitest": "^5.0.0"
   }
 }