pi-provider-litellm 3.4.0 → 4.0.0

package/README.md

@@ -1,6 +1,6 @@
 # pi-provider-litellm
 
-LiteLLM proxy native Provider extension for [Pi](https://pi.dev). Pi 0.83.0+ is required; the MCP tool exposure settings need Pi 0.99.0+.
+LiteLLM proxy native Provider extension for [Pi](https://pi.dev). Pi 0.99.2+ is required; on older Pi, install `npm:pi-provider-litellm@3.4.0`.
 
 Discovers models from self-hosted LiteLLM proxies and registers them under Pi providers. The default provider is `litellm`; optional aliases can register additional LiteLLM providers with separate credentials. Supports `/login litellm`, LiteLLM MCP tools, LiteLLM Skills Gateway prompt injection, and Google ADC token auth. Tries `/model/info` first (admin endpoint with rich metadata), falls back to `/v1/models` (OpenAI-compatible) on 401/403/404, then tries `/health` plus per-endpoint `/model/info` for older LiteLLM proxies.
 
@@ -177,17 +177,17 @@ LiteLLM Skills and MCP integration are enabled by default. Disable either featur
       "enabled": true,
       "exposure": "deferred",
       "toolExposure": {
-        "delete_*": "hidden",
-        "github/search_code": "direct"
+        "*delete*": "hidden",
+        "github-search_code": "direct"
       }
     }
   }
 }
 ```
 
-Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` disables LiteLLM MCP discovery and tool registration. Restart Pi after changing these settings so previously registered tools are removed.
+Setting `skills.enabled` to `false` disables the Skills Gateway management tools, skill fetching, and system-prompt injection. Setting `mcp.enabled` to `false` stops the extension from registering the proxy's MCP server. Restart Pi after changing these settings.
 
-`mcp.exposure` and `mcp.toolExposure` take the values of a Pi 0.99 `mcp.json` server entry: `direct` (the default; every tool's name, description, and schema is sent with every request), `codemode` or `codemode-deferred` (callable only from Pi's `codemode` scripts; Pi 0.99.2 and later treat `codemode-deferred` as an alias of `codemode` for its own servers, while here the two stay distinct: `codemode` lists the tools in the codemode description and `codemode-deferred` does not), `deferred` (declared to the model only after Pi's `tool_search` loads it), or `hidden` (registered but unreachable). `toolExposure` keys are tool names as the server offers them, or `<server>/<tool>` (a key containing `/` is always read as `<server>/<tool>`, so a tool whose own name contains `/` must be addressed in that qualified form); `*` matches any characters, exact keys win over patterns, and the first matching pattern wins. When a discovery pass registers non-direct tools, the extension activates Pi's `codemode` or `tool_search` tool, as Pi's own MCP support does for its servers, and warns once when neither is available (for example under `--no-extensions`, which since Pi 0.99 also disables the built-in extensions). Pi before 0.99 ignores the setting; the extension says so once and the tools stay direct. Large shared catalogs are the reason to prefer `deferred` or `codemode`; an unusable value is reported and falls back to `direct`.
+`mcp.exposure` and `mcp.toolExposure` are passed to Pi unchanged and mean what they mean in a Pi `mcp.json` server entry: `codemode` (Pi's default when `exposure` is unset; callable from Pi's `codemode` scripts), `deferred` (declared to the model once Pi's `tool_search` loads it), `direct` (declared on every request), or `hidden`. `toolExposure` keys are tool names as LiteLLM's `/mcp` endpoint offers them, which LiteLLM prefixes with the server name, for example `github-search_code`; `*` matches any characters. Pi validates both settings, and an unusable value is reported once.
 
 Treat the configured LiteLLM proxy as trusted: Skills can add instructions to the system prompt, and MCP can expose tools the agent may call. Disable these integrations when the proxy, its administrators, or its configured content are not fully trusted.
 
@@ -197,7 +197,7 @@ Treat the configured LiteLLM proxy as trusted: Skills can add instructions to th
 /model
 ```
 
-To refresh catalogs on demand, run `/litellm-refresh` for every configured LiteLLM provider, or `/litellm-refresh <provider>` for one alias. It works with `PI_OFFLINE` set, which stops Pi's own `/model` refresh from contacting the network. It contacts only the configured proxies: `LITELLM_OFFLINE=1` and `LITELLM_DISCOVERY_TIMEOUT_MS=0` still disable it, and models.dev enrichment stays off under `PI_OFFLINE`. Each provider reports its model count, the failure, or missing credentials. With LiteLLM MCP enabled, the refresh also re-checks that provider's MCP tools.
+To refresh catalogs on demand, run `/litellm-refresh` for every configured LiteLLM provider, or `/litellm-refresh <provider>` for one alias. It works with `PI_OFFLINE` set, which stops Pi's own `/model` refresh from contacting the network. It contacts only the configured proxies: `LITELLM_OFFLINE=1` and `LITELLM_DISCOVERY_TIMEOUT_MS=0` still disable it, and models.dev enrichment stays off under `PI_OFFLINE`. Each provider reports its model count, the failure, or missing credentials. MCP tools are not part of this refresh; run `/mcp reconnect <provider>` to reconnect a provider's MCP server.
 
 Every request routed through a configured LiteLLM provider includes Pi's canonical session ID in the
 `x-litellm-session-id` header. This groups Chat Completions, Responses, and native Messages requests in LiteLLM without
@@ -221,7 +221,7 @@ Native Messages authenticates with `x-api-key`; every transport carries the `x-l
 | `PI_OFFLINE` | unset | Pi documents it as disabling automatic network activity, including model catalog refreshes. Through Pi 1.0.0, any set value (including `0` or an empty string) disables startup model discovery and `/model` network refreshes. Run `/litellm-refresh`, or unset it, to discover; cached models can still be used offline. |
 | `LITELLM_DISCOVERY_TIMEOUT_MS` | `5000` | Background and explicit discovery fetch timeout in ms; `0` disables automatic discovery |
 | `LITELLM_CLI_JWT_EXPIRATION_HOURS` | `24` | CLI SSO token lifetime fallback for older proxies whose poll response omits `expires_in`; mirror a non-default proxy setting locally |
-| `LITELLM_VERBOSE_DISCOVERY` | unset | If `1`, enable progress messages during model and MCP discovery (login, refresh, startup), including startup skip reasons, defaulted `/model/info` context windows, and MCP prepared/registered/dropped counts. Progress messages are off by default; MCP safety diagnostics (see below) are always reported regardless of this setting |
+| `LITELLM_VERBOSE_DISCOVERY` | unset | If `1`, enable progress messages during model discovery (login, refresh, startup), including startup skip reasons and defaulted `/model/info` context windows. Progress messages are off by default |
 | `LITELLM_DEFAULT_CONTEXT_WINDOW` | `128000` | Context window assumed when neither LiteLLM's `/model/info` nor the catalog reports `max_input_tokens`. Set a positive integer for proxies whose custom aliases carry no metadata; an unset or unusable value keeps 128K |
 | `LITELLM_MODELS_DEV` | unset | If `1`, enrich discovered metadata (limits, prices, effort lists) from models.dev, cached for 28 days in `litellm-models-dev.json`. Off by default because LiteLLM's `/model/info` is authoritative; use it when LiteLLM's model map lacks a model's metadata |
 
@@ -245,60 +245,17 @@ Only `authorized_user` ADC files are supported. Service account JSON files are r
 
 ## LiteLLM MCP tools
 
-If your LiteLLM proxy exposes MCP REST endpoints, this extension discovers tools from:
+LiteLLM serves the MCP servers it gateways over streamable HTTP at `/mcp`. For each configured provider, the extension registers that endpoint as a Pi MCP server named after the provider (`litellm`, or the alias name), and Pi's built-in MCP client connects to it, lists its tools, and calls them. Tools are named `mcp__<provider>__<tool>` and appear in `/mcp`, where the server's connection state and errors are shown.
 
-- `GET /mcp-rest/tools/list`
-- `POST /mcp-rest/tools/call`
+The registration does not copy the provider's API key or token. It uses Pi's `auth: { provider }` setting, so Pi sends the provider's current credential as `Authorization: Bearer` on every request, and token refreshes, Google ADC, and key helpers apply without registering again. Custom headers (`LITELLM_HEADERS`, or an alias's `headers`) are registered as configured, after `$` and a leading `!` are escaped so Pi does not expand them a second time. Pi sends provider credentials only over https or to a loopback host, so a proxy reached over plain http elsewhere (`allowInsecureHttp`) gets no MCP server, and the refusal is reported once.
 
-An MCP access denial pauses discovery until a successful `/login litellm` (SSO or API key); an alias provider, which has no login, stays paused until its credentials, base URL, or headers change. The pause survives restarts in `~/.pi/agent/litellm-mcp-pauses/`, scoped to the proxy, login, and headers with salted credential fingerprints. A new API key or header configuration has its own scope. Each explicit login creates a fresh opaque `litellmMcpSession` identifier in Pi's stored credential, even when the key is unchanged; automatic OAuth refresh preserves that identifier and its pause. Older Pi processes cannot pause the new login. Pause files contain no credentials or proxy error text. Model discovery and chat continue normally. Grant MCP access on the proxy before signing in again. An explicit `litellm.mcp.enabled: false` setting remains disabled after login.
+The extension registers at startup and checks again before each agent turn. `/login litellm` disconnects the server as soon as it starts, before the new credential is stored, and the next turn connects again with the new credential and proxy root; a logout that leaves no root removes the server. `LITELLM_OFFLINE=1`, `LITELLM_DISCOVERY_TIMEOUT_MS=0`, and `PI_OFFLINE` register nothing. A server with the same name in your own `mcp.json` takes precedence over the extension's.
 
-Pi 0.99 ships its own MCP client, which reads `mcp.json` and names tools `mcp__<server>__<tool>`. This extension names its tools `mcp_<server>_<tool>_<hash>` and registers none of `/mcp`, `codemode`, or `tool_search`, so both load side by side without replacing each other. Each tool carries a `namespace` (`mcp_<server>_<hash>`, or `mcp_<alias>_<server>_<hash>` for an alias provider) built from the server name plus a 6-character hash of the server identity so distinct servers never share a group, with the server description LiteLLM reports, and the boolean MCP annotation hints the proxy passes through (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`), which permission extensions can read from `pi.getAllTools()`; other annotation fields are dropped. To let Pi's client talk to LiteLLM's streamable HTTP MCP endpoint instead, add its `/mcp` URL to `mcp.json` with the header LiteLLM documents for its MCP gateway and set `litellm.mcp.enabled` to `false`; that path uses a static key and Pi's own client rather than this extension's credentials, access-denied pauses, and schema guard.
+Pi reads the provider's credential when it sends each request, not when the server was registered. If another Pi process stores a login for a different proxy, this session's open connection can send that credential to the previous proxy until the next turn moves the server.
 
-Each discovered tool is registered as a native Pi tool named `mcp_<server>_<tool>_<hash>`, at most 64 characters, using only `[a-z0-9_]`. The trailing 10-character hash is derived from the tool's identity (`server_id`, `server_name`, `name`) and is always present, so a name depends only on that tool and never on which other tools happen to be in the same catalog — adding or removing a sibling never renames a survivor. When the readable prefix would overflow 64 characters it is truncated from the right, so a long server name can leave little or none of the tool name visible; the hash is what distinguishes such tools. Tools from different servers therefore never overwrite each other, and exact duplicate identities are registered once. Each alias provider discovers its own catalog: its tools are named `mcp_<alias>_<server>_<tool>_<hash>`, with the alias name also folded into the hash, and are called with that alias's credentials, so two providers exposing the same server never replace each other's tools. The default `litellm` provider keeps unprefixed names, and diagnostics for an alias read `LiteLLM MCP ("<alias>"):`.
+Earlier versions discovered tools through `/mcp-rest/tools/list` and registered them as `mcp_<server>_<tool>_<hash>`. Those names and the `<server>/<tool>` form of `toolExposure` keys no longer apply, and the `~/.pi/agent/litellm-mcp-pauses/` directory is no longer used.
 
-MCP discovery runs once for every configured provider before the first agent turn (so `-p` and other non-interactive modes get MCP tools), after Pi refreshes LiteLLM models, and after `/login litellm`; extension activation never waits for it. Discovery accepts at most a 5 MiB response and registers at most 512 tools. The response must be a JSON array of tools or an object with a `tools` array; an `unexpected_error` or unknown non-null `error` in the object envelope, an object without `tools`, and every other shape are reported as discovery failures rather than as empty catalogs. Failure reports use extension-owned text and never include LiteLLM's proxy-supplied tag or `message`. A `partial_failure` envelope keeps and registers the returned tools while reporting one bounded generated diagnostic. Each tool is isolated during normalization, so one bad tool never hides valid siblings. A tool is accepted only when its schema has a root of exactly `"type": "object"` (a bare `properties` object with no `type`, or a type union such as `["object","null"]`, is refused), plain-object `properties`, at most 16 levels of JSON nesting (about eight nested schema objects when each level uses `properties`), and a serialized size of at most 64 KiB. Descriptions are limited to 4 KiB, and tool labels and result metadata to 256 bytes, each with a truncation marker.
-
-### Which schema a tool ends up with
-
-A tool registers with one of two parameter shapes.
-
-- **The extension-owned `args` envelope** — a single object-valued `args` property, required at execution. Used when the proxy supplied no schema at all (absent or `{}`), and used as a safe substitute when the supplied schema could not be proven safe (below). Nothing in this shape originates from the proxy.
-- **The supplied schema, passed through unchanged.** Pi validates arguments against it before the call and may first coerce supported primitive values: omitting a property listed in `required` is rejected, while a value of the wrong type is rejected only when it cannot be coerced to the declared type. Unknown extra properties are accepted by default, but an `additionalProperties` schema can validate and coerce them, and constraints such as `additionalProperties: false` or `unevaluatedProperties: false` can reject them. Passing through is not the same as skipping validation. A schema of exactly `{"type": "object"}` with no `properties` therefore accepts any arguments at the top level rather than nesting them under `args`. Real schema properties named `args`, `properties`, or `required` remain intact.
-
-An `inputSchema` that is present but is not a JSON object (a string, number, or array) is a malformed tool and is dropped as `invalid-schema`. It is not treated as schemaless, so a proxy emitting junk cannot obtain the permissive envelope.
-
-### Regexes and references in supplied schemas
-
-`pattern`, and the keys of `patternProperties`, are compiled into a backtracking regular expression and executed with no time limit, so a proxy-supplied expression can block Pi for minutes on a single tool call. A `$ref` resolves an arbitrary JSON pointer, which means an expression can be parked under any key at all, including data-only positions such as `default` or `examples`. Enumerating "positions where a subschema may appear" is therefore not a sound defence.
-
-Instead, the whole supplied document is walked — every key and every value — and the tool keeps its schema only if that walk completes and finds nothing dangerous. A walk stops short, and the tool is degraded, when it finds a `pattern` or `patternProperties` keyword at all — whatever its value type, since refusing to vouch for it does not depend on an upstream type guard staying as it is — a `$dynamicRef` or `$recursiveRef`, a `$ref` pointing outside the document, or a graph too deep or too large to inspect within budget. A local `#/...` pointer is accepted only when the walk covers its target *and* that target resolves to something usable as a subschema, with the reference chain proven acyclic — covering the document proves no expression hides behind a pointer, but not that the pointer names a schema at all. A pointer to a non-schema (`#/description`), a pointer with no target (`#/$defs/missing`), an `$anchor` reference (`#name`), and a mutual `$defs` cycle are each degraded rather than forwarded, because the validator would otherwise throw, exhaust the stack, or produce a tool that can never accept any argument. A key that is an argument *name* rather than a keyword — a property literally called `pattern` or `patternProperties` — is left alone.
-
-**Degrading replaces the schema, not the tool.** Because regexes are common in real MCP schemas, a tool in this position still registers, with the `args` envelope in place of its schema, and is reported under the `schema-envelope` class. The compatibility cost is real and worth knowing: such a tool loses validation against the proxy-supplied schema but retains Pi's validation of the extension-owned envelope, so its arguments must be passed as an object under `args` and the model sees a less specific contract than the server documents. The server still validates authoritatively, so an argument that violates the proxy-specific contract becomes a server-side tool error rather than a local one.
-
-### What you see when a tool is missing or degraded
-
-Every raw entry the proxy returns is accounted for exactly once, and the counts reconcile: `raw = prepared + dropped + beyond-the-cap`, where *prepared* is what will be registered. If a registration pass is refused partway, the shortfall between prepared and registered is reported separately by that pass's own diagnostic. A tool is **dropped** for one of four reasons — `invalid-tool` (no usable name or server identity), `duplicate-identity`, `invalid-schema`, or `name-collision` — or discarded for being beyond the 512-tool limit (`tool-cap`). A tool is **degraded** rather than dropped under `schema-envelope`. Each class is reported with a count and a bounded sample of names through Pi notifications when a UI is active, or stderr in non-interactive mode. MCP messages emitted before the TUI context is ready are buffered until session start. For example:
-
-```text
-LiteLLM MCP: kept 2 MCP tools but replaced their schema with a safe args envelope, because of a schema that could not be proven free of proxy-supplied regexes or references: mcp_srv_lookup_1c97861c1f, mcp_srv_matcher_3ee4c31a5a.
-LiteLLM MCP: dropped 1 MCP tool with a missing name or server identity: entry_3.
-```
-
-A class is re-reported whenever its membership changes, including a change beyond the names shown in the sample, and stays quiet while unchanged — so a stable proxy does not repeat itself across refreshes, while a changing one is always surfaced. A class that stops occurring is cleared, so its recurrence is reported rather than suppressed. These lines never contain the proxy's schema, its response body, its descriptions, or your API key; only generated names, positional placeholders, and counts. They are reported regardless of `LITELLM_VERBOSE_DISCOVERY`; set that variable to also see per-refresh raw, prepared, enveloped, and registered counts. The enveloped count covers every tool carrying the envelope, whether because the proxy supplied no schema or because its schema was degraded, so it is the number of tools with no proxy-specific argument contract.
-
-### Registration lifecycle
-
-Pi registers a tool synchronously, replacing any existing tool of the same name, and exposes no way to unregister. It has no notion of rejecting one tool while accepting another: the only failure is a staleness check that fires once the extension instance has been superseded, and that check is never reset. A refused registration therefore ends the whole pass rather than skipping one tool.
-
-So a pass that is refused partway leaves the tools registered up to that point, reports one bounded diagnostic, and makes no further attempt from that extension instance — retrying would re-run discovery on every refresh and could never succeed. A reload creates a fresh instance, which starts clean on its own. Because names are stable and registration replaces by name, any retry that does happen is idempotent.
-
-Network discovery is separate and remains retryable: a catalog that yields no registrable tool is not recorded as settled, so a later refresh tries again, and the empty result is reported rather than passing in silence. A tool that disappears from the proxy's catalog is, on the next discovery pass, re-registered as a hidden withdrawal stub and removed from the active tools on Pi 0.99. Older Pi ignores `hidden`, so the stub stays visible but reports the withdrawal instead of calling the proxy.
-
-MCP tools run in Pi's parallel tool mode. Each side-effecting `POST /mcp-rest/tools/call` is attempted exactly once: timeouts, connection failures, HTTP errors, and malformed responses are returned to Pi as tool errors rather than retried. Pi cancellation aborts an in-flight call and preserves its original cancellation reason. Tool-call response bodies are limited to 5 MiB before JSON parsing, and returned result or error text to 64 KiB with a truncation marker. Results also carry `structuredContent` (the MCP `CallToolResult`, or a text wrapper when the proxy returns a bare value) limited to 1 MiB (larger results fall back to the text wrapper), under a declared `outputSchema`, so Pi 0.99 codemode scripts receive data rather than the truncated text; server `isError` results are returned as error results that keep their content (Pi before 0.99 raises them as tool errors instead).
-
-A passed-through schema's `format` keyword is evaluated, using the validator's own built-in expressions rather than anything the proxy supplies.
-
-Tools discovered this way are only as trustworthy as the MCP servers behind your proxy: their descriptions and results are text the model reads.
+Tools are only as trustworthy as the MCP servers behind your proxy: their descriptions and results are text the model reads.
 
 ## LiteLLM Skill Hub
 
@@ -433,9 +390,8 @@ The development probe runs against minimized snapshots with `npm run probe:proxy
 | Enterprise CLI SSO start/poll fails | Check the proxy logs and verify `/sso/cli/start` and `/sso/cli/poll/{login_id}` are reachable; only 404/405 falls back to legacy login |
 | Enterprise SSO login shows "virtual key generation failed" | The LiteLLM instance may lack a database (`/key/generate` requires one), your user account may lack key-generation permission, or the request timed out; the JWT is used directly as a fallback |
 | Enterprise SSO token prompt fails with "SSO token is required" | The token field was left empty — paste the token copied from the LiteLLM UI |
-| MCP tools not showing | Verify the proxy exposes `/mcp-rest/tools/list` and open `/model` after fixing the proxy |
-| MCP tools registered but the model cannot call them | Check `litellm.mcp.exposure`: `codemode` and `codemode-deferred` tools need Pi's `codemode` tool and `deferred` tools need `tool_search`, both built into Pi 0.99 and disabled by `--no-extensions`. Use `direct` to declare the tools on every request |
-| Some MCP tools missing or unvalidated | Check Pi notifications (stderr in non-interactive mode) for `LiteLLM MCP:` messages. Dropped tools are reported under `invalid-tool`, `duplicate-identity`, `invalid-schema`, `name-collision`, or `tool-cap`, each with a count and a bounded sample of generated names. A tool reported under `schema-envelope` is still present but uses the `args` envelope instead of its own schema. A refused registration pass is reported once. Set `LITELLM_VERBOSE_DISCOVERY=1` for per-refresh raw/prepared/enveloped/registered counts |
+| MCP tools not showing | Run `/mcp` and check the provider's server: its state shows a connection or sign-in error. Verify the proxy serves `/mcp` and that the key has MCP access |
+| MCP tools listed but the model cannot call them | `codemode` and `deferred` tools are reached through Pi's `codemode` and `tool_search` tools. Check that neither is disabled; `--no-extensions` disables them and Pi's MCP client alike. Setting `litellm.mcp.exposure` to `direct` declares the tools on every request instead |
 | Skills not affecting prompts | Verify the proxy exposes `/claude-code/marketplace.json` or `/v1/skills` and returns enabled skills |
 
 ### `No models available` at startup

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-provider-litellm",
-  "version": "3.4.0",
+  "version": "4.0.0",
   "description": "LiteLLM proxy provider extension for Pi",
   "type": "module",
   "license": "MIT",
@@ -48,8 +48,8 @@
     "node": ">=22.19.0"
   },
   "peerDependencies": {
-    "@earendil-works/pi-ai": ">=0.83.0",
-    "@earendil-works/pi-coding-agent": ">=0.83.0"
+    "@earendil-works/pi-ai": ">=0.99.2",
+    "@earendil-works/pi-coding-agent": ">=0.99.2"
   },
   "peerDependenciesMeta": {
     "@earendil-works/pi-ai": {