pi-provider-litellm 4.7.0 → 4.7.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -357,7 +357,7 @@ LiteLLM 1.104.0 and later answer such a request with 422, and earlier releases w
357
357
 
358
358
  A `/model/info` route group uses native Anthropic `/v1/messages` only when every deployment explicitly reports Chat mode, identifies a Claude backend through an Anthropic-, Bedrock-, or Vertex-capable adapter, and every declared backend identity resolves the same Anthropic compatibility policy. A `claude-<name>-<major>-<minor>` backend id resolves through its `claude-<name>-<major>` entry when the exact id is absent; an unknown generation stays on Chat Completions. When `supported_endpoints` is a list, it must include `/v1/messages`; an explicit endpoint list without it keeps the group on Chat Completions. A `null` value, which LiteLLM v1.100 and later report for models absent from their model map, counts as absent. An unresolved routing or base-model identity denies native Messages rather than being discarded from the unanimity check. Mixed-generation Claude groups can use Messages when their serializer policies match, with reasoning levels restricted to their common supported set. Groups with incompatible policies, mixed families, or unknown backend evidence remain on Chat Completions; unanimous explicit Responses mode takes precedence. Routes discovered through `/health` never select native Messages: Messages candidates are downgraded to Chat Completions with reasoning controls closed, while correlated Responses detail remains on Responses. Evidence-free `/v1/models` and wholly health-only `/health` groups take transport and presentation metadata from the bounded Pi catalog lookup: `openai-responses` when the catalog says so, otherwise Chat Completions. They expose no speculative reasoning selector and never select native Messages. Catalog identity such as Amazon Bedrock remains separate from the selected wire protocol.
359
359
 
360
- Native Messages authenticates with `x-api-key`; every transport carries the `x-litellm-session-id` header described above. Strict JSON-schema tools are enabled separately from Messages transport and thinking compatibility: every deployment must affirm support for the actual route/API. Generic `supports_response_schema` and `supports_native_structured_output` metadata does not grant strict tools. Only deployments that LiteLLM reports with the `anthropic` adapter carry that evidence today. Claude served through Bedrock or Vertex therefore uses ordinary tool serialization, and a tool that requires strict sampling fails before the request is sent.
360
+ Native Messages authenticates with `x-api-key`; every transport carries the `x-litellm-session-id` header described above. Strict JSON-schema tools are enabled separately from Messages transport and thinking compatibility: every deployment must affirm support for the actual route/API. Generic `supports_response_schema` and `supports_native_structured_output` metadata does not grant strict tools. Only deployments that LiteLLM reports with the `anthropic` adapter carry that evidence today. Claude served through Bedrock or Vertex therefore uses ordinary tool serialization, and a tool that requires strict sampling fails before the request is sent. The same routing rule applies to managed effort, which adds `thinking.block_binding` and mid-conversation effort markers under their `anthropic-beta` flags: hosted Claude endpoints such as Azure reject that field, so a Messages route with any deployment not routed only through Anthropic declares `supportsMidConvoEffort: false`. Thinking is otherwise unchanged: an adaptive-thinking Claude model still sends `thinking.type: "adaptive"` with the selected level as `output_config.effort`.
361
361
 
362
362
  ## Image generation
363
363
 
@@ -494,7 +494,7 @@ routes.
494
494
 
495
495
  The `LiteLLM Smoke` GitHub Actions workflow starts VidaiMock and a real LiteLLM proxy on the runner. LiteLLM exposes route-distinct Chat, Responses, native Messages, and mixed-deployment models whose upstreams are served by VidaiMock. The smoke runner discovers those models through LiteLLM, asserts each model's expected API, exercises `/v1/chat/completions`, `/v1/responses`, and `/v1/messages`, verifies the expected `x-litellm-response-cost` behavior, and proves endpoint coverage from captured LiteLLM request logs rather than response text.
496
496
 
497
- This keeps the LiteLLM integration path under test but does not call real LLM APIs. No provider API keys or GitHub Models permission are required. The smoke runner also asserts that discovery came from `/model/info` (`LITELLM_SMOKE_EXPECT_SOURCE`) so a silent fallback to `/v1/models` fails the run. The workflow also runs auth checks plus optional Postgres-backed auth checks when `LITELLM_LICENSE` is configured for virtual-key and admin-route behavior, then runs a non-interactive Pi CLI smoke with `--list-models` and `-p` against both the OpenAI-compatible and Anthropic-backed routes, so extension loading, model discovery, and real completion paths are covered without opening the TUI. It also runs an interactive Pi TUI smoke covering `/login litellm` and Pi's native `/model` refresh. VidaiMock returns fixed responses, so the real-proxy smoke does not exercise a model-originated tool call or thinking block; the native Messages compatibility suite covers thinking, tool use, tool results, and replay across turns.
497
+ This keeps the LiteLLM integration path under test but does not call real LLM APIs. No provider API keys or GitHub Models permission are required. The manual `LiteLLM Live Smoke` workflow (`.github/workflows/litellm-live-smoke.yml`) runs the same proxy and smoke runner with a real Anthropic Console key from the repository secret named in that file, publishing cheap Claude backends as native Messages, Chat Completions, and Responses routes; each run spends a few hundred tokens of real credits. The smoke runner also asserts that discovery came from `/model/info` (`LITELLM_SMOKE_EXPECT_SOURCE`) so a silent fallback to `/v1/models` fails the run. The workflow also runs auth checks plus optional Postgres-backed auth checks when `LITELLM_LICENSE` is configured for virtual-key and admin-route behavior, then runs a non-interactive Pi CLI smoke with `--list-models` and `-p` against both the OpenAI-compatible and Anthropic-backed routes, so extension loading, model discovery, and real completion paths are covered without opening the TUI. It also runs an interactive Pi TUI smoke covering `/login litellm` and Pi's native `/model` refresh. VidaiMock returns fixed responses, so the real-proxy smoke does not exercise a model-originated tool call or thinking block; the native Messages compatibility suite covers thinking, tool use, tool results, and replay across turns.
498
498
 
499
499
  ## Development
500
500
 
@@ -558,7 +558,7 @@ Each distinct availability diagnostic is written once per session on stderr. `LI
558
558
 
559
559
  ### Protocols and prompt caching
560
560
 
561
- Discovery selects native Messages for proven Claude groups under the [Model transport](#model-transport) rules, Responses for OpenAI-family backends, and Chat Completions for other OpenAI-compatible backends. Azure deployments stay on Chat Completions unless an explicit Responses mode or a `supported_endpoints` list containing `/v1/responses` authorizes Responses; `api_version` alone does not prove support. Azure AI (`azure_ai/` prefix or `custom_llm_provider: azure_ai`) deployments additionally depend on the proxy version. On LiteLLM before v1.103.0, or when the version is unknown, they ignore a `supported_endpoints` list that contains `/v1/responses` and need an explicit `mode: "responses"`: those releases have no native Responses config for that provider, so they bridge `/v1/responses` through Chat Completions, and the bridge fails streaming with `list index out of range` on Azure's empty-choices chunks (BerriAI/litellm#34455) while the endpoint list, copied from the model cost map, still advertises `/v1/responses`. From v1.103.0 the list is honored again. Discovery reads the version from the `x-litellm-version` header of one extra request, `GET /v1/responses/resp_version_probe`, sent only when such a deployment is published; LiteLLM answers it with an error and calls no model. A pre-release counts as older than the release it names, and a proxy that omits the header keeps these deployments on Chat Completions. Azure is detected from the adapter field, an `azure/` or `azure_ai/` model prefix, or the reported provider; other adapters follow the backend family because LiteLLM bridges `/v1/responses` to Chat Completions when the provider has no native Responses config (`litellm/responses/main.py`, `_bridges_to_chat_completions`). The family comes from the model id, not from LiteLLM's `openai/`, `openai_like/` or `custom_openai/` adapter prefix, so `openai/qwen3` stays on Chat Completions while `openai/gpt-5.5` uses Responses; a deployment opts into Responses with `mode: "responses"` or a `supported_endpoints` list containing `/v1/responses`. A route whose model prefix and `custom_llm_provider` name different providers is treated as unidentified and stays on Chat Completions, so it also does not get the local 128-tool preflight. When `/model/info` supplies `supported_endpoints`, that list takes precedence, and `mode: "responses"` remains an explicit Responses signal. An evidence-free fallback entry—a bare `/v1/models` id or a `/health` entry with no detail row—takes its protocol from the Pi catalog entry for its id; both evidence-free fallback paths also inherit presentation metadata from the bounded catalog lookup. An unknown id stays on Chat Completions. The route name alone authorizes nothing. A concrete id expanded from wildcard `/model/info` routes inherits the deployment evidence of the wildcard route LiteLLM would select for it, with the requested id substituted into the row's `model` the way LiteLLM serves it; the id is omitted when that selected route is not a published chat-style route. Chat Completions and Responses use `<root>/v1`; native Messages uses the proxy root directly. All three protocols pass through the provider's host guard. When an evidence-free cached `(no metadata)` entry can be enriched from the Pi catalog, it is restored on Responses only for a catalog Responses model and otherwise on Chat Completions, never Anthropic Messages. A model supplied only through `models.json` whose `api` this provider does not implement takes Pi's global API fallback rather than this provider's host guard; the credential-root pin (see [Model host enforcement](#model-host-enforcement)) still keeps the credential on the active proxy.
561
+ Discovery selects native Messages for proven Claude groups under the [Model transport](#model-transport) rules, Responses for OpenAI-family backends, and Chat Completions for other OpenAI-compatible backends. Azure deployments stay on Chat Completions unless an explicit Responses mode or a `supported_endpoints` list containing `/v1/responses` authorizes Responses; `api_version` alone does not prove support. Azure AI (`azure_ai/` prefix or `custom_llm_provider: azure_ai`) deployments additionally depend on the proxy version. On LiteLLM before v1.103.0, or when the version is unknown, they ignore a `supported_endpoints` list that contains `/v1/responses` and need an explicit `mode: "responses"`: those releases have no native Responses config for that provider, so they bridge `/v1/responses` through Chat Completions, and the bridge fails streaming with `list index out of range` on Azure's empty-choices chunks (BerriAI/litellm#34455) while the endpoint list, copied from the model cost map, still advertises `/v1/responses`. From v1.103.0 the list is honored again. Discovery reads the version from the `x-litellm-version` header of one extra request, `GET /v1/responses/resp_version_probe`, sent only when such a deployment is published; LiteLLM answers it with an error and calls no model. A pre-release counts as older than the release it names, and a proxy that omits the header keeps these deployments on Chat Completions. Azure is detected from the adapter field, an `azure/` or `azure_ai/` model prefix, or the reported provider; other adapters follow the backend family because LiteLLM bridges `/v1/responses` to Chat Completions when the provider has no native Responses config (`litellm/responses/main.py`, `_bridges_to_chat_completions`). The family comes from the model id, not from LiteLLM's `openai/`, `openai_like/` or `custom_openai/` adapter prefix, so `openai/qwen3` stays on Chat Completions while `openai/gpt-5.5` uses Responses; a deployment opts into Responses with `mode: "responses"` or a `supported_endpoints` list containing `/v1/responses`. A route whose model prefix and `custom_llm_provider` name different providers is treated as unidentified and stays on Chat Completions, so it also does not get the local 128-tool preflight. When `/model/info` supplies `supported_endpoints`, that list takes precedence, and `mode: "responses"` remains an explicit Responses signal. An evidence-free fallback entry—a bare `/v1/models` id or a `/health` entry with no detail row—takes its protocol from the Pi catalog entry for its id; both evidence-free fallback paths also inherit presentation metadata from the bounded catalog lookup. An unknown id stays on Chat Completions. The route name alone authorizes nothing. A concrete id expanded from wildcard `/model/info` routes inherits the deployment evidence of the wildcard route LiteLLM would select for it, with the requested id substituted into the row's `model` the way LiteLLM serves it; the id is omitted when that selected route is not a published chat-style route. Chat Completions and Responses use `<root>/v1`; native Messages uses the proxy root directly. All three protocols pass through the provider's host guard. A model that reaches the provider with Pi's `azure-openai-responses` API, for example a stored entry taken from Pi's Azure catalog, is served as `openai-responses` at `<root>/v1` with the proxy credential, never with Azure URLs or `api-key` auth; any other API the proxy does not serve is still rejected before a request is sent. When an evidence-free cached `(no metadata)` entry can be enriched from the Pi catalog, it is restored on Responses only for a catalog Responses model and otherwise on Chat Completions, never Anthropic Messages. A model supplied only through `models.json` whose `api` this provider does not implement takes Pi's global API fallback rather than this provider's host guard; the credential-root pin (see [Model host enforcement](#model-host-enforcement)) still keeps the credential on the active proxy.
562
562
 
563
563
  `cacheControlFormat: "anthropic"` applies only to Chat Completions, where Pi adds Anthropic `cache_control` markers. The Responses transport has a different compatibility type and uses native `prompt_cache_key` (plus supported retention fields), so Responses models must not receive `cacheControlFormat`. Freshly discovered OpenAI-family models forced onto Chat Completions are rejected locally when a request contains more than 128 tools; route them through Responses or reduce enabled extensions. Moonshot request-side reasoning suppression follows LiteLLM's routing provider, not `model_info.base_model`; response-side `<think>` normalization follows discovered Kimi-family evidence. Opaque aliases retain display normalization without sending Moonshot-only parameters to Azure- or Bedrock-hosted Kimi models. Strict tool-message repair requires every deployment to identify Kimi; Gemini effort normalization likewise requires consistent backend-family evidence. Claude-backed routes on native Messages or Chat Completions declare Pi's `promptCache` lifetimes (`short: 300`, `long: 3600` seconds, the values Pi's built-in Anthropic models use), so Pi's `cacheWarming` can keep their caches warm. Responses routes whose every deployment is GPT-5.6 or later and routes through LiteLLM's `openai`, `azure` or `azure_ai` provider declare `short: 1800`, the 30-minute default minimum lifetime OpenAI and Azure document for those models, and no `long` value. Claude routes on Responses, Chat routes to OpenAI models, earlier GPT models, GPT models served through a relay such as OpenRouter or GitHub Copilot, routes identified only by their name, and other backend families declare none, so Pi does not warm them. A `modelOverrides` entry for the route in the global `models.json` can change the values, and `cacheWarming: "off"` disables warming.
564
564
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-provider-litellm",
3
- "version": "4.7.0",
3
+ "version": "4.7.2",
4
4
  "description": "LiteLLM proxy provider extension for Pi",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -1,4 +1,4 @@
1
- export const LITELLM_DISCOVERY_VERSION = 10 as const;
1
+ export const LITELLM_DISCOVERY_VERSION = 11 as const;
2
2
 
3
3
  export type BackendFamily = "claude" | "deepseek" | "gemini" | "kimi" | "openai";
4
4
 
package/src/discover.ts CHANGED
@@ -650,7 +650,12 @@ export async function fetchJson<T>(
650
650
 
651
651
  const DIAGNOSTIC_ROUTE_SAMPLE = 3;
652
652
 
653
+ type Report = (message: string) => void;
654
+
655
+ const writeStderr: Report = (message) => process.stderr.write(`${message}\n`);
656
+
653
657
  function reportBoundedRoutes(
658
+ report: Report,
654
659
  reported: Set<string>,
655
660
  routes: readonly string[],
656
661
  describe: (count: number) => string,
@@ -660,7 +665,7 @@ function reportBoundedRoutes(
660
665
  for (const route of unreported) reported.add(route);
661
666
  const hidden = unreported.length - DIAGNOSTIC_ROUTE_SAMPLE;
662
667
  const sample = unreported.slice(0, DIAGNOSTIC_ROUTE_SAMPLE).join(", ");
663
- process.stderr.write(`${describe(unreported.length)}: ${sample}${hidden > 0 ? ` (+${hidden} more)` : ""}\n`);
668
+ report(`${describe(unreported.length)}: ${sample}${hidden > 0 ? ` (+${hidden} more)` : ""}`);
664
669
  }
665
670
 
666
671
  // Reported routes, so a persistent misconfiguration is announced once rather than on
@@ -673,8 +678,9 @@ const reportedAmbiguousRoutes = new Set<string>();
673
678
  // supplies complete prices; limits and other catalog-derived metadata may still use
674
679
  // conservative defaults. Report that degradation regardless of
675
680
  // LITELLM_VERBOSE_DISCOVERY, carrying only a count and bounded public route ids.
676
- function reportAmbiguousCatalogAuthority(routes: readonly string[]): void {
681
+ function reportAmbiguousCatalogAuthority(report: Report, routes: readonly string[]): void {
677
682
  reportBoundedRoutes(
683
+ report,
678
684
  reportedAmbiguousRoutes,
679
685
  routes,
680
686
  (count) =>
@@ -685,8 +691,9 @@ function reportAmbiguousCatalogAuthority(routes: readonly string[]): void {
685
691
 
686
692
  const reportedIncompatibleModeRoutes = new Set<string>();
687
693
 
688
- function reportIncompatibleDeploymentModes(routes: readonly string[]): void {
694
+ function reportIncompatibleDeploymentModes(report: Report, routes: readonly string[]): void {
689
695
  reportBoundedRoutes(
696
+ report,
690
697
  reportedIncompatibleModeRoutes,
691
698
  routes,
692
699
  (count) =>
@@ -762,6 +769,11 @@ function mapFromModelInfoGroup(
762
769
  ? {
763
770
  ...reduced.messagesCompat,
764
771
  ...(reduced.messagesStrictTools !== undefined ? { supportsStrictTools: reduced.messagesStrictTools } : {}),
772
+ // Managed effort adds thinking.block_binding, which hosted Claude endpoints (Azure among them) reject.
773
+ // LiteLLM reports no per-route support, so only a group routed solely through Anthropic may keep it.
774
+ ...(entries.every((entry) => routesOnlyThrough(entry, ANTHROPIC_ROUTES))
775
+ ? {}
776
+ : { supportsMidConvoEffort: false }),
765
777
  }
766
778
  : undefined;
767
779
  const policy = closeSerializerPolicy({
@@ -893,9 +905,10 @@ function mapFromModelsList(entry: ModelsListEntry): DiscoveredModel | undefined
893
905
  async function discoverFromHealth(
894
906
  base: string,
895
907
  apiKey: string,
896
- options: DiscoveryOptions & { onProgress?: (message: string) => void; silent?: boolean },
908
+ options: DiscoveryOptions & { onProgress?: (message: string) => void; report?: Report; silent?: boolean },
897
909
  ): Promise<DiscoveredModel[]> {
898
910
  const progress = options.silent ? undefined : options.onProgress;
911
+ const report = options.report ?? writeStderr;
899
912
  progress?.("Querying /health endpoint...");
900
913
  const healthResult = await fetchJson<HealthResponse>(`${base}/health`, apiKey, options);
901
914
  if (!healthResult.ok) return [];
@@ -971,11 +984,11 @@ async function discoverFromHealth(
971
984
  });
972
985
  })
973
986
  .filter((model): model is DiscoveredModel => model !== undefined);
974
- reportIncompatibleDeploymentModes(incompatibleModeRoutes);
975
- reportAmbiguousCatalogAuthority(ambiguousRoutes);
976
- reportConflictingFamilyEvidence(conflictingFamilyRoutes);
977
- reportWithheldToolRepair(withheldRepairRoutes);
978
- reportWithheldToolRepairForModels(discovered);
987
+ reportIncompatibleDeploymentModes(report, incompatibleModeRoutes);
988
+ reportAmbiguousCatalogAuthority(report, ambiguousRoutes);
989
+ reportConflictingFamilyEvidence(report, conflictingFamilyRoutes);
990
+ reportWithheldToolRepair(report, withheldRepairRoutes);
991
+ reportWithheldToolRepairForModels(report, discovered);
979
992
  return discovered;
980
993
  }
981
994
 
@@ -1210,10 +1223,11 @@ function mapFromWildcardExpansion(
1210
1223
  export async function discoverModels(
1211
1224
  baseUrl: string,
1212
1225
  apiKey: string,
1213
- options: DiscoveryOptions & { onProgress?: (message: string) => void; silent?: boolean } = {},
1226
+ options: DiscoveryOptions & { onProgress?: (message: string) => void; report?: Report; silent?: boolean } = {},
1214
1227
  ): Promise<DiscoveryResult> {
1215
1228
  const base = normalizeBaseUrl(baseUrl, options.allowInsecureHttp);
1216
1229
  const progress = options.silent ? undefined : options.onProgress;
1230
+ const report = options.report ?? writeStderr;
1217
1231
  progress?.("Querying /model/info endpoint...");
1218
1232
  const infoResult = await fetchJson<ModelInfoResponse>(`${base}/model/info`, apiKey, options);
1219
1233
  // An empty `/model/info` list is treated like an unavailable endpoint: some proxies
@@ -1265,10 +1279,10 @@ export async function discoverModels(
1265
1279
  let models = reducedGroups
1266
1280
  .map(({ model }) => model)
1267
1281
  .filter((model): model is DiscoveredModel => model !== undefined);
1268
- reportIncompatibleDeploymentModes(incompatibleModeRoutes);
1269
- reportAmbiguousCatalogAuthority(ambiguousRoutes);
1270
- reportConflictingFamilyEvidence(conflictingFamilyRoutes);
1271
- reportWithheldToolRepair(withheldRepairRoutes);
1282
+ reportIncompatibleDeploymentModes(report, incompatibleModeRoutes);
1283
+ reportAmbiguousCatalogAuthority(report, ambiguousRoutes);
1284
+ reportConflictingFamilyEvidence(report, conflictingFamilyRoutes);
1285
+ reportWithheldToolRepair(report, withheldRepairRoutes);
1272
1286
  // LiteLLM's /model/info does NOT expand wildcard model_name entries (e.g.
1273
1287
  // "lemonade/*" backed by model: openai/* + check_provider_endpoint: true)
1274
1288
  // — it returns the literal wildcard only. The discovered ids live in
@@ -1338,7 +1352,7 @@ export async function discoverModels(
1338
1352
  models = [...models, ...expanded];
1339
1353
  }
1340
1354
  }
1341
- reportWithheldToolRepair(withheldRepairRoutes);
1355
+ reportWithheldToolRepair(report, withheldRepairRoutes);
1342
1356
  models = deduplicateModels(models);
1343
1357
  if (process.env.LITELLM_VERBOSE_DISCOVERY === "1") {
1344
1358
  const defaultedWildcards = wildcardRoutes.filter(({ route }) => defaultedContextRoutes.has(route));
@@ -1352,6 +1366,7 @@ export async function discoverModels(
1352
1366
  ),
1353
1367
  );
1354
1368
  reportBoundedRoutes(
1369
+ report,
1355
1370
  reportedDefaultedContextRoutes,
1356
1371
  defaulted.map((model) => JSON.stringify(model.id)),
1357
1372
  (count) =>
@@ -1536,8 +1551,9 @@ function deploymentFamily(entry: ModelInfoEntry): CatalogResolution["semanticFam
1536
1551
  return adapterFamily === identityFamily ? identityFamily : "conflicting";
1537
1552
  }
1538
1553
 
1539
- function reportConflictingFamilyEvidence(routes: readonly string[]): void {
1554
+ function reportConflictingFamilyEvidence(report: Report, routes: readonly string[]): void {
1540
1555
  reportBoundedRoutes(
1556
+ report,
1541
1557
  reportedConflictingFamilyRoutes,
1542
1558
  routes,
1543
1559
  (count) =>
@@ -1546,8 +1562,9 @@ function reportConflictingFamilyEvidence(routes: readonly string[]): void {
1546
1562
  );
1547
1563
  }
1548
1564
 
1549
- function reportWithheldToolRepair(routes: readonly string[]): void {
1565
+ function reportWithheldToolRepair(report: Report, routes: readonly string[]): void {
1550
1566
  reportBoundedRoutes(
1567
+ report,
1551
1568
  reportedWithheldRepairRoutes,
1552
1569
  routes,
1553
1570
  (count) =>
@@ -1558,8 +1575,9 @@ function reportWithheldToolRepair(routes: readonly string[]): void {
1558
1575
  );
1559
1576
  }
1560
1577
 
1561
- function reportWithheldToolRepairForModels(models: readonly DiscoveredModel[]): void {
1578
+ function reportWithheldToolRepairForModels(report: Report, models: readonly DiscoveredModel[]): void {
1562
1579
  reportWithheldToolRepair(
1580
+ report,
1563
1581
  models
1564
1582
  .filter(
1565
1583
  (model) =>
package/src/index.ts CHANGED
@@ -2449,6 +2449,7 @@ export default async function (pi: ExtensionAPI): Promise<void> {
2449
2449
  let sessionStarted = false;
2450
2450
  const pendingMcpMessages = new Map<string, "info" | "warning">();
2451
2451
 
2452
+ // Also carries provider and discovery diagnostics: raw stderr would paint over the running TUI.
2452
2453
  function notifyMcp(message: string, level: "info" | "warning" = "warning"): void {
2453
2454
  const text = message.trimEnd();
2454
2455
  if (mcpUI) mcpUI.notify(text, level);
@@ -2701,6 +2702,7 @@ export default async function (pi: ExtensionAPI): Promise<void> {
2701
2702
  modelOverrides: await readModelLimitOverrides(),
2702
2703
  silent: !isVerboseDiscovery(),
2703
2704
  onProgress: isVerboseDiscovery() ? (message) => process.stderr.write(`LiteLLM: ${message}\n`) : undefined,
2705
+ report: (message) => notifyMcp(message),
2704
2706
  });
2705
2707
  signal?.throwIfAborted();
2706
2708
  return { ...result, baseUrl: normalizeBaseUrl(auth.baseUrl, auth.allowInsecureHttp) };
@@ -2824,6 +2826,7 @@ export default async function (pi: ExtensionAPI): Promise<void> {
2824
2826
  if (disabledReason) throw new Error(`discovery disabled (${disabledReason})`);
2825
2827
  return runDiscovery(await authForCredential(definition, credential), signal);
2826
2828
  },
2829
+ report: (message) => notifyMcp(message),
2827
2830
  });
2828
2831
  Object.assign(provider, { headers: resolveHeaders(definition) });
2829
2832
  const refreshModels = provider.refreshModels!;
@@ -447,7 +447,12 @@ export function closeSerializerPolicy(input: {
447
447
  } = input;
448
448
  if (api === "anthropic-messages") {
449
449
  const messagesCompat = vendorCompat as Model<"anthropic-messages">["compat"];
450
- const fields = ["forceAdaptiveThinking", "supportsTemperature", "supportsStrictTools"] as const;
450
+ const fields = [
451
+ "forceAdaptiveThinking",
452
+ "supportsTemperature",
453
+ "supportsStrictTools",
454
+ "supportsMidConvoEffort",
455
+ ] as const;
451
456
  const entries = fields
452
457
  .filter((field) => messagesCompat?.[field] !== undefined)
453
458
  .map((field) => [field, messagesCompat?.[field]]);
package/src/protocols.ts CHANGED
@@ -29,6 +29,16 @@ export function isLiteLLMApi(api: unknown): api is LiteLLMApi {
29
29
  return typeof api === "string" && Object.hasOwn(LITELLM_PROTOCOLS, api);
30
30
  }
31
31
 
32
+ // Pi's Azure Responses API is the OpenAI Responses wire format behind Azure URLs and `api-key` auth. A model can
33
+ // carry it from Pi's Azure catalog, but the proxy serves Responses itself at `<root>/v1` with its own credential.
34
+ const PROXY_API_ALIASES: Readonly<Record<string, LiteLLMApi>> = { "azure-openai-responses": "openai-responses" };
35
+
36
+ /** The protocol the proxy speaks for a model's API, or undefined when the proxy does not serve it. */
37
+ export function toLiteLLMApi(api: unknown): LiteLLMApi | undefined {
38
+ if (isLiteLLMApi(api)) return api;
39
+ return typeof api === "string" && Object.hasOwn(PROXY_API_ALIASES, api) ? PROXY_API_ALIASES[api] : undefined;
40
+ }
41
+
32
42
  export function resolveModelBaseUrl(baseUrl: string, api: LiteLLMApi, allowInsecureHttp = false): string {
33
43
  return LITELLM_PROTOCOLS[api].modelBaseUrl(normalizeBaseUrl(baseUrl, allowInsecureHttp));
34
44
  }
package/src/provider.ts CHANGED
@@ -34,7 +34,7 @@ import {
34
34
  LITELLM_IMAGES_API,
35
35
  type LiteLLMImageModel,
36
36
  } from "./images.js";
37
- import { createLiteLLMProtocolApis, isLiteLLMApi, LITELLM_API_NAMES, resolveModelBaseUrl } from "./protocols.js";
37
+ import { createLiteLLMProtocolApis, LITELLM_API_NAMES, resolveModelBaseUrl, toLiteLLMApi } from "./protocols.js";
38
38
  import type { DiscoveredModel, DiscoveryResult, LiteLLMApi, LiteLLMModel } from "./types.js";
39
39
 
40
40
  export type LiteLLMProviderOptions = {
@@ -48,6 +48,8 @@ export type LiteLLMProviderOptions = {
48
48
  allowInsecureHttp?: boolean;
49
49
  resolveCredentialRoot: (credential?: Credential, requestBaseUrl?: string, apiKey?: string) => string | undefined;
50
50
  discover(credential: Credential, signal?: AbortSignal): Promise<DiscoveryResult & { baseUrl?: string }>;
51
+ /** Shows a diagnostic line. Defaults to stderr, which paints over Pi's TUI once it is running. */
52
+ report?: (message: string) => void;
51
53
  };
52
54
 
53
55
  // Chat models may omit `type`.
@@ -147,7 +149,7 @@ function modelRootError(
147
149
  active: { root: string; canonical: string; hostname: string },
148
150
  allowInsecureHttp = false,
149
151
  ): Error | undefined {
150
- if (!isLiteLLMApi(model.api)) {
152
+ if (!toLiteLLMApi(model.api)) {
151
153
  return new Error(
152
154
  `LiteLLM model ${model.id} declares unsupported protocol "${String(model.api)}"; ` +
153
155
  `set "api" to one of ${LITELLM_API_NAMES.join(", ")} in models.json`,
@@ -202,8 +204,9 @@ function requestModel(
202
204
  const active = activeCredentialRoot(credentialRoot, allowInsecureHttp);
203
205
  const error = modelRootError(model, active, allowInsecureHttp);
204
206
  if (error) throw error;
207
+ const api = toLiteLLMApi(model.api)!;
205
208
  const toolCount =
206
- model.api === "openai-completions" && (model as LiteLLMModel).litellmBackendFamily === "openai"
209
+ api === "openai-completions" && (model as LiteLLMModel).litellmBackendFamily === "openai"
207
210
  ? getCurrentTools(context.messages).length
208
211
  : 0;
209
212
  if (toolCount > CHAT_TOOL_CAP) {
@@ -212,7 +215,7 @@ function requestModel(
212
215
  `${CHAT_TOOL_CAP}-tool cap; route this model via Responses or reduce enabled extensions`,
213
216
  );
214
217
  }
215
- return { ...model, provider, baseUrl: resolveModelBaseUrl(active.root, model.api, allowInsecureHttp) };
218
+ return { ...model, api, provider, baseUrl: resolveModelBaseUrl(active.root, api, allowInsecureHttp) };
216
219
  }
217
220
 
218
221
  export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider<LiteLLMApi> {
@@ -220,7 +223,9 @@ export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider
220
223
  const reportUnavailable = (message: string): void => {
221
224
  if (reportedAvailabilityDiagnostics.has(message)) return;
222
225
  reportedAvailabilityDiagnostics.add(message);
223
- process.stderr.write(`LiteLLM (${options.id}): ${message}\n`);
226
+ const line = `LiteLLM (${options.id}): ${message}`;
227
+ if (options.report) options.report(line);
228
+ else process.stderr.write(`${line}\n`);
224
229
  };
225
230
  type ActiveRoot = { root: string; canonical: string; hostname: string };
226
231
  const resolveActiveRoot = (credential: Credential | undefined, hiddenCount: number): ActiveRoot | undefined => {
@@ -249,10 +254,8 @@ export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider
249
254
  reportUnavailable(error.message);
250
255
  continue;
251
256
  }
252
- available.push({
253
- ...model,
254
- baseUrl: resolveModelBaseUrl(active.root, model.api, options.allowInsecureHttp),
255
- });
257
+ const api = toLiteLLMApi(model.api)!;
258
+ available.push({ ...model, api, baseUrl: resolveModelBaseUrl(active.root, api, options.allowInsecureHttp) });
256
259
  }
257
260
  return available;
258
261
  };
@@ -409,7 +412,11 @@ export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider
409
412
  refreshModels: async (context) => {
410
413
  // Pi 0.99 persists models of every type; the policy helpers below read chat-only fields, so
411
414
  // chat, image and classifier entries are restored separately. Other types are not this provider's.
412
- const storedModels = (context.stored?.models ?? []).filter(isChatModel);
415
+ // A stored entry can carry Pi's Azure Responses API; its policy is restored for the protocol the proxy serves.
416
+ const storedModels = (context.stored?.models ?? []).filter(isChatModel).map((model) => {
417
+ const api = toLiteLLMApi(model.api);
418
+ return api && api !== model.api ? { ...model, api } : model;
419
+ });
413
420
  // Image and classifier entries carry no request policy to restore; a stale one is only kept
414
421
  // when startup discovery proved the same route, otherwise the network refresh replaces it.
415
422
  const restoreOperationModels = <T extends AnyModel>(is: (model: AnyModel) => model is T): T[] =>