pi-provider-litellm 4.7.1 → 4.7.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -5
- package/package.json +1 -1
- package/src/backend-identity.ts +15 -12
- package/src/discover.ts +5 -0
- package/src/model-groups.ts +21 -2
- package/src/protocols.ts +10 -0
- package/src/provider.ts +12 -9
- package/src/skills.ts +1 -2
package/README.md
CHANGED
|
@@ -357,7 +357,7 @@ LiteLLM 1.104.0 and later answer such a request with 422, and earlier releases w
|
|
|
357
357
|
|
|
358
358
|
A `/model/info` route group uses native Anthropic `/v1/messages` only when every deployment explicitly reports Chat mode, identifies a Claude backend through an Anthropic-, Bedrock-, or Vertex-capable adapter, and every declared backend identity resolves the same Anthropic compatibility policy. A `claude-<name>-<major>-<minor>` backend id resolves through its `claude-<name>-<major>` entry when the exact id is absent; an unknown generation stays on Chat Completions. When `supported_endpoints` is a list, it must include `/v1/messages`; an explicit endpoint list without it keeps the group on Chat Completions. A `null` value, which LiteLLM v1.100 and later report for models absent from their model map, counts as absent. An unresolved routing or base-model identity denies native Messages rather than being discarded from the unanimity check. Mixed-generation Claude groups can use Messages when their serializer policies match, with reasoning levels restricted to their common supported set. Groups with incompatible policies, mixed families, or unknown backend evidence remain on Chat Completions; unanimous explicit Responses mode takes precedence. Routes discovered through `/health` never select native Messages: Messages candidates are downgraded to Chat Completions with reasoning controls closed, while correlated Responses detail remains on Responses. Evidence-free `/v1/models` and wholly health-only `/health` groups take transport and presentation metadata from the bounded Pi catalog lookup: `openai-responses` when the catalog says so, otherwise Chat Completions. They expose no speculative reasoning selector and never select native Messages. Catalog identity such as Amazon Bedrock remains separate from the selected wire protocol.
|
|
359
359
|
|
|
360
|
-
Native Messages authenticates with `x-api-key`; every transport carries the `x-litellm-session-id` header described above. Strict JSON-schema tools are enabled separately from Messages transport and thinking compatibility: every deployment must affirm support for the actual route/API. Generic `supports_response_schema` and `supports_native_structured_output` metadata does not grant strict tools. Only deployments that LiteLLM reports with the `anthropic` adapter carry that evidence today. Claude served through Bedrock or Vertex therefore uses ordinary tool serialization, and a tool that requires strict sampling fails before the request is sent.
|
|
360
|
+
Native Messages authenticates with `x-api-key`; every transport carries the `x-litellm-session-id` header described above. Strict JSON-schema tools are enabled separately from Messages transport and thinking compatibility: every deployment must affirm support for the actual route/API. Generic `supports_response_schema` and `supports_native_structured_output` metadata does not grant strict tools. Only deployments that LiteLLM reports with the `anthropic` adapter carry that evidence today. Claude served through Bedrock or Vertex therefore uses ordinary tool serialization, and a tool that requires strict sampling fails before the request is sent. The same routing rule applies to managed effort, which adds `thinking.block_binding` and mid-conversation effort markers under their `anthropic-beta` flags: hosted Claude endpoints such as Azure reject that field, so a Messages route with any deployment not routed only through Anthropic declares `supportsMidConvoEffort: false`. Thinking is otherwise unchanged: an adaptive-thinking Claude model still sends `thinking.type: "adaptive"` with the selected level as `output_config.effort`.
|
|
361
361
|
|
|
362
362
|
## Image generation
|
|
363
363
|
|
|
@@ -558,7 +558,7 @@ Each distinct availability diagnostic is written once per session on stderr. `LI
|
|
|
558
558
|
|
|
559
559
|
### Protocols and prompt caching
|
|
560
560
|
|
|
561
|
-
Discovery selects native Messages for proven Claude groups under the [Model transport](#model-transport) rules, Responses for OpenAI-family backends, and Chat Completions for other OpenAI-compatible backends. Azure deployments stay on Chat Completions unless an explicit Responses mode or a `supported_endpoints` list containing `/v1/responses` authorizes Responses; `api_version` alone does not prove support. Azure AI (`azure_ai/` prefix or `custom_llm_provider: azure_ai`) deployments additionally depend on the proxy version. On LiteLLM before v1.103.0, or when the version is unknown, they ignore a `supported_endpoints` list that contains `/v1/responses` and need an explicit `mode: "responses"`: those releases have no native Responses config for that provider, so they bridge `/v1/responses` through Chat Completions, and the bridge fails streaming with `list index out of range` on Azure's empty-choices chunks (BerriAI/litellm#34455) while the endpoint list, copied from the model cost map, still advertises `/v1/responses`. From v1.103.0 the list is honored again. Discovery reads the version from the `x-litellm-version` header of one extra request, `GET /v1/responses/resp_version_probe`, sent only when such a deployment is published; LiteLLM answers it with an error and calls no model. A pre-release counts as older than the release it names, and a proxy that omits the header keeps these deployments on Chat Completions. Azure is detected from the adapter field, an `azure/` or `azure_ai/` model prefix, or the reported provider; other adapters follow the backend family because LiteLLM bridges `/v1/responses` to Chat Completions when the provider has no native Responses config (`litellm/responses/main.py`, `_bridges_to_chat_completions`). The family comes from the model id, not from LiteLLM's `openai/`, `openai_like/` or `custom_openai/` adapter prefix, so `openai/qwen3` stays on Chat Completions while `openai/gpt-5.5` uses Responses; a deployment opts into Responses with `mode: "responses"` or a `supported_endpoints` list containing `/v1/responses`. A route whose
|
|
561
|
+
Discovery selects native Messages for proven Claude groups under the [Model transport](#model-transport) rules, Responses for OpenAI-family backends, and Chat Completions for other OpenAI-compatible backends. Azure deployments stay on Chat Completions unless an explicit Responses mode or a `supported_endpoints` list containing `/v1/responses` authorizes Responses; `api_version` alone does not prove support. Azure AI (`azure_ai/` prefix or `custom_llm_provider: azure_ai`) deployments additionally depend on the proxy version. On LiteLLM before v1.103.0, or when the version is unknown, they ignore a `supported_endpoints` list that contains `/v1/responses` and need an explicit `mode: "responses"`: those releases have no native Responses config for that provider, so they bridge `/v1/responses` through Chat Completions, and the bridge fails streaming with `list index out of range` on Azure's empty-choices chunks (BerriAI/litellm#34455) while the endpoint list, copied from the model cost map, still advertises `/v1/responses`. From v1.103.0 the list is honored again. Discovery reads the version from the `x-litellm-version` header of one extra request, `GET /v1/responses/resp_version_probe`, sent only when such a deployment is published; LiteLLM answers it with an error and calls no model. A pre-release counts as older than the release it names, and a proxy that omits the header keeps these deployments on Chat Completions. Azure is detected from the adapter field, an `azure/` or `azure_ai/` model prefix, or the reported provider; other adapters follow the backend family because LiteLLM bridges `/v1/responses` to Chat Completions when the provider has no native Responses config (`litellm/responses/main.py`, `_bridges_to_chat_completions`). The family comes from the model id, not from LiteLLM's `openai/`, `openai_like/` or `custom_openai/` adapter prefix, so `openai/qwen3` stays on Chat Completions while `openai/gpt-5.5` uses Responses; a deployment opts into Responses with `mode: "responses"` or a `supported_endpoints` list containing `/v1/responses`. A route whose `model_info.base_model` prefix and `custom_llm_provider` name different providers is treated as unidentified and stays on Chat Completions, so it also does not get the local 128-tool preflight. When `/model/info` supplies `supported_endpoints`, that list takes precedence, and `mode: "responses"` remains an explicit Responses signal. An evidence-free fallback entry—a bare `/v1/models` id or a `/health` entry with no detail row—takes its protocol from the Pi catalog entry for its id; both evidence-free fallback paths also inherit presentation metadata from the bounded catalog lookup. An unknown id stays on Chat Completions. The route name alone authorizes nothing. A concrete id expanded from wildcard `/model/info` routes inherits the deployment evidence of the wildcard route LiteLLM would select for it, with the requested id substituted into the row's `model` the way LiteLLM serves it; the id is omitted when that selected route is not a published chat-style route. Chat Completions and Responses use `<root>/v1`; native Messages uses the proxy root directly. All three protocols pass through the provider's host guard. A model that reaches the provider with Pi's `azure-openai-responses` API, for example a stored entry taken from Pi's Azure catalog, is served as `openai-responses` at `<root>/v1` with the proxy credential, never with Azure URLs or `api-key` auth; any other API the proxy does not serve is still rejected before a request is sent. When an evidence-free cached `(no metadata)` entry can be enriched from the Pi catalog, it is restored on Responses only for a catalog Responses model and otherwise on Chat Completions, never Anthropic Messages. A model supplied only through `models.json` whose `api` this provider does not implement takes Pi's global API fallback rather than this provider's host guard; the credential-root pin (see [Model host enforcement](#model-host-enforcement)) still keeps the credential on the active proxy.
|
|
562
562
|
|
|
563
563
|
`cacheControlFormat: "anthropic"` applies only to Chat Completions, where Pi adds Anthropic `cache_control` markers. The Responses transport has a different compatibility type and uses native `prompt_cache_key` (plus supported retention fields), so Responses models must not receive `cacheControlFormat`. Freshly discovered OpenAI-family models forced onto Chat Completions are rejected locally when a request contains more than 128 tools; route them through Responses or reduce enabled extensions. Moonshot request-side reasoning suppression follows LiteLLM's routing provider, not `model_info.base_model`; response-side `<think>` normalization follows discovered Kimi-family evidence. Opaque aliases retain display normalization without sending Moonshot-only parameters to Azure- or Bedrock-hosted Kimi models. Strict tool-message repair requires every deployment to identify Kimi; Gemini effort normalization likewise requires consistent backend-family evidence. Claude-backed routes on native Messages or Chat Completions declare Pi's `promptCache` lifetimes (`short: 300`, `long: 3600` seconds, the values Pi's built-in Anthropic models use), so Pi's `cacheWarming` can keep their caches warm. Responses routes whose every deployment is GPT-5.6 or later and routes through LiteLLM's `openai`, `azure` or `azure_ai` provider declare `short: 1800`, the 30-minute default minimum lifetime OpenAI and Azure document for those models, and no `long` value. Claude routes on Responses, Chat routes to OpenAI models, earlier GPT models, GPT models served through a relay such as OpenRouter or GitHub Copilot, routes identified only by their name, and other backend families declare none, so Pi does not warm them. A `modelOverrides` entry for the route in the global `models.json` can change the values, and `cacheWarming: "off"` disables warming.
|
|
564
564
|
|
|
@@ -583,17 +583,17 @@ LiteLLM may load-balance one public `model_name` across deployments with differe
|
|
|
583
583
|
```
|
|
584
584
|
|
|
585
585
|
Pi catalogue lookup prefers the deployment's provider-specific entry before falling back to the generic vendor entry; Azure and Azure AI therefore retain Azure-specific limits rather than being capped by a different OpenAI catalogue entry. When models.dev supplies a sparse vendor record, missing fields still use the provider-specific Pi catalogue.
|
|
586
|
-
- Each displayed base-price field uses the maximum only when every deployment resolves that field; unresolved fields remain zero and the model name is suffixed with ` (incomplete metadata)`.
|
|
586
|
+
- Each displayed base-price field uses the maximum only when every deployment resolves that field; unresolved fields remain zero and the model name is suffixed with ` (incomplete metadata)`. LiteLLM omits `cache_creation_input_token_cost` for backends that bill no cache write, so a deployment's missing cache-write price resolves to zero when its own catalog entry prices cache writes at zero (even when the group withholds catalog authority), or, when no catalog entry prices them, when its backend is DeepSeek. Any other missing cache-write price, Claude and GPT-5.6+ included, stays unresolved.
|
|
587
587
|
- When complete base pricing comes from the catalog under unanimous catalog authority, tiered pricing is the conservative worst-case envelope. Its usable finite non-negative thresholds are the sorted union of every deployment's thresholds, and each price field at each interval is the maximum applicable rate across all deployments, floored by each deployment's base rate. Differing tier breakpoints are therefore preserved rather than treated as incompatible. An explicit `/model/info` base-price field replaces catalog pricing for that field at every threshold; catalog tiers remain for unaffected fields and are omitted only when every base-price field is explicit.
|
|
588
588
|
- Reduced route groups omit tier ladders when complete base-pricing or catalog authority is unavailable. Wildcard expansion retains every known tier from matching groups even when another match has incomplete metadata: suppressing a complete sibling's higher tier could understate the known worst-case rate. The ` (incomplete metadata)` marker remains because the resulting envelope is only as complete as the available ladders.
|
|
589
589
|
- Wildcard routes expand through `/v1/models`; dropped exact or wildcard groups suppress matching expanded ids.
|
|
590
590
|
- Expanded ids inherit conservative metadata bounds from every matching wildcard group, including the same union-threshold, maximum-applicable-rate tier envelope. Protocol and backend-family evidence follow the most-specific route LiteLLM selects, with every deployment of that selected route voting together.
|
|
591
|
-
- Catalog metadata is accepted only from unanimous deployment identities resolved from `model_info.base_model` first and `litellm_params.model` second; the public `model_name` route is never backend authority. `litellm_params.custom_llm_provider` is provider evidence for the selected backend model
|
|
591
|
+
- Catalog metadata is accepted only from unanimous deployment identities resolved from `model_info.base_model` first and `litellm_params.model` second; the public `model_name` route is never backend authority. `litellm_params.custom_llm_provider` is provider evidence for the selected backend model, while generic transport adapters are ignored. LiteLLM routes by it and prepends it to a configured model whose first segment differs, so without `model_info.base_model` a configured `org/model`, such as `openai/gpt-4o` with `custom_llm_provider: "openrouter"` or `Qwen/Qwen3-32B` with `custom_llm_provider: "hosted_vllm"`, is equivalent to `<provider>/org/model`: the full ID is preserved, routing and catalog authority stay with that provider, and family evidence comes from the model ID. This does not rewrite the public route or outgoing requests, or grant direct-provider-only features. A non-generic provider that conflicts with the prefix of `model_info.base_model`, or of the route name when nothing else identifies the backend, still withholds identity. Recognized provider prefixes remain identity evidence even when the named model is absent from that provider's catalog. Different concrete catalog models conflict even within one provider; an unresolved Azure deployment name may still use its resolved `base_model`. Conflicts across a group trigger the bounded ambiguous-authority diagnostic; ambiguous groups are not matched across all Pi provider catalogs. Cross-host Claude routes spanning providers such as Vertex and Bedrock may intentionally trigger this withholding: explicit router metadata remains usable, but the route does not borrow either host's catalog metadata.
|
|
592
592
|
- `/v1/models` and `/health` do not provide deployment-level backend identity. For `/v1/models`, an unqualified ID is resolved only within an explicitly recognized `owned_by` provider or through the bounded alias rules documented here (currently Anthropic aliases), and is never searched across every Pi provider catalog. Both `/v1/models` fallback entries and `/health` entries without deployment detail take their protocol from the matching Pi catalog model: `openai-responses` selects Responses, and every other or missing catalog API selects Chat. Wholly health-only groups also inherit presentation metadata from that bounded Pi catalog lookup; a bare row mixed with deployment details grants no catalog authority to the group. A route name substituted for an unreadable deployment `model_name` also denies router-reported thinking levels. Unresolved evidence-free entries retain the ` (no metadata)` marker; groups containing deployment details remain permanently incomplete when their evidence is insufficient.
|
|
593
593
|
|
|
594
594
|
The ` (no metadata)` suffix is reserved for unresolved, evidence-free entries mapped by the full `/v1/models` fallback or wholly health-only `/health` groups. During wildcard expansion after a successful `/model/info` response, IDs that match no surviving wildcard route are discarded. A matched wildcard expansion inherits its matching groups' authority and is either unmarked or marked ` (incomplete metadata)`, never ` (no metadata)`. A recognized `owned_by` provider that conflicts with a provider-qualified model ID instead receives ` (incomplete metadata)`, permanently withholding cache enrichment rather than falling through to the ID prefix. Only an unchanged evidence-free sentinel shape is eligible for bounded catalog enrichment on a later cache read, when it also refreshes provider compatibility metadata from the model ID, adopts Responses only for a catalog `openai-responses` model, and otherwise uses Chat.
|
|
595
595
|
|
|
596
|
-
The ` (incomplete metadata)` suffix marks reduced `/model/info` groups, incomplete matched wildcard expansions, or `/health` groups with deployment details and permanently prevents route-name cache enrichment. It means at least one metadata field is unknown, including cache pricing that the proxy omitted; known input/output prices may still be shown alongside the suffix.
|
|
596
|
+
The ` (incomplete metadata)` suffix marks reduced `/model/info` groups, incomplete matched wildcard expansions, or `/health` groups with deployment details and permanently prevents route-name cache enrichment. It means at least one metadata field is unknown, including cache pricing that the proxy omitted (bar the zero cache-write evidence above); known input/output prices may still be shown alongside the suffix.
|
|
597
597
|
|
|
598
598
|
### Reasoning controls
|
|
599
599
|
|
package/package.json
CHANGED
package/src/backend-identity.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
export const LITELLM_DISCOVERY_VERSION =
|
|
1
|
+
export const LITELLM_DISCOVERY_VERSION = 13 as const;
|
|
2
2
|
|
|
3
3
|
export type BackendFamily = "claude" | "deepseek" | "gemini" | "kimi" | "openai";
|
|
4
4
|
|
|
@@ -31,8 +31,8 @@ const PROVIDER_FAMILIES: Readonly<Record<string, BackendFamily>> = {
|
|
|
31
31
|
// `openai` is absent for the same reason as in PROVIDER_FAMILIES: it is an adapter, not a model.
|
|
32
32
|
const KNOWN_VENDOR_PREFIXES = new Set(Object.keys(PROVIDER_FAMILIES));
|
|
33
33
|
// First segments that are known to be model path, never a provider: Fireworks' account-scoped
|
|
34
|
-
// ids ("accounts/fireworks/models/x").
|
|
35
|
-
// recognized provider or not, stays a conflict.
|
|
34
|
+
// ids ("accounts/fireworks/models/x"). Only base_model and the route name need it, as LiteLLM
|
|
35
|
+
// never routes them; any other differing prefix there, recognized provider or not, stays a conflict.
|
|
36
36
|
const MODEL_PATH_SEGMENTS = new Set(["accounts"]);
|
|
37
37
|
// `o\d` (OpenAI's o1/o3/o4-mini reasoning models) is only trustworthy at the start of the id or
|
|
38
38
|
// right after a provider path segment — an interior "-o1-" is as likely to be an unrelated
|
|
@@ -64,23 +64,26 @@ export function isOpenAIBackend(id: string): boolean {
|
|
|
64
64
|
}
|
|
65
65
|
|
|
66
66
|
export function resolveBackendIdentity(row: BackendIdentityRow): BackendIdentity | undefined {
|
|
67
|
-
const
|
|
68
|
-
|
|
67
|
+
const baseModel = wireString(row.model_info?.base_model);
|
|
68
|
+
const configuredModel = wireString(row.litellm_params?.model);
|
|
69
|
+
let raw = baseModel ?? configuredModel ?? wireString(row.model_name);
|
|
69
70
|
if (!raw) return undefined;
|
|
70
71
|
|
|
71
72
|
let slash = raw.indexOf("/");
|
|
72
73
|
const prefix = slash > 0 ? raw.slice(0, slash).trim().toLowerCase() : undefined;
|
|
73
74
|
let prefixProvider = prefix && !GENERIC_ADAPTERS.has(prefix) ? prefix : undefined;
|
|
74
|
-
// `custom_llm_provider` is how LiteLLM routes
|
|
75
|
-
//
|
|
76
|
-
//
|
|
77
|
-
//
|
|
75
|
+
// `custom_llm_provider` is how LiteLLM routes a model, and it prepends itself to a configured
|
|
76
|
+
// model whose first segment differs (`_get_llm_provider`). That segment is model namespace,
|
|
77
|
+
// so resolve that prefixed form: OpenRouter's "openai/gpt-4o", vLLM's "Qwen/Qwen3-32B".
|
|
78
|
+
// base_model and the route name are not routed, so there a differing prefix is a conflict.
|
|
79
|
+
// A generic adapter prefix ("azure_ai/") is not a namespace and stays stripped.
|
|
78
80
|
const custom = wireString(row.litellm_params?.custom_llm_provider)?.toLowerCase();
|
|
79
81
|
const customProvider = custom && !GENERIC_ADAPTERS.has(custom) ? custom : undefined;
|
|
80
82
|
if (prefixProvider && customProvider && prefixProvider !== customProvider) {
|
|
81
|
-
if (!MODEL_PATH_SEGMENTS.has(prefixProvider)) return undefined;
|
|
82
|
-
|
|
83
|
-
slash =
|
|
83
|
+
if ((baseModel || !configuredModel) && !MODEL_PATH_SEGMENTS.has(prefixProvider)) return undefined;
|
|
84
|
+
raw = `${customProvider}/${raw}`;
|
|
85
|
+
slash = customProvider.length;
|
|
86
|
+
prefixProvider = customProvider;
|
|
84
87
|
}
|
|
85
88
|
const provider = prefixProvider ?? customProvider;
|
|
86
89
|
const modelId = slash > 0 ? raw.slice(slash + 1) : raw;
|
package/src/discover.ts
CHANGED
|
@@ -769,6 +769,11 @@ function mapFromModelInfoGroup(
|
|
|
769
769
|
? {
|
|
770
770
|
...reduced.messagesCompat,
|
|
771
771
|
...(reduced.messagesStrictTools !== undefined ? { supportsStrictTools: reduced.messagesStrictTools } : {}),
|
|
772
|
+
// Managed effort adds thinking.block_binding, which hosted Claude endpoints (Azure among them) reject.
|
|
773
|
+
// LiteLLM reports no per-route support, so only a group routed solely through Anthropic may keep it.
|
|
774
|
+
...(entries.every((entry) => routesOnlyThrough(entry, ANTHROPIC_ROUTES))
|
|
775
|
+
? {}
|
|
776
|
+
: { supportsMidConvoEffort: false }),
|
|
772
777
|
}
|
|
773
778
|
: undefined;
|
|
774
779
|
const policy = closeSerializerPolicy({
|
package/src/model-groups.ts
CHANGED
|
@@ -447,7 +447,12 @@ export function closeSerializerPolicy(input: {
|
|
|
447
447
|
} = input;
|
|
448
448
|
if (api === "anthropic-messages") {
|
|
449
449
|
const messagesCompat = vendorCompat as Model<"anthropic-messages">["compat"];
|
|
450
|
-
const fields = [
|
|
450
|
+
const fields = [
|
|
451
|
+
"forceAdaptiveThinking",
|
|
452
|
+
"supportsTemperature",
|
|
453
|
+
"supportsStrictTools",
|
|
454
|
+
"supportsMidConvoEffort",
|
|
455
|
+
] as const;
|
|
451
456
|
const entries = fields
|
|
452
457
|
.filter((field) => messagesCompat?.[field] !== undefined)
|
|
453
458
|
.map((field) => [field, messagesCompat?.[field]]);
|
|
@@ -656,6 +661,16 @@ function resolvedCost(
|
|
|
656
661
|
return explicitCost(entry, field) ?? catalog?.cost?.[field];
|
|
657
662
|
}
|
|
658
663
|
|
|
664
|
+
// LiteLLM omits `cache_creation_input_token_cost` for backends that bill no cache write, but
|
|
665
|
+
// so does an unpriced deployment, so a missing write price is zero only with evidence: the
|
|
666
|
+
// deployment's own catalog entry prices it at zero (read even when the route withholds
|
|
667
|
+
// catalog authority), or, without a catalog write price, a DeepSeek backend, which no host
|
|
668
|
+
// in Pi's catalog charges for cache writes.
|
|
669
|
+
function unbilledCacheWrite(catalog: CatalogResolution | undefined): boolean {
|
|
670
|
+
const declared = catalog?.cost?.cacheWrite;
|
|
671
|
+
return declared === undefined ? catalog?.semanticFamily === "deepseek" : declared === 0;
|
|
672
|
+
}
|
|
673
|
+
|
|
659
674
|
function min(values: readonly number[]): number {
|
|
660
675
|
return Math.min(...values);
|
|
661
676
|
}
|
|
@@ -845,7 +860,11 @@ export function reduceModelGroup(
|
|
|
845
860
|
const maxTokens = min(maxTokensEvidence.map((value) => value ?? DEFAULT_MAX_TOKENS));
|
|
846
861
|
|
|
847
862
|
const costValues = COST_FIELDS.map((field) =>
|
|
848
|
-
deployments.map(
|
|
863
|
+
deployments.map(
|
|
864
|
+
(entry, index) =>
|
|
865
|
+
resolvedCost(entry, catalogAuthority[index], field) ??
|
|
866
|
+
(field === "cacheWrite" && unbilledCacheWrite(catalogs[index]) ? 0 : undefined),
|
|
867
|
+
),
|
|
849
868
|
);
|
|
850
869
|
const completeCostFields = costValues.map((values) => values.every((value) => value !== undefined));
|
|
851
870
|
const hasCompleteCost = completeCostFields.every(Boolean);
|
package/src/protocols.ts
CHANGED
|
@@ -29,6 +29,16 @@ export function isLiteLLMApi(api: unknown): api is LiteLLMApi {
|
|
|
29
29
|
return typeof api === "string" && Object.hasOwn(LITELLM_PROTOCOLS, api);
|
|
30
30
|
}
|
|
31
31
|
|
|
32
|
+
// Pi's Azure Responses API is the OpenAI Responses wire format behind Azure URLs and `api-key` auth. A model can
|
|
33
|
+
// carry it from Pi's Azure catalog, but the proxy serves Responses itself at `<root>/v1` with its own credential.
|
|
34
|
+
const PROXY_API_ALIASES: Readonly<Record<string, LiteLLMApi>> = { "azure-openai-responses": "openai-responses" };
|
|
35
|
+
|
|
36
|
+
/** The protocol the proxy speaks for a model's API, or undefined when the proxy does not serve it. */
|
|
37
|
+
export function toLiteLLMApi(api: unknown): LiteLLMApi | undefined {
|
|
38
|
+
if (isLiteLLMApi(api)) return api;
|
|
39
|
+
return typeof api === "string" && Object.hasOwn(PROXY_API_ALIASES, api) ? PROXY_API_ALIASES[api] : undefined;
|
|
40
|
+
}
|
|
41
|
+
|
|
32
42
|
export function resolveModelBaseUrl(baseUrl: string, api: LiteLLMApi, allowInsecureHttp = false): string {
|
|
33
43
|
return LITELLM_PROTOCOLS[api].modelBaseUrl(normalizeBaseUrl(baseUrl, allowInsecureHttp));
|
|
34
44
|
}
|
package/src/provider.ts
CHANGED
|
@@ -34,7 +34,7 @@ import {
|
|
|
34
34
|
LITELLM_IMAGES_API,
|
|
35
35
|
type LiteLLMImageModel,
|
|
36
36
|
} from "./images.js";
|
|
37
|
-
import { createLiteLLMProtocolApis,
|
|
37
|
+
import { createLiteLLMProtocolApis, LITELLM_API_NAMES, resolveModelBaseUrl, toLiteLLMApi } from "./protocols.js";
|
|
38
38
|
import type { DiscoveredModel, DiscoveryResult, LiteLLMApi, LiteLLMModel } from "./types.js";
|
|
39
39
|
|
|
40
40
|
export type LiteLLMProviderOptions = {
|
|
@@ -149,7 +149,7 @@ function modelRootError(
|
|
|
149
149
|
active: { root: string; canonical: string; hostname: string },
|
|
150
150
|
allowInsecureHttp = false,
|
|
151
151
|
): Error | undefined {
|
|
152
|
-
if (!
|
|
152
|
+
if (!toLiteLLMApi(model.api)) {
|
|
153
153
|
return new Error(
|
|
154
154
|
`LiteLLM model ${model.id} declares unsupported protocol "${String(model.api)}"; ` +
|
|
155
155
|
`set "api" to one of ${LITELLM_API_NAMES.join(", ")} in models.json`,
|
|
@@ -204,8 +204,9 @@ function requestModel(
|
|
|
204
204
|
const active = activeCredentialRoot(credentialRoot, allowInsecureHttp);
|
|
205
205
|
const error = modelRootError(model, active, allowInsecureHttp);
|
|
206
206
|
if (error) throw error;
|
|
207
|
+
const api = toLiteLLMApi(model.api)!;
|
|
207
208
|
const toolCount =
|
|
208
|
-
|
|
209
|
+
api === "openai-completions" && (model as LiteLLMModel).litellmBackendFamily === "openai"
|
|
209
210
|
? getCurrentTools(context.messages).length
|
|
210
211
|
: 0;
|
|
211
212
|
if (toolCount > CHAT_TOOL_CAP) {
|
|
@@ -214,7 +215,7 @@ function requestModel(
|
|
|
214
215
|
`${CHAT_TOOL_CAP}-tool cap; route this model via Responses or reduce enabled extensions`,
|
|
215
216
|
);
|
|
216
217
|
}
|
|
217
|
-
return { ...model, provider, baseUrl: resolveModelBaseUrl(active.root,
|
|
218
|
+
return { ...model, api, provider, baseUrl: resolveModelBaseUrl(active.root, api, allowInsecureHttp) };
|
|
218
219
|
}
|
|
219
220
|
|
|
220
221
|
export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider<LiteLLMApi> {
|
|
@@ -253,10 +254,8 @@ export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider
|
|
|
253
254
|
reportUnavailable(error.message);
|
|
254
255
|
continue;
|
|
255
256
|
}
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
baseUrl: resolveModelBaseUrl(active.root, model.api, options.allowInsecureHttp),
|
|
259
|
-
});
|
|
257
|
+
const api = toLiteLLMApi(model.api)!;
|
|
258
|
+
available.push({ ...model, api, baseUrl: resolveModelBaseUrl(active.root, api, options.allowInsecureHttp) });
|
|
260
259
|
}
|
|
261
260
|
return available;
|
|
262
261
|
};
|
|
@@ -413,7 +412,11 @@ export function createLiteLLMProvider(options: LiteLLMProviderOptions): Provider
|
|
|
413
412
|
refreshModels: async (context) => {
|
|
414
413
|
// Pi 0.99 persists models of every type; the policy helpers below read chat-only fields, so
|
|
415
414
|
// chat, image and classifier entries are restored separately. Other types are not this provider's.
|
|
416
|
-
|
|
415
|
+
// A stored entry can carry Pi's Azure Responses API; its policy is restored for the protocol the proxy serves.
|
|
416
|
+
const storedModels = (context.stored?.models ?? []).filter(isChatModel).map((model) => {
|
|
417
|
+
const api = toLiteLLMApi(model.api);
|
|
418
|
+
return api && api !== model.api ? { ...model, api } : model;
|
|
419
|
+
});
|
|
417
420
|
// Image and classifier entries carry no request policy to restore; a stale one is only kept
|
|
418
421
|
// when startup discovery proved the same route, otherwise the network refresh replaces it.
|
|
419
422
|
const restoreOperationModels = <T extends AnyModel>(is: (model: AnyModel) => model is T): T[] =>
|
package/src/skills.ts
CHANGED
|
@@ -1,6 +1,5 @@
|
|
|
1
|
-
import type { Static } from "@earendil-works/pi-ai";
|
|
2
|
-
import { Type } from "@earendil-works/pi-ai";
|
|
3
1
|
import { defineTool, type ExtensionContext, type ToolDefinition } from "@earendil-works/pi-coding-agent";
|
|
2
|
+
import { type Static, Type } from "typebox";
|
|
4
3
|
import { normalizeBaseUrl } from "./discover.js";
|
|
5
4
|
import type { LiteLLMRuntimeAuth, LiteLLMSkill } from "./types.js";
|
|
6
5
|
|