@oh-my-pi/pi-ai 16.2.13 → 16.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/README.md +8 -1
- package/dist/types/dialect/demotion.d.ts +8 -7
- package/dist/types/providers/anthropic-wire.d.ts +57 -1
- package/dist/types/providers/anthropic.d.ts +22 -2
- package/dist/types/providers/openai-responses.d.ts +2 -3
- package/dist/types/providers/openai-shared.d.ts +5 -3
- package/dist/types/types.d.ts +26 -2
- package/dist/types/utils/leaked-thinking-stream.d.ts +8 -4
- package/package.json +4 -4
- package/src/auth-broker/remote-store.ts +11 -1
- package/src/dialect/demotion.ts +13 -8
- package/src/providers/anthropic-wire.ts +51 -2
- package/src/providers/anthropic.ts +222 -5
- package/src/providers/azure-openai-responses.ts +2 -1
- package/src/providers/openai-responses.ts +12 -40
- package/src/providers/openai-shared.ts +65 -46
- package/src/providers/transform-messages.ts +26 -0
- package/src/registry/coreweave.ts +4 -3
- package/src/registry/oauth/xiaomi.ts +1 -1
- package/src/stream.ts +69 -7
- package/src/types.ts +23 -2
- package/src/utils/leaked-thinking-stream.ts +8 -4
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,32 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [16.3.1] - 2026-07-02
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- Removed automated injection of reasoning suppression prompts in OpenAI responses
|
|
10
|
+
|
|
11
|
+
## [16.3.0] - 2026-07-02
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- Added opt-in support for Anthropic's server-side fallback beta (server-side-fallback-2026-06-01) on the anthropic-messages provider, including support for AnthropicOptions.fallbacks and automatic filtering of fallback blocks during cross-provider message transformations.
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
|
|
19
|
+
- Improved stream healing for official first-party endpoints (Anthropic, OpenAI, and OpenAI Codex) by skipping leaked-thinking healing, preventing misfires on legitimate code blocks while maintaining healing for third-party gateways and custom base URLs.
|
|
20
|
+
- Updated CoreWeave Serverless Inference login instructions to clarify persisting COREWEAVE_PROJECT in shell startup files.
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- Fixed an issue where same-model Anthropic message replays incorrectly demoted unsigned thinking into textual content during API calls
|
|
25
|
+
- Fixed a performance issue where broker usage fetch failures were not cached, causing redundant network requests when the broker is offline.
|
|
26
|
+
- Fixed Xiaomi MiMo API key validation to use the supported mimo-v2.5 model.
|
|
27
|
+
- Fixed certificate verification errors for custom gateways behind private CA bundles by ensuring NODE_EXTRA_CA_CERTS is respected across all provider fetches.
|
|
28
|
+
- Fixed Claude Fable demoted-thinking replay to use markdown-italic assistant prose instead of <thinking> tags, preventing context issues after model switches.
|
|
29
|
+
- Fixed OpenAI Responses replay errors (400 Bad Request) caused by missing reasoning items during history replay.
|
|
30
|
+
|
|
5
31
|
## [16.2.13] - 2026-07-01
|
|
6
32
|
|
|
7
33
|
### Fixed
|
package/README.md
CHANGED
|
@@ -965,7 +965,14 @@ For Cloudflare AI Gateway models, use provider base URL format
|
|
|
965
965
|
|
|
966
966
|
For Anthropic Foundry routing, set `CLAUDE_CODE_USE_FOUNDRY=true` plus:
|
|
967
967
|
`FOUNDRY_BASE_URL`, `ANTHROPIC_FOUNDRY_API_KEY`, optional `ANTHROPIC_CUSTOM_HEADERS`,
|
|
968
|
-
and optional mTLS material (`CLAUDE_CODE_CLIENT_CERT`, `CLAUDE_CODE_CLIENT_KEY
|
|
968
|
+
and optional mTLS material (`CLAUDE_CODE_CLIENT_CERT`, `CLAUDE_CODE_CLIENT_KEY`).
|
|
969
|
+
|
|
970
|
+
`NODE_EXTRA_CA_CERTS` (PEM file path or inline PEM, mirroring Node's contract)
|
|
971
|
+
is honoured on every provider fetch — OpenAI-compatible, Codex, Ollama, Azure
|
|
972
|
+
Responses, Google, and Anthropic alike — for corporate relays or private CA
|
|
973
|
+
bundles. Bun's `fetch` does not consume the env var natively, so omp injects
|
|
974
|
+
the bundle into `RequestInit.tls.ca` and seeds the system root store
|
|
975
|
+
alongside it.
|
|
969
976
|
|
|
970
977
|
Provider endpoint defaults for the current OpenAI-compatible integrations:
|
|
971
978
|
|
|
@@ -5,13 +5,14 @@
|
|
|
5
5
|
* replayed unsigned `thought` part is schema-accepted but silently discarded —
|
|
6
6
|
* neither recalled nor influencing generation).
|
|
7
7
|
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
8
|
+
* Fable is the exception: replaying prior reasoning inside `<thinking>` /
|
|
9
|
+
* `antml:thinking`-style assistant text is treated as a reasoning-extraction
|
|
10
|
+
* attempt and can train the next turn to leak thoughts, so Fable receives the
|
|
11
|
+
* reasoning as markdown-italic assistant prose instead. Harmony and Gemma are
|
|
12
|
+
* also exceptions: their `renderThinking` emits chat-template control tokens
|
|
13
|
+
* (`<|channel|>analysis`, `<|channel>thought`) that must not appear inside a
|
|
14
|
+
* structured native message, so they fall back to a plain `<think>` block. Every
|
|
15
|
+
* other dialect's thinking form is inline-safe XML tags or a markdown fence.
|
|
15
16
|
*
|
|
16
17
|
* The result ends with a trailing newline so the block stays separated from the
|
|
17
18
|
* turn's reply text when the wire encoder concatenates parts.
|
|
@@ -66,7 +66,22 @@ export type RedactedThinkingBlockParam = {
|
|
|
66
66
|
type: "redacted_thinking";
|
|
67
67
|
data: string;
|
|
68
68
|
};
|
|
69
|
-
|
|
69
|
+
/**
|
|
70
|
+
* Server-side fallback beta boundary marker (server-side-fallback-2026-06-01).
|
|
71
|
+
* Emitted by the API mid-stream when a classifier block on the requested
|
|
72
|
+
* model is retried on a fallback model. Only the official Anthropic
|
|
73
|
+
* endpoint accepts this block on replay — cross-provider hops MUST strip it.
|
|
74
|
+
*/
|
|
75
|
+
export type FallbackBlockParam = {
|
|
76
|
+
type: "fallback";
|
|
77
|
+
from: {
|
|
78
|
+
model: string;
|
|
79
|
+
};
|
|
80
|
+
to: {
|
|
81
|
+
model: string;
|
|
82
|
+
};
|
|
83
|
+
};
|
|
84
|
+
export type ContentBlockParam = TextBlockParam | ImageBlockParam | ToolUseBlockParam | ToolResultBlockParam | ThinkingBlockParam | RedactedThinkingBlockParam | FallbackBlockParam;
|
|
70
85
|
/**
|
|
71
86
|
* A single conversation turn.
|
|
72
87
|
*
|
|
@@ -135,6 +150,18 @@ export type OutputConfig = {
|
|
|
135
150
|
/** Task-budgets beta. */
|
|
136
151
|
task_budget?: TokenTaskBudget | null;
|
|
137
152
|
};
|
|
153
|
+
/**
|
|
154
|
+
* Per-attempt override entry in `MessageCreateParams.fallbacks`
|
|
155
|
+
* (server-side-fallback-2026-06-01 beta). Every field except `model`
|
|
156
|
+
* mirrors a top-level control the beta allows re-specifying per attempt.
|
|
157
|
+
*/
|
|
158
|
+
export type FallbackParam = {
|
|
159
|
+
model: string;
|
|
160
|
+
max_tokens?: number;
|
|
161
|
+
thinking?: ThinkingConfigParam;
|
|
162
|
+
output_config?: OutputConfig;
|
|
163
|
+
speed?: "fast";
|
|
164
|
+
};
|
|
138
165
|
/** Claude Code context-management beta payload. */
|
|
139
166
|
export type ContextManagement = {
|
|
140
167
|
edits: Array<{
|
|
@@ -161,6 +188,12 @@ export type MessageCreateParams = {
|
|
|
161
188
|
speed?: "fast";
|
|
162
189
|
/** Claude Code context-management beta. */
|
|
163
190
|
context_management?: ContextManagement;
|
|
191
|
+
/**
|
|
192
|
+
* Server-side fallback beta chain — up to three fallback models the API
|
|
193
|
+
* retries when a classifier blocks the primary. Required companion beta
|
|
194
|
+
* header: `server-side-fallback-2026-06-01`.
|
|
195
|
+
*/
|
|
196
|
+
fallbacks?: FallbackParam[];
|
|
164
197
|
};
|
|
165
198
|
export type MessageCreateParamsStreaming = MessageCreateParams & {
|
|
166
199
|
stream: true;
|
|
@@ -174,6 +207,20 @@ export type ServerToolUsage = {
|
|
|
174
207
|
web_search_requests?: number | null;
|
|
175
208
|
web_fetch_requests?: number | null;
|
|
176
209
|
};
|
|
210
|
+
/**
|
|
211
|
+
* Per-attempt token accounting inside a multi-run turn
|
|
212
|
+
* (server-side-fallback-2026-06-01). Populated whenever a fallback chain
|
|
213
|
+
* ran, including sticky-served turns with no `fallback` content block.
|
|
214
|
+
* A `fallback_message` entry is the definitive "served by fallback" signal.
|
|
215
|
+
*/
|
|
216
|
+
export type UsageIteration = {
|
|
217
|
+
type?: "message" | "fallback_message" | string;
|
|
218
|
+
model?: string | null;
|
|
219
|
+
input_tokens?: number | null;
|
|
220
|
+
output_tokens?: number | null;
|
|
221
|
+
cache_read_input_tokens?: number | null;
|
|
222
|
+
cache_creation_input_tokens?: number | null;
|
|
223
|
+
};
|
|
177
224
|
export type Usage = {
|
|
178
225
|
input_tokens?: number | null;
|
|
179
226
|
output_tokens?: number | null;
|
|
@@ -181,6 +228,7 @@ export type Usage = {
|
|
|
181
228
|
cache_creation_input_tokens?: number | null;
|
|
182
229
|
cache_creation?: CacheCreation | null;
|
|
183
230
|
server_tool_use?: ServerToolUsage | null;
|
|
231
|
+
iterations?: UsageIteration[] | null;
|
|
184
232
|
};
|
|
185
233
|
/** The `message` envelope carried by `message_start`. */
|
|
186
234
|
export type ResponseMessage = {
|
|
@@ -209,6 +257,14 @@ export type ResponseContentBlock = {
|
|
|
209
257
|
id: string;
|
|
210
258
|
name: string;
|
|
211
259
|
input?: Record<string, unknown> | null;
|
|
260
|
+
} | {
|
|
261
|
+
type: "fallback";
|
|
262
|
+
from: {
|
|
263
|
+
model: string;
|
|
264
|
+
};
|
|
265
|
+
to: {
|
|
266
|
+
model: string;
|
|
267
|
+
};
|
|
212
268
|
};
|
|
213
269
|
export type ContentBlockDelta = {
|
|
214
270
|
type: "text_delta";
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import type { FetchImpl, Message, Model, ProviderSessionState, ServiceTier, SimpleStreamOptions, StreamFunction, StreamOptions, Usage } from "../types";
|
|
2
2
|
import { type AnthropicFetchOptions, type AnthropicMessagesClientLike } from "./anthropic-client";
|
|
3
|
-
import type { MessageParam, TextBlockParam } from "./anthropic-wire";
|
|
3
|
+
import type { FallbackParam, MessageParam, TextBlockParam } from "./anthropic-wire";
|
|
4
4
|
export type AnthropicHeaderOptions = {
|
|
5
5
|
apiKey: string;
|
|
6
6
|
baseUrl?: string;
|
|
@@ -135,6 +135,15 @@ export interface AnthropicOptions extends StreamOptions {
|
|
|
135
135
|
* including SDK clients such as `AnthropicVertex`.
|
|
136
136
|
*/
|
|
137
137
|
client?: AnthropicMessagesClientLike;
|
|
138
|
+
/**
|
|
139
|
+
* Server-side fallback beta chain (`server-side-fallback-2026-06-01`).
|
|
140
|
+
* When set, `fallbacks` is forwarded on the request body and the beta
|
|
141
|
+
* header is auto-attached; the response parser then honors mid-stream
|
|
142
|
+
* `fallback` content blocks and `usage.iterations` for served-model
|
|
143
|
+
* promotion and per-attempt pricing. Opt-in ONLY — leaving this
|
|
144
|
+
* undefined preserves the pre-fallback behavior on every code path.
|
|
145
|
+
*/
|
|
146
|
+
fallbacks?: FallbackParam[];
|
|
138
147
|
}
|
|
139
148
|
export type AnthropicClientOptionsArgs = {
|
|
140
149
|
model: Model<"anthropic-messages">;
|
|
@@ -223,6 +232,17 @@ export declare function buildAnthropicClientOptions(args: AnthropicClientOptions
|
|
|
223
232
|
* `system` role (Opus 4.8+ and Fable/Mythos 5).
|
|
224
233
|
*/
|
|
225
234
|
export type AnthropicMessageParam = MessageParam;
|
|
226
|
-
|
|
235
|
+
/**
|
|
236
|
+
* Serialize omp {@link Message}s to Anthropic wire messages.
|
|
237
|
+
*
|
|
238
|
+
* `opts.serverSideFallbackEnabled` — when the CURRENT request itself
|
|
239
|
+
* opts into the server-side-fallback beta chain. Only then may a persisted
|
|
240
|
+
* `fallback` content block from a prior turn be replayed on the wire;
|
|
241
|
+
* otherwise the block is dropped to avoid a 400 on non-fallback requests
|
|
242
|
+
* that don't send the beta.
|
|
243
|
+
*/
|
|
244
|
+
export declare function convertAnthropicMessages(messages: Message[], model: Model<"anthropic-messages">, isOAuthToken: boolean, opts?: {
|
|
245
|
+
serverSideFallbackEnabled?: boolean;
|
|
246
|
+
}): AnthropicMessageParam[];
|
|
227
247
|
export declare function normalizeAnthropicToolSchema(schema: unknown): unknown;
|
|
228
248
|
export {};
|
|
@@ -64,8 +64,8 @@ interface OpenAIResponsesProviderSessionState extends ProviderSessionState, Open
|
|
|
64
64
|
}
|
|
65
65
|
interface OpenAIResponsesChainState {
|
|
66
66
|
/**
|
|
67
|
-
* Wire params of the last successful turn
|
|
68
|
-
*
|
|
67
|
+
* Wire params of the last successful turn; never carries
|
|
68
|
+
* `previous_response_id`.
|
|
69
69
|
*/
|
|
70
70
|
lastParams?: OpenAIResponsesSamplingParams;
|
|
71
71
|
lastResponseId?: string;
|
|
@@ -108,7 +108,6 @@ type OpenAIResponsesSamplingParams = ResponseCreateParamsStreaming & {
|
|
|
108
108
|
export declare const streamOpenAIResponses: StreamFunction<"openai-responses">;
|
|
109
109
|
export declare function buildParams(model: Model<"openai-responses">, context: Context, options: OpenAIResponsesOptions | undefined, providerSessionState: OpenAIResponsesProviderSessionState | undefined, strictToolsScope?: OpenAIStrictToolsScope, disableStrictToolsOverride?: boolean): {
|
|
110
110
|
params: OpenAIResponsesSamplingParams;
|
|
111
|
-
trailingScaffoldingItems: number;
|
|
112
111
|
strictToolsApplied: boolean;
|
|
113
112
|
};
|
|
114
113
|
/**
|
|
@@ -359,9 +359,11 @@ export interface BuildResponsesInputOptions<TApi extends Api> {
|
|
|
359
359
|
includeThinkingSignatures?: boolean;
|
|
360
360
|
developerStringContent?: boolean;
|
|
361
361
|
repairOrphanOutputs?: boolean;
|
|
362
|
+
/** Preserve assistant message item IDs from text signatures during fallback replay. */
|
|
363
|
+
preserveAssistantMessageIds?: boolean;
|
|
362
364
|
}
|
|
363
365
|
export declare function buildResponsesInput<TApi extends Api>(options: BuildResponsesInputOptions<TApi>): ResponseInput;
|
|
364
|
-
export declare function convertResponsesAssistantMessage<TApi extends Api>(assistantMsg: AssistantMessage, model: Model<TApi>, msgIndex: number, knownCallIds: Set<string>, includeThinkingSignatures?: boolean, customCallIds?: Set<string
|
|
366
|
+
export declare function convertResponsesAssistantMessage<TApi extends Api>(assistantMsg: AssistantMessage, model: Model<TApi>, msgIndex: number, knownCallIds: Set<string>, includeThinkingSignatures?: boolean, customCallIds?: Set<string>, preserveMessageIds?: boolean): ResponseInput;
|
|
365
367
|
export declare function appendResponsesToolResultMessages<TApi extends Api>(messages: ResponseInput, toolResult: ToolResultMessage, model: Model<TApi>, strictResponsesPairing: boolean, supportsImageDetailOriginal: boolean, knownCallIds: ReadonlySet<string>, customCallIds?: ReadonlySet<string>): void;
|
|
366
368
|
/**
|
|
367
369
|
* Per-block accumulation helpers shared by the two Responses decode loops —
|
|
@@ -463,12 +465,12 @@ export interface ApplyResponsesCompatPolicyOptions {
|
|
|
463
465
|
reasoningSummary?: "auto" | "detailed" | "concise" | null;
|
|
464
466
|
mapEffort?: (effort: string) => string;
|
|
465
467
|
}
|
|
466
|
-
export declare function applyResponsesCompatPolicy<P extends ResponseCreateParamsStreaming>(params: P,
|
|
468
|
+
export declare function applyResponsesCompatPolicy<P extends ResponseCreateParamsStreaming>(params: P, policy: OpenAICompatPolicy, options: ApplyResponsesCompatPolicyOptions | undefined): void;
|
|
467
469
|
/**
|
|
468
470
|
* Apply reasoning-related Responses parameters. Default behavior comes from
|
|
469
471
|
* catalog compat; include/omit arguments are explicit adapter-wrapper overrides.
|
|
470
472
|
*/
|
|
471
|
-
export declare function applyResponsesReasoningParams<P extends ResponseCreateParamsStreaming>(params: P, model: Model<"openai-responses" | "azure-openai-responses" | "openai-codex-responses">, options: ReasoningOptions | undefined,
|
|
473
|
+
export declare function applyResponsesReasoningParams<P extends ResponseCreateParamsStreaming>(params: P, model: Model<"openai-responses" | "azure-openai-responses" | "openai-codex-responses">, options: ReasoningOptions | undefined, mapEffort?: (effort: string) => string, includeEncryptedReasoning?: boolean, omitReasoningEffort?: boolean): void;
|
|
472
474
|
/** Populate `output.usage` from a Responses-API `response.usage` payload. Does not invoke `calculateCost`. */
|
|
473
475
|
export declare function populateResponsesUsageFromResponse(output: AssistantMessage, usage: {
|
|
474
476
|
input_tokens?: number | null;
|
package/dist/types/types.d.ts
CHANGED
|
@@ -8,7 +8,7 @@ import type { ZodType, z } from "zod/v4";
|
|
|
8
8
|
import type { ApiKey } from "./auth-retry";
|
|
9
9
|
import type { BedrockOptions } from "./providers/amazon-bedrock";
|
|
10
10
|
import type { AnthropicOptions } from "./providers/anthropic";
|
|
11
|
-
import type { StopDetails } from "./providers/anthropic-wire";
|
|
11
|
+
import type { FallbackParam, StopDetails } from "./providers/anthropic-wire";
|
|
12
12
|
import type { AzureOpenAIResponsesOptions } from "./providers/azure-openai-responses";
|
|
13
13
|
import type { CursorOptions } from "./providers/cursor";
|
|
14
14
|
import type { DevinOptions } from "./providers/devin";
|
|
@@ -384,6 +384,13 @@ export interface SimpleStreamOptions extends Omit<StreamOptions, "apiKey"> {
|
|
|
384
384
|
openrouterVariant?: string;
|
|
385
385
|
/** Antigravity endpoint routing mode: "auto" (default with failover), "production", "sandbox". */
|
|
386
386
|
antigravityEndpointMode?: "auto" | "production" | "sandbox";
|
|
387
|
+
/**
|
|
388
|
+
* Anthropic `server-side-fallback-2026-06-01` fallback chain (top-level
|
|
389
|
+
* `fallbacks` request field). Opt-in ONLY — leaving this undefined is
|
|
390
|
+
* the default and preserves the pre-fallback behavior on every
|
|
391
|
+
* provider. Non-Anthropic providers ignore the field.
|
|
392
|
+
*/
|
|
393
|
+
fallbacks?: FallbackParam[];
|
|
387
394
|
}
|
|
388
395
|
export type StreamFunction<TApi extends Api> = (model: Model<TApi>, context: Context, options: OptionsForApi<TApi>) => AssistantMessageEventStream;
|
|
389
396
|
export interface TextSignatureV1 {
|
|
@@ -406,6 +413,23 @@ export interface RedactedThinkingContent {
|
|
|
406
413
|
type: "redactedThinking";
|
|
407
414
|
data: string;
|
|
408
415
|
}
|
|
416
|
+
/**
|
|
417
|
+
* Anthropic server-side-fallback boundary marker persisted on assistant
|
|
418
|
+
* turns whose provider request opted into
|
|
419
|
+
* `AnthropicOptions.fallbacks`. Consumers other than the Anthropic
|
|
420
|
+
* provider MUST ignore it — `transformMessages` strips the block on any
|
|
421
|
+
* cross-provider hop and on non-official Anthropic replays, so downstream
|
|
422
|
+
* converters never see it.
|
|
423
|
+
*/
|
|
424
|
+
export interface AnthropicFallbackContent {
|
|
425
|
+
type: "fallback";
|
|
426
|
+
from: {
|
|
427
|
+
model: string;
|
|
428
|
+
};
|
|
429
|
+
to: {
|
|
430
|
+
model: string;
|
|
431
|
+
};
|
|
432
|
+
}
|
|
409
433
|
export interface ImageContent {
|
|
410
434
|
type: "image";
|
|
411
435
|
data: string;
|
|
@@ -476,7 +500,7 @@ export interface ContextSnapshot {
|
|
|
476
500
|
}
|
|
477
501
|
export interface AssistantMessage {
|
|
478
502
|
role: "assistant";
|
|
479
|
-
content: (TextContent | ThinkingContent | RedactedThinkingContent | ToolCall)[];
|
|
503
|
+
content: (TextContent | ThinkingContent | RedactedThinkingContent | AnthropicFallbackContent | ToolCall)[];
|
|
480
504
|
api: Api;
|
|
481
505
|
provider: Provider;
|
|
482
506
|
model: string;
|
|
@@ -3,11 +3,15 @@
|
|
|
3
3
|
*
|
|
4
4
|
* Some providers emit their canonical reasoning idioms (` ```thinking `,
|
|
5
5
|
* `<think>`, Gemma/Harmony channels, …) into the *visible* text stream instead
|
|
6
|
-
* of a structured thinking part. {@link wrapLeakedThinkingStream} re-projects
|
|
6
|
+
* of a structured thinking part. {@link wrapLeakedThinkingStream} re-projects a
|
|
7
7
|
* provider stream into a fresh {@link AssistantMessageEventStream}, splitting the
|
|
8
|
-
* leaked fences out into proper `thinking` blocks *live* as deltas arrive
|
|
9
|
-
*
|
|
10
|
-
*
|
|
8
|
+
* leaked fences out into proper `thinking` blocks *live* as deltas arrive.
|
|
9
|
+
*
|
|
10
|
+
* Applied to every provider stream *except* official first-party endpoints
|
|
11
|
+
* (the official Anthropic API and the official OpenAI / OpenAI-Codex endpoints),
|
|
12
|
+
* which return structured thinking and never leak — `healLeakedThinking` in
|
|
13
|
+
* `../stream.ts` gates the wrap so the healer cannot misfire on legitimate
|
|
14
|
+
* fenced content those models emit as visible text.
|
|
11
15
|
*
|
|
12
16
|
* The healing is idempotent: a second pass over already-clean text finds no
|
|
13
17
|
* fences, so wrapping a provider that already heals (or wrapping twice) is a
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"type": "module",
|
|
3
3
|
"name": "@oh-my-pi/pi-ai",
|
|
4
|
-
"version": "16.2
|
|
4
|
+
"version": "16.3.2",
|
|
5
5
|
"description": "Unified LLM API with automatic model discovery and provider configuration",
|
|
6
6
|
"homepage": "https://omp.sh",
|
|
7
7
|
"author": "Can Boluk",
|
|
@@ -38,9 +38,9 @@
|
|
|
38
38
|
},
|
|
39
39
|
"dependencies": {
|
|
40
40
|
"@bufbuild/protobuf": "^2.12.0",
|
|
41
|
-
"@oh-my-pi/pi-catalog": "16.2
|
|
42
|
-
"@oh-my-pi/pi-utils": "16.2
|
|
43
|
-
"@oh-my-pi/pi-wire": "16.2
|
|
41
|
+
"@oh-my-pi/pi-catalog": "16.3.2",
|
|
42
|
+
"@oh-my-pi/pi-utils": "16.3.2",
|
|
43
|
+
"@oh-my-pi/pi-wire": "16.3.2",
|
|
44
44
|
"arktype": "^2.2.0",
|
|
45
45
|
"zod": "^4"
|
|
46
46
|
},
|
|
@@ -58,7 +58,13 @@ interface CacheEntry {
|
|
|
58
58
|
}
|
|
59
59
|
|
|
60
60
|
interface UsageCacheEntry {
|
|
61
|
-
|
|
61
|
+
/**
|
|
62
|
+
* `null` means the last aggregate `/v1/usage` fetch failed. Callers treat
|
|
63
|
+
* this the same as a successful empty-report response ("no usage signal
|
|
64
|
+
* for this cycle"), and the same 15s TTL applies so transient broker
|
|
65
|
+
* outages don't turn every ranking pass into a broker retry storm.
|
|
66
|
+
*/
|
|
67
|
+
reports: UsageReport[] | null;
|
|
62
68
|
fetchedAt: number;
|
|
63
69
|
}
|
|
64
70
|
|
|
@@ -573,6 +579,10 @@ export class RemoteAuthCredentialStore implements AuthCredentialStore {
|
|
|
573
579
|
})
|
|
574
580
|
.catch(error => {
|
|
575
581
|
logger.warn("auth-broker usage fetch failed", { error: String(error) });
|
|
582
|
+
// Documented 15s TTL fallback: cache the null so sequential callers
|
|
583
|
+
// don't re-hit the broker while it's still down. See
|
|
584
|
+
// docs/auth-broker-gateway.md § "Client-side single-flight".
|
|
585
|
+
this.#usageCache = { reports: null, fetchedAt: Date.now() };
|
|
576
586
|
return null;
|
|
577
587
|
})
|
|
578
588
|
.finally(() => {
|
package/src/dialect/demotion.ts
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
|
-
import { preferredDialect } from "@oh-my-pi/pi-catalog/identity";
|
|
1
|
+
import { bareModelId, preferredDialect } from "@oh-my-pi/pi-catalog/identity";
|
|
2
2
|
import { getDialectDefinition } from "./factory";
|
|
3
3
|
|
|
4
|
+
const CLAUDE_FABLE_ID = /(?:^|[./])claude[-.]fable(?:[-.]|$)/i;
|
|
5
|
+
|
|
4
6
|
/**
|
|
5
7
|
* Wrap a prior-turn reasoning string for demotion into native conversation
|
|
6
8
|
* history — the cross-provider / cross-model case where the target cannot replay
|
|
@@ -8,13 +10,14 @@ import { getDialectDefinition } from "./factory";
|
|
|
8
10
|
* replayed unsigned `thought` part is schema-accepted but silently discarded —
|
|
9
11
|
* neither recalled nor influencing generation).
|
|
10
12
|
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
17
|
-
*
|
|
13
|
+
* Fable is the exception: replaying prior reasoning inside `<thinking>` /
|
|
14
|
+
* `antml:thinking`-style assistant text is treated as a reasoning-extraction
|
|
15
|
+
* attempt and can train the next turn to leak thoughts, so Fable receives the
|
|
16
|
+
* reasoning as markdown-italic assistant prose instead. Harmony and Gemma are
|
|
17
|
+
* also exceptions: their `renderThinking` emits chat-template control tokens
|
|
18
|
+
* (`<|channel|>analysis`, `<|channel>thought`) that must not appear inside a
|
|
19
|
+
* structured native message, so they fall back to a plain `<think>` block. Every
|
|
20
|
+
* other dialect's thinking form is inline-safe XML tags or a markdown fence.
|
|
18
21
|
*
|
|
19
22
|
* The result ends with a trailing newline so the block stays separated from the
|
|
20
23
|
* turn's reply text when the wire encoder concatenates parts.
|
|
@@ -25,7 +28,9 @@ import { getDialectDefinition } from "./factory";
|
|
|
25
28
|
export function renderDemotedThinking(modelId: string, text: string): string {
|
|
26
29
|
if (!text) return "";
|
|
27
30
|
text = text.toWellFormed();
|
|
31
|
+
const canonicalId = bareModelId(modelId);
|
|
28
32
|
const dialect = preferredDialect(modelId);
|
|
33
|
+
if (CLAUDE_FABLE_ID.test(canonicalId)) return `_Hmm. ${text}_\n`;
|
|
29
34
|
if (dialect === "harmony" || dialect === "gemma") return `<think>\n${text}\n</think>\n`;
|
|
30
35
|
return `${getDialectDefinition(dialect).renderThinking(text)}\n`;
|
|
31
36
|
}
|
|
@@ -76,13 +76,26 @@ export type RedactedThinkingBlockParam = {
|
|
|
76
76
|
data: string;
|
|
77
77
|
};
|
|
78
78
|
|
|
79
|
+
/**
|
|
80
|
+
* Server-side fallback beta boundary marker (server-side-fallback-2026-06-01).
|
|
81
|
+
* Emitted by the API mid-stream when a classifier block on the requested
|
|
82
|
+
* model is retried on a fallback model. Only the official Anthropic
|
|
83
|
+
* endpoint accepts this block on replay — cross-provider hops MUST strip it.
|
|
84
|
+
*/
|
|
85
|
+
export type FallbackBlockParam = {
|
|
86
|
+
type: "fallback";
|
|
87
|
+
from: { model: string };
|
|
88
|
+
to: { model: string };
|
|
89
|
+
};
|
|
90
|
+
|
|
79
91
|
export type ContentBlockParam =
|
|
80
92
|
| TextBlockParam
|
|
81
93
|
| ImageBlockParam
|
|
82
94
|
| ToolUseBlockParam
|
|
83
95
|
| ToolResultBlockParam
|
|
84
96
|
| ThinkingBlockParam
|
|
85
|
-
| RedactedThinkingBlockParam
|
|
97
|
+
| RedactedThinkingBlockParam
|
|
98
|
+
| FallbackBlockParam;
|
|
86
99
|
|
|
87
100
|
/**
|
|
88
101
|
* A single conversation turn.
|
|
@@ -151,6 +164,19 @@ export type OutputConfig = {
|
|
|
151
164
|
task_budget?: TokenTaskBudget | null;
|
|
152
165
|
};
|
|
153
166
|
|
|
167
|
+
/**
|
|
168
|
+
* Per-attempt override entry in `MessageCreateParams.fallbacks`
|
|
169
|
+
* (server-side-fallback-2026-06-01 beta). Every field except `model`
|
|
170
|
+
* mirrors a top-level control the beta allows re-specifying per attempt.
|
|
171
|
+
*/
|
|
172
|
+
export type FallbackParam = {
|
|
173
|
+
model: string;
|
|
174
|
+
max_tokens?: number;
|
|
175
|
+
thinking?: ThinkingConfigParam;
|
|
176
|
+
output_config?: OutputConfig;
|
|
177
|
+
speed?: "fast";
|
|
178
|
+
};
|
|
179
|
+
|
|
154
180
|
/** Claude Code context-management beta payload. */
|
|
155
181
|
export type ContextManagement = {
|
|
156
182
|
edits: Array<{ type: "clear_thinking_20251015"; keep: "all" }>;
|
|
@@ -175,6 +201,12 @@ export type MessageCreateParams = {
|
|
|
175
201
|
speed?: "fast";
|
|
176
202
|
/** Claude Code context-management beta. */
|
|
177
203
|
context_management?: ContextManagement;
|
|
204
|
+
/**
|
|
205
|
+
* Server-side fallback beta chain — up to three fallback models the API
|
|
206
|
+
* retries when a classifier blocks the primary. Required companion beta
|
|
207
|
+
* header: `server-side-fallback-2026-06-01`.
|
|
208
|
+
*/
|
|
209
|
+
fallbacks?: FallbackParam[];
|
|
178
210
|
};
|
|
179
211
|
|
|
180
212
|
export type MessageCreateParamsStreaming = MessageCreateParams & { stream: true };
|
|
@@ -201,6 +233,21 @@ export type ServerToolUsage = {
|
|
|
201
233
|
web_fetch_requests?: number | null;
|
|
202
234
|
};
|
|
203
235
|
|
|
236
|
+
/**
|
|
237
|
+
* Per-attempt token accounting inside a multi-run turn
|
|
238
|
+
* (server-side-fallback-2026-06-01). Populated whenever a fallback chain
|
|
239
|
+
* ran, including sticky-served turns with no `fallback` content block.
|
|
240
|
+
* A `fallback_message` entry is the definitive "served by fallback" signal.
|
|
241
|
+
*/
|
|
242
|
+
export type UsageIteration = {
|
|
243
|
+
type?: "message" | "fallback_message" | string;
|
|
244
|
+
model?: string | null;
|
|
245
|
+
input_tokens?: number | null;
|
|
246
|
+
output_tokens?: number | null;
|
|
247
|
+
cache_read_input_tokens?: number | null;
|
|
248
|
+
cache_creation_input_tokens?: number | null;
|
|
249
|
+
};
|
|
250
|
+
|
|
204
251
|
export type Usage = {
|
|
205
252
|
input_tokens?: number | null;
|
|
206
253
|
output_tokens?: number | null;
|
|
@@ -208,6 +255,7 @@ export type Usage = {
|
|
|
208
255
|
cache_creation_input_tokens?: number | null;
|
|
209
256
|
cache_creation?: CacheCreation | null;
|
|
210
257
|
server_tool_use?: ServerToolUsage | null;
|
|
258
|
+
iterations?: UsageIteration[] | null;
|
|
211
259
|
};
|
|
212
260
|
|
|
213
261
|
/** The `message` envelope carried by `message_start`. */
|
|
@@ -229,7 +277,8 @@ export type ResponseContentBlock =
|
|
|
229
277
|
| { type: "text"; text: string }
|
|
230
278
|
| { type: "thinking"; thinking: string; signature?: string }
|
|
231
279
|
| { type: "redacted_thinking"; data: string }
|
|
232
|
-
| { type: "tool_use"; id: string; name: string; input?: Record<string, unknown> | null }
|
|
280
|
+
| { type: "tool_use"; id: string; name: string; input?: Record<string, unknown> | null }
|
|
281
|
+
| { type: "fallback"; from: { model: string }; to: { model: string } };
|
|
233
282
|
|
|
234
283
|
export type ContentBlockDelta =
|
|
235
284
|
| { type: "text_delta"; text: string }
|