ai 7.0.101 → 7.0.102

package/CHANGELOG.md

@@ -1,5 +1,35 @@
 # ai
 
+## 7.0.102
+
+### Patch Changes
+
+- 5c0054d: Add optional browser-direct WebRTC for experimental client-delegated Live conversations alongside the existing WebSocket path. Exchange SDP through an application endpoint with `api.session`, configure server-owned data-channel permissions, and preserve committed React session ownership. Capture follows the selected sender track, borrowed tracks remain caller-owned, and disconnect recovery and finalization stay bounded. Applications continue to handle client delegation and submit context; Live session updates and Responses delegation remain unsupported.
+
+  Serialize microphone sender changes and close the peer if detachment fails, without stopping borrowed tracks. Validate nonempty SDP setup answers with a bounded response body, and document the same-origin broker authentication contract.
+
+- 39535af: Add experimental OpenAI Live provider support through the unified `openai.experimental_realtime` factory for server WebSocket sessions with client delegation. Applications own their agents and tools, receive continuous audio/transcript events and delegation metadata, and return context through validated channels. Support immutable startup options, microphone mute controls, graceful session close, and cumulative voice usage. Responses delegation and Live session updates reject before sending.
+
+  Route known Live model IDs to Live, allow an OpenAI-specific `api` override for early-access models, and preserve legacy Realtime defaults for unknown IDs. Token minting follows the same selection rules and rejects Live before requesting unsupported credentials. Extend the realtime v4 specification with optional server WebSocket configuration, per-connection raw-event parsers, and model-wide startup/finalization capabilities. This provider layer supplies connection settings and protocol mapping for server adapters; browser lifecycle and UI integration belong to the core runtime and framework hooks.
+
+  Preserve Realtime client event IDs for session updates and audio appends, and correlate server errors with the originating client event.
+
+- 8b92ba9: fix(ai): settle automatically denied tool calls in UI streams
+- 4b306c2: Report abnormal realtime WebSocket close codes and reasons through the session error path.
+- 4b306c2: Add the client-delegation-first realtime WebSocket runtime with session lifecycle, exact final duration, bounded event queues and command acknowledgement tracking, transcripts, context append, and reversible capture. Continuous sessions support PCM16 audio-chunk playback over an application relay with a recoverable live-edge buffer policy. Legacy turn-based WebSockets retain queued playback and frontend tool handling.
+
+  Remove the deferred managed Responses coordinator, autoContinueTools option, backend usage state, and Live tool-result aliases. Continuous text and delegation handling belong to the application; sendTextMessage and addToolOutput reject continuous sessions. Server-confirmed provider delegation and turn-based audio-delta on the continuous profile fail explicitly. maxPlaybackBufferSeconds is supported only for continuous PCM sessions.
+
+  Fence callbacks, startup publications, tool results, and teardown by connection attempt. Validate token setup before opening a client-secret socket, require explicit relay transport mode, and include WebSocket URL, protocol values, and runtime timeouts/buffer settings in React session configuration keys.
+
+  Retain immediate local capture for explicit legacy preconnect streams, including owned-track cleanup and capture continuity through asynchronous setup. Signal transport closing before draining accepted terminal events, share reentrant close settlement, and cancel stale readiness and deadlines. Enforce 128 KiB frame and combined buffered-send budgets only for Live continuous sessions, using actual encoded wire bytes; oversized manual operations remain recoverable while startup and automatic audio failures close the session. Legacy startup, text, tool output, and audio retain their pre-Live behavior without these byte caps. Live pending-audio overflow drains accepted terminal usage within the existing one-second drain window. Validate final provider-transformed WebSocket URL syntax without removing native authentication query parameters.
+
+- Updated dependencies [5c0054d]
+- Updated dependencies [39535af]
+  - @ai-sdk/provider@4.0.15
+  - @ai-sdk/gateway@4.0.82
+  - @ai-sdk/provider-utils@5.0.41
+
 ## 7.0.101
 
 ### Patch Changes

package/dist/index.d.ts

@@ -8940,23 +8940,63 @@ declare function getRealtimeToolDefinitions<TOOLS extends ToolSet>({ tools, tool
     toolsContext?: InferToolSetContext<TOOLS>;
 }): Promise<RealtimeToolDefinition[]>;
 
-type RealtimeStatus = 'disconnected' | 'connecting' | 'connected' | 'error';
+type RealtimeSessionState = {
+    delegationMode?: 'client' | 'provider';
+    sessionId?: string;
+    transcripts: Extract<RealtimeServerEvent, {
+        type: 'transcript-fragment';
+    }>[];
+    delegations: Extract<RealtimeServerEvent, {
+        type: 'delegation-created';
+    }>[];
+    usage?: {
+        seconds: number;
+    };
+    finalization: 'pending' | 'confirmed' | 'unconfirmed';
+    terminationReason?: string;
+    isInputMuted: boolean;
+};
+
+type RealtimeStatus = 'disconnected' | 'connecting' | 'connected' | 'closing' | 'error';
 interface RealtimeState {
     status: RealtimeStatus;
     messages: UIMessage[];
     events: RealtimeServerEvent[];
     isCapturing: boolean;
     isPlaying: boolean;
+    session?: RealtimeSessionState;
 }
 
 type RealtimeSessionOptions = {
     model: RealtimeModel;
+    /** websocket uses a raw-protocol relay; session exchanges SDP; token fetches client-secret setup. */
     api: {
         token: string;
+        session?: never;
+        websocket?: never;
+        protocols?: never;
+    } | {
+        session: string;
+        token?: never;
+        websocket?: never;
+        protocols?: never;
+    } | {
+        websocket: string;
+        protocols?: string[];
+        token?: never;
+        session?: never;
     };
     sessionConfig?: Partial<RealtimeSessionConfig>;
     sampleRate?: number;
     maxEvents?: number;
+    /** Maximum time to establish transport and receive readiness. Default: 30s. */
+    startupTimeoutMs?: number;
+    /** Maximum wait for final usage after close(). Default: 15s. */
+    closeTimeoutMs?: number;
+    /** WebRTC peer disconnect recovery grace period. Default: 5s. */
+    rtcDisconnectTimeoutMs?: number;
+    /** Continuous PCM only: pause at this budget until resumePlayback(). Default: 2s. */
+    maxPlaybackBufferSeconds?: number;
     onToolCall?: (args: {
         toolCall: {
             toolCallId: string;
@@ -8968,50 +9008,72 @@ type RealtimeSessionOptions = {
     onError?: (error: Error) => void;
 };
 declare abstract class AbstractRealtimeSession {
+    private readonly options;
     protected state: RealtimeState;
     protected maxEvents: number;
     onToolCall: RealtimeSessionOptions['onToolCall'];
-    onEvent: ((event: RealtimeServerEvent) => void) | undefined;
-    onError: ((error: Error) => void) | undefined;
-    private readonly model;
-    private readonly api;
-    private readonly sessionConfig;
-    private readonly reducer;
-    private readonly transport;
-    private readonly audio;
+    onEvent: RealtimeSessionOptions['onEvent'];
+    onError: RealtimeSessionOptions['onError'];
+    private reducer;
+    private readonly sessionLifecycle;
+    private readonly continuous;
+    private attempt?;
+    private commands?;
+    private publication;
+    private transport?;
+    private audio?;
+    private pcm?;
+    private rtc?;
+    private suppliedStream?;
+    private captureGeneration;
+    private captureRequested;
     private currentResponseItemId;
     private readonly toolCallsInResponse;
     private readonly submittedToolOutputs;
     private responseToolCallsClosed;
     protected abstract setState<K extends keyof RealtimeState>(key: K, value: RealtimeState[K]): void;
     constructor(options: RealtimeSessionOptions);
+    private validateConnection;
     connect(): Promise<void>;
+    connect(options: {
+        stream?: MediaStream;
+        capture?: boolean;
+    }): Promise<void>;
+    private fail;
+    private drainAttempt;
+    private finishAttempt;
+    private closeFailed;
     disconnect(): void;
-    sendEvent(event: RealtimeClientEvent): void;
+    /** Wait for final usage when the model supports a session-close acknowledgement. */
+    close(options?: {
+        eventId?: string;
+    }): Promise<void>;
+    private sendTransport;
+    sendEvent(event: RealtimeClientEvent): Promise<void>;
     sendTextMessage(text: string): void;
-    sendAudio(base64Audio: string): void;
+    sendAudio(audio: string): void;
     commitAudio(): void;
     clearAudioBuffer(): void;
     requestResponse(options?: {
         modalities?: string[];
     }): void;
     cancelResponse(): void;
+    private sendAutomaticAudio;
     addToolOutput(callId: string, result: unknown): void;
-    /**
-     * Requests a single response once the tool-bearing response has finished
-     * delivering its tool calls and every one of them has an output. Requesting a
-     * response after each individual output can cause the model to continue
-     * without the full tool context on multi-tool turns.
-     */
     private maybeRequestToolResponse;
     startAudioCapture(stream: MediaStream): void;
+    resumeAudioCapture(): Promise<void>;
     stopAudioCapture(): void;
     stopPlayback(): void;
+    resumePlayback(): Promise<void>;
     dispose(): void;
+    private ensureAudio;
     private applyState;
-    private executeToolCall;
+    private executeTool;
     private handleServerEvent;
     private handleReducerEffect;
+    private reportError;
+    private notifyEvent;
 }
 
 /**
@@ -9023,7 +9085,7 @@ type RealtimeSetupResponse = {
     token: string;
     url: string;
     expiresAt?: number;
-    tools: RealtimeToolDefinition[];
+    tools?: RealtimeToolDefinition[];
 };
 
 /**
@@ -9876,4 +9938,4 @@ declare function uploadSkill({ api, files, displayTitle, providerOptions, }: {
     files: UploadSkillFile[];
 }): Promise<UploadSkillResult>;
 
-export { AI_SDK_TELEMETRY_TRACING_CHANNEL, AbstractChat, type ActiveTools, type Agent, type AgentCallParameters, type AgentStreamParameters, type AsyncIterableStream, type CallSettings, type CallWarning, type ChatAddToolApproveResponseFunction, type ChatAddToolOutputFunction, type ChatInit, type ChatOnDataCallback, type ChatOnErrorCallback, type ChatOnFinishCallback, type ChatOnToolCallCallback, type ChatRequestOptions, type ChatState, type ChatStatus, type ChatTransport, type ChunkDetector, type CompletionRequestOptions, type ContentPart, type CreateUIMessage, type CustomContentUIPart, type DataUIPart, type DeepPartial, DefaultChatTransport, DefaultGeneratedFile, DirectChatTransport, type DirectChatTransportOptions, type DynamicToolCall, type DynamicToolError, type DynamicToolResult, type DynamicToolUIPart, type EmbedEndEvent, type EmbedManyResult, type EmbedResult, type EmbedStartEvent, type Embedding, type EmbeddingModel, type EmbeddingModelCallEndEvent, type EmbeddingModelCallStartEvent, type EmbeddingModelMiddleware, type EmbeddingModelUsage, type ErrorHandler, AbstractRealtimeSession as Experimental_AbstractRealtimeSession, ToolLoopAgent as Experimental_Agent, type ToolLoopAgentSettings as Experimental_AgentSettings, type Batch as Experimental_Batch, type BatchError as Experimental_BatchError, type BatchItemResult as Experimental_BatchItemResult, type BatchProvider as Experimental_BatchProvider, type BatchReference as Experimental_BatchReference, type BatchRequest as Experimental_BatchRequest, type BatchStatus as Experimental_BatchStatus, type CancelBatchOptions as Experimental_CancelBatchOptions, type CancelBatchResult as Experimental_CancelBatchResult, type DownloadFunction as Experimental_DownloadFunction, type Experimental_GeneratedImage, type GetBatchResultsOptions as Experimental_GetBatchResultsOptions, type GetBatchStatusOptions as Experimental_GetBatchStatusOptions, type ImageBatchGenerationResult as Experimental_ImageBatchGenerationResult, type ImageBatchItemResult as Experimental_ImageBatchItemResult, type ImageBatchRequest as Experimental_ImageBatchRequest, type InferAgentUIMessage as Experimental_InferAgentUIMessage, type LanguageModelStreamPart as Experimental_LanguageModelStreamPart, type ListBatchesOptions as Experimental_ListBatchesOptions, type ListBatchesResult as Experimental_ListBatchesResult, type LogWarningsFunction as Experimental_LogWarningsFunction, type RealtimeClientEvent as Experimental_RealtimeClientEvent, type RealtimeFactory as Experimental_RealtimeFactory, type RealtimeFactoryGetTokenOptions as Experimental_RealtimeFactoryGetTokenOptions, type RealtimeFactoryGetTokenResult as Experimental_RealtimeFactoryGetTokenResult, type RealtimeModel as Experimental_RealtimeModel, type RealtimeServerEvent as Experimental_RealtimeServerEvent, type RealtimeSessionConfig as Experimental_RealtimeSessionConfig, type RealtimeSessionOptions as Experimental_RealtimeSessionOptions, type RealtimeSetupResponse as Experimental_RealtimeSetupResponse, type RealtimeState as Experimental_RealtimeState, type RealtimeStatus as Experimental_RealtimeStatus, type RealtimeToolDefinition as Experimental_RealtimeToolDefinition, type Experimental_SpeechResult, type StartBatchOptions as Experimental_StartBatchOptions, type StartBatchResult as Experimental_StartBatchResult, type StreamTranslationResult as Experimental_StreamTranslationResult, type TextBatchGenerationResult as Experimental_TextBatchGenerationResult, type TextBatchItemResult as Experimental_TextBatchItemResult, type TextBatchRequest as Experimental_TextBatchRequest, type Experimental_ToolCallers, type Experimental_TranscriptionResult, type TranslationStreamPart as Experimental_TranslationStreamPart, type FileUIPart, type FinishReason, type GenerateImageCall, type GenerateImageResult, type GenerateObjectEndEvent, type GenerateObjectResult, type GenerateObjectStartEvent, type GenerateObjectStepEndEvent, type GenerateObjectStepStartEvent, type GenerateTextAbortEvent, type GenerateTextEndEvent, type GenerateTextInclude, type GenerateTextOnAbortCallback, type GenerateTextOnEndCallback, type GenerateTextOnFinishCallback, type GenerateTextOnStartCallback, type GenerateTextOnStepEndCallback, type GenerateTextOnStepFinishCallback, type GenerateTextOnStepStartCallback, type GenerateTextResult, type GenerateTextStartEvent, type GenerateTextStepEndEvent, type GenerateTextStepStartEvent, type GenerateVideoPrompt, type GenerateVideoResult, type GeneratedAudioFile, type GeneratedFile, type GenericToolApprovalFunction, type GetVideoStatusResult, HttpChatTransport, type HttpChatTransportInitOptions, type ImageModel, type ImageModelMiddleware, type ImageModelProviderMetadata, type ImageModelResponseMetadata, type ImageModelUsage, type InferAgentUIMessage, type InferCompleteOutput as InferGenerateOutput, type InferPartialOutput as InferStreamOutput, type InferTelemetryEvent, type InferUIDataParts, type InferUIMessageChunk, type InferUITool, type InferUITools, type Instructions, InvalidArgumentError, InvalidDataContentError, InvalidMessageRoleError, InvalidStreamPartError, InvalidToolApprovalError, InvalidToolApprovalSignatureError, InvalidToolInputError, type JSONValue, JsonToSseTransformStream, type LanguageModel, type LanguageModelCallEndEvent, type LanguageModelCallOptions, type LanguageModelCallStartEvent, type LanguageModelMiddleware, type LanguageModelRequestMetadata, type LanguageModelResponseMetadata, type LanguageModelUsage, type LogWarningsFunction, MessageConversionError, MissingToolResultsError, type ModelInfo, NoImageGeneratedError, NoObjectGeneratedError, NoOutputGeneratedError, NoSpeechGeneratedError, NoSuchProviderError, NoSuchToolError, NoTranscriptGeneratedError, NoTranslationGeneratedError, NoVideoGeneratedError, type ObjectStreamPart, type OnFinishEvent, type OnLanguageModelCallEndCallback, type OnLanguageModelCallStartCallback, type OnStartEvent, type OnStepFinishEvent, type OnStepStartEvent, type OnToolCallFinishEvent, type OnToolCallStartEvent, type OnToolExecutionEndCallback, type OnToolExecutionStartCallback, output as Output, type OutputChunkTimingStats, type Output as OutputInterface, type PrepareReconnectToStreamRequest, type PrepareSendMessagesRequest, type PrepareStepFunction, type PrepareStepResult, type Prompt, type Provider, type ProviderMetadata, type ProviderReference, type ProviderRegistryProvider, type ReasoningFileOutput, type ReasoningFileUIPart, type ReasoningOutput, type ReasoningUIPart, type RepairTextFunction, type RequestOptions, type RerankEndEvent, type RerankResult, type RerankStartEvent, type RerankingModel, type RerankingModelCallEndEvent, type RerankingModelCallStartEvent, RetryError, type SafeValidateUIMessagesResult, SerialJobExecutor, type SingleToolApprovalFunction, type SourceDocumentUIPart, type SourceUrlUIPart, type SpeechModel, type SpeechModelResponseMetadata, type SpeechResult, type StartVideoResult, type StaticToolCall, type StaticToolError, type StaticToolOutputDenied, type StaticToolResult, type StepResult, type StepResultPerformance, type StepStartUIPart, type StopCondition, type StreamObjectOnFinishCallback, type StreamObjectResult, StreamProviderError, type StreamTextEndEvent, type StreamTextInclude, type StreamTextOnChunkCallback, type StreamTextOnEndCallback, type StreamTextOnErrorCallback, type StreamTextOnErrorResult, type StreamTextOnErrorRetryCallback, type StreamTextResult, type StreamTextTransform, type StreamTranscriptionResult, type Telemetry, type TelemetryOptions, type TelemetryTracingChannelMessage, type TelemetryTracingEventType, TextStreamChatTransport, type TextStreamPart, type TextUIPart, type TimeoutConfiguration, type ToUIMessageChunkOptions, type ToolApprovalConfiguration, type ToolApprovalRequestOutput, type ToolApprovalResponseOutput, type ToolApprovalStatus, ToolCallNotFoundForApprovalError, ToolCallRepairError, type ToolCallRepairFunction, type ToolChoice, ToolChoiceViolationError, type ToolExecutionEndEvent, type ToolExecutionStartEvent, type ToolInputRefinement, ToolLoopAgent, type ToolLoopAgentSettings, type ToolOrder, type ToolOutputErrorUIPart, type ToolUIPart, type TranscriptionModel, type TranscriptionModelResponseMetadata, type TranscriptionResult, type TranscriptionStreamPart, type TypedToolCall, type TypedToolError, type TypedToolOutputDenied, type TypedToolResult, type UIDataPartSchemas, type UIDataTypes, type UIMessage, type UIMessageChunk, type UIMessagePart, UIMessageStreamError, type UIMessageStreamOnEndCallback, type UIMessageStreamOnFinishCallback, type UIMessageStreamOnStepEndCallback, type UIMessageStreamOnStepFinishCallback, type UIMessageStreamOptions, type UIMessageStreamOutcome, type UIMessageStreamWriter, type UIMessageStreamWriterWithOutcome, type UITool, type UIToolInvocation, type UITools, UI_MESSAGE_STREAM_HEADERS, UnsupportedModelVersionError, type UploadFileResult, type UploadSkillResult, type UseCompletionOptions, type Warning, addToolInputExamplesMiddleware, assistantModelMessageSchema, callCompletionApi, consumeStream, convertDataContentToBase64String, convertFileListToFileUIParts, convertToModelMessages, cosineSimilarity, createAgentUIStream, createAgentUIStreamResponse, createDownload, createProviderRegistry, createTextStreamResponse, createUIMessageStream, createUIMessageStreamResponse, customProvider, defaultEmbeddingSettingsMiddleware, defaultInstructionsMiddleware, defaultSettingsMiddleware, detectToolDrift, embed, embedMany, cancelBatch as experimental_cancelBatch, experimental_createProviderRegistry, decodeRealtimeAudio as experimental_decodeRealtimeAudio, encodeRealtimeAudio as experimental_encodeRealtimeAudio, filterActiveTools as experimental_filterActiveTools, experimental_generateSpeech, experimental_generateVideo, getBatchResults as experimental_getBatchResults, getBatchStatus as experimental_getBatchStatus, getRealtimeToolDefinitions as experimental_getRealtimeToolDefinitions, experimental_getVideoStatus, listBatches as experimental_listBatches, resampleAudio as experimental_resampleAudio, startBatch as experimental_startBatch, experimental_startVideo, streamLanguageModelCall as experimental_streamLanguageModelCall, streamTranscribe as experimental_streamTranscribe, streamTranslate as experimental_streamTranslate, experimental_transcribe, extractJsonMiddleware, extractReasoningMiddleware, fingerprintTools, generateImage, generateObject, generateSpeech, generateText, getChunkTimeoutMs, getFirstChunkTimeoutMs, getStaticToolName, getStepTimeoutMs, getTextFromDataUrl, getToolName, getToolOrDynamicToolName, getToolTimeoutMs, getTotalTimeoutMs, hasToolCall, isCustomContentUIPart, isDataUIPart, isDeepEqualData, isDynamicToolUIPart, isFileUIPart, isLoopFinished, isReasoningFileUIPart, isReasoningUIPart, isStaticToolUIPart, isStepCount, isTextUIPart, isToolOutputErrorUIPart, isToolUIPart, lastAssistantMessageIsCompleteWithApprovalResponses, lastAssistantMessageIsCompleteWithToolCalls, modelMessageSchema, parsePartialJson, pipeAgentUIStreamToResponse, pipeTextStreamToResponse, pipeUIMessageStreamToResponse, pruneMessages, readUIMessageStream, registerTelemetry, rerank, safeValidateUIMessages, simulateReadableStream, simulateStreamingMiddleware, smoothStream, isStepCount as stepCountIs, streamObject, streamText, systemModelMessageSchema, toTextStream, toUIMessageChunk, toUIMessageStream, toolModelMessageSchema, transcribe, uiMessageChunkSchema, uploadFile, uploadSkill, userModelMessageSchema, validateUIMessages, wrapEmbeddingModel, wrapImageModel, wrapLanguageModel, wrapProvider };
+export { AI_SDK_TELEMETRY_TRACING_CHANNEL, AbstractChat, type ActiveTools, type Agent, type AgentCallParameters, type AgentStreamParameters, type AsyncIterableStream, type CallSettings, type CallWarning, type ChatAddToolApproveResponseFunction, type ChatAddToolOutputFunction, type ChatInit, type ChatOnDataCallback, type ChatOnErrorCallback, type ChatOnFinishCallback, type ChatOnToolCallCallback, type ChatRequestOptions, type ChatState, type ChatStatus, type ChatTransport, type ChunkDetector, type CompletionRequestOptions, type ContentPart, type CreateUIMessage, type CustomContentUIPart, type DataUIPart, type DeepPartial, DefaultChatTransport, DefaultGeneratedFile, DirectChatTransport, type DirectChatTransportOptions, type DynamicToolCall, type DynamicToolError, type DynamicToolResult, type DynamicToolUIPart, type EmbedEndEvent, type EmbedManyResult, type EmbedResult, type EmbedStartEvent, type Embedding, type EmbeddingModel, type EmbeddingModelCallEndEvent, type EmbeddingModelCallStartEvent, type EmbeddingModelMiddleware, type EmbeddingModelUsage, type ErrorHandler, AbstractRealtimeSession as Experimental_AbstractRealtimeSession, ToolLoopAgent as Experimental_Agent, type ToolLoopAgentSettings as Experimental_AgentSettings, type Batch as Experimental_Batch, type BatchError as Experimental_BatchError, type BatchItemResult as Experimental_BatchItemResult, type BatchProvider as Experimental_BatchProvider, type BatchReference as Experimental_BatchReference, type BatchRequest as Experimental_BatchRequest, type BatchStatus as Experimental_BatchStatus, type CancelBatchOptions as Experimental_CancelBatchOptions, type CancelBatchResult as Experimental_CancelBatchResult, type DownloadFunction as Experimental_DownloadFunction, type Experimental_GeneratedImage, type GetBatchResultsOptions as Experimental_GetBatchResultsOptions, type GetBatchStatusOptions as Experimental_GetBatchStatusOptions, type ImageBatchGenerationResult as Experimental_ImageBatchGenerationResult, type ImageBatchItemResult as Experimental_ImageBatchItemResult, type ImageBatchRequest as Experimental_ImageBatchRequest, type InferAgentUIMessage as Experimental_InferAgentUIMessage, type LanguageModelStreamPart as Experimental_LanguageModelStreamPart, type ListBatchesOptions as Experimental_ListBatchesOptions, type ListBatchesResult as Experimental_ListBatchesResult, type LogWarningsFunction as Experimental_LogWarningsFunction, type RealtimeClientEvent as Experimental_RealtimeClientEvent, type RealtimeFactory as Experimental_RealtimeFactory, type RealtimeFactoryGetTokenOptions as Experimental_RealtimeFactoryGetTokenOptions, type RealtimeFactoryGetTokenResult as Experimental_RealtimeFactoryGetTokenResult, type RealtimeModel as Experimental_RealtimeModel, type RealtimeServerEvent as Experimental_RealtimeServerEvent, type RealtimeSessionConfig as Experimental_RealtimeSessionConfig, type RealtimeSessionOptions as Experimental_RealtimeSessionOptions, type RealtimeSessionState as Experimental_RealtimeSessionState, type RealtimeSetupResponse as Experimental_RealtimeSetupResponse, type RealtimeState as Experimental_RealtimeState, type RealtimeStatus as Experimental_RealtimeStatus, type RealtimeToolDefinition as Experimental_RealtimeToolDefinition, type Experimental_SpeechResult, type StartBatchOptions as Experimental_StartBatchOptions, type StartBatchResult as Experimental_StartBatchResult, type StreamTranslationResult as Experimental_StreamTranslationResult, type TextBatchGenerationResult as Experimental_TextBatchGenerationResult, type TextBatchItemResult as Experimental_TextBatchItemResult, type TextBatchRequest as Experimental_TextBatchRequest, type Experimental_ToolCallers, type Experimental_TranscriptionResult, type TranslationStreamPart as Experimental_TranslationStreamPart, type FileUIPart, type FinishReason, type GenerateImageCall, type GenerateImageResult, type GenerateObjectEndEvent, type GenerateObjectResult, type GenerateObjectStartEvent, type GenerateObjectStepEndEvent, type GenerateObjectStepStartEvent, type GenerateTextAbortEvent, type GenerateTextEndEvent, type GenerateTextInclude, type GenerateTextOnAbortCallback, type GenerateTextOnEndCallback, type GenerateTextOnFinishCallback, type GenerateTextOnStartCallback, type GenerateTextOnStepEndCallback, type GenerateTextOnStepFinishCallback, type GenerateTextOnStepStartCallback, type GenerateTextResult, type GenerateTextStartEvent, type GenerateTextStepEndEvent, type GenerateTextStepStartEvent, type GenerateVideoPrompt, type GenerateVideoResult, type GeneratedAudioFile, type GeneratedFile, type GenericToolApprovalFunction, type GetVideoStatusResult, HttpChatTransport, type HttpChatTransportInitOptions, type ImageModel, type ImageModelMiddleware, type ImageModelProviderMetadata, type ImageModelResponseMetadata, type ImageModelUsage, type InferAgentUIMessage, type InferCompleteOutput as InferGenerateOutput, type InferPartialOutput as InferStreamOutput, type InferTelemetryEvent, type InferUIDataParts, type InferUIMessageChunk, type InferUITool, type InferUITools, type Instructions, InvalidArgumentError, InvalidDataContentError, InvalidMessageRoleError, InvalidStreamPartError, InvalidToolApprovalError, InvalidToolApprovalSignatureError, InvalidToolInputError, type JSONValue, JsonToSseTransformStream, type LanguageModel, type LanguageModelCallEndEvent, type LanguageModelCallOptions, type LanguageModelCallStartEvent, type LanguageModelMiddleware, type LanguageModelRequestMetadata, type LanguageModelResponseMetadata, type LanguageModelUsage, type LogWarningsFunction, MessageConversionError, MissingToolResultsError, type ModelInfo, NoImageGeneratedError, NoObjectGeneratedError, NoOutputGeneratedError, NoSpeechGeneratedError, NoSuchProviderError, NoSuchToolError, NoTranscriptGeneratedError, NoTranslationGeneratedError, NoVideoGeneratedError, type ObjectStreamPart, type OnFinishEvent, type OnLanguageModelCallEndCallback, type OnLanguageModelCallStartCallback, type OnStartEvent, type OnStepFinishEvent, type OnStepStartEvent, type OnToolCallFinishEvent, type OnToolCallStartEvent, type OnToolExecutionEndCallback, type OnToolExecutionStartCallback, output as Output, type OutputChunkTimingStats, type Output as OutputInterface, type PrepareReconnectToStreamRequest, type PrepareSendMessagesRequest, type PrepareStepFunction, type PrepareStepResult, type Prompt, type Provider, type ProviderMetadata, type ProviderReference, type ProviderRegistryProvider, type ReasoningFileOutput, type ReasoningFileUIPart, type ReasoningOutput, type ReasoningUIPart, type RepairTextFunction, type RequestOptions, type RerankEndEvent, type RerankResult, type RerankStartEvent, type RerankingModel, type RerankingModelCallEndEvent, type RerankingModelCallStartEvent, RetryError, type SafeValidateUIMessagesResult, SerialJobExecutor, type SingleToolApprovalFunction, type SourceDocumentUIPart, type SourceUrlUIPart, type SpeechModel, type SpeechModelResponseMetadata, type SpeechResult, type StartVideoResult, type StaticToolCall, type StaticToolError, type StaticToolOutputDenied, type StaticToolResult, type StepResult, type StepResultPerformance, type StepStartUIPart, type StopCondition, type StreamObjectOnFinishCallback, type StreamObjectResult, StreamProviderError, type StreamTextEndEvent, type StreamTextInclude, type StreamTextOnChunkCallback, type StreamTextOnEndCallback, type StreamTextOnErrorCallback, type StreamTextOnErrorResult, type StreamTextOnErrorRetryCallback, type StreamTextResult, type StreamTextTransform, type StreamTranscriptionResult, type Telemetry, type TelemetryOptions, type TelemetryTracingChannelMessage, type TelemetryTracingEventType, TextStreamChatTransport, type TextStreamPart, type TextUIPart, type TimeoutConfiguration, type ToUIMessageChunkOptions, type ToolApprovalConfiguration, type ToolApprovalRequestOutput, type ToolApprovalResponseOutput, type ToolApprovalStatus, ToolCallNotFoundForApprovalError, ToolCallRepairError, type ToolCallRepairFunction, type ToolChoice, ToolChoiceViolationError, type ToolExecutionEndEvent, type ToolExecutionStartEvent, type ToolInputRefinement, ToolLoopAgent, type ToolLoopAgentSettings, type ToolOrder, type ToolOutputErrorUIPart, type ToolUIPart, type TranscriptionModel, type TranscriptionModelResponseMetadata, type TranscriptionResult, type TranscriptionStreamPart, type TypedToolCall, type TypedToolError, type TypedToolOutputDenied, type TypedToolResult, type UIDataPartSchemas, type UIDataTypes, type UIMessage, type UIMessageChunk, type UIMessagePart, UIMessageStreamError, type UIMessageStreamOnEndCallback, type UIMessageStreamOnFinishCallback, type UIMessageStreamOnStepEndCallback, type UIMessageStreamOnStepFinishCallback, type UIMessageStreamOptions, type UIMessageStreamOutcome, type UIMessageStreamWriter, type UIMessageStreamWriterWithOutcome, type UITool, type UIToolInvocation, type UITools, UI_MESSAGE_STREAM_HEADERS, UnsupportedModelVersionError, type UploadFileResult, type UploadSkillResult, type UseCompletionOptions, type Warning, addToolInputExamplesMiddleware, assistantModelMessageSchema, callCompletionApi, consumeStream, convertDataContentToBase64String, convertFileListToFileUIParts, convertToModelMessages, cosineSimilarity, createAgentUIStream, createAgentUIStreamResponse, createDownload, createProviderRegistry, createTextStreamResponse, createUIMessageStream, createUIMessageStreamResponse, customProvider, defaultEmbeddingSettingsMiddleware, defaultInstructionsMiddleware, defaultSettingsMiddleware, detectToolDrift, embed, embedMany, cancelBatch as experimental_cancelBatch, experimental_createProviderRegistry, decodeRealtimeAudio as experimental_decodeRealtimeAudio, encodeRealtimeAudio as experimental_encodeRealtimeAudio, filterActiveTools as experimental_filterActiveTools, experimental_generateSpeech, experimental_generateVideo, getBatchResults as experimental_getBatchResults, getBatchStatus as experimental_getBatchStatus, getRealtimeToolDefinitions as experimental_getRealtimeToolDefinitions, experimental_getVideoStatus, listBatches as experimental_listBatches, resampleAudio as experimental_resampleAudio, startBatch as experimental_startBatch, experimental_startVideo, streamLanguageModelCall as experimental_streamLanguageModelCall, streamTranscribe as experimental_streamTranscribe, streamTranslate as experimental_streamTranslate, experimental_transcribe, extractJsonMiddleware, extractReasoningMiddleware, fingerprintTools, generateImage, generateObject, generateSpeech, generateText, getChunkTimeoutMs, getFirstChunkTimeoutMs, getStaticToolName, getStepTimeoutMs, getTextFromDataUrl, getToolName, getToolOrDynamicToolName, getToolTimeoutMs, getTotalTimeoutMs, hasToolCall, isCustomContentUIPart, isDataUIPart, isDeepEqualData, isDynamicToolUIPart, isFileUIPart, isLoopFinished, isReasoningFileUIPart, isReasoningUIPart, isStaticToolUIPart, isStepCount, isTextUIPart, isToolOutputErrorUIPart, isToolUIPart, lastAssistantMessageIsCompleteWithApprovalResponses, lastAssistantMessageIsCompleteWithToolCalls, modelMessageSchema, parsePartialJson, pipeAgentUIStreamToResponse, pipeTextStreamToResponse, pipeUIMessageStreamToResponse, pruneMessages, readUIMessageStream, registerTelemetry, rerank, safeValidateUIMessages, simulateReadableStream, simulateStreamingMiddleware, smoothStream, isStepCount as stepCountIs, streamObject, streamText, systemModelMessageSchema, toTextStream, toUIMessageChunk, toUIMessageStream, toolModelMessageSchema, transcribe, uiMessageChunkSchema, uploadFile, uploadSkill, userModelMessageSchema, validateUIMessages, wrapEmbeddingModel, wrapImageModel, wrapLanguageModel, wrapProvider };