ai 7.0.23 → 7.0.26
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -0
- package/dist/index.js +10 -4
- package/dist/index.js.map +1 -1
- package/dist/internal/index.js +1 -1
- package/docs/03-ai-sdk-core/36-transcription.mdx +7 -4
- package/docs/06-advanced/11-secure-url-fetching.mdx +117 -0
- package/docs/07-reference/01-ai-sdk-core/11-stream-transcribe.mdx +13 -1
- package/package.json +3 -3
- package/src/generate-text/stream-text.ts +14 -2
- package/src/transcribe/stream-transcribe.ts +13 -4
package/dist/internal/index.js
CHANGED
|
@@ -83,10 +83,13 @@ const durationInSeconds = await result.durationInSeconds; // duration in seconds
|
|
|
83
83
|
The `audio` stream must contain raw audio chunks. `Uint8Array` chunks are raw bytes; `string` chunks are base64-encoded raw bytes. Always set `inputAudioFormat` to match the chunks you send.
|
|
84
84
|
|
|
85
85
|
<Note>
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
86
|
+
String model IDs resolve through the global provider (AI Gateway by
|
|
87
|
+
default). AI Gateway supports streaming transcription for supported models
|
|
88
|
+
(e.g. `openai/gpt-realtime-whisper`, `xai/grok-stt`), so string IDs work:
|
|
89
|
+
`experimental_streamTranscribe({ model: 'openai/gpt-realtime-whisper', ...
|
|
90
|
+
})`. You can also pass a provider model instance (e.g.
|
|
91
|
+
`openai.transcription('gpt-realtime-whisper')`) to stream directly against
|
|
92
|
+
the provider.
|
|
90
93
|
</Note>
|
|
91
94
|
|
|
92
95
|
OpenAI streaming transcription uses `openai.transcription('gpt-realtime-whisper')`. xAI uses the same `xai.transcription()` model for request/response and streaming transcription; `experimental_streamTranscribe` uses xAI's WebSocket STT transport under the hood.
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Secure URL Fetching
|
|
3
|
+
description: How the AI SDK protects server-side fetches of URLs returned by model providers, and how to harden your deployment further.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Secure URL Fetching
|
|
7
|
+
|
|
8
|
+
Many providers return a **URL in their response body** — a generated image,
|
|
9
|
+
audio, or video to download, or a polling URL to check job status. The AI SDK
|
|
10
|
+
fetches these server-side and returns the result to your code. Because that URL
|
|
11
|
+
comes from an external service, a malicious or compromised provider (or anyone
|
|
12
|
+
able to tamper with the response) could point it at an internal address such as
|
|
13
|
+
a cloud-metadata endpoint (`http://169.254.169.254/…`), a private host
|
|
14
|
+
(`http://10.0.0.5/…`), or `localhost`.
|
|
15
|
+
|
|
16
|
+
To prevent that, the SDK validates every response-supplied URL before fetching
|
|
17
|
+
it. This happens automatically inside the provider packages — you don't need to
|
|
18
|
+
configure anything.
|
|
19
|
+
|
|
20
|
+
## What the SDK protects against
|
|
21
|
+
|
|
22
|
+
When the SDK fetches a URL taken from a provider response, it:
|
|
23
|
+
|
|
24
|
+
- **Rejects private, loopback, and link-local targets** — IPv4 (`10/8`,
|
|
25
|
+
`172.16/12`, `192.168/16`, `127/8`, `169.254/16`, CGNAT, multicast, …) and the
|
|
26
|
+
equivalent IPv6 ranges, plus `localhost` and `.local`. Non-`http(s)` schemes
|
|
27
|
+
are rejected too.
|
|
28
|
+
- **Re-validates every redirect hop** — a URL that passes but then redirects to
|
|
29
|
+
an internal address is blocked; the redirect is never followed blindly.
|
|
30
|
+
- **Strips risky request headers** — proxy-forwarding, cloud-metadata, and
|
|
31
|
+
cookie headers are removed before the request.
|
|
32
|
+
- **Drops credentials across origins** — caller headers (`Authorization`,
|
|
33
|
+
`Cookie`, and provider-specific API-key headers alike) are not sent to a host
|
|
34
|
+
on a different origin than the provider's; a redirect that crosses origin
|
|
35
|
+
drops all of them except the user-agent.
|
|
36
|
+
|
|
37
|
+
A blocked URL surfaces as a `DownloadError`.
|
|
38
|
+
|
|
39
|
+
## Self-hosted and local endpoints
|
|
40
|
+
|
|
41
|
+
URLs that are same-origin with the provider endpoint **you configured** (e.g. a
|
|
42
|
+
custom `baseURL` pointing at a self-hosted or `localhost` deployment) are
|
|
43
|
+
exempt from these checks — they target exactly the host you told the SDK to
|
|
44
|
+
talk to. Any redirect off that origin is still validated.
|
|
45
|
+
|
|
46
|
+
## Limitation: DNS resolution and DNS rebinding
|
|
47
|
+
|
|
48
|
+
The built-in guard inspects the URL **as a string**. It deliberately does
|
|
49
|
+
**not resolve DNS**, so two attacks remain out of scope at this layer:
|
|
50
|
+
|
|
51
|
+
1. **Hostname that resolves to a private IP** — a literal host that looks public
|
|
52
|
+
but whose DNS record points at an internal address.
|
|
53
|
+
2. **DNS rebinding** — a host that resolves to a public IP when validated and a
|
|
54
|
+
private IP a moment later when the socket actually connects (a
|
|
55
|
+
time-of-check/time-of-use window).
|
|
56
|
+
|
|
57
|
+
### Why this isn't built in
|
|
58
|
+
|
|
59
|
+
Closing these requires resolving DNS and pinning the resolved IP **at connect
|
|
60
|
+
time** — Node-only capabilities (`node:dns`, a custom `undici` dispatcher). The
|
|
61
|
+
SDK's provider utilities are **cross-runtime**: they run on the edge, in the
|
|
62
|
+
browser, and on Bun/Deno, with no Node-only dependencies, so those APIs aren't
|
|
63
|
+
available there. The threat is also specifically a **server-side** one — on the
|
|
64
|
+
edge and in the browser, outbound `fetch` cannot reach a host's internal network
|
|
65
|
+
or metadata endpoint in the first place. So connect-time IP pinning is only
|
|
66
|
+
meaningful, and only available, on a Node server — which is exactly where you
|
|
67
|
+
can add it yourself.
|
|
68
|
+
|
|
69
|
+
## Hardening your deployment
|
|
70
|
+
|
|
71
|
+
If your server fetches provider-supplied URLs and you want to close the DNS
|
|
72
|
+
gaps, use one (ideally both) of these:
|
|
73
|
+
|
|
74
|
+
### 1. Restrict outbound egress at the network layer
|
|
75
|
+
|
|
76
|
+
Deny your server's network egress to `169.254.0.0/16`, RFC-1918 ranges, and
|
|
77
|
+
loopback. This is the most robust control and is independent of application
|
|
78
|
+
code.
|
|
79
|
+
|
|
80
|
+
### 2. Inject a hardened `fetch`
|
|
81
|
+
|
|
82
|
+
Every provider accepts a custom `fetch`. On Node, back it with an `undici`
|
|
83
|
+
`Agent` whose `connect.lookup` validates the resolved IP and lets the socket
|
|
84
|
+
connect only to a safe address — closing both the hostname-to-private and the
|
|
85
|
+
DNS-rebinding windows:
|
|
86
|
+
|
|
87
|
+
```ts
|
|
88
|
+
import { Agent, fetch as undiciFetch } from 'undici';
|
|
89
|
+
import { lookup } from 'node:dns';
|
|
90
|
+
|
|
91
|
+
// Your own check that returns true for private/loopback/link-local addresses.
|
|
92
|
+
declare function isUnsafeAddress(ip: string): boolean;
|
|
93
|
+
|
|
94
|
+
const safeLookup: typeof lookup = (hostname, options, callback) => {
|
|
95
|
+
lookup(hostname, options as any, (err, address, family) => {
|
|
96
|
+
if (!err && typeof address === 'string' && isUnsafeAddress(address)) {
|
|
97
|
+
callback(new Error(`Refusing to connect to ${address}`), '', 0);
|
|
98
|
+
return;
|
|
99
|
+
}
|
|
100
|
+
(callback as any)(err, address, family);
|
|
101
|
+
});
|
|
102
|
+
};
|
|
103
|
+
|
|
104
|
+
const safeDispatcher = new Agent({ connect: { lookup: safeLookup } });
|
|
105
|
+
|
|
106
|
+
const safeFetch: typeof fetch = (input, init) =>
|
|
107
|
+
undiciFetch(input, { ...init, dispatcher: safeDispatcher }) as any;
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
```ts
|
|
111
|
+
import { createFal } from '@ai-sdk/fal';
|
|
112
|
+
|
|
113
|
+
const fal = createFal({ fetch: safeFetch });
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
The SDK's built-in validation and your connect-time pinning are complementary —
|
|
117
|
+
keep both.
|
|
@@ -48,7 +48,7 @@ console.log(await result.text);
|
|
|
48
48
|
name: 'model',
|
|
49
49
|
type: 'TranscriptionModelV4',
|
|
50
50
|
description:
|
|
51
|
-
|
|
51
|
+
"The transcription model to use. The model must support streaming (`doStream`). String model IDs resolve through the global provider (AI Gateway by default), which supports streaming transcription for supported models (e.g. `openai/gpt-realtime-whisper`, `xai/grok-stt`): `experimental_streamTranscribe({ model: 'openai/gpt-realtime-whisper', ... })`.",
|
|
52
52
|
},
|
|
53
53
|
{
|
|
54
54
|
name: 'audio',
|
|
@@ -147,3 +147,15 @@ console.log(await result.text);
|
|
|
147
147
|
`fullStream` early (e.g. `break` out of the loop), the underlying provider
|
|
148
148
|
connection is closed and pending result promises reject.
|
|
149
149
|
</Note>
|
|
150
|
+
|
|
151
|
+
## Wire format (experimental)
|
|
152
|
+
|
|
153
|
+
Streaming transcription over WebSocket is serialized with the experimental
|
|
154
|
+
transcription-stream envelope defined in `@ai-sdk/provider-utils`
|
|
155
|
+
(`experimental_parseTranscriptionStreamClientFrame`,
|
|
156
|
+
`experimental_serializeTranscriptionStreamPart`,
|
|
157
|
+
`experimental_parseTranscriptionStreamPart`): the client sends one
|
|
158
|
+
`transcription-stream.start` TEXT frame, audio as BINARY frames, and a
|
|
159
|
+
`transcription-stream.audio-done` TEXT frame; each server TEXT frame is one
|
|
160
|
+
JSON-serialized transcription stream part. AI Gateway implements the server
|
|
161
|
+
side of this envelope.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ai",
|
|
3
|
-
"version": "7.0.
|
|
3
|
+
"version": "7.0.26",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "AI SDK by Vercel - build apps like ChatGPT, Claude, Gemini, and more with a single interface for any model using the Vercel AI Gateway or go direct to OpenAI, Anthropic, Google, or any other model provider.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -42,9 +42,9 @@
|
|
|
42
42
|
}
|
|
43
43
|
},
|
|
44
44
|
"dependencies": {
|
|
45
|
-
"@ai-sdk/gateway": "4.0.
|
|
45
|
+
"@ai-sdk/gateway": "4.0.19",
|
|
46
46
|
"@ai-sdk/provider": "4.0.3",
|
|
47
|
-
"@ai-sdk/provider-utils": "5.0.
|
|
47
|
+
"@ai-sdk/provider-utils": "5.0.9"
|
|
48
48
|
},
|
|
49
49
|
"devDependencies": {
|
|
50
50
|
"@edge-runtime/vm": "^5.0.0",
|
|
@@ -1589,6 +1589,19 @@ class DefaultStreamTextResult<
|
|
|
1589
1589
|
// Re-enter the streamText tracing context after stream setup returns.
|
|
1590
1590
|
const runInStreamTextTracingChannelContext = <T>(execute: () => T): T =>
|
|
1591
1591
|
streamTextTracingChannelContext?.run(execute) ?? execute();
|
|
1592
|
+
const runInTracingChannelSpanInStreamText =
|
|
1593
|
+
telemetryDispatcher.runInTracingChannelSpan == null
|
|
1594
|
+
? undefined
|
|
1595
|
+
: <T>(
|
|
1596
|
+
options: Parameters<
|
|
1597
|
+
NonNullable<TelemetryDispatcher['runInTracingChannelSpan']>
|
|
1598
|
+
>[0] & {
|
|
1599
|
+
execute: () => PromiseLike<T>;
|
|
1600
|
+
},
|
|
1601
|
+
) =>
|
|
1602
|
+
runInStreamTextTracingChannelContext(() =>
|
|
1603
|
+
telemetryDispatcher.runInTracingChannelSpan!(options),
|
|
1604
|
+
);
|
|
1592
1605
|
|
|
1593
1606
|
await notify({
|
|
1594
1607
|
event: startEvent,
|
|
@@ -1676,8 +1689,7 @@ class DefaultStreamTextResult<
|
|
|
1676
1689
|
telemetryDispatcher.onToolExecutionEnd,
|
|
1677
1690
|
),
|
|
1678
1691
|
executeToolInTelemetryContext: telemetryDispatcher.executeTool,
|
|
1679
|
-
runInTracingChannelSpan:
|
|
1680
|
-
telemetryDispatcher.runInTracingChannelSpan,
|
|
1692
|
+
runInTracingChannelSpan: runInTracingChannelSpanInStreamText,
|
|
1681
1693
|
onPreliminaryToolResult: result => {
|
|
1682
1694
|
toolExecutionStepStreamController?.enqueue(result);
|
|
1683
1695
|
},
|
|
@@ -15,6 +15,7 @@ import type { TranscriptionModel } from '../types/transcription-model';
|
|
|
15
15
|
import type { TranscriptionModelResponseMetadata } from '../types/transcription-model-response-metadata';
|
|
16
16
|
import type { Warning } from '../types/warning';
|
|
17
17
|
import { asAsyncIterableStream } from '../util/async-iterable-stream';
|
|
18
|
+
import { mergeAbortSignals } from '../util/merge-abort-signals';
|
|
18
19
|
import { VERSION } from '../version';
|
|
19
20
|
import type {
|
|
20
21
|
StreamTranscriptionResult,
|
|
@@ -113,9 +114,10 @@ export function streamTranscribe({
|
|
|
113
114
|
message:
|
|
114
115
|
`The ${resolvedModel.provider} model "${resolvedModel.modelId}" does not support streaming transcription.` +
|
|
115
116
|
(typeof model === 'string'
|
|
116
|
-
? ' String model IDs resolve through the global provider (AI Gateway by default)
|
|
117
|
-
'
|
|
118
|
-
"
|
|
117
|
+
? ' String model IDs resolve through the global provider (AI Gateway by default).' +
|
|
118
|
+
' If that provider does not support streaming transcription, pass a provider model' +
|
|
119
|
+
" instance instead (e.g. openai.transcription('gpt-realtime-whisper'))" +
|
|
120
|
+
' or upgrade @ai-sdk/gateway to a version with streaming transcription support.'
|
|
119
121
|
: ''),
|
|
120
122
|
});
|
|
121
123
|
}
|
|
@@ -253,7 +255,8 @@ export function streamTranscribe({
|
|
|
253
255
|
audio,
|
|
254
256
|
inputAudioFormat,
|
|
255
257
|
providerOptions,
|
|
256
|
-
|
|
258
|
+
// merged so cancelling fullStream also aborts a still-pending doStream
|
|
259
|
+
abortSignal: mergeAbortSignals(abortSignal, pipeAbortController.signal),
|
|
257
260
|
headers: headersWithUserAgent,
|
|
258
261
|
includeRawChunks,
|
|
259
262
|
});
|
|
@@ -271,6 +274,12 @@ export function streamTranscribe({
|
|
|
271
274
|
const reason =
|
|
272
275
|
error ?? new Error('Transcription stream was cancelled or errored.');
|
|
273
276
|
rejectPendingPromises(reason);
|
|
277
|
+
// When `doStream` rejects before the model stream exists (e.g. auth or
|
|
278
|
+
// header resolution failure), nothing has taken ownership of `audio` yet,
|
|
279
|
+
// so cancel it directly — otherwise an upstream producer piping into it
|
|
280
|
+
// hangs forever. When the model did take a reader, `audio` is locked and
|
|
281
|
+
// the cancel rejects, which is fine: the model's cleanup owns it then.
|
|
282
|
+
audio.cancel(reason).catch(() => {});
|
|
274
283
|
transform.writable.abort(reason).catch(() => {
|
|
275
284
|
// the writable is already errored when the model stream failed mid-pipe
|
|
276
285
|
});
|