@ai-sdk/google 3.0.130 → 3.0.132
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -0
- package/dist/index.d.mts +29 -3
- package/dist/index.d.ts +29 -3
- package/dist/index.js +225 -37
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +225 -37
- package/dist/index.mjs.map +1 -1
- package/dist/internal/index.d.mts +19 -2
- package/dist/internal/index.d.ts +19 -2
- package/dist/internal/index.js +224 -36
- package/dist/internal/index.js.map +1 -1
- package/dist/internal/index.mjs +224 -36
- package/dist/internal/index.mjs.map +1 -1
- package/docs/15-google-generative-ai.mdx +143 -1
- package/package.json +3 -3
- package/src/continue-google-stream.ts +75 -0
- package/src/convert-google-generative-ai-usage.ts +75 -0
- package/src/google-generative-ai-image-settings.ts +1 -0
- package/src/google-generative-ai-language-model.ts +150 -24
- package/src/google-generative-ai-options.ts +13 -0
- package/src/google-generative-ai-prompt.ts +18 -0
- package/src/interactions/google-interactions-language-model-options.ts +2 -0
- package/src/interactions/google-interactions-language-model.ts +6 -6
- package/src/map-google-generative-ai-finish-reason.ts +3 -0
|
@@ -267,6 +267,12 @@ The following optional provider options are available for Google Generative AI m
|
|
|
267
267
|
[Priority inference](https://ai.google.dev/gemini-api/docs/priority-inference)
|
|
268
268
|
and [Flex inference](https://ai.google.dev/gemini-api/docs/flex-inference).
|
|
269
269
|
|
|
270
|
+
- **continuationToken** _string_
|
|
271
|
+
|
|
272
|
+
Optional. Resumes a response that stopped because it used up its
|
|
273
|
+
`maxOutputTokens` budget (Gemini 4 long decode). Send it with the original
|
|
274
|
+
request and a higher `maxOutputTokens`. See [Long decode](#long-decode).
|
|
275
|
+
|
|
270
276
|
- **threshold** _string_
|
|
271
277
|
|
|
272
278
|
Optional. Standalone threshold setting that can be used independently of `safetySettings`.
|
|
@@ -332,6 +338,136 @@ console.log(text);
|
|
|
332
338
|
console.log(reasoning); // Reasoning summary
|
|
333
339
|
```
|
|
334
340
|
|
|
341
|
+
### Long decode
|
|
342
|
+
|
|
343
|
+
Gemini 4 models such as `gemini-4-argon` can keep thinking and writing past a
|
|
344
|
+
single request's limits, up to the shared 1M token context window (input,
|
|
345
|
+
thinking, and output tokens together). Use it for hard math, science, proofs,
|
|
346
|
+
and long coding or agentic tasks.
|
|
347
|
+
|
|
348
|
+
- A single request generates at most 32,768 tokens without streaming and 65,536
|
|
349
|
+
tokens with streaming (`streamText`). Thinking tokens count towards this cap.
|
|
350
|
+
- `maxOutputTokens` is a **cumulative budget across requests**, not a target
|
|
351
|
+
length. When a request reaches the per-request cap while budget remains, the
|
|
352
|
+
provider automatically sends the same request again with a continuation token
|
|
353
|
+
and combines the results into one response. Set `maxOutputTokens` above the
|
|
354
|
+
per-request cap (up to 1M minus the input tokens) to enable this. If you do
|
|
355
|
+
not set it, the call ends after one request.
|
|
356
|
+
- A lower `maxOutputTokens` does not make the model think less, it ends the
|
|
357
|
+
response early (possibly before any answer) and you are still charged for the
|
|
358
|
+
tokens. Use `thinkingConfig.thinkingLevel` to control how much the model
|
|
359
|
+
thinks. `thinkingBudget` is not supported by these models.
|
|
360
|
+
- Complex requests can run for several minutes. Make sure your HTTP client
|
|
361
|
+
allows read timeouts of at least 15 to 20 minutes.
|
|
362
|
+
|
|
363
|
+
**Usage across requests.** Each continued request re-sends the prompt plus all
|
|
364
|
+
output so far as input, so its prompt token count includes everything
|
|
365
|
+
generated before it. `usage` is the sum of all requests, which matches what
|
|
366
|
+
Google bills (re-sent tokens are billed at a discount through implicit
|
|
367
|
+
caching). Do not use the summed input tokens for context window checks or
|
|
368
|
+
compaction: they can be several times larger than the real context. When a
|
|
369
|
+
response spanned more than one request,
|
|
370
|
+
`providerMetadata.google.initialPromptTokenCount` holds the prompt token count
|
|
371
|
+
of the first request (the google-genai SDKs expose the same value as
|
|
372
|
+
`initial_prompt_token_count`):
|
|
373
|
+
|
|
374
|
+
```ts
|
|
375
|
+
const result = await generateText({
|
|
376
|
+
model: google('gemini-4-argon'),
|
|
377
|
+
prompt,
|
|
378
|
+
maxOutputTokens: 500_000,
|
|
379
|
+
});
|
|
380
|
+
|
|
381
|
+
// Billing: summed across all requests
|
|
382
|
+
console.log(result.usage.inputTokens.total);
|
|
383
|
+
|
|
384
|
+
// Context window: the first request's prompt plus the generated output
|
|
385
|
+
const promptTokens =
|
|
386
|
+
result.providerMetadata?.google?.initialPromptTokenCount ??
|
|
387
|
+
result.usage.inputTokens.total;
|
|
388
|
+
const contextTokens = promptTokens + (result.usage.outputTokens.total ?? 0);
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
**Resuming after the budget is used up.** When the cumulative generated tokens
|
|
392
|
+
reach `maxOutputTokens`, the finish reason is `length` and the provider returns
|
|
393
|
+
a continuation token on `result.providerMetadata.google.continuationToken`. To
|
|
394
|
+
continue, send the **original request unchanged** (same prompt, tools, and
|
|
395
|
+
settings; do not append the partial output as a message) together with the
|
|
396
|
+
token as `providerOptions.google.continuationToken` and a **higher**
|
|
397
|
+
`maxOutputTokens`. Resending with the same `maxOutputTokens` fails because the
|
|
398
|
+
budget is used up. Each response contains only the newly generated content.
|
|
399
|
+
|
|
400
|
+
```ts
|
|
401
|
+
import { google } from '@ai-sdk/google';
|
|
402
|
+
import { generateText } from 'ai';
|
|
403
|
+
|
|
404
|
+
const prompt = 'Prove that sqrt(2) is irrational, in detail.';
|
|
405
|
+
|
|
406
|
+
const result = await generateText({
|
|
407
|
+
model: google('gemini-4-argon'),
|
|
408
|
+
prompt,
|
|
409
|
+
maxOutputTokens: 128_000,
|
|
410
|
+
});
|
|
411
|
+
|
|
412
|
+
const continuationToken =
|
|
413
|
+
result.providerMetadata?.google?.continuationToken;
|
|
414
|
+
|
|
415
|
+
if (result.finishReason === 'length' && typeof continuationToken === 'string') {
|
|
416
|
+
// The budget was used up. Raise it to keep going.
|
|
417
|
+
const continued = await generateText({
|
|
418
|
+
model: google('gemini-4-argon'),
|
|
419
|
+
prompt,
|
|
420
|
+
maxOutputTokens: 256_000,
|
|
421
|
+
providerOptions: { google: { continuationToken } },
|
|
422
|
+
});
|
|
423
|
+
|
|
424
|
+
console.log(result.text + continued.text);
|
|
425
|
+
}
|
|
426
|
+
```
|
|
427
|
+
|
|
428
|
+
Things to keep in mind:
|
|
429
|
+
|
|
430
|
+
- **Dropped streams.** Streamed parts are forwarded as they arrive and are not
|
|
431
|
+
buffered between the server's checkpoints, so a stream that drops mid-request
|
|
432
|
+
is not resumed automatically. Retry the request from the start.
|
|
433
|
+
- **Provider metadata is per call.** If you store the conversation history,
|
|
434
|
+
combine the content of all calls into one model message only after the final
|
|
435
|
+
call finishes. Keep thought signatures unchanged, including the ones on empty
|
|
436
|
+
trailing text parts.
|
|
437
|
+
- **Zero Data Retention.** Continuation works with `store: false` because the
|
|
438
|
+
state travels encrypted inside the token.
|
|
439
|
+
|
|
440
|
+
#### Long decode with the Interactions API
|
|
441
|
+
|
|
442
|
+
With the [Interactions API](#interactions-api), long decode requires
|
|
443
|
+
`background: true`. The server runs every request itself, so no client-side
|
|
444
|
+
continuation is needed, and `usage` covers all requests. Set an explicit
|
|
445
|
+
`maxOutputTokens` above 65,536 (without it, you get a single 64K request) and
|
|
446
|
+
keep `store` enabled (the default). The provider polls the interaction (or
|
|
447
|
+
streams it, reconnecting when the connection drops) until it finishes.
|
|
448
|
+
|
|
449
|
+
```ts
|
|
450
|
+
import { google } from '@ai-sdk/google';
|
|
451
|
+
import { generateText } from 'ai';
|
|
452
|
+
|
|
453
|
+
const result = await generateText({
|
|
454
|
+
model: google.interactions('gemini-4-argon'),
|
|
455
|
+
prompt: 'Prove that sqrt(2) is irrational, in detail.',
|
|
456
|
+
maxOutputTokens: 500_000,
|
|
457
|
+
providerOptions: {
|
|
458
|
+
google: {
|
|
459
|
+
background: true,
|
|
460
|
+
pollingTimeoutMs: 60 * 60 * 1000, // default: 30 minutes
|
|
461
|
+
},
|
|
462
|
+
},
|
|
463
|
+
});
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
Without `background: true`, an interaction generates at most 65,536 tokens.
|
|
467
|
+
Requests with a higher `maxOutputTokens` are rejected by the API, and an
|
|
468
|
+
interaction that reaches the cap finishes with `length`. Long decode is not
|
|
469
|
+
available with the Vertex AI Interactions API.
|
|
470
|
+
|
|
335
471
|
### File Inputs
|
|
336
472
|
|
|
337
473
|
The Google Generative AI provider supports file inputs, e.g. PDF files.
|
|
@@ -672,7 +808,7 @@ The [File Search tool](https://ai.google.dev/gemini-api/docs/file-search) lets G
|
|
|
672
808
|
import { google } from '@ai-sdk/google';
|
|
673
809
|
import { generateText } from 'ai';
|
|
674
810
|
|
|
675
|
-
const { text, sources } = await generateText({
|
|
811
|
+
const { text, sources, providerMetadata } = await generateText({
|
|
676
812
|
model: google('gemini-2.5-pro'),
|
|
677
813
|
tools: {
|
|
678
814
|
file_search: google.tools.fileSearch({
|
|
@@ -689,6 +825,9 @@ const { text, sources } = await generateText({
|
|
|
689
825
|
|
|
690
826
|
File Search responses include citations via the normal `sources` field and expose raw [grounding metadata](#google-search) in `providerMetadata.google.groundingMetadata`.
|
|
691
827
|
|
|
828
|
+
Custom metadata attached to the indexed documents is available in
|
|
829
|
+
`groundingMetadata.groundingChunks[].retrievedContext.customMetadata`.
|
|
830
|
+
|
|
692
831
|
### URL Context
|
|
693
832
|
|
|
694
833
|
Google provides a provider-defined URL context tool.
|
|
@@ -1082,6 +1221,8 @@ The following Zod features are known to not work with Google Generative AI:
|
|
|
1082
1221
|
|
|
1083
1222
|
| Model | Image Input | Object Generation | Tool Usage | Tool Streaming | Google Search | URL Context |
|
|
1084
1223
|
| ------------------------------------- | ------------------- | ------------------- | ------------------- | ------------------- | ------------------- | ------------------- |
|
|
1224
|
+
| `gemini-nano-banana-2.1` | <Check size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Cross size={18} /> | <Check size={18} /> | <Cross size={18} /> |
|
|
1225
|
+
| `gemini-4-argon` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
|
|
1085
1226
|
| `gemini-3.8-flash` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
|
|
1086
1227
|
| `gemini-3.7-flash` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
|
|
1087
1228
|
| `gemini-3.6-flash` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
|
|
@@ -1965,6 +2106,7 @@ console.log(result.providerMetadata?.google?.groundingMetadata);
|
|
|
1965
2106
|
| `gemini-2.5-flash-image` | <Check size={18} /> | <Check size={18} /> | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 |
|
|
1966
2107
|
| `gemini-3-pro-image-preview` | <Check size={18} /> | <Check size={18} /> | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 |
|
|
1967
2108
|
| `gemini-3.1-flash-image-preview` | <Check size={18} /> | <Check size={18} /> | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9 |
|
|
2109
|
+
| `gemini-nano-banana-2.1` | <Check size={18} /> | <Check size={18} /> | 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9, 1:8, 8:1, 1:4, 4:1 |
|
|
1968
2110
|
|
|
1969
2111
|
<Note>
|
|
1970
2112
|
`gemini-3-pro-image-preview` supports additional features including up to 14
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ai-sdk/google",
|
|
3
|
-
"version": "3.0.
|
|
3
|
+
"version": "3.0.132",
|
|
4
4
|
"license": "Apache-2.0",
|
|
5
5
|
"sideEffects": false,
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -36,8 +36,8 @@
|
|
|
36
36
|
}
|
|
37
37
|
},
|
|
38
38
|
"dependencies": {
|
|
39
|
-
"@ai-sdk/provider": "
|
|
40
|
-
"@ai-sdk/provider
|
|
39
|
+
"@ai-sdk/provider-utils": "4.0.58",
|
|
40
|
+
"@ai-sdk/provider": "3.0.18"
|
|
41
41
|
},
|
|
42
42
|
"devDependencies": {
|
|
43
43
|
"@types/node": "20.17.24",
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
import type { ParseResult } from '@ai-sdk/provider-utils';
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Concatenates the event streams of a long decode response that spans several
|
|
5
|
+
* requests (Gemini 4 long decode).
|
|
6
|
+
*
|
|
7
|
+
* Events of the current stream are passed through unchanged. When an event
|
|
8
|
+
* signals that the per-request cap was reached while budget remains
|
|
9
|
+
* (`getContinuationToken` returns a token), the next request is sent with that
|
|
10
|
+
* token once the current stream ends, and its events are appended. If the
|
|
11
|
+
* continuation request fails, the error is emitted as a failed parse result
|
|
12
|
+
* and the stream ends.
|
|
13
|
+
*/
|
|
14
|
+
export function continueGoogleStream<T>({
|
|
15
|
+
stream,
|
|
16
|
+
getContinuationToken,
|
|
17
|
+
continueStream,
|
|
18
|
+
}: {
|
|
19
|
+
stream: ReadableStream<ParseResult<T>>;
|
|
20
|
+
getContinuationToken: (value: T) => string | undefined;
|
|
21
|
+
continueStream: (
|
|
22
|
+
continuationToken: string,
|
|
23
|
+
) => Promise<ReadableStream<ParseResult<T>>>;
|
|
24
|
+
}): ReadableStream<ParseResult<T>> {
|
|
25
|
+
let reader = stream.getReader();
|
|
26
|
+
let pendingContinuationToken: string | undefined;
|
|
27
|
+
|
|
28
|
+
return new ReadableStream<ParseResult<T>>({
|
|
29
|
+
async pull(controller) {
|
|
30
|
+
while (true) {
|
|
31
|
+
const { done, value } = await reader.read();
|
|
32
|
+
|
|
33
|
+
if (!done) {
|
|
34
|
+
if (value.success) {
|
|
35
|
+
const token = getContinuationToken(value.value);
|
|
36
|
+
if (token != null) {
|
|
37
|
+
pendingContinuationToken = token;
|
|
38
|
+
}
|
|
39
|
+
}
|
|
40
|
+
controller.enqueue(value);
|
|
41
|
+
return;
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
if (pendingContinuationToken == null) {
|
|
45
|
+
controller.close();
|
|
46
|
+
return;
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
const continuationToken = pendingContinuationToken;
|
|
50
|
+
pendingContinuationToken = undefined;
|
|
51
|
+
|
|
52
|
+
try {
|
|
53
|
+
reader = (await continueStream(continuationToken)).getReader();
|
|
54
|
+
} catch (error) {
|
|
55
|
+
// Surfaced as an `error` stream part, followed by the `finish` part
|
|
56
|
+
// with the last continuation token so the caller can resume.
|
|
57
|
+
controller.enqueue({
|
|
58
|
+
success: false,
|
|
59
|
+
error: error as Extract<
|
|
60
|
+
ParseResult<T>,
|
|
61
|
+
{ success: false }
|
|
62
|
+
>['error'],
|
|
63
|
+
rawValue: undefined,
|
|
64
|
+
});
|
|
65
|
+
controller.close();
|
|
66
|
+
return;
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
},
|
|
70
|
+
|
|
71
|
+
async cancel(reason) {
|
|
72
|
+
await reader.cancel(reason);
|
|
73
|
+
},
|
|
74
|
+
});
|
|
75
|
+
}
|
|
@@ -60,3 +60,78 @@ export function convertGoogleGenerativeAIUsage(
|
|
|
60
60
|
raw: usage,
|
|
61
61
|
};
|
|
62
62
|
}
|
|
63
|
+
|
|
64
|
+
/**
|
|
65
|
+
* Sums the usage metadata of two requests, e.g. the hops of a long decode
|
|
66
|
+
* response that was continued automatically. Non-numeric fields are taken
|
|
67
|
+
* from the later request.
|
|
68
|
+
*/
|
|
69
|
+
export function addGoogleGenerativeAIUsageMetadata<
|
|
70
|
+
T extends GoogleGenerativeAIUsageMetadata,
|
|
71
|
+
>(a: T | undefined, b: T | undefined): T | undefined {
|
|
72
|
+
if (a == null || b == null) {
|
|
73
|
+
return b ?? a;
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
return {
|
|
77
|
+
...a,
|
|
78
|
+
...b,
|
|
79
|
+
promptTokenCount: addCounts(a.promptTokenCount, b.promptTokenCount),
|
|
80
|
+
candidatesTokenCount: addCounts(
|
|
81
|
+
a.candidatesTokenCount,
|
|
82
|
+
b.candidatesTokenCount,
|
|
83
|
+
),
|
|
84
|
+
toolUsePromptTokenCount: addCounts(
|
|
85
|
+
a.toolUsePromptTokenCount,
|
|
86
|
+
b.toolUsePromptTokenCount,
|
|
87
|
+
),
|
|
88
|
+
totalTokenCount: addCounts(a.totalTokenCount, b.totalTokenCount),
|
|
89
|
+
cachedContentTokenCount: addCounts(
|
|
90
|
+
a.cachedContentTokenCount,
|
|
91
|
+
b.cachedContentTokenCount,
|
|
92
|
+
),
|
|
93
|
+
thoughtsTokenCount: addCounts(a.thoughtsTokenCount, b.thoughtsTokenCount),
|
|
94
|
+
promptTokensDetails: addTokenDetails(
|
|
95
|
+
a.promptTokensDetails,
|
|
96
|
+
b.promptTokensDetails,
|
|
97
|
+
),
|
|
98
|
+
cacheTokensDetails: addTokenDetails(
|
|
99
|
+
a.cacheTokensDetails,
|
|
100
|
+
b.cacheTokensDetails,
|
|
101
|
+
),
|
|
102
|
+
candidatesTokensDetails: addTokenDetails(
|
|
103
|
+
a.candidatesTokensDetails,
|
|
104
|
+
b.candidatesTokensDetails,
|
|
105
|
+
),
|
|
106
|
+
toolUsePromptTokensDetails: addTokenDetails(
|
|
107
|
+
a.toolUsePromptTokensDetails,
|
|
108
|
+
b.toolUsePromptTokensDetails,
|
|
109
|
+
),
|
|
110
|
+
};
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
function addCounts(
|
|
114
|
+
a: number | null | undefined,
|
|
115
|
+
b: number | null | undefined,
|
|
116
|
+
): number | null | undefined {
|
|
117
|
+
return a == null && b == null ? (b ?? a) : (a ?? 0) + (b ?? 0);
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
function addTokenDetails(
|
|
121
|
+
a: GoogleGenerativeAITokenDetail[] | null | undefined,
|
|
122
|
+
b: GoogleGenerativeAITokenDetail[] | null | undefined,
|
|
123
|
+
): GoogleGenerativeAITokenDetail[] | null | undefined {
|
|
124
|
+
if (a == null || b == null) {
|
|
125
|
+
return b ?? a;
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
const tokenCounts = new Map<string, number>();
|
|
129
|
+
for (const { modality, tokenCount } of [...a, ...b]) {
|
|
130
|
+
tokenCounts.set(modality, (tokenCounts.get(modality) ?? 0) + tokenCount);
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
return Array.from(tokenCounts, ([modality, tokenCount]) => ({
|
|
134
|
+
modality,
|
|
135
|
+
tokenCount,
|
|
136
|
+
}));
|
|
137
|
+
}
|
|
@@ -27,7 +27,9 @@ import {
|
|
|
27
27
|
type Resolvable,
|
|
28
28
|
} from '@ai-sdk/provider-utils';
|
|
29
29
|
import { z } from 'zod/v4';
|
|
30
|
+
import { continueGoogleStream } from './continue-google-stream';
|
|
30
31
|
import {
|
|
32
|
+
addGoogleGenerativeAIUsageMetadata,
|
|
31
33
|
convertGoogleGenerativeAIUsage,
|
|
32
34
|
type GoogleGenerativeAIUsageMetadata,
|
|
33
35
|
} from './convert-google-generative-ai-usage';
|
|
@@ -376,6 +378,9 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
376
378
|
cachedContent: googleOptions?.cachedContent,
|
|
377
379
|
labels: googleOptions?.labels,
|
|
378
380
|
serviceTier: bodyServiceTier,
|
|
381
|
+
...(googleOptions?.continuationToken != null && {
|
|
382
|
+
continuationToken: googleOptions.continuationToken,
|
|
383
|
+
}),
|
|
379
384
|
},
|
|
380
385
|
warnings: [...warnings, ...toolWarnings],
|
|
381
386
|
providerOptionsName,
|
|
@@ -395,21 +400,60 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
395
400
|
extraHeaders,
|
|
396
401
|
);
|
|
397
402
|
|
|
398
|
-
const
|
|
403
|
+
const generateContent = (body: typeof args) =>
|
|
404
|
+
postJsonToApi({
|
|
405
|
+
url: `${this.config.baseURL}/${getModelPath(
|
|
406
|
+
this.modelId,
|
|
407
|
+
)}:generateContent`,
|
|
408
|
+
headers: mergedHeaders,
|
|
409
|
+
body,
|
|
410
|
+
failedResponseHandler: googleFailedResponseHandler,
|
|
411
|
+
successfulResponseHandler: createJsonResponseHandler(responseSchema),
|
|
412
|
+
abortSignal: options.abortSignal,
|
|
413
|
+
fetch: this.config.fetch,
|
|
414
|
+
});
|
|
415
|
+
|
|
416
|
+
let {
|
|
399
417
|
responseHeaders,
|
|
400
418
|
value: response,
|
|
401
419
|
rawValue: rawResponse,
|
|
402
|
-
} = await
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
420
|
+
} = await generateContent(args);
|
|
421
|
+
|
|
422
|
+
// Gemini 4 long decode: a request that reaches the per-request cap while
|
|
423
|
+
// `maxOutputTokens` budget remains finishes with `CONTINUATION`. Resend
|
|
424
|
+
// the same request with the continuation token (the server tracks the
|
|
425
|
+
// cumulative tokens) and combine the parts and usage of all requests.
|
|
426
|
+
//
|
|
427
|
+
// Usage is summed across requests (matching billing). Each request re-sends
|
|
428
|
+
// the prompt plus all output so far as input, so the first request's
|
|
429
|
+
// prompt token count is kept separately for context window checks.
|
|
430
|
+
const continuedParts: ContentPartSchema[] = [];
|
|
431
|
+
let continuedUsageMetadata: GoogleGenerativeAIUsageMetadata | undefined;
|
|
432
|
+
const firstPromptTokenCount = response.usageMetadata?.promptTokenCount;
|
|
433
|
+
|
|
434
|
+
while (true) {
|
|
435
|
+
const candidate = response.candidates?.[0];
|
|
436
|
+
const continuationToken =
|
|
437
|
+
candidate?.finishReason === 'CONTINUATION'
|
|
438
|
+
? candidate.continuationToken
|
|
439
|
+
: undefined;
|
|
440
|
+
|
|
441
|
+
if (continuationToken == null) {
|
|
442
|
+
break;
|
|
443
|
+
}
|
|
444
|
+
|
|
445
|
+
continuedParts.push(...(candidate?.content?.parts ?? []));
|
|
446
|
+
continuedUsageMetadata = addGoogleGenerativeAIUsageMetadata(
|
|
447
|
+
continuedUsageMetadata,
|
|
448
|
+
response.usageMetadata ?? undefined,
|
|
449
|
+
);
|
|
450
|
+
|
|
451
|
+
({
|
|
452
|
+
responseHeaders,
|
|
453
|
+
value: response,
|
|
454
|
+
rawValue: rawResponse,
|
|
455
|
+
} = await generateContent({ ...args, continuationToken }));
|
|
456
|
+
}
|
|
413
457
|
|
|
414
458
|
const candidate = response.candidates?.[0];
|
|
415
459
|
const promptBlockReason = response.promptFeedback?.blockReason;
|
|
@@ -425,9 +469,12 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
425
469
|
const content: Array<LanguageModelV3Content> = [];
|
|
426
470
|
|
|
427
471
|
// map ordered parts to content:
|
|
428
|
-
const parts = candidate?.content?.parts ?? [];
|
|
472
|
+
const parts = [...continuedParts, ...(candidate?.content?.parts ?? [])];
|
|
429
473
|
|
|
430
|
-
const usageMetadata =
|
|
474
|
+
const usageMetadata = addGoogleGenerativeAIUsageMetadata(
|
|
475
|
+
continuedUsageMetadata,
|
|
476
|
+
response.usageMetadata ?? undefined,
|
|
477
|
+
);
|
|
431
478
|
|
|
432
479
|
// Associates code execution results with their preceding call.
|
|
433
480
|
let lastCodeExecutionToolCallId: string | undefined;
|
|
@@ -600,6 +647,12 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
600
647
|
usageMetadata: usageMetadata ?? null,
|
|
601
648
|
finishMessage: candidate?.finishMessage ?? null,
|
|
602
649
|
serviceTier: usageMetadata?.serviceTier ?? null,
|
|
650
|
+
...(candidate?.continuationToken != null
|
|
651
|
+
? { continuationToken: candidate.continuationToken }
|
|
652
|
+
: {}),
|
|
653
|
+
...(continuedUsageMetadata != null && firstPromptTokenCount != null
|
|
654
|
+
? { initialPromptTokenCount: firstPromptTokenCount }
|
|
655
|
+
: {}),
|
|
603
656
|
} satisfies GoogleGenerativeAIProviderMetadata,
|
|
604
657
|
},
|
|
605
658
|
request: { body: args },
|
|
@@ -624,16 +677,37 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
624
677
|
extraHeaders,
|
|
625
678
|
);
|
|
626
679
|
|
|
627
|
-
const
|
|
628
|
-
|
|
629
|
-
this.
|
|
630
|
-
|
|
631
|
-
|
|
632
|
-
|
|
633
|
-
|
|
634
|
-
|
|
635
|
-
|
|
636
|
-
|
|
680
|
+
const streamGenerateContent = (body: typeof args) =>
|
|
681
|
+
postJsonToApi({
|
|
682
|
+
url: `${this.config.baseURL}/${getModelPath(
|
|
683
|
+
this.modelId,
|
|
684
|
+
)}:streamGenerateContent?alt=sse`,
|
|
685
|
+
headers,
|
|
686
|
+
body,
|
|
687
|
+
failedResponseHandler: googleFailedResponseHandler,
|
|
688
|
+
successfulResponseHandler:
|
|
689
|
+
createEventSourceResponseHandler(chunkSchema),
|
|
690
|
+
abortSignal: options.abortSignal,
|
|
691
|
+
fetch: this.config.fetch,
|
|
692
|
+
});
|
|
693
|
+
|
|
694
|
+
const { responseHeaders, value: firstResponse } =
|
|
695
|
+
await streamGenerateContent(args);
|
|
696
|
+
|
|
697
|
+
// Gemini 4 long decode: when a stream reaches the per-request cap while
|
|
698
|
+
// `maxOutputTokens` budget remains, its last chunk finishes with
|
|
699
|
+
// `CONTINUATION`. Resend the same request with the continuation token and
|
|
700
|
+
// append the new stream, which contains only newly generated content.
|
|
701
|
+
const response = continueGoogleStream({
|
|
702
|
+
stream: firstResponse,
|
|
703
|
+
getContinuationToken: chunk => {
|
|
704
|
+
const candidate = chunk.candidates?.[0];
|
|
705
|
+
return candidate?.finishReason === 'CONTINUATION'
|
|
706
|
+
? (candidate.continuationToken ?? undefined)
|
|
707
|
+
: undefined;
|
|
708
|
+
},
|
|
709
|
+
continueStream: async continuationToken =>
|
|
710
|
+
(await streamGenerateContent({ ...args, continuationToken })).value,
|
|
637
711
|
});
|
|
638
712
|
|
|
639
713
|
let finishReason: LanguageModelV3FinishReason = {
|
|
@@ -641,11 +715,16 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
641
715
|
raw: undefined,
|
|
642
716
|
};
|
|
643
717
|
let usage: GoogleGenerativeAIUsageMetadata | undefined = undefined;
|
|
718
|
+
// Usage of the completed requests of an automatically continued response.
|
|
719
|
+
let continuedUsage: GoogleGenerativeAIUsageMetadata | undefined = undefined;
|
|
720
|
+
// Prompt token count of the first request, for context window checks.
|
|
721
|
+
let initialPromptTokenCount: number | null | undefined = undefined;
|
|
644
722
|
let promptFeedback: PromptFeedbackSchema | null = null;
|
|
645
723
|
let lastGroundingMetadata: GroundingMetadataSchema | null = null;
|
|
646
724
|
let lastUrlContextMetadata: UrlContextMetadataSchema | null = null;
|
|
647
725
|
let lastSafetyRatings: SafetyRatingSchema[] | null = null;
|
|
648
726
|
let lastFinishMessage: string | null = null;
|
|
727
|
+
let lastContinuationToken: string | null = null;
|
|
649
728
|
let confirmedPromptBlockReason: string | undefined;
|
|
650
729
|
|
|
651
730
|
const generateId = this.config.generateId;
|
|
@@ -775,6 +854,9 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
775
854
|
if (candidate.finishMessage != null) {
|
|
776
855
|
lastFinishMessage = candidate.finishMessage;
|
|
777
856
|
}
|
|
857
|
+
if (candidate.continuationToken != null) {
|
|
858
|
+
lastContinuationToken = candidate.continuationToken;
|
|
859
|
+
}
|
|
778
860
|
}
|
|
779
861
|
|
|
780
862
|
// A confirmed prompt block is terminal for generated content, but
|
|
@@ -1162,6 +1244,19 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
1162
1244
|
}
|
|
1163
1245
|
}
|
|
1164
1246
|
|
|
1247
|
+
if (candidate.finishReason === 'CONTINUATION') {
|
|
1248
|
+
// Usage metadata is per request. The next request reports its
|
|
1249
|
+
// own usage, so bank the usage of this one.
|
|
1250
|
+
if (continuedUsage == null) {
|
|
1251
|
+
initialPromptTokenCount = usage?.promptTokenCount;
|
|
1252
|
+
}
|
|
1253
|
+
continuedUsage = addGoogleGenerativeAIUsageMetadata(
|
|
1254
|
+
continuedUsage,
|
|
1255
|
+
usage,
|
|
1256
|
+
);
|
|
1257
|
+
usage = undefined;
|
|
1258
|
+
}
|
|
1259
|
+
|
|
1165
1260
|
if (candidate.finishReason != null) {
|
|
1166
1261
|
finishReason = {
|
|
1167
1262
|
unified: mapGoogleGenerativeAIFinishReason({
|
|
@@ -1174,6 +1269,8 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
1174
1269
|
},
|
|
1175
1270
|
|
|
1176
1271
|
flush(controller) {
|
|
1272
|
+
usage = addGoogleGenerativeAIUsageMetadata(continuedUsage, usage);
|
|
1273
|
+
|
|
1177
1274
|
if (currentTextBlockId !== null) {
|
|
1178
1275
|
controller.enqueue({
|
|
1179
1276
|
type: 'text-end',
|
|
@@ -1200,6 +1297,17 @@ export class GoogleGenerativeAILanguageModel implements LanguageModelV3 {
|
|
|
1200
1297
|
usageMetadata: usage ?? null,
|
|
1201
1298
|
finishMessage: lastFinishMessage,
|
|
1202
1299
|
serviceTier: usage?.serviceTier ?? null,
|
|
1300
|
+
// Only surface the token when the response stopped early
|
|
1301
|
+
// (budget used up, or continuing failed), not for the
|
|
1302
|
+
// checkpoint tokens of a stream that finished normally.
|
|
1303
|
+
...(lastContinuationToken != null &&
|
|
1304
|
+
(finishReason.raw === 'MAX_TOKENS' ||
|
|
1305
|
+
finishReason.raw === 'CONTINUATION')
|
|
1306
|
+
? { continuationToken: lastContinuationToken }
|
|
1307
|
+
: {}),
|
|
1308
|
+
...(initialPromptTokenCount != null
|
|
1309
|
+
? { initialPromptTokenCount }
|
|
1310
|
+
: {}),
|
|
1203
1311
|
} satisfies GoogleGenerativeAIProviderMetadata,
|
|
1204
1312
|
},
|
|
1205
1313
|
});
|
|
@@ -1383,6 +1491,18 @@ export const getGroundingMetadataSchema = () =>
|
|
|
1383
1491
|
title: z.string().nullish(),
|
|
1384
1492
|
text: z.string().nullish(),
|
|
1385
1493
|
fileSearchStore: z.string().nullish(),
|
|
1494
|
+
customMetadata: z
|
|
1495
|
+
.array(
|
|
1496
|
+
z.object({
|
|
1497
|
+
key: z.string(),
|
|
1498
|
+
stringValue: z.string().nullish(),
|
|
1499
|
+
numericValue: z.number().nullish(),
|
|
1500
|
+
stringListValue: z
|
|
1501
|
+
.object({ values: z.array(z.string()).nullish() })
|
|
1502
|
+
.nullish(),
|
|
1503
|
+
}),
|
|
1504
|
+
)
|
|
1505
|
+
.nullish(),
|
|
1386
1506
|
})
|
|
1387
1507
|
.nullish(),
|
|
1388
1508
|
maps: z
|
|
@@ -1559,6 +1679,7 @@ const responseSchema = lazySchema(() =>
|
|
|
1559
1679
|
content: getContentSchema().nullish().or(z.object({}).strict()),
|
|
1560
1680
|
finishReason: z.string().nullish(),
|
|
1561
1681
|
finishMessage: z.string().nullish(),
|
|
1682
|
+
continuationToken: z.string().nullish(),
|
|
1562
1683
|
safetyRatings: z.array(getSafetyRatingSchema()).nullish(),
|
|
1563
1684
|
groundingMetadata: getGroundingMetadataSchema().nullish(),
|
|
1564
1685
|
urlContextMetadata: getUrlContextMetadataSchema().nullish(),
|
|
@@ -1582,6 +1703,10 @@ type CandidateSchema = NonNullable<
|
|
|
1582
1703
|
|
|
1583
1704
|
type ContentSchema = NonNullable<CandidateSchema['content']>;
|
|
1584
1705
|
|
|
1706
|
+
type ContentPartSchema = NonNullable<
|
|
1707
|
+
Extract<ContentSchema, { parts?: unknown }>['parts']
|
|
1708
|
+
>[number];
|
|
1709
|
+
|
|
1585
1710
|
export type GroundingMetadataSchema = NonNullable<
|
|
1586
1711
|
CandidateSchema['groundingMetadata']
|
|
1587
1712
|
>;
|
|
@@ -1618,6 +1743,7 @@ const chunkSchema = lazySchema(() =>
|
|
|
1618
1743
|
content: getContentSchema().nullish(),
|
|
1619
1744
|
finishReason: z.string().nullish(),
|
|
1620
1745
|
finishMessage: z.string().nullish(),
|
|
1746
|
+
continuationToken: z.string().nullish(),
|
|
1621
1747
|
safetyRatings: z.array(getSafetyRatingSchema()).nullish(),
|
|
1622
1748
|
groundingMetadata: getGroundingMetadataSchema().nullish(),
|
|
1623
1749
|
urlContextMetadata: getUrlContextMetadataSchema().nullish(),
|
|
@@ -35,6 +35,8 @@ export type GoogleGenerativeAIModelId =
|
|
|
35
35
|
| 'gemini-3.6-flash'
|
|
36
36
|
| 'gemini-3.7-flash'
|
|
37
37
|
| 'gemini-3.8-flash'
|
|
38
|
+
| 'gemini-4-argon'
|
|
39
|
+
| 'gemini-nano-banana-2.1'
|
|
38
40
|
// latest version
|
|
39
41
|
// https://ai.google.dev/gemini-api/docs/models#latest
|
|
40
42
|
| 'gemini-pro-latest'
|
|
@@ -256,6 +258,17 @@ export const googleLanguageModelOptions = lazySchema(() =>
|
|
|
256
258
|
*/
|
|
257
259
|
serviceTier: z.enum(['standard', 'flex', 'priority']).optional(),
|
|
258
260
|
|
|
261
|
+
/**
|
|
262
|
+
* Optional. Resumes a response that stopped with `MAX_TOKENS` because it
|
|
263
|
+
* used up its `maxOutputTokens` budget (Gemini 4 long decode). Pass the
|
|
264
|
+
* `continuationToken` from the previous response's provider metadata
|
|
265
|
+
* together with the original prompt and a higher `maxOutputTokens` (the
|
|
266
|
+
* budget is cumulative across requests). Requests that reach the
|
|
267
|
+
* per-request cap while budget remains (`CONTINUATION`) are continued
|
|
268
|
+
* automatically. Sent as the `continuationToken` body field.
|
|
269
|
+
*/
|
|
270
|
+
continuationToken: z.string().optional(),
|
|
271
|
+
|
|
259
272
|
/**
|
|
260
273
|
* Optional. Vertex AI only. Sent as the
|
|
261
274
|
* `X-Vertex-AI-LLM-Shared-Request-Type` request header to select a
|