@combycode/llm-sdk 1.6.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +103 -0
- package/dist/bus/hook-map.d.ts +11 -0
- package/dist/index.browser.js +179 -70
- package/dist/index.js +179 -70
- package/dist/llm/moderation/native.d.ts +5 -4
- package/dist/llm/providers/google/constants.d.ts +17 -2
- package/dist/llm/providers/openai/completions.d.ts +9 -2
- package/dist/llm/providers/xai/completions.d.ts +2 -2
- package/dist/llm/providers/xai/media.d.ts +8 -0
- package/dist/llm/types/request.d.ts +8 -0
- package/dist/llm/types/tools.d.ts +10 -1
- package/dist/plugins/media/source-image.d.ts +9 -0
- package/dist/plugins/media/types.d.ts +21 -0
- package/dist/plugins/model-catalog/catalog.d.ts +3 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,108 @@ All notable changes to `@combycode/llm-sdk` are documented here. The format foll
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [1.7.0] - 2026-07-16
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
- **Video extend + edit (xAI grok-imagine-video).** `VideoGenRequest` gains `sourceVideo?: DataSource`
|
|
13
|
+
and `params.videoMode?: 'extend' | 'edit'`. When a source video is present the xAI adapter routes to
|
|
14
|
+
the right endpoint instead of plain generation: `extend` (default) → `POST /v1/videos/extensions`
|
|
15
|
+
(continues from the last frame; takes `duration`, ignores aspect/resolution), `edit` →
|
|
16
|
+
`POST /v1/videos/edits` (prompt + video only). The clip is passed as a public URL, a Files-API id, or
|
|
17
|
+
an inline base64 data-URL. `MediaCapabilities` gains `videoExtension`; `MediaOutput.generateVideo`
|
|
18
|
+
throws if a `sourceVideo` is sent to a provider that doesn't support it (rather than silently
|
|
19
|
+
generating). `generateVideo(req)` is unchanged — the new fields flow through. Verified live end-to-end
|
|
20
|
+
on `grok-imagine-video`: generate → extend → edit all return video bytes. (Note: extend/edit require
|
|
21
|
+
`grok-imagine-video`, not `grok-imagine-video-1.5`, which is generation-only.)
|
|
22
|
+
- **`onMediaProgress` hook.** Long-running async video ops (generate/extend/edit) now emit an
|
|
23
|
+
`onMediaProgress` event once per poll (`{ type, provider, operationId, progress, model }`), so a UI can
|
|
24
|
+
render a progress bar. Progress values confirmed live (0→100).
|
|
25
|
+
- **`RawMediaResult.sourceUrl` + `MediaMeta.sourceUrl`.** Async video results now carry the provider's
|
|
26
|
+
hosted URL, so callers can render or re-submit the asset without holding the bytes.
|
|
27
|
+
- **Unified reasoning visibility.** `ThinkingConfig` gains `visibility: 'full' (default) | 'summary' |
|
|
28
|
+
'hidden'`, mapping to Anthropic `enabled.display`, OpenAI Responses `summary`, and Google
|
|
29
|
+
`includeThoughts` — one knob for "how much reasoning comes back" across providers (best-effort; a
|
|
30
|
+
provider without a middle state degrades `summary` to full). Default `full` = prior behaviour.
|
|
31
|
+
- **OpenAI reasoning execution mode.** `providerOptions.reasoningMode: 'standard' | 'pro'` maps to
|
|
32
|
+
`reasoning.mode` on the OpenAI **Responses** path (chat-completions rejects it). Kept in
|
|
33
|
+
`providerOptions` rather than a first-class knob since only one provider/API honours it.
|
|
34
|
+
- **Google `translationConfig` passthrough.** `providerOptions.translationConfig` forwards to
|
|
35
|
+
`generationConfig.translationConfig` on generateContent (Gemini Developer API; live-verified 200).
|
|
36
|
+
- **OpenAI native moderation blocking.** `providerOptions.moderationPolicy`
|
|
37
|
+
(`{ input?: { mode: 'score'|'block' }, output?: {…} }`) forwards to OpenAI's `moderation.policy` on
|
|
38
|
+
Responses + chat for server-side blocking. Unified moderation stays report-only by design (blocking is
|
|
39
|
+
`moderationGuardrail` at the agent layer); this is an OpenAI-specific opt-in, so it lives in
|
|
40
|
+
`providerOptions`. Live-verified the field is accepted.
|
|
41
|
+
- **OpenAI explicit prompt caching.** `providerOptions.promptCacheOptions`
|
|
42
|
+
(`{ mode: 'implicit'|'explicit', ttl: '30m' }`) forwards to `prompt_cache_options` (gpt-5.6+),
|
|
43
|
+
true-OpenAI only (xai/openrouter inherit the builder and don't emit it). Note: OpenAI caches
|
|
44
|
+
**implicitly by default**, so the unified `cache` config already works there with no config — this
|
|
45
|
+
passthrough is for manual control. (`prompt_cache_retention` is deprecated upstream in favour of
|
|
46
|
+
`prompt_cache_options.ttl`.)
|
|
47
|
+
- **OpenAI programmatic tool calling (Responses).** `BuiltinTool` gains `programmatic_tool_calling`, and
|
|
48
|
+
`FunctionTool` gains `allowedCallers?: ('direct'|'programmatic')[]` + `outputSchema?` — emitted on the
|
|
49
|
+
OpenAI Responses path only. Live-verified on gpt-5.6 (tool calls succeed; older models reject the
|
|
50
|
+
builtin, which is model-gated). (Surfacing the `program`/`program_output` output items is deferred; the
|
|
51
|
+
parser already tolerates them without error.)
|
|
52
|
+
|
|
53
|
+
### Fixed
|
|
54
|
+
- **OpenAI prompt-cache write tokens were dropped.** Both usage parsers hardcoded `cacheWriteTokens: 0`;
|
|
55
|
+
they now read `input_tokens_details.cache_write_tokens` (Responses) / `prompt_tokens_details.
|
|
56
|
+
cache_write_tokens` (chat), so cost accounting no longer under-reports explicit prompt caching.
|
|
57
|
+
- **Google reasoning was live-broken (two paths).** generateContent sent `thinkingLevel`, which the
|
|
58
|
+
Gemini Developer API 400s on **2.5** models (it is 3.x-only) — now routed per series: 2.5 →
|
|
59
|
+
`thinkingBudget` (token count), 3.x → `thinkingLevel`. The Interactions path wrapped `thinking_config`
|
|
60
|
+
(rejected outright) and used uppercase values — it takes `thinking_level` **flat** on `generation_config`
|
|
61
|
+
and **lowercase** (`minimal/low/medium/high`). Both live-verified across gemini-2.5 + 3.5.
|
|
62
|
+
- **Google Interactions rejected sampling penalties.** We emitted `presence_penalty`/`frequency_penalty`
|
|
63
|
+
on the Interactions path, which the API 400s ("Unknown parameter") — upstream removed them from its
|
|
64
|
+
Interactions config. No longer emitted there (still valid on generateContent).
|
|
65
|
+
- **Streamed tool-call id collision on OpenAI-compatible backends.** The chat-completions stream parser
|
|
66
|
+
keyed tool-call fragments by `id ?? ''`, so parallel calls from backends that omit ids (LiteLLM/Bedrock,
|
|
67
|
+
some OpenRouter routes) merged into one. Fragments are now correlated by `index`, with a stable
|
|
68
|
+
`call_<uuid>` synthesized once per index when the backend omits ids.
|
|
69
|
+
- **`content_filter` finish reason was flattened to `stop`/`length`.** The chat-completions stream reason
|
|
70
|
+
map lacked `content_filter`, `AgentLoop` derived every normal-completion finish as `stop`, and the
|
|
71
|
+
Responses parser mapped `status: 'incomplete'` to `length` regardless of `incomplete_details.reason` —
|
|
72
|
+
all discarding the provider's actual reason. Now the stream map, the loop, and the Responses parser
|
|
73
|
+
(reading `incomplete_details.reason`) all surface `content_filter` (and `length`) to consumers — so a
|
|
74
|
+
moderation/safety block is distinguishable from a token cap. (Our loop already terminates on non-tool
|
|
75
|
+
finishes, so it never retry-looped on an empty filtered turn.)
|
|
76
|
+
- **Browser: xAI video result was unusable (CORS).** The generated clip lives on a cross-origin bucket
|
|
77
|
+
(`vidgen.x.ai`) that sends no `Access-Control-Allow-Origin`, so `downloadVideo`'s programmatic
|
|
78
|
+
byte-fetch was blocked in the browser and video generation failed outright. In the browser the adapter
|
|
79
|
+
now returns the hosted URL (via `sourceUrl`) with empty bytes instead of fetching — `<video src>` plays
|
|
80
|
+
it cross-origin without CORS, and it can be re-submitted as a `sourceVideo`. Node/Bun still download the
|
|
81
|
+
bytes. (Node-only tests couldn't surface this; CORS isn't enforced off-browser.)
|
|
82
|
+
- **Google `editImage` aspect ratio / size (same bug, second code path).** The 1.6.1 fix moved
|
|
83
|
+
`aspectRatio` / `imageSize` to `generationConfig.imageConfig` only in `generateImage`; the sibling
|
|
84
|
+
`editImage` method still wrote `generationConfig.responseFormat.image` and so 400'd on any edit that
|
|
85
|
+
passed an aspect ratio or size. Now both image paths use `imageConfig`. Verified live end-to-end:
|
|
86
|
+
generate → edit round-trip both return an image at `16:9` / `2K`. Locked with a unit regression on the
|
|
87
|
+
`editImage` request body.
|
|
88
|
+
- **xAI video generation polled forever after the job finished.** `getVideoStatus` only treated
|
|
89
|
+
`status: "completed"`/`"ready"` (or a `download_url`) as done, but xAI reports terminal success as
|
|
90
|
+
`status: "done"` with the URL under `video.url` — so a finished job kept polling until the wait cap and
|
|
91
|
+
never returned. `downloadVideo` likewise read `download_url`/`url` and missed `video.url` (and duration
|
|
92
|
+
under `video.duration`). Both now read the real `video.*` shape (flat fallbacks kept), and terminal
|
|
93
|
+
`status: "done"`/`"expired"` are handled. Progress is now carried on the processing status. Verified
|
|
94
|
+
live end-to-end (grok-imagine-video-1.5 image-to-video: progress 0→75→100 → downloaded 2.5 MB); locked
|
|
95
|
+
with a unit test replaying the real server payload.
|
|
96
|
+
|
|
97
|
+
## [1.6.1] - 2026-07-13
|
|
98
|
+
|
|
99
|
+
### Fixed
|
|
100
|
+
- **Google gemini-image aspect ratio / size (broken image generation).** The `generateContent` image
|
|
101
|
+
path put `aspectRatio` / `imageSize` under `generationConfig.responseFormat.image`, which the API
|
|
102
|
+
rejects (`Invalid value at 'generation_config.response_format.image.aspect_ratio'`) — so any image
|
|
103
|
+
request that passed an aspect ratio 400'd. They belong under `generationConfig.imageConfig`. Verified
|
|
104
|
+
live: `imageConfig.aspectRatio` returns an image for every ratio. (Regressed into view once the 1.6.0
|
|
105
|
+
catalog started advertising `aspectRatio` media params, so the sandbox began sending it.)
|
|
106
|
+
- **Per-model image sizes in the catalog.** The gemini-image `imageSize` options were a blanket
|
|
107
|
+
`512/1K/2K/4K`, but the models 400 on sizes they don't support. Narrowed per model
|
|
108
|
+
(live-probed + confirmed against the official docs): `gemini-3.1-flash-lite-image` → `1K` only;
|
|
109
|
+
`gemini-3-pro-image` / `nano-banana-pro` → `1K/2K/4K` (no 512); `gemini-3.1-flash-image` keeps all four.
|
|
110
|
+
|
|
9
111
|
## [1.6.0] - 2026-07-11
|
|
10
112
|
|
|
11
113
|
### Added
|
|
@@ -238,6 +340,7 @@ First public release.
|
|
|
238
340
|
- Service tiers end to end (request → bill → cost).
|
|
239
341
|
- Cross-environment: runs on Node, Bun, and the browser. ESM, zero runtime deps.
|
|
240
342
|
|
|
343
|
+
[1.6.1]: https://github.com/combycode/llm-sdk-ts/releases/tag/v1.6.1
|
|
241
344
|
[1.6.0]: https://github.com/combycode/llm-sdk-ts/releases/tag/v1.6.0
|
|
242
345
|
[1.2.0]: https://github.com/combycode/llm-sdk-ts/releases/tag/v1.2.0
|
|
243
346
|
[1.1.0]: https://github.com/combycode/llm-sdk-ts/releases/tag/v1.1.0
|
package/dist/bus/hook-map.d.ts
CHANGED
|
@@ -467,6 +467,16 @@ export interface MediaErrorContext {
|
|
|
467
467
|
/** For async ops (video). */
|
|
468
468
|
operationId?: string;
|
|
469
469
|
}
|
|
470
|
+
/** Progress ping for a long-running async media op (video generation/extend/
|
|
471
|
+
* edit) — emitted once per poll while the job runs, so a UI can show a bar. */
|
|
472
|
+
export interface MediaProgressContext {
|
|
473
|
+
type: 'video';
|
|
474
|
+
provider: string;
|
|
475
|
+
operationId: string;
|
|
476
|
+
/** 0–100 when the provider reports it; omitted otherwise. */
|
|
477
|
+
progress?: number;
|
|
478
|
+
model?: string;
|
|
479
|
+
}
|
|
470
480
|
export interface ServerRequestContext {
|
|
471
481
|
serverId: string;
|
|
472
482
|
requestId: string;
|
|
@@ -581,6 +591,7 @@ export interface HookMap {
|
|
|
581
591
|
onContextMeasure: ContextMeasureContext;
|
|
582
592
|
onMediaGenerated: MediaGeneratedContext;
|
|
583
593
|
onMediaError: MediaErrorContext;
|
|
594
|
+
onMediaProgress: MediaProgressContext;
|
|
584
595
|
onInternalToolCallStart: InternalToolCallStartContext;
|
|
585
596
|
onInternalToolCallComplete: InternalToolCallCompleteContext;
|
|
586
597
|
onInternalToolCallError: InternalToolCallErrorContext;
|
package/dist/index.browser.js
CHANGED
|
@@ -4306,7 +4306,6 @@ var catalog_default2 = {
|
|
|
4306
4306
|
},
|
|
4307
4307
|
imageSize: {
|
|
4308
4308
|
values: [
|
|
4309
|
-
"512",
|
|
4310
4309
|
"1K",
|
|
4311
4310
|
"2K",
|
|
4312
4311
|
"4K"
|
|
@@ -4471,7 +4470,6 @@ var catalog_default2 = {
|
|
|
4471
4470
|
},
|
|
4472
4471
|
imageSize: {
|
|
4473
4472
|
values: [
|
|
4474
|
-
"512",
|
|
4475
4473
|
"1K",
|
|
4476
4474
|
"2K",
|
|
4477
4475
|
"4K"
|
|
@@ -6026,10 +6024,7 @@ var catalog_default2 = {
|
|
|
6026
6024
|
},
|
|
6027
6025
|
imageSize: {
|
|
6028
6026
|
values: [
|
|
6029
|
-
"
|
|
6030
|
-
"1K",
|
|
6031
|
-
"2K",
|
|
6032
|
-
"4K"
|
|
6027
|
+
"1K"
|
|
6033
6028
|
],
|
|
6034
6029
|
default: "1K"
|
|
6035
6030
|
}
|
|
@@ -23496,7 +23491,8 @@ var catalog_default5 = {
|
|
|
23496
23491
|
video: false,
|
|
23497
23492
|
imageGeneration: false,
|
|
23498
23493
|
audioGeneration: false,
|
|
23499
|
-
videoGeneration: true
|
|
23494
|
+
videoGeneration: true,
|
|
23495
|
+
videoExtension: true
|
|
23500
23496
|
},
|
|
23501
23497
|
reasoning: {
|
|
23502
23498
|
supported: false,
|
|
@@ -25179,7 +25175,9 @@ var AnthropicAdapter = class {
|
|
|
25179
25175
|
if (req.thinking.mode === "off") {
|
|
25180
25176
|
} else {
|
|
25181
25177
|
const budget = req.thinking.effort ? ANTHROPIC_THINKING_BUDGETS[req.thinking.effort] ?? DEFAULT_ANTHROPIC_THINKING_BUDGET : DEFAULT_ANTHROPIC_THINKING_BUDGET;
|
|
25182
|
-
|
|
25178
|
+
const thinking = { type: "enabled", budget_tokens: budget };
|
|
25179
|
+
if (req.thinking.visibility === "hidden") thinking.display = "omitted";
|
|
25180
|
+
body.thinking = thinking;
|
|
25183
25181
|
if (body.max_tokens <= budget) body.max_tokens = budget + 1024;
|
|
25184
25182
|
}
|
|
25185
25183
|
}
|
|
@@ -25696,6 +25694,21 @@ var GOOGLE_THINKING_LEVELS = {
|
|
|
25696
25694
|
high: "HIGH",
|
|
25697
25695
|
max: "HIGH"
|
|
25698
25696
|
};
|
|
25697
|
+
var GOOGLE_THINKING_BUDGETS = {
|
|
25698
|
+
low: 2048,
|
|
25699
|
+
medium: 8192,
|
|
25700
|
+
high: 16384,
|
|
25701
|
+
max: 24576
|
|
25702
|
+
};
|
|
25703
|
+
function googleUsesThinkingBudget(model) {
|
|
25704
|
+
return /gemini-2\.5/.test(model);
|
|
25705
|
+
}
|
|
25706
|
+
var GOOGLE_INTERACTION_THINKING_LEVELS = {
|
|
25707
|
+
low: "low",
|
|
25708
|
+
medium: "medium",
|
|
25709
|
+
high: "high",
|
|
25710
|
+
max: "high"
|
|
25711
|
+
};
|
|
25699
25712
|
|
|
25700
25713
|
// src/llm/providers/google/generate.ts
|
|
25701
25714
|
var GoogleAdapter = class {
|
|
@@ -25776,9 +25789,16 @@ var GoogleAdapter = class {
|
|
|
25776
25789
|
body.toolConfig = { functionCallingConfig: { mode } };
|
|
25777
25790
|
}
|
|
25778
25791
|
if (req.thinking && req.thinking.mode !== "off") {
|
|
25779
|
-
|
|
25780
|
-
|
|
25792
|
+
const effort = req.thinking.effort ?? "high";
|
|
25793
|
+
const thinkingConfig = {
|
|
25794
|
+
includeThoughts: req.thinking.visibility !== "hidden"
|
|
25781
25795
|
};
|
|
25796
|
+
if (googleUsesThinkingBudget(req.model)) {
|
|
25797
|
+
thinkingConfig.thinkingBudget = GOOGLE_THINKING_BUDGETS[effort] ?? GOOGLE_THINKING_BUDGETS.high;
|
|
25798
|
+
} else {
|
|
25799
|
+
thinkingConfig.thinkingLevel = GOOGLE_THINKING_LEVELS[effort] ?? "HIGH";
|
|
25800
|
+
}
|
|
25801
|
+
config.thinkingConfig = thinkingConfig;
|
|
25782
25802
|
}
|
|
25783
25803
|
if (req.structured) {
|
|
25784
25804
|
config.responseMimeType = "application/json";
|
|
@@ -25794,6 +25814,9 @@ var GoogleAdapter = class {
|
|
|
25794
25814
|
if (req.providerOptions.imageConfig) {
|
|
25795
25815
|
config.imageConfig = req.providerOptions.imageConfig;
|
|
25796
25816
|
}
|
|
25817
|
+
if (req.providerOptions.translationConfig) {
|
|
25818
|
+
config.translationConfig = req.providerOptions.translationConfig;
|
|
25819
|
+
}
|
|
25797
25820
|
}
|
|
25798
25821
|
return {
|
|
25799
25822
|
body,
|
|
@@ -26144,8 +26167,6 @@ var GoogleInteractionsAdapter = class {
|
|
|
26144
26167
|
if (req.maxTokens) genConfig.max_output_tokens = req.maxTokens;
|
|
26145
26168
|
if (req.temperature !== void 0) genConfig.temperature = req.temperature;
|
|
26146
26169
|
if (req.topP !== void 0) genConfig.top_p = req.topP;
|
|
26147
|
-
if (req.presencePenalty !== void 0) genConfig.presence_penalty = req.presencePenalty;
|
|
26148
|
-
if (req.frequencyPenalty !== void 0) genConfig.frequency_penalty = req.frequencyPenalty;
|
|
26149
26170
|
if (req.stop) genConfig.stop_sequences = req.stop;
|
|
26150
26171
|
if (req.tools?.length) {
|
|
26151
26172
|
body.tools = req.tools.filter(isFunctionTool).map((t) => ({
|
|
@@ -26156,9 +26177,7 @@ var GoogleInteractionsAdapter = class {
|
|
|
26156
26177
|
}));
|
|
26157
26178
|
}
|
|
26158
26179
|
if (req.thinking && req.thinking.mode !== "off") {
|
|
26159
|
-
genConfig.
|
|
26160
|
-
thinking_level: GOOGLE_THINKING_LEVELS[req.thinking.effort ?? "high"] ?? "HIGH"
|
|
26161
|
-
};
|
|
26180
|
+
genConfig.thinking_level = GOOGLE_INTERACTION_THINKING_LEVELS[req.thinking.effort ?? "high"] ?? "high";
|
|
26162
26181
|
}
|
|
26163
26182
|
if (Object.keys(genConfig).length > 0) body.generation_config = genConfig;
|
|
26164
26183
|
const cachedContent = req.providerOptions?.cachedContent;
|
|
@@ -26461,6 +26480,24 @@ function openaiImageRef(ref) {
|
|
|
26461
26480
|
function xaiImageRef(ref) {
|
|
26462
26481
|
return ref.fileId ? { file_id: ref.fileId } : { url: toDataUrl(ref) };
|
|
26463
26482
|
}
|
|
26483
|
+
function xaiVideoRef(src) {
|
|
26484
|
+
switch (src.type) {
|
|
26485
|
+
case "url":
|
|
26486
|
+
return { url: src.url };
|
|
26487
|
+
case "file":
|
|
26488
|
+
return { file_id: src.fileId };
|
|
26489
|
+
case "provider_ref":
|
|
26490
|
+
return { file_id: src.refId };
|
|
26491
|
+
case "base64":
|
|
26492
|
+
return { url: `data:${src.mimeType};base64,${src.data}` };
|
|
26493
|
+
case "buffer":
|
|
26494
|
+
return { url: `data:${src.mimeType};base64,${bytesToBase64(src.data)}` };
|
|
26495
|
+
case "path":
|
|
26496
|
+
throw new Error(
|
|
26497
|
+
"media source video: `path` DataSource is not supported here \u2014 read the file and pass base64/buffer."
|
|
26498
|
+
);
|
|
26499
|
+
}
|
|
26500
|
+
}
|
|
26464
26501
|
function googleImagePart(ref) {
|
|
26465
26502
|
const mimeType = ref.mimeType ?? "image/png";
|
|
26466
26503
|
if (ref.base64) return { inline_data: { mime_type: mimeType, data: ref.base64 } };
|
|
@@ -26547,7 +26584,7 @@ var GoogleMediaAdapter = class {
|
|
|
26547
26584
|
const image = {};
|
|
26548
26585
|
if (req.params?.aspectRatio) image.aspectRatio = req.params.aspectRatio;
|
|
26549
26586
|
if (req.params?.imageSize) image.imageSize = req.params.imageSize;
|
|
26550
|
-
if (Object.keys(image).length) generationConfig.
|
|
26587
|
+
if (Object.keys(image).length) generationConfig.imageConfig = image;
|
|
26551
26588
|
const { items, usage } = await this.generateContentMedia(
|
|
26552
26589
|
model,
|
|
26553
26590
|
req.prompt,
|
|
@@ -26611,7 +26648,7 @@ var GoogleMediaAdapter = class {
|
|
|
26611
26648
|
const image = {};
|
|
26612
26649
|
if (req.params?.aspectRatio) image.aspectRatio = req.params.aspectRatio;
|
|
26613
26650
|
if (req.params?.imageSize) image.imageSize = req.params.imageSize;
|
|
26614
|
-
if (Object.keys(image).length) generationConfig.
|
|
26651
|
+
if (Object.keys(image).length) generationConfig.imageConfig = image;
|
|
26615
26652
|
const imagePart = googleImagePart(normalizeImageSource(req.sourceImage));
|
|
26616
26653
|
const { items, usage } = await this.generateContentMedia(
|
|
26617
26654
|
model,
|
|
@@ -27015,8 +27052,10 @@ var OpenAIBatchAdapter = class {
|
|
|
27015
27052
|
};
|
|
27016
27053
|
|
|
27017
27054
|
// src/llm/moderation/native.ts
|
|
27018
|
-
function buildNativeModeration(mod) {
|
|
27019
|
-
|
|
27055
|
+
function buildNativeModeration(mod, policy) {
|
|
27056
|
+
const out = { model: mod?.model ?? MODERATION_DEFAULT_MODEL };
|
|
27057
|
+
if (policy && typeof policy === "object") out.policy = policy;
|
|
27058
|
+
return out;
|
|
27020
27059
|
}
|
|
27021
27060
|
function parseNativeModeration(raw) {
|
|
27022
27061
|
if (!raw || typeof raw !== "object") return void 0;
|
|
@@ -27126,8 +27165,12 @@ var OpenAIAdapter = class {
|
|
|
27126
27165
|
if (req.stop) body.stop = req.stop;
|
|
27127
27166
|
const tier = openaiRequestTier(req.serviceTier);
|
|
27128
27167
|
if (tier) body.service_tier = tier;
|
|
27129
|
-
|
|
27130
|
-
|
|
27168
|
+
const modPolicy = req.providerOptions?.moderationPolicy;
|
|
27169
|
+
if (req.moderation && req.moderation.mode !== "emulate" || modPolicy) {
|
|
27170
|
+
body.moderation = buildNativeModeration(req.moderation, modPolicy);
|
|
27171
|
+
}
|
|
27172
|
+
if (this.name === "openai" && req.providerOptions?.promptCacheOptions) {
|
|
27173
|
+
body.prompt_cache_options = req.providerOptions.promptCacheOptions;
|
|
27131
27174
|
}
|
|
27132
27175
|
const hasAudioInput = req.messages.some(
|
|
27133
27176
|
(m) => Array.isArray(m.content) && m.content.some((p) => p.type === "audio")
|
|
@@ -27303,7 +27346,7 @@ var OpenAIAdapter = class {
|
|
|
27303
27346
|
raw
|
|
27304
27347
|
};
|
|
27305
27348
|
}
|
|
27306
|
-
parseStreamEvent(event) {
|
|
27349
|
+
parseStreamEvent(event, state) {
|
|
27307
27350
|
const data = JSON.parse(event.data);
|
|
27308
27351
|
if (data.moderation) {
|
|
27309
27352
|
const report = parseNativeModeration(data.moderation);
|
|
@@ -27329,29 +27372,32 @@ var OpenAIAdapter = class {
|
|
|
27329
27372
|
if (delta.content) {
|
|
27330
27373
|
events.push({ type: "text", text: delta.content });
|
|
27331
27374
|
}
|
|
27375
|
+
const toolIdByIndex = state?.toolIdByIndex ?? /* @__PURE__ */ new Map();
|
|
27332
27376
|
const toolCalls = delta.tool_calls ?? [];
|
|
27333
27377
|
for (const tc of toolCalls) {
|
|
27378
|
+
const index = tc.index ?? 0;
|
|
27379
|
+
let id = toolIdByIndex.get(index);
|
|
27380
|
+
if (id === void 0) {
|
|
27381
|
+
id = tc.id || `call_${crypto.randomUUID()}`;
|
|
27382
|
+
toolIdByIndex.set(index, id);
|
|
27383
|
+
}
|
|
27334
27384
|
const fn = tc.function;
|
|
27335
27385
|
if (fn?.name) {
|
|
27336
|
-
events.push({
|
|
27337
|
-
type: "tool_call_start",
|
|
27338
|
-
id: tc.id ?? "",
|
|
27339
|
-
name: fn.name
|
|
27340
|
-
});
|
|
27386
|
+
events.push({ type: "tool_call_start", id, name: fn.name });
|
|
27341
27387
|
}
|
|
27342
27388
|
if (fn?.arguments) {
|
|
27343
|
-
events.push({
|
|
27344
|
-
type: "tool_call_delta",
|
|
27345
|
-
id: tc.id ?? "",
|
|
27346
|
-
arguments: fn.arguments
|
|
27347
|
-
});
|
|
27389
|
+
events.push({ type: "tool_call_delta", id, arguments: fn.arguments });
|
|
27348
27390
|
}
|
|
27349
27391
|
}
|
|
27350
27392
|
const fr = choice.finish_reason;
|
|
27351
27393
|
if (fr) {
|
|
27352
27394
|
events.push({
|
|
27353
27395
|
type: "done",
|
|
27354
|
-
finishReason: extractFinishReason(false, fr, {
|
|
27396
|
+
finishReason: extractFinishReason(false, fr, {
|
|
27397
|
+
tool_calls: "tool_use",
|
|
27398
|
+
length: "length",
|
|
27399
|
+
content_filter: "content_filter"
|
|
27400
|
+
})
|
|
27355
27401
|
});
|
|
27356
27402
|
}
|
|
27357
27403
|
if (data.usage) {
|
|
@@ -27359,9 +27405,11 @@ var OpenAIAdapter = class {
|
|
|
27359
27405
|
}
|
|
27360
27406
|
return events;
|
|
27361
27407
|
}
|
|
27362
|
-
/**
|
|
27408
|
+
/** Per-stream: correlates streamed tool-call fragments by index and synthesizes
|
|
27409
|
+
* a stable id for backends that omit tool-call ids (see `parseStreamEvent`). */
|
|
27363
27410
|
createStreamParser() {
|
|
27364
|
-
|
|
27411
|
+
const state = { toolIdByIndex: /* @__PURE__ */ new Map() };
|
|
27412
|
+
return (event) => this.parseStreamEvent(event, state);
|
|
27365
27413
|
}
|
|
27366
27414
|
parseUsage(u) {
|
|
27367
27415
|
if (!u) return emptyUsage();
|
|
@@ -27374,7 +27422,7 @@ var OpenAIAdapter = class {
|
|
|
27374
27422
|
outputTokens: output,
|
|
27375
27423
|
totalTokens: input + output,
|
|
27376
27424
|
cachedTokens: details.cached_tokens ?? 0,
|
|
27377
|
-
cacheWriteTokens: 0,
|
|
27425
|
+
cacheWriteTokens: details.cache_write_tokens ?? 0,
|
|
27378
27426
|
reasoningTokens: outDetails.reasoning_tokens ?? 0
|
|
27379
27427
|
};
|
|
27380
27428
|
}
|
|
@@ -28020,8 +28068,12 @@ var OpenAIResponsesAdapter = class {
|
|
|
28020
28068
|
if (req.topP !== void 0) body.top_p = req.topP;
|
|
28021
28069
|
const tier = openaiRequestTier(req.serviceTier);
|
|
28022
28070
|
if (tier) body.service_tier = tier;
|
|
28023
|
-
|
|
28024
|
-
|
|
28071
|
+
const modPolicy = req.providerOptions?.moderationPolicy;
|
|
28072
|
+
if (req.moderation && req.moderation.mode !== "emulate" || modPolicy) {
|
|
28073
|
+
body.moderation = buildNativeModeration(req.moderation, modPolicy);
|
|
28074
|
+
}
|
|
28075
|
+
if (this.name === "openai" && req.providerOptions?.promptCacheOptions) {
|
|
28076
|
+
body.prompt_cache_options = req.providerOptions.promptCacheOptions;
|
|
28025
28077
|
}
|
|
28026
28078
|
if (req.tools?.length) {
|
|
28027
28079
|
body.tools = req.tools.map((t) => {
|
|
@@ -28031,7 +28083,10 @@ var OpenAIResponsesAdapter = class {
|
|
|
28031
28083
|
name: t.name,
|
|
28032
28084
|
description: t.description,
|
|
28033
28085
|
parameters: ensureAdditionalProperties(t.parameters),
|
|
28034
|
-
strict: t.strict ?? true
|
|
28086
|
+
strict: t.strict ?? true,
|
|
28087
|
+
// Programmatic tool calling (Responses): who may call it + return schema.
|
|
28088
|
+
...t.allowedCallers ? { allowed_callers: t.allowedCallers } : {},
|
|
28089
|
+
...t.outputSchema ? { output_schema: t.outputSchema } : {}
|
|
28035
28090
|
};
|
|
28036
28091
|
}
|
|
28037
28092
|
const builtin = { type: t.type, ...t.params };
|
|
@@ -28059,9 +28114,13 @@ var OpenAIResponsesAdapter = class {
|
|
|
28059
28114
|
};
|
|
28060
28115
|
}
|
|
28061
28116
|
if (req.thinking && req.thinking.mode !== "off") {
|
|
28117
|
+
const visibility = req.thinking.visibility ?? "full";
|
|
28118
|
+
const summary = visibility === "hidden" ? null : visibility === "summary" ? "concise" : "auto";
|
|
28119
|
+
const mode = req.providerOptions?.reasoningMode;
|
|
28062
28120
|
body.reasoning = {
|
|
28063
28121
|
effort: req.thinking.effort ?? "medium",
|
|
28064
|
-
summary:
|
|
28122
|
+
...summary !== null ? { summary } : {},
|
|
28123
|
+
...mode ? { mode } : {},
|
|
28065
28124
|
// Cross-turn reasoning persistence (gpt-5/o-series, Responses only).
|
|
28066
28125
|
...req.thinking.context ? { context: req.thinking.context } : {}
|
|
28067
28126
|
};
|
|
@@ -28205,9 +28264,8 @@ var OpenAIResponsesAdapter = class {
|
|
|
28205
28264
|
}
|
|
28206
28265
|
}
|
|
28207
28266
|
const status = r.status;
|
|
28208
|
-
const
|
|
28209
|
-
|
|
28210
|
-
});
|
|
28267
|
+
const incompleteReason = r.incomplete_details?.reason;
|
|
28268
|
+
const finishReason = incompleteReason === "content_filter" ? "content_filter" : extractFinishReason(toolCalls.length > 0, status, { incomplete: "length" });
|
|
28211
28269
|
if (!text && typeof r.output_text === "string") {
|
|
28212
28270
|
text = r.output_text;
|
|
28213
28271
|
if (text && content.length === 0) content.push({ type: "text", text });
|
|
@@ -28327,7 +28385,7 @@ var OpenAIResponsesAdapter = class {
|
|
|
28327
28385
|
outputTokens: output,
|
|
28328
28386
|
totalTokens: u.total_tokens ?? input + output,
|
|
28329
28387
|
cachedTokens: inputDetails.cached_tokens ?? 0,
|
|
28330
|
-
cacheWriteTokens: 0,
|
|
28388
|
+
cacheWriteTokens: inputDetails.cache_write_tokens ?? 0,
|
|
28331
28389
|
reasoningTokens: outputDetails.reasoning_tokens ?? 0
|
|
28332
28390
|
};
|
|
28333
28391
|
}
|
|
@@ -28417,8 +28475,9 @@ var OpenRouterAdapter = class extends OpenAIAdapter {
|
|
|
28417
28475
|
* `url_citation` annotations appear in the stream (the `:online` search signal). */
|
|
28418
28476
|
createStreamParser() {
|
|
28419
28477
|
let webSearchEmitted = false;
|
|
28478
|
+
const state = { toolIdByIndex: /* @__PURE__ */ new Map() };
|
|
28420
28479
|
return (event) => {
|
|
28421
|
-
const events = this.parseStreamEvent(event);
|
|
28480
|
+
const events = this.parseStreamEvent(event, state);
|
|
28422
28481
|
if (!webSearchEmitted) {
|
|
28423
28482
|
const choice = JSON.parse(event.data).choices?.[0];
|
|
28424
28483
|
const annotations = choice?.delta?.annotations ?? choice?.message?.annotations;
|
|
@@ -28589,8 +28648,8 @@ var XAIAdapter = class extends OpenAIAdapter {
|
|
|
28589
28648
|
}
|
|
28590
28649
|
return result;
|
|
28591
28650
|
}
|
|
28592
|
-
parseStreamEvent(event) {
|
|
28593
|
-
const events = super.parseStreamEvent(event);
|
|
28651
|
+
parseStreamEvent(event, state) {
|
|
28652
|
+
const events = super.parseStreamEvent(event, state);
|
|
28594
28653
|
try {
|
|
28595
28654
|
const data = JSON.parse(event.data);
|
|
28596
28655
|
const choices = data.choices ?? [];
|
|
@@ -28713,7 +28772,8 @@ var XAIMediaAdapter = class {
|
|
|
28713
28772
|
imageEditing: true,
|
|
28714
28773
|
audioGeneration: true,
|
|
28715
28774
|
videoGeneration: true,
|
|
28716
|
-
audioStreaming: true
|
|
28775
|
+
audioStreaming: true,
|
|
28776
|
+
videoExtension: true
|
|
28717
28777
|
};
|
|
28718
28778
|
}
|
|
28719
28779
|
authHeaders() {
|
|
@@ -28827,13 +28887,9 @@ var XAIMediaAdapter = class {
|
|
|
28827
28887
|
}
|
|
28828
28888
|
async submitVideo(req, fetch2) {
|
|
28829
28889
|
const model = req.model ?? "grok-imagine-video";
|
|
28830
|
-
const
|
|
28831
|
-
if (req.params?.duration) body.duration = req.params.duration;
|
|
28832
|
-
if (req.params?.aspectRatio) body.aspect_ratio = req.params.aspectRatio;
|
|
28833
|
-
if (req.params?.resolution) body.resolution = req.params.resolution;
|
|
28834
|
-
if (req.sourceImage) body.image = xaiImageRef(normalizeImageSource(req.sourceImage));
|
|
28890
|
+
const { url, body } = this.buildVideoSubmit(req, model);
|
|
28835
28891
|
const res = await fetch2({
|
|
28836
|
-
url
|
|
28892
|
+
url,
|
|
28837
28893
|
method: "POST",
|
|
28838
28894
|
headers: this.authHeaders(),
|
|
28839
28895
|
body,
|
|
@@ -28844,6 +28900,30 @@ var XAIMediaAdapter = class {
|
|
|
28844
28900
|
const data = res.body;
|
|
28845
28901
|
return data.request_id ?? data.id ?? "";
|
|
28846
28902
|
}
|
|
28903
|
+
/** Route a video request to the right xAI endpoint by input + mode:
|
|
28904
|
+
* - no `sourceVideo` → `/v1/videos/generations` (text/image-to-video)
|
|
28905
|
+
* - `sourceVideo` + `videoMode:'extend'` (default) → `/v1/videos/extensions`
|
|
28906
|
+
* — continues from the last frame; takes `duration`, NOT aspect/resolution.
|
|
28907
|
+
* - `sourceVideo` + `videoMode:'edit'` → `/v1/videos/edits` — prompt + video
|
|
28908
|
+
* only (no duration/aspect/resolution).
|
|
28909
|
+
* All three return a `request_id` polled via the same status endpoint. */
|
|
28910
|
+
buildVideoSubmit(req, model) {
|
|
28911
|
+
if (req.sourceVideo) {
|
|
28912
|
+
const video = xaiVideoRef(req.sourceVideo);
|
|
28913
|
+
if ((req.params?.videoMode ?? "extend") === "edit") {
|
|
28914
|
+
return { url: `${this.baseURL}/v1/videos/edits`, body: { model, prompt: req.prompt, video } };
|
|
28915
|
+
}
|
|
28916
|
+
const body2 = { model, prompt: req.prompt, video };
|
|
28917
|
+
if (req.params?.duration) body2.duration = req.params.duration;
|
|
28918
|
+
return { url: `${this.baseURL}/v1/videos/extensions`, body: body2 };
|
|
28919
|
+
}
|
|
28920
|
+
const body = { model, prompt: req.prompt };
|
|
28921
|
+
if (req.params?.duration) body.duration = req.params.duration;
|
|
28922
|
+
if (req.params?.aspectRatio) body.aspect_ratio = req.params.aspectRatio;
|
|
28923
|
+
if (req.params?.resolution) body.resolution = req.params.resolution;
|
|
28924
|
+
if (req.sourceImage) body.image = xaiImageRef(normalizeImageSource(req.sourceImage));
|
|
28925
|
+
return { url: `${this.baseURL}/v1/videos/generations`, body };
|
|
28926
|
+
}
|
|
28847
28927
|
async getVideoStatus(operationId, fetch2) {
|
|
28848
28928
|
const res = await fetch2({
|
|
28849
28929
|
url: `${this.baseURL}/v1/videos/${operationId}`,
|
|
@@ -28857,13 +28937,15 @@ var XAIMediaAdapter = class {
|
|
|
28857
28937
|
if (res.status >= 400) return { status: "failed", error: `HTTP ${res.status}` };
|
|
28858
28938
|
const data = res.body;
|
|
28859
28939
|
const state = data.status ?? "";
|
|
28860
|
-
|
|
28861
|
-
|
|
28940
|
+
const video = data.video;
|
|
28941
|
+
const progress = data.progress;
|
|
28942
|
+
if (state === "done" || state === "completed" || state === "ready" || video?.url || data.download_url) {
|
|
28943
|
+
return { status: "completed", progress };
|
|
28862
28944
|
}
|
|
28863
|
-
if (state === "failed" || state === "error") {
|
|
28945
|
+
if (state === "failed" || state === "error" || state === "expired") {
|
|
28864
28946
|
return { status: "failed", error: data.error ?? "Unknown error" };
|
|
28865
28947
|
}
|
|
28866
|
-
return { status: "processing", progress
|
|
28948
|
+
return { status: "processing", progress };
|
|
28867
28949
|
}
|
|
28868
28950
|
async downloadVideo(operationId, fetch2) {
|
|
28869
28951
|
const statusRes = await fetch2({
|
|
@@ -28879,8 +28961,19 @@ var XAIMediaAdapter = class {
|
|
|
28879
28961
|
throw new Error(`xAI video download failed: HTTP ${statusRes.status}`);
|
|
28880
28962
|
}
|
|
28881
28963
|
const data = statusRes.body;
|
|
28882
|
-
const
|
|
28964
|
+
const video = data.video;
|
|
28965
|
+
const downloadUrl = video?.url ?? data.download_url ?? data.url;
|
|
28883
28966
|
if (!downloadUrl) throw new Error("No download URL in video response");
|
|
28967
|
+
const durationSec = video?.duration ?? data.duration;
|
|
28968
|
+
const base = {
|
|
28969
|
+
data: new Uint8Array(0),
|
|
28970
|
+
mimeType: "video/mp4",
|
|
28971
|
+
sourceUrl: downloadUrl,
|
|
28972
|
+
durationMs: durationSec ? durationSec * 1e3 : void 0,
|
|
28973
|
+
// Provider-reported cost (usage.cost_in_usd_ticks), when present.
|
|
28974
|
+
providerMeta: data.usage ? { usage: data.usage } : void 0
|
|
28975
|
+
};
|
|
28976
|
+
if (isBrowser()) return base;
|
|
28884
28977
|
const videoRes = await fetch2({
|
|
28885
28978
|
url: downloadUrl,
|
|
28886
28979
|
method: "GET",
|
|
@@ -28890,13 +28983,7 @@ var XAIMediaAdapter = class {
|
|
|
28890
28983
|
model: "",
|
|
28891
28984
|
responseType: "arraybuffer"
|
|
28892
28985
|
});
|
|
28893
|
-
return {
|
|
28894
|
-
data: videoRes.body,
|
|
28895
|
-
mimeType: "video/mp4",
|
|
28896
|
-
durationMs: data.duration ? data.duration * 1e3 : void 0,
|
|
28897
|
-
// Provider-reported cost (usage.cost_in_usd_ticks), when present.
|
|
28898
|
-
providerMeta: data.usage ? { usage: data.usage } : void 0
|
|
28899
|
-
};
|
|
28986
|
+
return { ...base, data: videoRes.body };
|
|
28900
28987
|
}
|
|
28901
28988
|
async cancelVideo(operationId, fetch2) {
|
|
28902
28989
|
await fetch2({
|
|
@@ -30309,7 +30396,11 @@ var AgentLoop = class _AgentLoop {
|
|
|
30309
30396
|
id: lastResponse?.id ?? `agent-${runId}`,
|
|
30310
30397
|
model: this.client.model,
|
|
30311
30398
|
content: finalContent,
|
|
30312
|
-
finishReason: reason === "done" ?
|
|
30399
|
+
finishReason: reason === "done" ? (
|
|
30400
|
+
// Ended because the model requested no tools — surface the provider's
|
|
30401
|
+
// actual reason (stop / content_filter / length), not a flat 'stop'.
|
|
30402
|
+
lastResponse?.finishReason ?? "stop"
|
|
30403
|
+
) : reason === "stopped" ? "stop" : reason === "guardrail" ? "stop" : reason === "max_steps" ? "length" : "error",
|
|
30313
30404
|
usage: totalUsage,
|
|
30314
30405
|
text: finalText,
|
|
30315
30406
|
toolCalls: lastResponse?.toolCalls ?? [],
|
|
@@ -30518,7 +30609,11 @@ var AgentLoop = class _AgentLoop {
|
|
|
30518
30609
|
id: lastResponse?.id ?? `agent-${runId}`,
|
|
30519
30610
|
model: this.client.model,
|
|
30520
30611
|
content: finalContent,
|
|
30521
|
-
finishReason: reason === "done" ?
|
|
30612
|
+
finishReason: reason === "done" ? (
|
|
30613
|
+
// Ended because the model requested no tools — surface the provider's
|
|
30614
|
+
// actual reason (stop / content_filter / length), not a flat 'stop'.
|
|
30615
|
+
lastResponse?.finishReason ?? "stop"
|
|
30616
|
+
) : reason === "stopped" ? "stop" : reason === "guardrail" ? "stop" : reason === "max_steps" ? "length" : "error",
|
|
30522
30617
|
usage: totalUsage,
|
|
30523
30618
|
text: finalText,
|
|
30524
30619
|
toolCalls: lastResponse?.toolCalls ?? [],
|
|
@@ -34843,9 +34938,13 @@ var MediaOutput = class {
|
|
|
34843
34938
|
}
|
|
34844
34939
|
async generateVideo(req) {
|
|
34845
34940
|
const adapter = this.getAdapter(req.provider);
|
|
34846
|
-
|
|
34941
|
+
const caps = adapter.capabilities();
|
|
34942
|
+
if (!caps.videoGeneration || !adapter.submitVideo) {
|
|
34847
34943
|
throw new Error(`Provider ${req.provider} does not support video generation`);
|
|
34848
34944
|
}
|
|
34945
|
+
if (req.sourceVideo && !caps.videoExtension) {
|
|
34946
|
+
throw new Error(`Provider ${req.provider} does not support video extension/editing`);
|
|
34947
|
+
}
|
|
34849
34948
|
const { trace, fetch: fetch2 } = this.tracedOp();
|
|
34850
34949
|
const operationId = await adapter.submitVideo(req, fetch2);
|
|
34851
34950
|
return this.pollVideoCompletion(adapter, operationId, req, fetch2, trace);
|
|
@@ -34878,7 +34977,8 @@ var MediaOutput = class {
|
|
|
34878
34977
|
width: raw.width,
|
|
34879
34978
|
height: raw.height,
|
|
34880
34979
|
durationMs: raw.durationMs,
|
|
34881
|
-
sampleRate: raw.sampleRate
|
|
34980
|
+
sampleRate: raw.sampleRate,
|
|
34981
|
+
sourceUrl: raw.sourceUrl
|
|
34882
34982
|
};
|
|
34883
34983
|
await this.mediaStore.save(id, raw.data, meta);
|
|
34884
34984
|
results.push({ id, type, mimeType: raw.mimeType, meta });
|
|
@@ -34911,6 +35011,15 @@ var MediaOutput = class {
|
|
|
34911
35011
|
const start = Date.now();
|
|
34912
35012
|
while (Date.now() - start < this.maxPollWaitMs) {
|
|
34913
35013
|
const status = await adapter.getVideoStatus(operationId, fetch2);
|
|
35014
|
+
if (status.status === "processing" || status.status === "pending") {
|
|
35015
|
+
await this.hooks.emit("onMediaProgress", {
|
|
35016
|
+
type: "video",
|
|
35017
|
+
provider: req.provider,
|
|
35018
|
+
operationId,
|
|
35019
|
+
progress: status.progress,
|
|
35020
|
+
model: req.model
|
|
35021
|
+
});
|
|
35022
|
+
}
|
|
34914
35023
|
if (status.status === "completed") {
|
|
34915
35024
|
const raw = await adapter.downloadVideo(operationId, fetch2);
|
|
34916
35025
|
const results = await this.saveResults(
|