@warlock.js/ai-google 4.14.0 → 4.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -1
- package/cjs/index.cjs +309 -37
- package/cjs/index.cjs.map +1 -1
- package/esm/config.type.d.mts +11 -5
- package/esm/config.type.d.mts.map +1 -1
- package/esm/gemini-image.d.mts +91 -0
- package/esm/gemini-image.d.mts.map +1 -0
- package/esm/gemini-image.mjs +231 -0
- package/esm/gemini-image.mjs.map +1 -0
- package/esm/index.d.mts +2 -1
- package/esm/index.mjs +2 -1
- package/esm/model.mjs +7 -25
- package/esm/model.mjs.map +1 -1
- package/esm/sdk.d.mts +16 -8
- package/esm/sdk.d.mts.map +1 -1
- package/esm/sdk.mjs +48 -8
- package/esm/sdk.mjs.map +1 -1
- package/esm/utils/apply-google-usage.mjs +30 -0
- package/esm/utils/apply-google-usage.mjs.map +1 -0
- package/esm/utils/index.mjs +1 -0
- package/llms-full.txt +50 -6
- package/llms.txt +1 -1
- package/package.json +6 -6
- package/skills/setup-google/SKILL.md +50 -6
package/CHANGELOG.md
CHANGED
|
@@ -4,7 +4,32 @@ All notable changes to `@warlock.js/ai-google` are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `@warlock.js/*` packages are released in lockstep — every package shares the same version number, so a version below may list only the changes that affected this package.
|
|
6
6
|
|
|
7
|
-
## 4.
|
|
7
|
+
## 4.15.0
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **`GeminiImageModel` — a Gemini-native image path over `ai.models.generateContent`** (new `src/gemini-image.ts`, exported as `GeminiImageModel`). Requests `responseModalities: ["TEXT", "IMAGE"]` (override the list verbatim with `options.responseModalities`), maps `aspectRatio` / `imageSize` / `personGeneration` onto Gemini's `config.imageConfig`, and reshapes inline image parts that come back into the **same** `GeneratedImage[]` (`{ type: "base64", base64, mediaType }`, `image/png` fallback) the Imagen path emits — so `ai.image()`'s envelope is unchanged for callers
|
|
12
|
+
- **Token usage is passed through on the Gemini image path instead of hard-zeroed.** Whatever `usageMetadata` Google attaches becomes `usage.input` / `output` / `total` (plus `cachedTokens` / `reasoningTokens` when reported `> 0`); only an absent block collapses to zeros. The Imagen path stays a flat zero because Imagen reports no tokens at all. Price these models with `{ input, output }` rather than `{ perImage }`, and check the first live `usage` — whether these models report tokens is not confirmed here. The mapping is now a shared `applyGoogleUsage` util used by both the chat model and the image model, so one rule decides what a Gemini token report means package-wide
|
|
13
|
+
- A response with **no image part is never a silent empty success**: a blocked prompt (`promptFeedback.blockReason`) or a safety/policy `finishReason` (`SAFETY`, `IMAGE_SAFETY`, `PROHIBITED_CONTENT`, `IMAGE_PROHIBITED_CONTENT`, `RECITATION`, `IMAGE_RECITATION`, `BLOCKLIST`, `SPII`) throws `ContentFilterError` carrying the reason; a text-only answer throws `ProviderError` **quoting the text the model returned**; anything else throws `ProviderError` naming the part count and finish reason
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
- **`google.image({ name: "gemini-…" })` no longer hits the endpoint that 404s it.** `ai.models.generateImages` routes to `{model}:predict` (`generateImages` → `generateImagesInternal` → `formatMap('{model}:predict', …)` in `@google/genai`'s bundle), which does not serve the Gemini image models — the call came back `404 models/… is not found for API version v1beta, or is not supported for predict`. `GoogleSDK.image()` now picks the transport from the id: a `gemini-` id (with an optional `models/` resource prefix) gets the new `generateContent` implementation, everything else keeps `GoogleImageModel` / `generateImages`. **Scope of the proof:** two levels. Measured here — on the new transport such an id got as far as a quota error (HTTP 429) instead of the 404, which establishes that the endpoint accepts the id. Reported by the maintainer — once billing was enabled on the project, the path returned an image end-to-end from an application running a locally linked build of this package. No test in this package calls Google; the suite proves the request shape and the error mapping, not the round trip
|
|
18
|
+
|
|
19
|
+
### Deprecated
|
|
20
|
+
|
|
21
|
+
- **Google has deprecated `generateImages`, the transport the `imagen-*` path still uses.** Verbatim from the `@google/genai` runtime warning: *"The generateImages method is deprecated and will be removed in the next major release (not before Jan. 1 2027). Please use the generateContent method with image models instead. See https://ai.google.dev/gemini-api/docs/deprecations#imagen-models and https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/image-generation#generate-images"* (`editImage` carries the same notice.) Nothing breaks today and the Imagen path is unchanged, but it is on a clock: new image work should prefer a `gemini-` id. The warning is emitted by `@google/genai` ≥ 2.17; with the bump below, this package now prints it whenever the `imagen-*` path is used
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
|
|
25
|
+
- **`@google/genai` moves from `^2.4.0` to `^2.17.1`** (2.17.1 is what installs today). The Gemini image path does not depend on the bump — `models.generateContent` exists in both — but the older range predates the deprecation notice above and predates `ai.interactions`, so staying on it meant documenting an SDK surface the package could not reach. The 11 suites / 149 specs in this package pass unchanged on 2.17.1. Note this re-resolved the whole workspace lockfile, not just this package's dependency
|
|
26
|
+
- `GoogleSDK.image()` returns `GeminiImageModel` for a `gemini-` id. This is **routing, not validation** — no id is rejected locally: an id matching neither family takes the `generateImages` route, the only route that existed before, so every id that reached Google before still reaches Google the same way and still fails (or succeeds) at the provider
|
|
27
|
+
|
|
28
|
+
### Not included
|
|
29
|
+
|
|
30
|
+
- **The `interactions` API is not used.** `@google/genai` ≥ 2.17 adds `ai.interactions.create({ model, input, response_format: { type: "image", … } })` with images at `interaction.output_image.data` and a different snake_case usage shape (`total_input_tokens` …). It is reachable now that the SDK is on 2.17.1, but nothing in this package calls it: it would need its own usage mapper and its own error surface, and its own request type already marks `response_modalities` / `response_mime_type` deprecated. If it lands it will be an **opt-in config flag**, not id routing
|
|
31
|
+
|
|
32
|
+
## 4.14.0
|
|
8
33
|
|
|
9
34
|
### Removed
|
|
10
35
|
|
package/cjs/index.cjs
CHANGED
|
@@ -3,6 +3,34 @@ let _google_genai = require("@google/genai");
|
|
|
3
3
|
let _warlock_js_ai = require("@warlock.js/ai");
|
|
4
4
|
let _warlock_js_logger = require("@warlock.js/logger");
|
|
5
5
|
|
|
6
|
+
//#region ../ai-google/src/utils/apply-google-usage.ts
|
|
7
|
+
/**
|
|
8
|
+
* Fold a Gemini `usageMetadata` block into a running neutral `Usage`
|
|
9
|
+
* accumulator. Shared by every `generateContent`-backed surface — the
|
|
10
|
+
* chat model's `complete()`, its streaming loop (where the final chunk
|
|
11
|
+
* carries cumulative totals), and the Gemini image model — so one
|
|
12
|
+
* mapping decides what a Gemini token report means package-wide.
|
|
13
|
+
*
|
|
14
|
+
* Cache-read hits (`cachedContentTokenCount`, implicit or explicit
|
|
15
|
+
* context caching) surface as `cachedTokens`; the thinking-phase tokens
|
|
16
|
+
* of a reasoning model (`thoughtsTokenCount`) surface as
|
|
17
|
+
* `reasoningTokens`. Both are emitted only when reported `> 0` so an
|
|
18
|
+
* absent channel leaves the field undefined rather than a false zero.
|
|
19
|
+
*
|
|
20
|
+
* `total` falls back to `input + output` when Google omits
|
|
21
|
+
* `totalTokenCount`.
|
|
22
|
+
*/
|
|
23
|
+
function applyGoogleUsage(usage, raw) {
|
|
24
|
+
usage.input = raw.promptTokenCount ?? usage.input;
|
|
25
|
+
usage.output = raw.candidatesTokenCount ?? usage.output;
|
|
26
|
+
usage.total = raw.totalTokenCount ?? usage.input + usage.output;
|
|
27
|
+
const cached = raw.cachedContentTokenCount;
|
|
28
|
+
if (cached && cached > 0) usage.cachedTokens = cached;
|
|
29
|
+
const reasoning = raw.thoughtsTokenCount;
|
|
30
|
+
if (reasoning && reasoning > 0) usage.reasoningTokens = reasoning;
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
//#endregion
|
|
6
34
|
//#region ../ai-google/src/utils/map-finish-reason.ts
|
|
7
35
|
const finishReasonMap = {
|
|
8
36
|
STOP: "stop",
|
|
@@ -301,7 +329,7 @@ function buildContext(shape) {
|
|
|
301
329
|
|
|
302
330
|
//#endregion
|
|
303
331
|
//#region ../ai-google/src/embedder.ts
|
|
304
|
-
const LOG_MODULE$
|
|
332
|
+
const LOG_MODULE$3 = "ai.google";
|
|
305
333
|
/**
|
|
306
334
|
* Token usage is not returned by Gemini's `embedContent`, so every
|
|
307
335
|
* embedding result reports a zeroed `EmbeddingUsage` (honest absence,
|
|
@@ -367,7 +395,7 @@ var GoogleEmbedder = class {
|
|
|
367
395
|
* and return the raw vectors in input order.
|
|
368
396
|
*/
|
|
369
397
|
async request(inputs) {
|
|
370
|
-
this.logger.debug(LOG_MODULE$
|
|
398
|
+
this.logger.debug(LOG_MODULE$3, "embedder.request", "embedContent", {
|
|
371
399
|
model: this.name,
|
|
372
400
|
count: inputs.length
|
|
373
401
|
});
|
|
@@ -380,7 +408,7 @@ var GoogleEmbedder = class {
|
|
|
380
408
|
});
|
|
381
409
|
} catch (thrown) {
|
|
382
410
|
const wrapped = wrapGoogleError(thrown);
|
|
383
|
-
this.logger.error(LOG_MODULE$
|
|
411
|
+
this.logger.error(LOG_MODULE$3, "embedder.error", wrapped.message, {
|
|
384
412
|
code: wrapped.code,
|
|
385
413
|
context: wrapped.context
|
|
386
414
|
});
|
|
@@ -388,7 +416,7 @@ var GoogleEmbedder = class {
|
|
|
388
416
|
}
|
|
389
417
|
const vectors = (response.embeddings ?? []).map((embedding) => embedding.values ?? []);
|
|
390
418
|
if (this.dimensions === 0 && vectors[0]) this.dimensions = vectors[0].length;
|
|
391
|
-
this.logger.debug(LOG_MODULE$
|
|
419
|
+
this.logger.debug(LOG_MODULE$3, "embedder.response", "embedContent returned", {
|
|
392
420
|
count: vectors.length,
|
|
393
421
|
dimensions: this.dimensions
|
|
394
422
|
});
|
|
@@ -396,6 +424,229 @@ var GoogleEmbedder = class {
|
|
|
396
424
|
}
|
|
397
425
|
};
|
|
398
426
|
|
|
427
|
+
//#endregion
|
|
428
|
+
//#region ../ai-google/src/gemini-image.ts
|
|
429
|
+
const LOG_MODULE$2 = "ai.google";
|
|
430
|
+
/**
|
|
431
|
+
* Response modalities requested when the caller names none.
|
|
432
|
+
*
|
|
433
|
+
* `IMAGE` is the modality this adapter extracts; `TEXT` rides along so
|
|
434
|
+
* a model that narrates what it drew is not answering outside the set
|
|
435
|
+
* it was granted (the narration is then dropped — only inline image
|
|
436
|
+
* parts become `GeneratedImage`s).
|
|
437
|
+
*
|
|
438
|
+
* *Unverified:* which pairing any individual Gemini image model
|
|
439
|
+
* requires is not established here — no spec or run in this package
|
|
440
|
+
* touches the live API. `options.responseModalities` replaces this list
|
|
441
|
+
* verbatim for a model that wants something else.
|
|
442
|
+
*/
|
|
443
|
+
const DEFAULT_RESPONSE_MODALITIES = ["TEXT", "IMAGE"];
|
|
444
|
+
/**
|
|
445
|
+
* Gemini `finishReason` values that mean generation was stopped by a
|
|
446
|
+
* safety / policy rule rather than by the model simply not drawing.
|
|
447
|
+
* Taken from the `FinishReason` enum in `@google/genai`'s own type
|
|
448
|
+
* declarations, whose doc comments describe each of these as content
|
|
449
|
+
* or image generation being "stopped" for safety, prohibited content,
|
|
450
|
+
* recitation, blocklist, or SPII.
|
|
451
|
+
*/
|
|
452
|
+
const FILTERED_FINISH_REASONS = new Set([
|
|
453
|
+
"SAFETY",
|
|
454
|
+
"IMAGE_SAFETY",
|
|
455
|
+
"PROHIBITED_CONTENT",
|
|
456
|
+
"IMAGE_PROHIBITED_CONTENT",
|
|
457
|
+
"RECITATION",
|
|
458
|
+
"IMAGE_RECITATION",
|
|
459
|
+
"BLOCKLIST",
|
|
460
|
+
"SPII"
|
|
461
|
+
]);
|
|
462
|
+
/** How much of a text-only answer to quote back inside the error message. */
|
|
463
|
+
const TEXT_EXCERPT_LIMIT = 200;
|
|
464
|
+
/**
|
|
465
|
+
* Gemini-native implementation of `ImageModelContract`, via
|
|
466
|
+
* `ai.models.generateContent` with `config.responseModalities`
|
|
467
|
+
* including `"IMAGE"`.
|
|
468
|
+
*
|
|
469
|
+
* **Why a second image adapter.** `GoogleImageModel` calls
|
|
470
|
+
* `ai.models.generateImages`, which the `@google/genai` bundle routes
|
|
471
|
+
* to `{model}:predict` (`generateImages` → `generateImagesInternal` →
|
|
472
|
+
* `formatMap('{model}:predict', …)`). A Gemini image model is not
|
|
473
|
+
* served there: asking for one returns Google's
|
|
474
|
+
* `404 … is not found for API version v1beta, or is not supported for
|
|
475
|
+
* predict`. `generateContent` is the SDK's own named replacement — its
|
|
476
|
+
* runtime deprecation notice for `generateImages` reads "Please use the
|
|
477
|
+
* generateContent method with image models instead" — so that is the
|
|
478
|
+
* transport this class speaks, hence a separate class rather than a
|
|
479
|
+
* branch inside `image.ts`.
|
|
480
|
+
*
|
|
481
|
+
* **Same envelope.** Inline image parts are mapped to the identical
|
|
482
|
+
* `GeneratedImage[]` shape `GoogleImageModel` produces, so `ai.image()`
|
|
483
|
+
* callers see no difference between the two paths.
|
|
484
|
+
*
|
|
485
|
+
* **Token usage is passed through, not zeroed.** The Imagen path
|
|
486
|
+
* returns a hard `{ 0, 0, 0 }` because Imagen reports no tokens at all;
|
|
487
|
+
* here, whatever `usageMetadata` Google attaches is mapped by the same
|
|
488
|
+
* {@link applyGoogleUsage} the chat model uses, and only an absent
|
|
489
|
+
* block collapses to zeros. Price accordingly.
|
|
490
|
+
*
|
|
491
|
+
* **No model-id validation.** `config.name` is forwarded to
|
|
492
|
+
* `generateContent` exactly as given; nothing here inspects it. An id
|
|
493
|
+
* Google does not serve fails at Google, wrapped into the typed
|
|
494
|
+
* `AIError` hierarchy — never with a local throw.
|
|
495
|
+
*
|
|
496
|
+
* **Evidence, in two tiers.** No spec in this package calls Google.
|
|
497
|
+
* *Measured here:* a `gemini-*` image id, which 404s on the `predict`
|
|
498
|
+
* transport, reached the model on this one and came back with a quota
|
|
499
|
+
* error (HTTP 429) — the endpoint accepts the id. *Reported by the
|
|
500
|
+
* maintainer:* once billing was enabled on the project, an image came
|
|
501
|
+
* back end-to-end from an application running a locally linked build.
|
|
502
|
+
* *Still unestablished:* whether these models report token usage — no
|
|
503
|
+
* `usageMetadata` from a successful image call has been observed, so
|
|
504
|
+
* the pass-through above is untested against a real response.
|
|
505
|
+
*
|
|
506
|
+
* @example
|
|
507
|
+
* const model = new GeminiImageModel(ai, { name: "gemini-3.1-flash-lite-image" });
|
|
508
|
+
* const { images, usage } = await model.generate("a red bicycle on a white background");
|
|
509
|
+
*/
|
|
510
|
+
var GeminiImageModel = class {
|
|
511
|
+
constructor(ai, config, provider = "google") {
|
|
512
|
+
this.logger = _warlock_js_logger.log;
|
|
513
|
+
this.ai = ai;
|
|
514
|
+
this.name = config.name;
|
|
515
|
+
this.provider = provider;
|
|
516
|
+
this.pricing = config.pricing;
|
|
517
|
+
}
|
|
518
|
+
async generate(prompt, options) {
|
|
519
|
+
const config = this.buildConfig(options);
|
|
520
|
+
this.logger.debug(LOG_MODULE$2, "image.request", "models.generateContent", {
|
|
521
|
+
model: this.name,
|
|
522
|
+
responseModalities: config.responseModalities
|
|
523
|
+
});
|
|
524
|
+
let response;
|
|
525
|
+
try {
|
|
526
|
+
response = await this.ai.models.generateContent({
|
|
527
|
+
model: this.name,
|
|
528
|
+
contents: prompt,
|
|
529
|
+
config
|
|
530
|
+
});
|
|
531
|
+
} catch (thrown) {
|
|
532
|
+
const wrapped = wrapGoogleError(thrown);
|
|
533
|
+
this.logger.error(LOG_MODULE$2, "image.error", wrapped.message, {
|
|
534
|
+
code: wrapped.code,
|
|
535
|
+
context: wrapped.context
|
|
536
|
+
});
|
|
537
|
+
throw wrapped;
|
|
538
|
+
}
|
|
539
|
+
const parts = collectParts(response);
|
|
540
|
+
const images = toGeneratedImages(parts);
|
|
541
|
+
if (images.length === 0) throw this.noImageError(response, parts);
|
|
542
|
+
const usage = {
|
|
543
|
+
input: 0,
|
|
544
|
+
output: 0,
|
|
545
|
+
total: 0
|
|
546
|
+
};
|
|
547
|
+
if (response.usageMetadata) applyGoogleUsage(usage, response.usageMetadata);
|
|
548
|
+
this.logger.debug(LOG_MODULE$2, "image.response", "models.generateContent succeeded", {
|
|
549
|
+
images: images.length,
|
|
550
|
+
usage
|
|
551
|
+
});
|
|
552
|
+
return {
|
|
553
|
+
images,
|
|
554
|
+
usage
|
|
555
|
+
};
|
|
556
|
+
}
|
|
557
|
+
/**
|
|
558
|
+
* Assemble the `GenerateContentConfig` for an image turn: the
|
|
559
|
+
* requested modalities, the image-specific knobs Gemini exposes under
|
|
560
|
+
* `imageConfig`, and the cancellation handle.
|
|
561
|
+
*
|
|
562
|
+
* Three neutral options are deliberately NOT forwarded, because
|
|
563
|
+
* `GenerateContentConfig` / `ImageConfig` in `@google/genai` expose
|
|
564
|
+
* no equivalent for them on this path: `count` (no per-request image
|
|
565
|
+
* count — every inline image part the model does return is mapped),
|
|
566
|
+
* `negativePrompt` (an Imagen-only field), and `format`
|
|
567
|
+
* (`ImageConfig.outputMimeType` is documented "not supported in
|
|
568
|
+
* Gemini API"). Fold those intentions into the prompt instead.
|
|
569
|
+
*/
|
|
570
|
+
buildConfig(options) {
|
|
571
|
+
const imageConfig = {};
|
|
572
|
+
if (options?.aspectRatio !== void 0) imageConfig.aspectRatio = options.aspectRatio;
|
|
573
|
+
if (typeof options?.imageSize === "string") imageConfig.imageSize = options.imageSize;
|
|
574
|
+
if (typeof options?.personGeneration === "string") imageConfig.personGeneration = options.personGeneration;
|
|
575
|
+
const requested = options?.responseModalities;
|
|
576
|
+
return {
|
|
577
|
+
responseModalities: Array.isArray(requested) ? requested : DEFAULT_RESPONSE_MODALITIES,
|
|
578
|
+
...Object.keys(imageConfig).length > 0 ? { imageConfig } : {},
|
|
579
|
+
...options?.signal ? { abortSignal: options.signal } : {}
|
|
580
|
+
};
|
|
581
|
+
}
|
|
582
|
+
/**
|
|
583
|
+
* Build the typed error for a response that carried no inline image
|
|
584
|
+
* part. Never a silent empty success: the caller asked for an image
|
|
585
|
+
* and got something else, so the error names what actually came back.
|
|
586
|
+
*
|
|
587
|
+
* - A blocked prompt (`promptFeedback.blockReason`) or a
|
|
588
|
+
* safety/policy `finishReason` → `ContentFilterError` carrying the
|
|
589
|
+
* reason, matching how the Imagen path reports `raiFilteredReason`.
|
|
590
|
+
* - A text-only answer → `ProviderError` quoting the text, so the
|
|
591
|
+
* log says what the model replied instead of guessing.
|
|
592
|
+
* - Anything else → `ProviderError` naming the finish reason and how
|
|
593
|
+
* many parts arrived.
|
|
594
|
+
*/
|
|
595
|
+
noImageError(response, parts) {
|
|
596
|
+
const blockReason = response.promptFeedback?.blockReason;
|
|
597
|
+
if (blockReason) return new _warlock_js_ai.ContentFilterError(`Gemini blocked the prompt for ${this.name}: ${blockReason}`, { reason: blockReason });
|
|
598
|
+
const finishReason = response.candidates?.[0]?.finishReason;
|
|
599
|
+
if (finishReason && FILTERED_FINISH_REASONS.has(finishReason)) return new _warlock_js_ai.ContentFilterError(`Gemini filtered the image for ${this.name}: ${finishReason}`, { reason: finishReason });
|
|
600
|
+
const text = collectText(parts);
|
|
601
|
+
if (text) return new _warlock_js_ai.ProviderError(`Gemini returned no image for ${this.name} — the response was text only: "${excerpt(text)}"`, { context: {
|
|
602
|
+
model: this.name,
|
|
603
|
+
...finishReason ? { finishReason } : {}
|
|
604
|
+
} });
|
|
605
|
+
return new _warlock_js_ai.ProviderError(`Gemini returned no image part for ${this.name} (parts: ${parts.length}${finishReason ? `, finishReason: ${finishReason}` : ""}).`, { context: {
|
|
606
|
+
model: this.name,
|
|
607
|
+
parts: parts.length
|
|
608
|
+
} });
|
|
609
|
+
}
|
|
610
|
+
};
|
|
611
|
+
/**
|
|
612
|
+
* Flatten every candidate's content parts into one list. Read off
|
|
613
|
+
* `candidates[].content.parts` rather than the response's convenience
|
|
614
|
+
* getters: `response.text` covers only the first candidate's text and
|
|
615
|
+
* there is no getter for inline image data at all.
|
|
616
|
+
*/
|
|
617
|
+
function collectParts(response) {
|
|
618
|
+
const parts = [];
|
|
619
|
+
for (const candidate of response.candidates ?? []) parts.push(...candidate.content?.parts ?? []);
|
|
620
|
+
return parts;
|
|
621
|
+
}
|
|
622
|
+
/**
|
|
623
|
+
* Map the inline image parts to the neutral `GeneratedImage[]` — the
|
|
624
|
+
* same `{ type: "base64", base64, mediaType }` shape the Imagen path
|
|
625
|
+
* emits, including its `image/png` fallback for a part that arrives
|
|
626
|
+
* without a declared mime type.
|
|
627
|
+
*/
|
|
628
|
+
function toGeneratedImages(parts) {
|
|
629
|
+
const images = [];
|
|
630
|
+
for (const part of parts) {
|
|
631
|
+
const data = part.inlineData?.data;
|
|
632
|
+
if (!data) continue;
|
|
633
|
+
images.push({
|
|
634
|
+
type: "base64",
|
|
635
|
+
base64: data,
|
|
636
|
+
mediaType: part.inlineData?.mimeType ?? "image/png"
|
|
637
|
+
});
|
|
638
|
+
}
|
|
639
|
+
return images;
|
|
640
|
+
}
|
|
641
|
+
/** Join the text parts of a response — what the model said instead of drawing. */
|
|
642
|
+
function collectText(parts) {
|
|
643
|
+
return parts.map((part) => part.text).filter((text) => typeof text === "string" && text.length > 0).join(" ").trim();
|
|
644
|
+
}
|
|
645
|
+
/** Trim a quoted model answer so an error message stays readable. */
|
|
646
|
+
function excerpt(text) {
|
|
647
|
+
return text.length > TEXT_EXCERPT_LIMIT ? `${text.slice(0, TEXT_EXCERPT_LIMIT)}…` : text;
|
|
648
|
+
}
|
|
649
|
+
|
|
399
650
|
//#endregion
|
|
400
651
|
//#region ../ai-google/src/image.ts
|
|
401
652
|
const LOG_MODULE$1 = "ai.google";
|
|
@@ -687,7 +938,7 @@ var GoogleModel = class {
|
|
|
687
938
|
}
|
|
688
939
|
const candidateFinish = chunk.candidates?.[0]?.finishReason;
|
|
689
940
|
if (candidateFinish) rawFinishReason = candidateFinish;
|
|
690
|
-
if (chunk.usageMetadata)
|
|
941
|
+
if (chunk.usageMetadata) applyGoogleUsage(usage, chunk.usageMetadata);
|
|
691
942
|
}
|
|
692
943
|
} catch (thrown) {
|
|
693
944
|
throw this.logAndWrap(thrown);
|
|
@@ -800,9 +1051,10 @@ var GoogleModel = class {
|
|
|
800
1051
|
};
|
|
801
1052
|
}
|
|
802
1053
|
/**
|
|
803
|
-
* Normalize Gemini's `usageMetadata` into the neutral `Usage` shape
|
|
804
|
-
*
|
|
805
|
-
*
|
|
1054
|
+
* Normalize Gemini's `usageMetadata` into the neutral `Usage` shape
|
|
1055
|
+
* via the shared {@link applyGoogleUsage} mapper (the same one the
|
|
1056
|
+
* streaming loop and the Gemini image model use). Absent usage
|
|
1057
|
+
* collapses to zeros.
|
|
806
1058
|
*/
|
|
807
1059
|
extractUsage(response) {
|
|
808
1060
|
const usage = {
|
|
@@ -810,30 +1062,10 @@ var GoogleModel = class {
|
|
|
810
1062
|
output: 0,
|
|
811
1063
|
total: 0
|
|
812
1064
|
};
|
|
813
|
-
if (response.usageMetadata)
|
|
1065
|
+
if (response.usageMetadata) applyGoogleUsage(usage, response.usageMetadata);
|
|
814
1066
|
return usage;
|
|
815
1067
|
}
|
|
816
1068
|
/**
|
|
817
|
-
* Fold a Gemini `usageMetadata` block into the running neutral
|
|
818
|
-
* `Usage` accumulator. Shared by `complete()` and the streaming
|
|
819
|
-
* loop (where the final chunk carries cumulative totals).
|
|
820
|
-
*
|
|
821
|
-
* Cache-read hits (`cachedContentTokenCount`, implicit or explicit
|
|
822
|
-
* context caching) surface as `cachedTokens`; the thinking-phase
|
|
823
|
-
* tokens of a reasoning model (`thoughtsTokenCount`) surface as
|
|
824
|
-
* `reasoningTokens`. Both are emitted only when reported `> 0` so an
|
|
825
|
-
* absent channel leaves the field undefined.
|
|
826
|
-
*/
|
|
827
|
-
applyUsage(usage, raw) {
|
|
828
|
-
usage.input = raw.promptTokenCount ?? usage.input;
|
|
829
|
-
usage.output = raw.candidatesTokenCount ?? usage.output;
|
|
830
|
-
usage.total = raw.totalTokenCount ?? usage.input + usage.output;
|
|
831
|
-
const cached = raw.cachedContentTokenCount;
|
|
832
|
-
if (cached && cached > 0) usage.cachedTokens = cached;
|
|
833
|
-
const reasoning = raw.thoughtsTokenCount;
|
|
834
|
-
if (reasoning && reasoning > 0) usage.reasoningTokens = reasoning;
|
|
835
|
-
}
|
|
836
|
-
/**
|
|
837
1069
|
* Wrap a thrown provider error into the typed `AIError` hierarchy
|
|
838
1070
|
* and emit the standard error log line before it propagates.
|
|
839
1071
|
*/
|
|
@@ -850,6 +1082,36 @@ var GoogleModel = class {
|
|
|
850
1082
|
//#endregion
|
|
851
1083
|
//#region ../ai-google/src/sdk.ts
|
|
852
1084
|
/**
|
|
1085
|
+
* Pick the transport for an image model id.
|
|
1086
|
+
*
|
|
1087
|
+
* `ai.models.generateImages` calls `{model}:predict`, and a `gemini-`
|
|
1088
|
+
* id sent there comes back `404 … is not supported for predict`
|
|
1089
|
+
* (observed verbatim from Google). `generateContent` is what the SDK
|
|
1090
|
+
* itself points `generateImages` users at — its deprecation notice
|
|
1091
|
+
* reads "Please use the generateContent method with image models
|
|
1092
|
+
* instead" — so the id has to choose the transport.
|
|
1093
|
+
*
|
|
1094
|
+
* Runs in this package establish where a `gemini-` id is ACCEPTED, not
|
|
1095
|
+
* what it returns: on this transport such an id got as far as a quota
|
|
1096
|
+
* error (HTTP 429) instead of the 404. That an image
|
|
1097
|
+
* comes back end-to-end once billing is enabled is reported by the
|
|
1098
|
+
* maintainer from a locally linked build, not measured here. Whether
|
|
1099
|
+
* these models report token usage is still unknown.
|
|
1100
|
+
*
|
|
1101
|
+
* This is ROUTING, not validation — no id is refused here. An id this
|
|
1102
|
+
* function does not recognize takes the `generateImages` route, the
|
|
1103
|
+
* only route that existed before Gemini image support landed, so every
|
|
1104
|
+
* id that reached Google before still reaches Google the same way and
|
|
1105
|
+
* still fails (or succeeds) at the provider.
|
|
1106
|
+
*
|
|
1107
|
+
* A leading `models/` resource prefix is tolerated, matching the id
|
|
1108
|
+
* shapes `inferVisionCapability` already accepts
|
|
1109
|
+
* (`models/gemini-1.5-flash-001`).
|
|
1110
|
+
*/
|
|
1111
|
+
function usesGeminiImageTransport(name) {
|
|
1112
|
+
return name.toLowerCase().replace(/^models\//, "").startsWith("gemini-");
|
|
1113
|
+
}
|
|
1114
|
+
/**
|
|
853
1115
|
* Google Gemini-backed implementation of `SDKAdapterContract`.
|
|
854
1116
|
*
|
|
855
1117
|
* **Role.** The package entry point for Gemini models via the
|
|
@@ -917,19 +1179,27 @@ var GoogleSDK = class {
|
|
|
917
1179
|
return new GoogleEmbedder(this.ai, config, this.provider);
|
|
918
1180
|
}
|
|
919
1181
|
/**
|
|
920
|
-
* Build
|
|
921
|
-
*
|
|
922
|
-
*
|
|
923
|
-
*
|
|
1182
|
+
* Build an image model bound to this SDK's client for use with
|
|
1183
|
+
* `ai.image({ model, prompt })`. `config.name` decides the transport
|
|
1184
|
+
* (see {@link usesGeminiImageTransport}) — a `gemini-` id gets the
|
|
1185
|
+
* `generateContent` implementation, everything else the Imagen
|
|
1186
|
+
* `generateImages` one. No id is rejected locally either way, so an
|
|
1187
|
+
* unsupported model fails at Google, not here.
|
|
1188
|
+
*
|
|
1189
|
+
* The two differ in what usage they can report, which is what the
|
|
1190
|
+
* caller must price for: the Imagen path always returns a zero token
|
|
1191
|
+
* `Usage` (Imagen reports none — price with `{ perImage }`), while the
|
|
1192
|
+
* Gemini path passes through whatever `usageMetadata` Google attaches
|
|
1193
|
+
* (price with `{ input, output }` when tokens come back).
|
|
924
1194
|
*
|
|
925
1195
|
* Pricing resolution mirrors `model()`: per-model `config.pricing`
|
|
926
1196
|
* wins, otherwise the SDK-level registry entry keyed by `config.name`,
|
|
927
|
-
* otherwise `undefined`.
|
|
928
|
-
* entry typically carries `{ perImage }`.
|
|
1197
|
+
* otherwise `undefined`.
|
|
929
1198
|
*
|
|
930
1199
|
* @example
|
|
931
|
-
* const
|
|
932
|
-
* const
|
|
1200
|
+
* const imagen = google.image({ name: "imagen-4.0-generate-001" });
|
|
1201
|
+
* const gemini = google.image({ name: "gemini-3.1-flash-lite-image" });
|
|
1202
|
+
* const { data } = await ai.image({ model: gemini, prompt: "a red bicycle" });
|
|
933
1203
|
*/
|
|
934
1204
|
image(config) {
|
|
935
1205
|
const resolvedPricing = config.pricing ?? this.pricing?.[config.name];
|
|
@@ -937,11 +1207,13 @@ var GoogleSDK = class {
|
|
|
937
1207
|
...config,
|
|
938
1208
|
pricing: resolvedPricing
|
|
939
1209
|
};
|
|
1210
|
+
if (usesGeminiImageTransport(config.name)) return new GeminiImageModel(this.ai, resolvedConfig, this.provider);
|
|
940
1211
|
return new GoogleImageModel(this.ai, resolvedConfig, this.provider);
|
|
941
1212
|
}
|
|
942
1213
|
};
|
|
943
1214
|
|
|
944
1215
|
//#endregion
|
|
1216
|
+
exports.GeminiImageModel = GeminiImageModel;
|
|
945
1217
|
exports.GoogleImageModel = GoogleImageModel;
|
|
946
1218
|
exports.GoogleSDK = GoogleSDK;
|
|
947
1219
|
//# sourceMappingURL=index.cjs.map
|