@warlock.js/ai-google 4.13.0 → 4.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +35 -0
- package/cjs/index.cjs +307 -65
- package/cjs/index.cjs.map +1 -1
- package/esm/config.type.d.mts +11 -3
- package/esm/config.type.d.mts.map +1 -1
- package/esm/gemini-image.d.mts +91 -0
- package/esm/gemini-image.d.mts.map +1 -0
- package/esm/gemini-image.mjs +231 -0
- package/esm/gemini-image.mjs.map +1 -0
- package/esm/image.d.mts +7 -4
- package/esm/image.d.mts.map +1 -1
- package/esm/image.mjs +8 -7
- package/esm/image.mjs.map +1 -1
- package/esm/index.d.mts +2 -2
- package/esm/index.mjs +2 -2
- package/esm/model.mjs +7 -25
- package/esm/model.mjs.map +1 -1
- package/esm/sdk.d.mts +16 -7
- package/esm/sdk.d.mts.map +1 -1
- package/esm/sdk.mjs +48 -7
- package/esm/sdk.mjs.map +1 -1
- package/esm/utils/apply-google-usage.mjs +30 -0
- package/esm/utils/apply-google-usage.mjs.map +1 -0
- package/esm/utils/index.mjs +1 -0
- package/llms-full.txt +50 -6
- package/llms.txt +1 -1
- package/package.json +6 -6
- package/skills/setup-google/SKILL.md +50 -6
- package/esm/known-image-models.d.mts +0 -30
- package/esm/known-image-models.d.mts.map +0 -1
- package/esm/known-image-models.mjs +0 -33
- package/esm/known-image-models.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,41 @@ All notable changes to `@warlock.js/ai-google` are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `@warlock.js/*` packages are released in lockstep — every package shares the same version number, so a version below may list only the changes that affected this package.
|
|
6
6
|
|
|
7
|
+
## 4.15.0
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **`GeminiImageModel` — a Gemini-native image path over `ai.models.generateContent`** (new `src/gemini-image.ts`, exported as `GeminiImageModel`). Requests `responseModalities: ["TEXT", "IMAGE"]` (override the list verbatim with `options.responseModalities`), maps `aspectRatio` / `imageSize` / `personGeneration` onto Gemini's `config.imageConfig`, and reshapes inline image parts that come back into the **same** `GeneratedImage[]` (`{ type: "base64", base64, mediaType }`, `image/png` fallback) the Imagen path emits — so `ai.image()`'s envelope is unchanged for callers
|
|
12
|
+
- **Token usage is passed through on the Gemini image path instead of hard-zeroed.** Whatever `usageMetadata` Google attaches becomes `usage.input` / `output` / `total` (plus `cachedTokens` / `reasoningTokens` when reported `> 0`); only an absent block collapses to zeros. The Imagen path stays a flat zero because Imagen reports no tokens at all. Price these models with `{ input, output }` rather than `{ perImage }`, and check the first live `usage` — whether these models report tokens is not confirmed here. The mapping is now a shared `applyGoogleUsage` util used by both the chat model and the image model, so one rule decides what a Gemini token report means package-wide
|
|
13
|
+
- A response with **no image part is never a silent empty success**: a blocked prompt (`promptFeedback.blockReason`) or a safety/policy `finishReason` (`SAFETY`, `IMAGE_SAFETY`, `PROHIBITED_CONTENT`, `IMAGE_PROHIBITED_CONTENT`, `RECITATION`, `IMAGE_RECITATION`, `BLOCKLIST`, `SPII`) throws `ContentFilterError` carrying the reason; a text-only answer throws `ProviderError` **quoting the text the model returned**; anything else throws `ProviderError` naming the part count and finish reason
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
- **`google.image({ name: "gemini-…" })` no longer hits the endpoint that 404s it.** `ai.models.generateImages` routes to `{model}:predict` (`generateImages` → `generateImagesInternal` → `formatMap('{model}:predict', …)` in `@google/genai`'s bundle), which does not serve the Gemini image models — the call came back `404 models/… is not found for API version v1beta, or is not supported for predict`. `GoogleSDK.image()` now picks the transport from the id: a `gemini-` id (with an optional `models/` resource prefix) gets the new `generateContent` implementation, everything else keeps `GoogleImageModel` / `generateImages`. **Scope of the proof:** two levels. Measured here — on the new transport such an id got as far as a quota error (HTTP 429) instead of the 404, which establishes that the endpoint accepts the id. Reported by the maintainer — once billing was enabled on the project, the path returned an image end-to-end from an application running a locally linked build of this package. No test in this package calls Google; the suite proves the request shape and the error mapping, not the round trip
|
|
18
|
+
|
|
19
|
+
### Deprecated
|
|
20
|
+
|
|
21
|
+
- **Google has deprecated `generateImages`, the transport the `imagen-*` path still uses.** Verbatim from the `@google/genai` runtime warning: *"The generateImages method is deprecated and will be removed in the next major release (not before Jan. 1 2027). Please use the generateContent method with image models instead. See https://ai.google.dev/gemini-api/docs/deprecations#imagen-models and https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/image-generation#generate-images"* (`editImage` carries the same notice.) Nothing breaks today and the Imagen path is unchanged, but it is on a clock: new image work should prefer a `gemini-` id. The warning is emitted by `@google/genai` ≥ 2.17; with the bump below, this package now prints it whenever the `imagen-*` path is used
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
|
|
25
|
+
- **`@google/genai` moves from `^2.4.0` to `^2.17.1`** (2.17.1 is what installs today). The Gemini image path does not depend on the bump — `models.generateContent` exists in both — but the older range predates the deprecation notice above and predates `ai.interactions`, so staying on it meant documenting an SDK surface the package could not reach. The 11 suites / 149 specs in this package pass unchanged on 2.17.1. Note this re-resolved the whole workspace lockfile, not just this package's dependency
|
|
26
|
+
- `GoogleSDK.image()` returns `GeminiImageModel` for a `gemini-` id. This is **routing, not validation** — no id is rejected locally: an id matching neither family takes the `generateImages` route, the only route that existed before, so every id that reached Google before still reaches Google the same way and still fails (or succeeds) at the provider
|
|
27
|
+
|
|
28
|
+
### Not included
|
|
29
|
+
|
|
30
|
+
- **The `interactions` API is not used.** `@google/genai` ≥ 2.17 adds `ai.interactions.create({ model, input, response_format: { type: "image", … } })` with images at `interaction.output_image.data` and a different snake_case usage shape (`total_input_tokens` …). It is reachable now that the SDK is on 2.17.1, but nothing in this package calls it: it would need its own usage mapper and its own error surface, and its own request type already marks `response_modalities` / `response_mime_type` deprecated. If it lands it will be an **opt-in config flag**, not id routing
|
|
31
|
+
|
|
32
|
+
## 4.14.0
|
|
33
|
+
|
|
34
|
+
### Removed
|
|
35
|
+
|
|
36
|
+
- **BREAKING — `isGoogleImageModel()` and `GOOGLE_IMAGE_MODEL_PREFIXES` are gone from the public API.** Both were dropped from the package entrypoint and the module deleted; importing either from `@warlock.js/ai-google` is now a compile error. With the construction-time guard gone (below) they enforced nothing and only invited callers to re-implement a model allow-list the framework does not own — a model id is the provider's to rule on, so there is nothing left for a local list to say. Callers that branched on the Imagen family should match on the id themselves (`name.startsWith("imagen-")`) or, better, stop branching and let the provider answer
|
|
37
|
+
|
|
38
|
+
### Changed
|
|
39
|
+
|
|
40
|
+
- `google.image({ name })` no longer rejects a non-`imagen-*` model id at construction — the id is passed through to `ai.models.generateImages` as given, so an id Google does not serve now fails as a typed provider error instead of a local `InvalidRequestError`
|
|
41
|
+
|
|
7
42
|
## 4.12.0
|
|
8
43
|
|
|
9
44
|
### Changed
|
package/cjs/index.cjs
CHANGED
|
@@ -3,6 +3,34 @@ let _google_genai = require("@google/genai");
|
|
|
3
3
|
let _warlock_js_ai = require("@warlock.js/ai");
|
|
4
4
|
let _warlock_js_logger = require("@warlock.js/logger");
|
|
5
5
|
|
|
6
|
+
//#region ../ai-google/src/utils/apply-google-usage.ts
|
|
7
|
+
/**
|
|
8
|
+
* Fold a Gemini `usageMetadata` block into a running neutral `Usage`
|
|
9
|
+
* accumulator. Shared by every `generateContent`-backed surface — the
|
|
10
|
+
* chat model's `complete()`, its streaming loop (where the final chunk
|
|
11
|
+
* carries cumulative totals), and the Gemini image model — so one
|
|
12
|
+
* mapping decides what a Gemini token report means package-wide.
|
|
13
|
+
*
|
|
14
|
+
* Cache-read hits (`cachedContentTokenCount`, implicit or explicit
|
|
15
|
+
* context caching) surface as `cachedTokens`; the thinking-phase tokens
|
|
16
|
+
* of a reasoning model (`thoughtsTokenCount`) surface as
|
|
17
|
+
* `reasoningTokens`. Both are emitted only when reported `> 0` so an
|
|
18
|
+
* absent channel leaves the field undefined rather than a false zero.
|
|
19
|
+
*
|
|
20
|
+
* `total` falls back to `input + output` when Google omits
|
|
21
|
+
* `totalTokenCount`.
|
|
22
|
+
*/
|
|
23
|
+
function applyGoogleUsage(usage, raw) {
|
|
24
|
+
usage.input = raw.promptTokenCount ?? usage.input;
|
|
25
|
+
usage.output = raw.candidatesTokenCount ?? usage.output;
|
|
26
|
+
usage.total = raw.totalTokenCount ?? usage.input + usage.output;
|
|
27
|
+
const cached = raw.cachedContentTokenCount;
|
|
28
|
+
if (cached && cached > 0) usage.cachedTokens = cached;
|
|
29
|
+
const reasoning = raw.thoughtsTokenCount;
|
|
30
|
+
if (reasoning && reasoning > 0) usage.reasoningTokens = reasoning;
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
//#endregion
|
|
6
34
|
//#region ../ai-google/src/utils/map-finish-reason.ts
|
|
7
35
|
const finishReasonMap = {
|
|
8
36
|
STOP: "stop",
|
|
@@ -301,7 +329,7 @@ function buildContext(shape) {
|
|
|
301
329
|
|
|
302
330
|
//#endregion
|
|
303
331
|
//#region ../ai-google/src/embedder.ts
|
|
304
|
-
const LOG_MODULE$
|
|
332
|
+
const LOG_MODULE$3 = "ai.google";
|
|
305
333
|
/**
|
|
306
334
|
* Token usage is not returned by Gemini's `embedContent`, so every
|
|
307
335
|
* embedding result reports a zeroed `EmbeddingUsage` (honest absence,
|
|
@@ -367,7 +395,7 @@ var GoogleEmbedder = class {
|
|
|
367
395
|
* and return the raw vectors in input order.
|
|
368
396
|
*/
|
|
369
397
|
async request(inputs) {
|
|
370
|
-
this.logger.debug(LOG_MODULE$
|
|
398
|
+
this.logger.debug(LOG_MODULE$3, "embedder.request", "embedContent", {
|
|
371
399
|
model: this.name,
|
|
372
400
|
count: inputs.length
|
|
373
401
|
});
|
|
@@ -380,7 +408,7 @@ var GoogleEmbedder = class {
|
|
|
380
408
|
});
|
|
381
409
|
} catch (thrown) {
|
|
382
410
|
const wrapped = wrapGoogleError(thrown);
|
|
383
|
-
this.logger.error(LOG_MODULE$
|
|
411
|
+
this.logger.error(LOG_MODULE$3, "embedder.error", wrapped.message, {
|
|
384
412
|
code: wrapped.code,
|
|
385
413
|
context: wrapped.context
|
|
386
414
|
});
|
|
@@ -388,7 +416,7 @@ var GoogleEmbedder = class {
|
|
|
388
416
|
}
|
|
389
417
|
const vectors = (response.embeddings ?? []).map((embedding) => embedding.values ?? []);
|
|
390
418
|
if (this.dimensions === 0 && vectors[0]) this.dimensions = vectors[0].length;
|
|
391
|
-
this.logger.debug(LOG_MODULE$
|
|
419
|
+
this.logger.debug(LOG_MODULE$3, "embedder.response", "embedContent returned", {
|
|
392
420
|
count: vectors.length,
|
|
393
421
|
dimensions: this.dimensions
|
|
394
422
|
});
|
|
@@ -397,34 +425,226 @@ var GoogleEmbedder = class {
|
|
|
397
425
|
};
|
|
398
426
|
|
|
399
427
|
//#endregion
|
|
400
|
-
//#region ../ai-google/src/
|
|
428
|
+
//#region ../ai-google/src/gemini-image.ts
|
|
429
|
+
const LOG_MODULE$2 = "ai.google";
|
|
430
|
+
/**
|
|
431
|
+
* Response modalities requested when the caller names none.
|
|
432
|
+
*
|
|
433
|
+
* `IMAGE` is the modality this adapter extracts; `TEXT` rides along so
|
|
434
|
+
* a model that narrates what it drew is not answering outside the set
|
|
435
|
+
* it was granted (the narration is then dropped — only inline image
|
|
436
|
+
* parts become `GeneratedImage`s).
|
|
437
|
+
*
|
|
438
|
+
* *Unverified:* which pairing any individual Gemini image model
|
|
439
|
+
* requires is not established here — no spec or run in this package
|
|
440
|
+
* touches the live API. `options.responseModalities` replaces this list
|
|
441
|
+
* verbatim for a model that wants something else.
|
|
442
|
+
*/
|
|
443
|
+
const DEFAULT_RESPONSE_MODALITIES = ["TEXT", "IMAGE"];
|
|
401
444
|
/**
|
|
402
|
-
*
|
|
403
|
-
*
|
|
404
|
-
*
|
|
405
|
-
*
|
|
406
|
-
*
|
|
407
|
-
*
|
|
408
|
-
* different surface (`generateContent` with `responseModalities`) and
|
|
409
|
-
* is intentionally NOT routed here — `google.image()` targets the
|
|
410
|
-
* dedicated Imagen endpoint only.
|
|
411
|
-
*
|
|
412
|
-
* Used by {@link isGoogleImageModel} for the construction-time guard so
|
|
413
|
-
* `google.image({ name: "gemini-2.5-flash" })` fails fast with a
|
|
414
|
-
* curated error rather than a downstream 400.
|
|
445
|
+
* Gemini `finishReason` values that mean generation was stopped by a
|
|
446
|
+
* safety / policy rule rather than by the model simply not drawing.
|
|
447
|
+
* Taken from the `FinishReason` enum in `@google/genai`'s own type
|
|
448
|
+
* declarations, whose doc comments describe each of these as content
|
|
449
|
+
* or image generation being "stopped" for safety, prohibited content,
|
|
450
|
+
* recitation, blocklist, or SPII.
|
|
415
451
|
*/
|
|
416
|
-
const
|
|
452
|
+
const FILTERED_FINISH_REASONS = new Set([
|
|
453
|
+
"SAFETY",
|
|
454
|
+
"IMAGE_SAFETY",
|
|
455
|
+
"PROHIBITED_CONTENT",
|
|
456
|
+
"IMAGE_PROHIBITED_CONTENT",
|
|
457
|
+
"RECITATION",
|
|
458
|
+
"IMAGE_RECITATION",
|
|
459
|
+
"BLOCKLIST",
|
|
460
|
+
"SPII"
|
|
461
|
+
]);
|
|
462
|
+
/** How much of a text-only answer to quote back inside the error message. */
|
|
463
|
+
const TEXT_EXCERPT_LIMIT = 200;
|
|
417
464
|
/**
|
|
418
|
-
*
|
|
419
|
-
*
|
|
420
|
-
*
|
|
465
|
+
* Gemini-native implementation of `ImageModelContract`, via
|
|
466
|
+
* `ai.models.generateContent` with `config.responseModalities`
|
|
467
|
+
* including `"IMAGE"`.
|
|
468
|
+
*
|
|
469
|
+
* **Why a second image adapter.** `GoogleImageModel` calls
|
|
470
|
+
* `ai.models.generateImages`, which the `@google/genai` bundle routes
|
|
471
|
+
* to `{model}:predict` (`generateImages` → `generateImagesInternal` →
|
|
472
|
+
* `formatMap('{model}:predict', …)`). A Gemini image model is not
|
|
473
|
+
* served there: asking for one returns Google's
|
|
474
|
+
* `404 … is not found for API version v1beta, or is not supported for
|
|
475
|
+
* predict`. `generateContent` is the SDK's own named replacement — its
|
|
476
|
+
* runtime deprecation notice for `generateImages` reads "Please use the
|
|
477
|
+
* generateContent method with image models instead" — so that is the
|
|
478
|
+
* transport this class speaks, hence a separate class rather than a
|
|
479
|
+
* branch inside `image.ts`.
|
|
480
|
+
*
|
|
481
|
+
* **Same envelope.** Inline image parts are mapped to the identical
|
|
482
|
+
* `GeneratedImage[]` shape `GoogleImageModel` produces, so `ai.image()`
|
|
483
|
+
* callers see no difference between the two paths.
|
|
484
|
+
*
|
|
485
|
+
* **Token usage is passed through, not zeroed.** The Imagen path
|
|
486
|
+
* returns a hard `{ 0, 0, 0 }` because Imagen reports no tokens at all;
|
|
487
|
+
* here, whatever `usageMetadata` Google attaches is mapped by the same
|
|
488
|
+
* {@link applyGoogleUsage} the chat model uses, and only an absent
|
|
489
|
+
* block collapses to zeros. Price accordingly.
|
|
490
|
+
*
|
|
491
|
+
* **No model-id validation.** `config.name` is forwarded to
|
|
492
|
+
* `generateContent` exactly as given; nothing here inspects it. An id
|
|
493
|
+
* Google does not serve fails at Google, wrapped into the typed
|
|
494
|
+
* `AIError` hierarchy — never with a local throw.
|
|
495
|
+
*
|
|
496
|
+
* **Evidence, in two tiers.** No spec in this package calls Google.
|
|
497
|
+
* *Measured here:* a `gemini-*` image id, which 404s on the `predict`
|
|
498
|
+
* transport, reached the model on this one and came back with a quota
|
|
499
|
+
* error (HTTP 429) — the endpoint accepts the id. *Reported by the
|
|
500
|
+
* maintainer:* once billing was enabled on the project, an image came
|
|
501
|
+
* back end-to-end from an application running a locally linked build.
|
|
502
|
+
* *Still unestablished:* whether these models report token usage — no
|
|
503
|
+
* `usageMetadata` from a successful image call has been observed, so
|
|
504
|
+
* the pass-through above is untested against a real response.
|
|
421
505
|
*
|
|
422
506
|
* @example
|
|
423
|
-
*
|
|
424
|
-
*
|
|
507
|
+
* const model = new GeminiImageModel(ai, { name: "gemini-3.1-flash-lite-image" });
|
|
508
|
+
* const { images, usage } = await model.generate("a red bicycle on a white background");
|
|
425
509
|
*/
|
|
426
|
-
|
|
427
|
-
|
|
510
|
+
var GeminiImageModel = class {
|
|
511
|
+
constructor(ai, config, provider = "google") {
|
|
512
|
+
this.logger = _warlock_js_logger.log;
|
|
513
|
+
this.ai = ai;
|
|
514
|
+
this.name = config.name;
|
|
515
|
+
this.provider = provider;
|
|
516
|
+
this.pricing = config.pricing;
|
|
517
|
+
}
|
|
518
|
+
async generate(prompt, options) {
|
|
519
|
+
const config = this.buildConfig(options);
|
|
520
|
+
this.logger.debug(LOG_MODULE$2, "image.request", "models.generateContent", {
|
|
521
|
+
model: this.name,
|
|
522
|
+
responseModalities: config.responseModalities
|
|
523
|
+
});
|
|
524
|
+
let response;
|
|
525
|
+
try {
|
|
526
|
+
response = await this.ai.models.generateContent({
|
|
527
|
+
model: this.name,
|
|
528
|
+
contents: prompt,
|
|
529
|
+
config
|
|
530
|
+
});
|
|
531
|
+
} catch (thrown) {
|
|
532
|
+
const wrapped = wrapGoogleError(thrown);
|
|
533
|
+
this.logger.error(LOG_MODULE$2, "image.error", wrapped.message, {
|
|
534
|
+
code: wrapped.code,
|
|
535
|
+
context: wrapped.context
|
|
536
|
+
});
|
|
537
|
+
throw wrapped;
|
|
538
|
+
}
|
|
539
|
+
const parts = collectParts(response);
|
|
540
|
+
const images = toGeneratedImages(parts);
|
|
541
|
+
if (images.length === 0) throw this.noImageError(response, parts);
|
|
542
|
+
const usage = {
|
|
543
|
+
input: 0,
|
|
544
|
+
output: 0,
|
|
545
|
+
total: 0
|
|
546
|
+
};
|
|
547
|
+
if (response.usageMetadata) applyGoogleUsage(usage, response.usageMetadata);
|
|
548
|
+
this.logger.debug(LOG_MODULE$2, "image.response", "models.generateContent succeeded", {
|
|
549
|
+
images: images.length,
|
|
550
|
+
usage
|
|
551
|
+
});
|
|
552
|
+
return {
|
|
553
|
+
images,
|
|
554
|
+
usage
|
|
555
|
+
};
|
|
556
|
+
}
|
|
557
|
+
/**
|
|
558
|
+
* Assemble the `GenerateContentConfig` for an image turn: the
|
|
559
|
+
* requested modalities, the image-specific knobs Gemini exposes under
|
|
560
|
+
* `imageConfig`, and the cancellation handle.
|
|
561
|
+
*
|
|
562
|
+
* Three neutral options are deliberately NOT forwarded, because
|
|
563
|
+
* `GenerateContentConfig` / `ImageConfig` in `@google/genai` expose
|
|
564
|
+
* no equivalent for them on this path: `count` (no per-request image
|
|
565
|
+
* count — every inline image part the model does return is mapped),
|
|
566
|
+
* `negativePrompt` (an Imagen-only field), and `format`
|
|
567
|
+
* (`ImageConfig.outputMimeType` is documented "not supported in
|
|
568
|
+
* Gemini API"). Fold those intentions into the prompt instead.
|
|
569
|
+
*/
|
|
570
|
+
buildConfig(options) {
|
|
571
|
+
const imageConfig = {};
|
|
572
|
+
if (options?.aspectRatio !== void 0) imageConfig.aspectRatio = options.aspectRatio;
|
|
573
|
+
if (typeof options?.imageSize === "string") imageConfig.imageSize = options.imageSize;
|
|
574
|
+
if (typeof options?.personGeneration === "string") imageConfig.personGeneration = options.personGeneration;
|
|
575
|
+
const requested = options?.responseModalities;
|
|
576
|
+
return {
|
|
577
|
+
responseModalities: Array.isArray(requested) ? requested : DEFAULT_RESPONSE_MODALITIES,
|
|
578
|
+
...Object.keys(imageConfig).length > 0 ? { imageConfig } : {},
|
|
579
|
+
...options?.signal ? { abortSignal: options.signal } : {}
|
|
580
|
+
};
|
|
581
|
+
}
|
|
582
|
+
/**
|
|
583
|
+
* Build the typed error for a response that carried no inline image
|
|
584
|
+
* part. Never a silent empty success: the caller asked for an image
|
|
585
|
+
* and got something else, so the error names what actually came back.
|
|
586
|
+
*
|
|
587
|
+
* - A blocked prompt (`promptFeedback.blockReason`) or a
|
|
588
|
+
* safety/policy `finishReason` → `ContentFilterError` carrying the
|
|
589
|
+
* reason, matching how the Imagen path reports `raiFilteredReason`.
|
|
590
|
+
* - A text-only answer → `ProviderError` quoting the text, so the
|
|
591
|
+
* log says what the model replied instead of guessing.
|
|
592
|
+
* - Anything else → `ProviderError` naming the finish reason and how
|
|
593
|
+
* many parts arrived.
|
|
594
|
+
*/
|
|
595
|
+
noImageError(response, parts) {
|
|
596
|
+
const blockReason = response.promptFeedback?.blockReason;
|
|
597
|
+
if (blockReason) return new _warlock_js_ai.ContentFilterError(`Gemini blocked the prompt for ${this.name}: ${blockReason}`, { reason: blockReason });
|
|
598
|
+
const finishReason = response.candidates?.[0]?.finishReason;
|
|
599
|
+
if (finishReason && FILTERED_FINISH_REASONS.has(finishReason)) return new _warlock_js_ai.ContentFilterError(`Gemini filtered the image for ${this.name}: ${finishReason}`, { reason: finishReason });
|
|
600
|
+
const text = collectText(parts);
|
|
601
|
+
if (text) return new _warlock_js_ai.ProviderError(`Gemini returned no image for ${this.name} — the response was text only: "${excerpt(text)}"`, { context: {
|
|
602
|
+
model: this.name,
|
|
603
|
+
...finishReason ? { finishReason } : {}
|
|
604
|
+
} });
|
|
605
|
+
return new _warlock_js_ai.ProviderError(`Gemini returned no image part for ${this.name} (parts: ${parts.length}${finishReason ? `, finishReason: ${finishReason}` : ""}).`, { context: {
|
|
606
|
+
model: this.name,
|
|
607
|
+
parts: parts.length
|
|
608
|
+
} });
|
|
609
|
+
}
|
|
610
|
+
};
|
|
611
|
+
/**
|
|
612
|
+
* Flatten every candidate's content parts into one list. Read off
|
|
613
|
+
* `candidates[].content.parts` rather than the response's convenience
|
|
614
|
+
* getters: `response.text` covers only the first candidate's text and
|
|
615
|
+
* there is no getter for inline image data at all.
|
|
616
|
+
*/
|
|
617
|
+
function collectParts(response) {
|
|
618
|
+
const parts = [];
|
|
619
|
+
for (const candidate of response.candidates ?? []) parts.push(...candidate.content?.parts ?? []);
|
|
620
|
+
return parts;
|
|
621
|
+
}
|
|
622
|
+
/**
|
|
623
|
+
* Map the inline image parts to the neutral `GeneratedImage[]` — the
|
|
624
|
+
* same `{ type: "base64", base64, mediaType }` shape the Imagen path
|
|
625
|
+
* emits, including its `image/png` fallback for a part that arrives
|
|
626
|
+
* without a declared mime type.
|
|
627
|
+
*/
|
|
628
|
+
function toGeneratedImages(parts) {
|
|
629
|
+
const images = [];
|
|
630
|
+
for (const part of parts) {
|
|
631
|
+
const data = part.inlineData?.data;
|
|
632
|
+
if (!data) continue;
|
|
633
|
+
images.push({
|
|
634
|
+
type: "base64",
|
|
635
|
+
base64: data,
|
|
636
|
+
mediaType: part.inlineData?.mimeType ?? "image/png"
|
|
637
|
+
});
|
|
638
|
+
}
|
|
639
|
+
return images;
|
|
640
|
+
}
|
|
641
|
+
/** Join the text parts of a response — what the model said instead of drawing. */
|
|
642
|
+
function collectText(parts) {
|
|
643
|
+
return parts.map((part) => part.text).filter((text) => typeof text === "string" && text.length > 0).join(" ").trim();
|
|
644
|
+
}
|
|
645
|
+
/** Trim a quoted model answer so an error message stays readable. */
|
|
646
|
+
function excerpt(text) {
|
|
647
|
+
return text.length > TEXT_EXCERPT_LIMIT ? `${text.slice(0, TEXT_EXCERPT_LIMIT)}…` : text;
|
|
428
648
|
}
|
|
429
649
|
|
|
430
650
|
//#endregion
|
|
@@ -445,10 +665,13 @@ function mediaTypeFor(format) {
|
|
|
445
665
|
* `ai.models.generateImages`. Imagen is per-image-metered and returns
|
|
446
666
|
* base64 image bytes (no hosted URL, no token usage).
|
|
447
667
|
*
|
|
448
|
-
* **
|
|
449
|
-
*
|
|
450
|
-
*
|
|
451
|
-
*
|
|
668
|
+
* **No model-id validation.** `config.name` is passed through to
|
|
669
|
+
* `ai.models.generateImages` exactly as given — the constructor never
|
|
670
|
+
* inspects it. Google adds and retires image model ids on its own
|
|
671
|
+
* schedule, so an id this adapter does not recognize is not the
|
|
672
|
+
* adapter's call to refuse; an unsupported id surfaces as a provider
|
|
673
|
+
* error from Google (wrapped into the typed `AIError` hierarchy by
|
|
674
|
+
* `generate()`), not as a local one.
|
|
452
675
|
*
|
|
453
676
|
* **Safety filtering.** When Imagen filters every candidate for safety
|
|
454
677
|
* (`raiFilteredReason`), this surfaces a typed `ContentFilterError`
|
|
@@ -461,7 +684,6 @@ function mediaTypeFor(format) {
|
|
|
461
684
|
var GoogleImageModel = class {
|
|
462
685
|
constructor(ai, config, provider = "google") {
|
|
463
686
|
this.logger = _warlock_js_logger.log;
|
|
464
|
-
if (!isGoogleImageModel(config.name)) throw new _warlock_js_ai.InvalidRequestError(`"${config.name}" is not a known Google Imagen model. Use an \`imagen-*\` model with google.image({ name }).`);
|
|
465
687
|
this.ai = ai;
|
|
466
688
|
this.name = config.name;
|
|
467
689
|
this.provider = provider;
|
|
@@ -716,7 +938,7 @@ var GoogleModel = class {
|
|
|
716
938
|
}
|
|
717
939
|
const candidateFinish = chunk.candidates?.[0]?.finishReason;
|
|
718
940
|
if (candidateFinish) rawFinishReason = candidateFinish;
|
|
719
|
-
if (chunk.usageMetadata)
|
|
941
|
+
if (chunk.usageMetadata) applyGoogleUsage(usage, chunk.usageMetadata);
|
|
720
942
|
}
|
|
721
943
|
} catch (thrown) {
|
|
722
944
|
throw this.logAndWrap(thrown);
|
|
@@ -829,9 +1051,10 @@ var GoogleModel = class {
|
|
|
829
1051
|
};
|
|
830
1052
|
}
|
|
831
1053
|
/**
|
|
832
|
-
* Normalize Gemini's `usageMetadata` into the neutral `Usage` shape
|
|
833
|
-
*
|
|
834
|
-
*
|
|
1054
|
+
* Normalize Gemini's `usageMetadata` into the neutral `Usage` shape
|
|
1055
|
+
* via the shared {@link applyGoogleUsage} mapper (the same one the
|
|
1056
|
+
* streaming loop and the Gemini image model use). Absent usage
|
|
1057
|
+
* collapses to zeros.
|
|
835
1058
|
*/
|
|
836
1059
|
extractUsage(response) {
|
|
837
1060
|
const usage = {
|
|
@@ -839,30 +1062,10 @@ var GoogleModel = class {
|
|
|
839
1062
|
output: 0,
|
|
840
1063
|
total: 0
|
|
841
1064
|
};
|
|
842
|
-
if (response.usageMetadata)
|
|
1065
|
+
if (response.usageMetadata) applyGoogleUsage(usage, response.usageMetadata);
|
|
843
1066
|
return usage;
|
|
844
1067
|
}
|
|
845
1068
|
/**
|
|
846
|
-
* Fold a Gemini `usageMetadata` block into the running neutral
|
|
847
|
-
* `Usage` accumulator. Shared by `complete()` and the streaming
|
|
848
|
-
* loop (where the final chunk carries cumulative totals).
|
|
849
|
-
*
|
|
850
|
-
* Cache-read hits (`cachedContentTokenCount`, implicit or explicit
|
|
851
|
-
* context caching) surface as `cachedTokens`; the thinking-phase
|
|
852
|
-
* tokens of a reasoning model (`thoughtsTokenCount`) surface as
|
|
853
|
-
* `reasoningTokens`. Both are emitted only when reported `> 0` so an
|
|
854
|
-
* absent channel leaves the field undefined.
|
|
855
|
-
*/
|
|
856
|
-
applyUsage(usage, raw) {
|
|
857
|
-
usage.input = raw.promptTokenCount ?? usage.input;
|
|
858
|
-
usage.output = raw.candidatesTokenCount ?? usage.output;
|
|
859
|
-
usage.total = raw.totalTokenCount ?? usage.input + usage.output;
|
|
860
|
-
const cached = raw.cachedContentTokenCount;
|
|
861
|
-
if (cached && cached > 0) usage.cachedTokens = cached;
|
|
862
|
-
const reasoning = raw.thoughtsTokenCount;
|
|
863
|
-
if (reasoning && reasoning > 0) usage.reasoningTokens = reasoning;
|
|
864
|
-
}
|
|
865
|
-
/**
|
|
866
1069
|
* Wrap a thrown provider error into the typed `AIError` hierarchy
|
|
867
1070
|
* and emit the standard error log line before it propagates.
|
|
868
1071
|
*/
|
|
@@ -879,6 +1082,36 @@ var GoogleModel = class {
|
|
|
879
1082
|
//#endregion
|
|
880
1083
|
//#region ../ai-google/src/sdk.ts
|
|
881
1084
|
/**
|
|
1085
|
+
* Pick the transport for an image model id.
|
|
1086
|
+
*
|
|
1087
|
+
* `ai.models.generateImages` calls `{model}:predict`, and a `gemini-`
|
|
1088
|
+
* id sent there comes back `404 … is not supported for predict`
|
|
1089
|
+
* (observed verbatim from Google). `generateContent` is what the SDK
|
|
1090
|
+
* itself points `generateImages` users at — its deprecation notice
|
|
1091
|
+
* reads "Please use the generateContent method with image models
|
|
1092
|
+
* instead" — so the id has to choose the transport.
|
|
1093
|
+
*
|
|
1094
|
+
* Runs in this package establish where a `gemini-` id is ACCEPTED, not
|
|
1095
|
+
* what it returns: on this transport such an id got as far as a quota
|
|
1096
|
+
* error (HTTP 429) instead of the 404. That an image
|
|
1097
|
+
* comes back end-to-end once billing is enabled is reported by the
|
|
1098
|
+
* maintainer from a locally linked build, not measured here. Whether
|
|
1099
|
+
* these models report token usage is still unknown.
|
|
1100
|
+
*
|
|
1101
|
+
* This is ROUTING, not validation — no id is refused here. An id this
|
|
1102
|
+
* function does not recognize takes the `generateImages` route, the
|
|
1103
|
+
* only route that existed before Gemini image support landed, so every
|
|
1104
|
+
* id that reached Google before still reaches Google the same way and
|
|
1105
|
+
* still fails (or succeeds) at the provider.
|
|
1106
|
+
*
|
|
1107
|
+
* A leading `models/` resource prefix is tolerated, matching the id
|
|
1108
|
+
* shapes `inferVisionCapability` already accepts
|
|
1109
|
+
* (`models/gemini-1.5-flash-001`).
|
|
1110
|
+
*/
|
|
1111
|
+
function usesGeminiImageTransport(name) {
|
|
1112
|
+
return name.toLowerCase().replace(/^models\//, "").startsWith("gemini-");
|
|
1113
|
+
}
|
|
1114
|
+
/**
|
|
882
1115
|
* Google Gemini-backed implementation of `SDKAdapterContract`.
|
|
883
1116
|
*
|
|
884
1117
|
* **Role.** The package entry point for Gemini models via the
|
|
@@ -946,18 +1179,27 @@ var GoogleSDK = class {
|
|
|
946
1179
|
return new GoogleEmbedder(this.ai, config, this.provider);
|
|
947
1180
|
}
|
|
948
1181
|
/**
|
|
949
|
-
* Build
|
|
950
|
-
*
|
|
951
|
-
*
|
|
1182
|
+
* Build an image model bound to this SDK's client for use with
|
|
1183
|
+
* `ai.image({ model, prompt })`. `config.name` decides the transport
|
|
1184
|
+
* (see {@link usesGeminiImageTransport}) — a `gemini-` id gets the
|
|
1185
|
+
* `generateContent` implementation, everything else the Imagen
|
|
1186
|
+
* `generateImages` one. No id is rejected locally either way, so an
|
|
1187
|
+
* unsupported model fails at Google, not here.
|
|
1188
|
+
*
|
|
1189
|
+
* The two differ in what usage they can report, which is what the
|
|
1190
|
+
* caller must price for: the Imagen path always returns a zero token
|
|
1191
|
+
* `Usage` (Imagen reports none — price with `{ perImage }`), while the
|
|
1192
|
+
* Gemini path passes through whatever `usageMetadata` Google attaches
|
|
1193
|
+
* (price with `{ input, output }` when tokens come back).
|
|
952
1194
|
*
|
|
953
1195
|
* Pricing resolution mirrors `model()`: per-model `config.pricing`
|
|
954
1196
|
* wins, otherwise the SDK-level registry entry keyed by `config.name`,
|
|
955
|
-
* otherwise `undefined`.
|
|
956
|
-
* entry typically carries `{ perImage }`.
|
|
1197
|
+
* otherwise `undefined`.
|
|
957
1198
|
*
|
|
958
1199
|
* @example
|
|
959
|
-
* const
|
|
960
|
-
* const
|
|
1200
|
+
* const imagen = google.image({ name: "imagen-4.0-generate-001" });
|
|
1201
|
+
* const gemini = google.image({ name: "gemini-3.1-flash-lite-image" });
|
|
1202
|
+
* const { data } = await ai.image({ model: gemini, prompt: "a red bicycle" });
|
|
961
1203
|
*/
|
|
962
1204
|
image(config) {
|
|
963
1205
|
const resolvedPricing = config.pricing ?? this.pricing?.[config.name];
|
|
@@ -965,13 +1207,13 @@ var GoogleSDK = class {
|
|
|
965
1207
|
...config,
|
|
966
1208
|
pricing: resolvedPricing
|
|
967
1209
|
};
|
|
1210
|
+
if (usesGeminiImageTransport(config.name)) return new GeminiImageModel(this.ai, resolvedConfig, this.provider);
|
|
968
1211
|
return new GoogleImageModel(this.ai, resolvedConfig, this.provider);
|
|
969
1212
|
}
|
|
970
1213
|
};
|
|
971
1214
|
|
|
972
1215
|
//#endregion
|
|
973
|
-
exports.
|
|
1216
|
+
exports.GeminiImageModel = GeminiImageModel;
|
|
974
1217
|
exports.GoogleImageModel = GoogleImageModel;
|
|
975
1218
|
exports.GoogleSDK = GoogleSDK;
|
|
976
|
-
exports.isGoogleImageModel = isGoogleImageModel;
|
|
977
1219
|
//# sourceMappingURL=index.cjs.map
|