@starci/ai-router 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README.md +33 -0
- package/dist/gateway/index.d.ts +2 -0
- package/dist/gateway/index.js +1 -0
- package/dist/media.d.ts +184 -0
- package/dist/media.js +363 -0
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.0 - 2026-11-17
|
|
4
|
+
|
|
5
|
+
Media beside chat: images, speech and video jobs, key-gated like the chat providers. Additive: nothing declared before changes, and `AiRouter` gains no method. The README section "Media: images, speech and video" lists each vendor page read 2026-11-17.
|
|
6
|
+
|
|
7
|
+
- Added: `createMedia({ env?, fetch?, timeoutMs?, baseUrls?, catalogue?, now? })` with `generateImage({ prompt, size?, n?, referenceImages?, model?, provider? })` (`{ images: [{ bytes, mime }], model, provider, usage: { units } }`), `synthesizeSpeech({ text, voice?, format? })` (`{ bytes, mime, model, provider, usage: { characters } }`), `startVideo({ prompt, seconds?, size? })` (`{ jobId, provider }`), `getVideo(jobId)` (`running`, `done` with bytes and mime, or `failed` with error and failureClass), `availableMediaModels()` (`{ kind, id, provider, label }`) and `auto(kind)`. A provider with no key (`OPENAI_API_KEY`, `GEMINI_API_KEY`, `ELEVENLABS_API_KEY`) is absent and nothing fails at boot.
|
|
8
|
+
- Added: images from OpenAI (`gpt-image-2.5-flare`, `gpt-image-2.5-sunburst`; generations, and edits with multipart `image[]` for reference images) and Google (`gemini-nano-banana-2.1`, `gemini-3-pro-image`, `gemini-3.1-flash-lite-image` through `POST /v1beta/interactions`); speech from OpenAI (`gpt-4o-mini-tts`, shut down 2027-01-06 per its page) and ElevenLabs (`eleven_multilingual_v2`); video from Google (`gemini-omni-1.2-flash`).
|
|
9
|
+
- Added: `MediaError` with `failureClass` (`refused`, `transient`, `auth`, `invalid_request`, `not_found`, `unavailable`), `retryable` and the code `AI_ROUTER_MEDIA_<CLASS>`; `classifyMediaStatus`, `MEDIA_CATALOGUE`, `AI_ROUTER_MEDIA_ERROR_CODES`. A call without `model` fails over across models and keys on `auth` and `transient`; `refused` and `invalid_request` stop at once.
|
|
10
|
+
- Left out because the pages do not support it: OpenAI video (Sora and the Videos API are shut down, per its guide), Google Veo (preview models shut down, per its video page), fal (its queue page documents no video model, input or output). Google's pages show no usage fields or blocked-result shape, so image usage is the images returned and an answer with no image is `refused`; Google video is documented as one synchronous request, so its job lives in this process (kept an hour after it finishes, lost on restart).
|
|
11
|
+
|
|
3
12
|
## 0.8.0 - 2026-11-16
|
|
4
13
|
|
|
5
14
|
The router knows the current Claude 5 and OpenAI models by id, offers OpenRouter's catalog at run time, lists what a caller may choose, and routes `auto-*` profiles by effort tier within the providers that have keys. Additive: nothing declared before changes.
|
package/README.md
CHANGED
|
@@ -110,6 +110,39 @@ await router.chat({ messages }, { profile: "auto-high" }) // opus first, then th
|
|
|
110
110
|
await router.chat({ model: "openrouter/vendor/model", messages }) // sent to OpenRouter as vendor/model
|
|
111
111
|
```
|
|
112
112
|
|
|
113
|
+
## Media: images, speech and video
|
|
114
|
+
|
|
115
|
+
Chat is not the only thing an app asks of a vendor. `createMedia` generates images, synthesizes speech and runs video jobs over the providers whose keys are in env, key-gated exactly like chat: a provider with no key is absent from `availableMediaModels()` and nothing fails at boot. It is separate from `createRouter`, so no `AiRouter` implementer changes.
|
|
116
|
+
|
|
117
|
+
```ts
|
|
118
|
+
import { createMedia } from "@starci/ai-router/gateway"
|
|
119
|
+
|
|
120
|
+
const media = createMedia() // reads OPENAI_API_KEY, GEMINI_API_KEY, ELEVENLABS_API_KEY (and _2.., _API_KEYS) from process.env
|
|
121
|
+
media.availableMediaModels() // [{ kind: "image", id: "gpt-image-2.5-flare", provider: "openai", label: "GPT Image 2.5 Flare" }, ...]
|
|
122
|
+
media.auto("image") // the first available model of the kind, the one a call without `model` tries first
|
|
123
|
+
|
|
124
|
+
const { images, model, usage } = await media.generateImage({ prompt: "a red fox", size: "1024x1024", n: 2, referenceImages: [{ bytes, mime: "image/png" }] })
|
|
125
|
+
const speech = await media.synthesizeSpeech({ text: "Hello", voice: "marin", format: "mp3" }) // { bytes, mime, model, provider, usage: { characters } }
|
|
126
|
+
const { jobId } = await media.startVideo({ prompt: "waves at dusk", seconds: 8, size: "16:9" })
|
|
127
|
+
const state = await media.getVideo(jobId) // { status: "running" } | { status: "done", bytes, mime, model } | { status: "failed", error, failureClass }
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
| kind | provider (key) | models | documentation read 2026-11-17 |
|
|
131
|
+
|---|---|---|---|
|
|
132
|
+
| image | OpenAI (`OPENAI_API_KEY`) | `gpt-image-2.5-flare`, `gpt-image-2.5-sunburst`; reference images through `/v1/images/edits` | developers.openai.com/api/reference/resources/images/methods/generate, developers.openai.com/api/docs/guides/image-generation |
|
|
133
|
+
| image | Google (`GEMINI_API_KEY`) | `gemini-nano-banana-2.1`, `gemini-3-pro-image`, `gemini-3.1-flash-lite-image` through the Interactions API | aistudio.google.com/docs/image-generation |
|
|
134
|
+
| speech | OpenAI | `gpt-4o-mini-tts` (the page says it shuts down 2027-01-06) | developers.openai.com/api/docs/guides/text-to-speech |
|
|
135
|
+
| speech | ElevenLabs (`ELEVENLABS_API_KEY`, `ELEVENLABS_VOICE_ID` for the default voice) | `eleven_multilingual_v2` | elevenlabs.io/docs/api-reference/text-to-speech/convert |
|
|
136
|
+
| video | Google | `gemini-omni-1.2-flash`, 3 to 15 seconds | aistudio.google.com/docs/omni |
|
|
137
|
+
|
|
138
|
+
A call without `model` tries the available models of its kind in the order above and each model's keys in order; a 401 or 403 moves to the next key, a 429, 5xx, timeout or network failure to the next model, and a content-policy refusal or an invalid request stops at once. `model` and `provider` pin a call. `OPENAI_BASE_URL` and `createMedia({ baseUrls })` move a base URL; `fetch` replaces the HTTP client (the specs use fakes).
|
|
139
|
+
|
|
140
|
+
Usage is what the vendor pages state: images report `units` (the images returned, plus OpenAI's token counts), speech reports `characters` (neither page documents a duration). `size` is `<width>x<height>` for OpenAI and read as an aspect ratio by Google (`16:9` is also accepted); video reads only its orientation.
|
|
141
|
+
|
|
142
|
+
Every failure is a `MediaError` with `failureClass`: `refused` (content policy: OpenAI's moderation block, Google answering text and no image, a blocked video prompt), `transient` (429, 408, 409, 425, 5xx, timeout, network; `retryable` is true), `auth` (401, 403), `invalid_request` (any other 4xx, a bad argument, a voice or format no available provider serves), `not_found` (an unknown video job) and `unavailable` (no provider with a key serves the kind). Its `code` is `AI_ROUTER_MEDIA_<CLASS>`; no message holds a key.
|
|
143
|
+
|
|
144
|
+
Video is a job: `startVideo` returns `{ jobId, provider }` at once and `getVideo` polls. Google's page documents the video call as one synchronous request with no background mode, so the job is that request held by this process: a restart loses a running job (`getVideo` then answers `not_found`), and a finished job is kept one hour. OpenAI video is not offered because the Sora models and the Videos API are shut down, Google Veo because its preview models are shut down, and fal because its queue page documents no video model, input or output.
|
|
145
|
+
|
|
113
146
|
## Run it as a gateway
|
|
114
147
|
|
|
115
148
|
One process (one container) holds the pool, the virtual keys and the admin API. The package declares no `bin` (the
|
package/dist/gateway/index.d.ts
CHANGED
|
@@ -7,6 +7,8 @@ export { keysFromEnv, poolFromEnv, providersFromEnv, virtualKeyFromEnv } from ".
|
|
|
7
7
|
export type { EnvPoolSpec, EnvProviderSpec, EnvSource, VirtualKeyEnvSpec } from "../env.js";
|
|
8
8
|
export { availableModels, clearOpenRouterCatalogCache, defaultProviderSpecs, EFFORT_TIERS, fetchOpenRouterModels, KNOWN_MODELS, knownModelConfigs, withAutoProfiles } from "../known-models.js";
|
|
9
9
|
export type { AvailableModel, DefaultProvidersOptions, EffortTier, KnownModel, OpenRouterCatalogOptions } from "../known-models.js";
|
|
10
|
+
export { AI_ROUTER_MEDIA_ERROR_CODES, classifyMediaStatus, createMedia, MEDIA_CATALOGUE, MediaError } from "../media.js";
|
|
11
|
+
export type { AvailableMediaModel, CatalogueEntry, ImageRequest, ImageResult, MediaBlob, MediaErrorCode, MediaFailureClass, MediaKind, MediaOptions, MediaRouter, SpeechRequest, SpeechResult, VideoJob, VideoRequest, VideoStatus } from "../media.js";
|
|
10
12
|
export { PROVIDER_PRESETS } from "../providers.js";
|
|
11
13
|
export type { Protocol, ProviderPreset, QuotaWindow } from "../providers.js";
|
|
12
14
|
export { openSqliteStateStore, redisStateStore, STATE_STORE_SCHEMA } from "../state-stores.js";
|
package/dist/gateway/index.js
CHANGED
|
@@ -3,6 +3,7 @@ export { createRouter } from "../router.js";
|
|
|
3
3
|
export { classifyFailure, readProviderError } from "../classify.js";
|
|
4
4
|
export { keysFromEnv, poolFromEnv, providersFromEnv, virtualKeyFromEnv } from "../env.js";
|
|
5
5
|
export { availableModels, clearOpenRouterCatalogCache, defaultProviderSpecs, EFFORT_TIERS, fetchOpenRouterModels, KNOWN_MODELS, knownModelConfigs, withAutoProfiles } from "../known-models.js";
|
|
6
|
+
export { AI_ROUTER_MEDIA_ERROR_CODES, classifyMediaStatus, createMedia, MEDIA_CATALOGUE, MediaError } from "../media.js";
|
|
6
7
|
export { PROVIDER_PRESETS } from "../providers.js";
|
|
7
8
|
export { openSqliteStateStore, redisStateStore, STATE_STORE_SCHEMA } from "../state-stores.js";
|
|
8
9
|
export { EMPTY_RECORD, MemoryStateStore, parseHealthRecord, toHealthRecord } from "../store.js";
|
package/dist/media.d.ts
ADDED
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
import { DomainError } from "@starci/common";
|
|
2
|
+
import type { EnvSource } from "./env.js";
|
|
3
|
+
import type { FetchLike } from "./types.js";
|
|
4
|
+
/**
|
|
5
|
+
* Media generation beside chat: images, speech and video, key-gated like the chat providers. A provider whose key is not in
|
|
6
|
+
* env is absent from `availableMediaModels()` and nothing fails at boot. Sources, read 2026-11-17 (a fact a page does not
|
|
7
|
+
* state is left out, never guessed):
|
|
8
|
+
*
|
|
9
|
+
* - OpenAI images, https://developers.openai.com/api/reference/resources/images/methods/generate (`POST /v1/images/generations`:
|
|
10
|
+
* `model`, `prompt`, `n` 1 to 10, `size`; GPT image models always answer `data[].b64_json` and `usage`) and
|
|
11
|
+
* https://developers.openai.com/api/docs/guides/image-generation (`gpt-image-2.5-flare`, `gpt-image-2.5-sunburst`; reference
|
|
12
|
+
* images go to `POST /v1/images/edits` as multipart `image[]`).
|
|
13
|
+
* - OpenAI speech, https://developers.openai.com/api/docs/guides/text-to-speech (`POST /v1/audio/speech`: `gpt-4o-mini-tts`,
|
|
14
|
+
* `input`, `voice`, `response_format` mp3|opus|aac|flac|wav|pcm, binary answer; the page says the model shuts down on
|
|
15
|
+
* 2027-01-06). The page gives no usage, so usage is the characters sent.
|
|
16
|
+
* - Google images, https://aistudio.google.com/docs/image-generation (`POST /v1beta/interactions`, header `x-goog-api-key`:
|
|
17
|
+
* `model`, `input` as text and `{type:"image", data, mime_type}` parts, `response_format {type:"image", aspect_ratio}`;
|
|
18
|
+
* the answer's `steps[]` of type `model_output` hold `content[]` blocks of type `image` with base64 `data`). The page shows
|
|
19
|
+
* no usage fields and no blocked-result shape: usage is the images returned, and an answer with no image is a refusal.
|
|
20
|
+
* - Google video, https://aistudio.google.com/docs/omni (`gemini-omni-1.2-flash`, same endpoint, `response_format {type:"video",
|
|
21
|
+
* aspect_ratio, duration:"<n>s"}` 3s to 15s; the answer's `model_output` step holds a `video/mp4` block with base64 `data`).
|
|
22
|
+
* The page documents this call as synchronous, with no background mode or polling, so the job lives in this process.
|
|
23
|
+
* - ElevenLabs, https://elevenlabs.io/docs/api-reference/text-to-speech/convert (`POST /v1/text-to-speech/{voice_id}`, header
|
|
24
|
+
* `xi-api-key`, body `text` and `model_id`, query `output_format` default `mp3_44100_128`, binary answer, 422 validation).
|
|
25
|
+
*
|
|
26
|
+
* Left out: OpenAI video (the Sora models and the Videos API are shut down, per https://developers.openai.com/api/docs/guides/video-generation),
|
|
27
|
+
* Google Veo (its preview models are shut down, per https://aistudio.google.com/docs/video) and fal (https://fal.ai/docs/documentation/model-apis/inference/queue
|
|
28
|
+
* documents the queue only; no video model id, input or output is stated there).
|
|
29
|
+
*/
|
|
30
|
+
export type MediaKind = "image" | "speech" | "video";
|
|
31
|
+
/** Why a media call failed: `refused` the content policy, `transient` a 429, 5xx, timeout or network failure, `auth` a refused key, `invalid_request` the caller's request, `not_found` an unknown job, `unavailable` no provider with a key serves the kind. */
|
|
32
|
+
export type MediaFailureClass = "refused" | "transient" | "auth" | "invalid_request" | "not_found" | "unavailable";
|
|
33
|
+
declare const MEDIA_CODES: {
|
|
34
|
+
readonly refused: "AI_ROUTER_MEDIA_REFUSED";
|
|
35
|
+
readonly transient: "AI_ROUTER_MEDIA_TRANSIENT";
|
|
36
|
+
readonly auth: "AI_ROUTER_MEDIA_AUTH";
|
|
37
|
+
readonly invalid_request: "AI_ROUTER_MEDIA_INVALID_REQUEST";
|
|
38
|
+
readonly not_found: "AI_ROUTER_MEDIA_NOT_FOUND";
|
|
39
|
+
readonly unavailable: "AI_ROUTER_MEDIA_UNAVAILABLE";
|
|
40
|
+
};
|
|
41
|
+
/** The stable code of a media error class. */
|
|
42
|
+
export declare const AI_ROUTER_MEDIA_ERROR_CODES: Readonly<Record<MediaFailureClass, (typeof MEDIA_CODES)[MediaFailureClass]>>;
|
|
43
|
+
/** A code of `MediaError`. */
|
|
44
|
+
export type MediaErrorCode = (typeof MEDIA_CODES)[MediaFailureClass];
|
|
45
|
+
/** The one error a media call throws; `failureClass` says why, `retryable` whether the same call may succeed later. */
|
|
46
|
+
export declare class MediaError extends DomainError<MediaErrorCode> {
|
|
47
|
+
readonly failureClass: MediaFailureClass;
|
|
48
|
+
readonly retryable: boolean;
|
|
49
|
+
readonly status: number | undefined;
|
|
50
|
+
readonly provider: string | undefined;
|
|
51
|
+
readonly model: string | undefined;
|
|
52
|
+
constructor(init: {
|
|
53
|
+
readonly failureClass: MediaFailureClass;
|
|
54
|
+
readonly message: string;
|
|
55
|
+
readonly status?: number;
|
|
56
|
+
readonly provider?: string;
|
|
57
|
+
readonly model?: string;
|
|
58
|
+
});
|
|
59
|
+
}
|
|
60
|
+
/** Raw media: bytes and their MIME type. */
|
|
61
|
+
export interface MediaBlob {
|
|
62
|
+
readonly bytes: Uint8Array;
|
|
63
|
+
readonly mime: string;
|
|
64
|
+
}
|
|
65
|
+
export interface ImageRequest {
|
|
66
|
+
readonly prompt: string;
|
|
67
|
+
/** `<width>x<height>` (`1024x1024`), or for Google an aspect ratio (`16:9`); Google reads only the shape of a `<width>x<height>`. */
|
|
68
|
+
readonly size?: string;
|
|
69
|
+
/** How many images, 1 to 10 (default 1). */
|
|
70
|
+
readonly n?: number;
|
|
71
|
+
/** Images the result is built from (OpenAI edits, Google input parts). */
|
|
72
|
+
readonly referenceImages?: readonly MediaBlob[];
|
|
73
|
+
/** A model of `availableMediaModels()`; absent or `auto`: the first available, with failover. */
|
|
74
|
+
readonly model?: string;
|
|
75
|
+
/** Keeps the call on one provider id. */
|
|
76
|
+
readonly provider?: string;
|
|
77
|
+
}
|
|
78
|
+
export interface ImageResult {
|
|
79
|
+
readonly images: readonly MediaBlob[];
|
|
80
|
+
readonly model: string;
|
|
81
|
+
readonly provider: string;
|
|
82
|
+
/** `units` is the number of images returned; OpenAI also reports its token counts. */
|
|
83
|
+
readonly usage: {
|
|
84
|
+
readonly units: number;
|
|
85
|
+
readonly inputTokens?: number;
|
|
86
|
+
readonly outputTokens?: number;
|
|
87
|
+
};
|
|
88
|
+
}
|
|
89
|
+
export interface SpeechRequest {
|
|
90
|
+
readonly text: string;
|
|
91
|
+
/** OpenAI: a voice name (default `marin`). ElevenLabs: a voice id (default `ELEVENLABS_VOICE_ID`). */
|
|
92
|
+
readonly voice?: string;
|
|
93
|
+
/** `mp3` (default), or OpenAI `opus|aac|flac|wav|pcm`, or an ElevenLabs `output_format` such as `mp3_44100_128`. */
|
|
94
|
+
readonly format?: string;
|
|
95
|
+
readonly model?: string;
|
|
96
|
+
readonly provider?: string;
|
|
97
|
+
}
|
|
98
|
+
export interface SpeechResult extends MediaBlob {
|
|
99
|
+
readonly model: string;
|
|
100
|
+
readonly provider: string;
|
|
101
|
+
/** The characters sent: neither provider's page documents a duration. */
|
|
102
|
+
readonly usage: {
|
|
103
|
+
readonly characters: number;
|
|
104
|
+
};
|
|
105
|
+
}
|
|
106
|
+
export interface VideoRequest {
|
|
107
|
+
readonly prompt: string;
|
|
108
|
+
/** 3 to 15. */
|
|
109
|
+
readonly seconds?: number;
|
|
110
|
+
/** `16:9`, `9:16` or `<width>x<height>`: only the orientation is sent. */
|
|
111
|
+
readonly size?: string;
|
|
112
|
+
readonly model?: string;
|
|
113
|
+
readonly provider?: string;
|
|
114
|
+
}
|
|
115
|
+
export interface VideoJob {
|
|
116
|
+
readonly jobId: string;
|
|
117
|
+
readonly provider: string;
|
|
118
|
+
}
|
|
119
|
+
export type VideoStatus = {
|
|
120
|
+
readonly status: "running";
|
|
121
|
+
} | ({
|
|
122
|
+
readonly status: "done";
|
|
123
|
+
} & MediaBlob & {
|
|
124
|
+
readonly model: string;
|
|
125
|
+
}) | {
|
|
126
|
+
readonly status: "failed";
|
|
127
|
+
readonly error: string;
|
|
128
|
+
readonly failureClass: MediaFailureClass;
|
|
129
|
+
};
|
|
130
|
+
/** One media model a caller may choose, from the providers whose keys are configured. */
|
|
131
|
+
export interface AvailableMediaModel {
|
|
132
|
+
readonly kind: MediaKind;
|
|
133
|
+
/** The name to ask for. */
|
|
134
|
+
readonly id: string;
|
|
135
|
+
readonly provider: string;
|
|
136
|
+
readonly label: string;
|
|
137
|
+
}
|
|
138
|
+
export interface MediaRouter {
|
|
139
|
+
generateImage(request: ImageRequest, signal?: AbortSignal): Promise<ImageResult>;
|
|
140
|
+
synthesizeSpeech(request: SpeechRequest, signal?: AbortSignal): Promise<SpeechResult>;
|
|
141
|
+
/** Starts a video and returns at once; poll `getVideo`. Jobs live in this process. */
|
|
142
|
+
startVideo(request: VideoRequest): Promise<VideoJob>;
|
|
143
|
+
getVideo(jobId: string): Promise<VideoStatus>;
|
|
144
|
+
/** The models whose provider has a key; empty for a kind nobody serves. */
|
|
145
|
+
availableMediaModels(): AvailableMediaModel[];
|
|
146
|
+
/** The model `auto` picks for a kind among the available ones, or undefined when none. */
|
|
147
|
+
auto(kind: MediaKind): AvailableMediaModel | undefined;
|
|
148
|
+
}
|
|
149
|
+
export interface MediaOptions {
|
|
150
|
+
readonly env?: EnvSource;
|
|
151
|
+
readonly fetch?: FetchLike;
|
|
152
|
+
/** Per HTTP call (default 120000; a video call 600000). */
|
|
153
|
+
readonly timeoutMs?: number;
|
|
154
|
+
/** Moves a provider's base URL: OpenAI otherwise reads `OPENAI_BASE_URL`. */
|
|
155
|
+
readonly baseUrls?: {
|
|
156
|
+
readonly openai?: string;
|
|
157
|
+
readonly google?: string;
|
|
158
|
+
readonly elevenlabs?: string;
|
|
159
|
+
};
|
|
160
|
+
/** Models offered, replacing the built-in catalogue, in `auto` order. */
|
|
161
|
+
readonly catalogue?: readonly CatalogueEntry[];
|
|
162
|
+
/** Clock for job expiry. */
|
|
163
|
+
readonly now?: () => number;
|
|
164
|
+
}
|
|
165
|
+
type ProviderId = "openai" | "google" | "elevenlabs";
|
|
166
|
+
/** A model the catalogue offers; `provider` is `openai`, `google` or `elevenlabs`. */
|
|
167
|
+
export interface CatalogueEntry {
|
|
168
|
+
readonly kind: MediaKind;
|
|
169
|
+
readonly provider: ProviderId;
|
|
170
|
+
readonly id: string;
|
|
171
|
+
readonly label: string;
|
|
172
|
+
}
|
|
173
|
+
/** The built-in catalogue in `auto` order (see the sources above). */
|
|
174
|
+
export declare const MEDIA_CATALOGUE: readonly CatalogueEntry[];
|
|
175
|
+
/** Classifies an HTTP failure: 401/403 auth, 408/409/425/429/5xx transient, a 400 or 422 naming a policy refused, any other 4xx an invalid request. */
|
|
176
|
+
export declare function classifyMediaStatus(status: number, body: unknown): MediaFailureClass;
|
|
177
|
+
/**
|
|
178
|
+
* Media over the providers whose keys are in env (`OPENAI_API_KEY`, `GEMINI_API_KEY`, `ELEVENLABS_API_KEY`, each with `_2`.. or
|
|
179
|
+
* `_API_KEYS` twins). With no key, a provider is absent and nothing fails at boot. A call without `model` tries the available
|
|
180
|
+
* models of its kind in catalogue order and its keys in order, moving on after an `auth` or `transient` failure; a `refused` or
|
|
181
|
+
* `invalid_request` stops at once, since another provider would not make it the caller's right.
|
|
182
|
+
*/
|
|
183
|
+
export declare function createMedia(options?: MediaOptions): MediaRouter;
|
|
184
|
+
export {};
|
package/dist/media.js
ADDED
|
@@ -0,0 +1,363 @@
|
|
|
1
|
+
import { DomainError, isPlainRecord } from "@starci/common";
|
|
2
|
+
import { keysFromEnv } from "./env.js";
|
|
3
|
+
import { scrub } from "./errors.js";
|
|
4
|
+
const MEDIA_CODES = {
|
|
5
|
+
refused: "AI_ROUTER_MEDIA_REFUSED",
|
|
6
|
+
transient: "AI_ROUTER_MEDIA_TRANSIENT",
|
|
7
|
+
auth: "AI_ROUTER_MEDIA_AUTH",
|
|
8
|
+
invalid_request: "AI_ROUTER_MEDIA_INVALID_REQUEST",
|
|
9
|
+
not_found: "AI_ROUTER_MEDIA_NOT_FOUND",
|
|
10
|
+
unavailable: "AI_ROUTER_MEDIA_UNAVAILABLE",
|
|
11
|
+
};
|
|
12
|
+
/** The stable code of a media error class. */
|
|
13
|
+
export const AI_ROUTER_MEDIA_ERROR_CODES = MEDIA_CODES;
|
|
14
|
+
/** The one error a media call throws; `failureClass` says why, `retryable` whether the same call may succeed later. */
|
|
15
|
+
export class MediaError extends DomainError {
|
|
16
|
+
failureClass;
|
|
17
|
+
retryable;
|
|
18
|
+
status;
|
|
19
|
+
provider;
|
|
20
|
+
model;
|
|
21
|
+
constructor(init) {
|
|
22
|
+
super({ code: MEDIA_CODES[init.failureClass] });
|
|
23
|
+
this.message = init.message;
|
|
24
|
+
this.name = "MediaError";
|
|
25
|
+
this.failureClass = init.failureClass;
|
|
26
|
+
this.retryable = init.failureClass === "transient";
|
|
27
|
+
this.status = init.status;
|
|
28
|
+
this.provider = init.provider;
|
|
29
|
+
this.model = init.model;
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
/** The built-in catalogue in `auto` order (see the sources above). */
|
|
33
|
+
export const MEDIA_CATALOGUE = Object.freeze([
|
|
34
|
+
{ kind: "image", provider: "openai", id: "gpt-image-2.5-flare", label: "GPT Image 2.5 Flare" },
|
|
35
|
+
{ kind: "image", provider: "openai", id: "gpt-image-2.5-sunburst", label: "GPT Image 2.5 Sunburst" },
|
|
36
|
+
{ kind: "image", provider: "google", id: "gemini-nano-banana-2.1", label: "Nano Banana 2.1" },
|
|
37
|
+
{ kind: "image", provider: "google", id: "gemini-3-pro-image", label: "Nano Banana Pro" },
|
|
38
|
+
{ kind: "image", provider: "google", id: "gemini-3.1-flash-lite-image", label: "Nano Banana 2 Lite" },
|
|
39
|
+
{ kind: "speech", provider: "openai", id: "gpt-4o-mini-tts", label: "GPT-4o mini TTS" },
|
|
40
|
+
{ kind: "speech", provider: "elevenlabs", id: "eleven_multilingual_v2", label: "ElevenLabs Multilingual v2" },
|
|
41
|
+
{ kind: "video", provider: "google", id: "gemini-omni-1.2-flash", label: "Gemini Omni 1.2 Flash" },
|
|
42
|
+
]);
|
|
43
|
+
const ENV_PREFIX = { openai: "OPENAI", google: "GEMINI", elevenlabs: "ELEVENLABS" };
|
|
44
|
+
const DEFAULT_URL = {
|
|
45
|
+
openai: "https://api.openai.com/v1",
|
|
46
|
+
google: "https://generativelanguage.googleapis.com/v1beta",
|
|
47
|
+
elevenlabs: "https://api.elevenlabs.io/v1",
|
|
48
|
+
};
|
|
49
|
+
const OPENAI_FORMATS = { mp3: "audio/mpeg", opus: "audio/opus", aac: "audio/aac", flac: "audio/flac", wav: "audio/wav", pcm: "audio/pcm" };
|
|
50
|
+
const REFUSAL = /moderation|content[ _-]?policy|safety|blocked|prohibited|violat|not allowed/i;
|
|
51
|
+
const JOB_TTL_MS = 3_600_000;
|
|
52
|
+
function transient(message, provider, model, status) {
|
|
53
|
+
return new MediaError({ failureClass: "transient", message, provider, model, ...(status === undefined ? {} : { status }) });
|
|
54
|
+
}
|
|
55
|
+
function bytesOf(base64) {
|
|
56
|
+
return new Uint8Array(Buffer.from(base64, "base64"));
|
|
57
|
+
}
|
|
58
|
+
function field(value, name) {
|
|
59
|
+
return isPlainRecord(value) ? value[name] : undefined;
|
|
60
|
+
}
|
|
61
|
+
function textOf(value) {
|
|
62
|
+
return typeof value === "string" ? value : "";
|
|
63
|
+
}
|
|
64
|
+
/** Reads the message and code of an error body in the OpenAI, Gemini or ElevenLabs shape. */
|
|
65
|
+
function errorText(body) {
|
|
66
|
+
const root = Array.isArray(body) ? body[0] : body;
|
|
67
|
+
const error = field(root, "error");
|
|
68
|
+
const detail = field(root, "detail");
|
|
69
|
+
const parts = [textOf(field(error, "code")), textOf(field(error, "type")), textOf(field(error, "status")), textOf(field(error, "message")), textOf(error), textOf(detail), textOf(field(detail, "status")), textOf(field(detail, "message")), textOf(field(root, "message"))];
|
|
70
|
+
if (Array.isArray(detail))
|
|
71
|
+
for (const item of detail)
|
|
72
|
+
parts.push(textOf(field(item, "msg")));
|
|
73
|
+
return parts.filter((part) => part !== "").join(" ");
|
|
74
|
+
}
|
|
75
|
+
/** Classifies an HTTP failure: 401/403 auth, 408/409/425/429/5xx transient, a 400 or 422 naming a policy refused, any other 4xx an invalid request. */
|
|
76
|
+
export function classifyMediaStatus(status, body) {
|
|
77
|
+
if (status === 401 || status === 403)
|
|
78
|
+
return "auth";
|
|
79
|
+
if (status === 408 || status === 409 || status === 425 || status === 429 || status >= 500)
|
|
80
|
+
return "transient";
|
|
81
|
+
if ((status === 400 || status === 422) && REFUSAL.test(errorText(body)))
|
|
82
|
+
return "refused";
|
|
83
|
+
return "invalid_request";
|
|
84
|
+
}
|
|
85
|
+
/**
|
|
86
|
+
* Media over the providers whose keys are in env (`OPENAI_API_KEY`, `GEMINI_API_KEY`, `ELEVENLABS_API_KEY`, each with `_2`.. or
|
|
87
|
+
* `_API_KEYS` twins). With no key, a provider is absent and nothing fails at boot. A call without `model` tries the available
|
|
88
|
+
* models of its kind in catalogue order and its keys in order, moving on after an `auth` or `transient` failure; a `refused` or
|
|
89
|
+
* `invalid_request` stops at once, since another provider would not make it the caller's right.
|
|
90
|
+
*/
|
|
91
|
+
export function createMedia(options = {}) {
|
|
92
|
+
const env = options.env ?? process.env;
|
|
93
|
+
const fetchFn = options.fetch ?? ((input, init) => fetch(input, init));
|
|
94
|
+
const timeoutMs = options.timeoutMs ?? 120_000;
|
|
95
|
+
const now = options.now ?? Date.now;
|
|
96
|
+
const catalogue = options.catalogue ?? MEDIA_CATALOGUE;
|
|
97
|
+
const providers = new Map();
|
|
98
|
+
for (const id of ["openai", "google", "elevenlabs"]) {
|
|
99
|
+
const keys = keysFromEnv(ENV_PREFIX[id], id, env).filter((key) => "resolve" in key);
|
|
100
|
+
if (keys.length === 0)
|
|
101
|
+
continue;
|
|
102
|
+
const own = options.baseUrls?.[id] ?? (id === "openai" ? env.OPENAI_BASE_URL : undefined);
|
|
103
|
+
providers.set(id, { id, keys, baseUrl: (own !== undefined && own.trim() !== "" ? own : DEFAULT_URL[id]).replace(/\/+$/, "") });
|
|
104
|
+
}
|
|
105
|
+
const jobs = new Map();
|
|
106
|
+
let counter = 0;
|
|
107
|
+
const available = () => catalogue.filter((entry) => providers.has(entry.provider)).map(({ kind, id, provider, label }) => ({ kind, id, provider, label }));
|
|
108
|
+
function candidates(kind, model, provider) {
|
|
109
|
+
const all = catalogue.filter((entry) => entry.kind === kind && providers.has(entry.provider));
|
|
110
|
+
if (all.length === 0)
|
|
111
|
+
throw new MediaError({ failureClass: "unavailable", message: `no provider with a key serves ${kind}` });
|
|
112
|
+
const named = model === undefined || model === "auto" ? undefined : model;
|
|
113
|
+
const out = all.filter((entry) => (named === undefined || entry.id === named) && (provider === undefined || entry.provider === provider));
|
|
114
|
+
if (out.length === 0)
|
|
115
|
+
throw new MediaError({ failureClass: "invalid_request", message: `no available ${kind} model matches ${named ?? "auto"}${provider === undefined ? "" : ` on ${provider}`}` });
|
|
116
|
+
return out;
|
|
117
|
+
}
|
|
118
|
+
async function http(entry, key, url, init, signal, limitMs) {
|
|
119
|
+
const timeout = AbortSignal.timeout(limitMs);
|
|
120
|
+
try {
|
|
121
|
+
return await fetchFn(url, { ...init, signal: signal === undefined ? timeout : AbortSignal.any([signal, timeout]) });
|
|
122
|
+
}
|
|
123
|
+
catch (error) {
|
|
124
|
+
if (signal?.aborted === true)
|
|
125
|
+
throw error;
|
|
126
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
127
|
+
throw transient(scrub(`${entry.provider} did not answer: ${message}`, [key]), entry.provider, entry.id);
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
async function ok(entry, key, response) {
|
|
131
|
+
if (response.ok)
|
|
132
|
+
return response;
|
|
133
|
+
const raw = await response.text().catch(() => "");
|
|
134
|
+
let body = raw;
|
|
135
|
+
try {
|
|
136
|
+
body = JSON.parse(raw);
|
|
137
|
+
}
|
|
138
|
+
catch {
|
|
139
|
+
/* a non-JSON error body stays text */
|
|
140
|
+
}
|
|
141
|
+
const failureClass = classifyMediaStatus(response.status, body);
|
|
142
|
+
const detail = scrub(errorText(body) || raw.slice(0, 300), [key]);
|
|
143
|
+
throw new MediaError({ failureClass, message: `${entry.provider} answered ${String(response.status)}${detail === "" ? "" : `: ${detail}`}`, status: response.status, provider: entry.provider, model: entry.id });
|
|
144
|
+
}
|
|
145
|
+
/** Tries each candidate with each of its keys: `skip` marks a candidate that cannot serve the request. */
|
|
146
|
+
async function attempt(kind, request, serve) {
|
|
147
|
+
let last;
|
|
148
|
+
let skipped = 0;
|
|
149
|
+
const list = candidates(kind, request.model, request.provider);
|
|
150
|
+
for (const entry of list) {
|
|
151
|
+
const resolved = providers.get(entry.provider);
|
|
152
|
+
if (resolved === undefined)
|
|
153
|
+
continue;
|
|
154
|
+
for (const keyConfig of resolved.keys) {
|
|
155
|
+
const key = await keyConfig.resolve();
|
|
156
|
+
if (key === undefined || key.trim() === "")
|
|
157
|
+
continue;
|
|
158
|
+
let pending;
|
|
159
|
+
try {
|
|
160
|
+
pending = serve(entry, resolved.baseUrl, key);
|
|
161
|
+
if (pending === "skip") {
|
|
162
|
+
skipped += 1;
|
|
163
|
+
break;
|
|
164
|
+
}
|
|
165
|
+
return await pending;
|
|
166
|
+
}
|
|
167
|
+
catch (error) {
|
|
168
|
+
if (!(error instanceof MediaError))
|
|
169
|
+
throw error;
|
|
170
|
+
last = error;
|
|
171
|
+
if (error.failureClass === "refused" || error.failureClass === "invalid_request")
|
|
172
|
+
throw error;
|
|
173
|
+
if (error.failureClass === "transient" && error.status === undefined)
|
|
174
|
+
break;
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
}
|
|
178
|
+
if (last !== undefined)
|
|
179
|
+
throw last;
|
|
180
|
+
if (skipped > 0)
|
|
181
|
+
throw new MediaError({ failureClass: "invalid_request", message: `no available ${kind} model supports this request (voice, format or size)` });
|
|
182
|
+
throw new MediaError({ failureClass: "unavailable", message: `no key resolved for ${kind}` });
|
|
183
|
+
}
|
|
184
|
+
function ratioOf(size) {
|
|
185
|
+
if (size === undefined)
|
|
186
|
+
return undefined;
|
|
187
|
+
if (/^\d+:\d+$/.test(size))
|
|
188
|
+
return size;
|
|
189
|
+
const match = /^(\d+)x(\d+)$/.exec(size);
|
|
190
|
+
if (match === null)
|
|
191
|
+
return undefined;
|
|
192
|
+
const [width, height] = [Number(match[1]), Number(match[2])];
|
|
193
|
+
const divisor = ((a, b) => { while (b !== 0)
|
|
194
|
+
[a, b] = [b, a % b]; return a; })(width, height);
|
|
195
|
+
return divisor === 0 ? undefined : `${String(width / divisor)}:${String(height / divisor)}`;
|
|
196
|
+
}
|
|
197
|
+
async function openaiImage(entry, base, key, request, signal) {
|
|
198
|
+
const refs = request.referenceImages ?? [];
|
|
199
|
+
let response;
|
|
200
|
+
if (refs.length === 0) {
|
|
201
|
+
const body = { model: entry.id, prompt: request.prompt, ...(request.n === undefined ? {} : { n: request.n }), ...(request.size === undefined ? {} : { size: request.size }) };
|
|
202
|
+
response = await http(entry, key, `${base}/images/generations`, { method: "POST", headers: { authorization: `Bearer ${key}`, "content-type": "application/json" }, body: JSON.stringify(body) }, signal, timeoutMs);
|
|
203
|
+
}
|
|
204
|
+
else {
|
|
205
|
+
const form = new FormData();
|
|
206
|
+
form.set("model", entry.id);
|
|
207
|
+
form.set("prompt", request.prompt);
|
|
208
|
+
if (request.n !== undefined)
|
|
209
|
+
form.set("n", String(request.n));
|
|
210
|
+
if (request.size !== undefined)
|
|
211
|
+
form.set("size", request.size);
|
|
212
|
+
for (const [index, ref] of refs.entries())
|
|
213
|
+
form.append("image[]", new Blob([ref.bytes], { type: ref.mime }), `reference-${String(index + 1)}`);
|
|
214
|
+
response = await http(entry, key, `${base}/images/edits`, { method: "POST", headers: { authorization: `Bearer ${key}` }, body: form }, signal, timeoutMs);
|
|
215
|
+
}
|
|
216
|
+
const json = await (await ok(entry, key, response)).json().catch(() => undefined);
|
|
217
|
+
const data = field(json, "data");
|
|
218
|
+
const images = [];
|
|
219
|
+
const format = textOf(field(json, "output_format")) || "png";
|
|
220
|
+
for (const item of Array.isArray(data) ? data : []) {
|
|
221
|
+
const encoded = textOf(field(item, "b64_json"));
|
|
222
|
+
if (encoded !== "")
|
|
223
|
+
images.push({ bytes: bytesOf(encoded), mime: `image/${format}` });
|
|
224
|
+
}
|
|
225
|
+
if (images.length === 0)
|
|
226
|
+
throw transient("openai answered no image", entry.provider, entry.id);
|
|
227
|
+
const usage = field(json, "usage");
|
|
228
|
+
const input = field(usage, "input_tokens");
|
|
229
|
+
const output = field(usage, "output_tokens");
|
|
230
|
+
return { images, model: entry.id, provider: entry.provider, usage: { units: images.length, ...(typeof input === "number" ? { inputTokens: input } : {}), ...(typeof output === "number" ? { outputTokens: output } : {}) } };
|
|
231
|
+
}
|
|
232
|
+
function googleOutputs(json, type) {
|
|
233
|
+
const blocks = [];
|
|
234
|
+
let text = "";
|
|
235
|
+
const steps = field(json, "steps");
|
|
236
|
+
for (const step of Array.isArray(steps) ? steps : []) {
|
|
237
|
+
if (field(step, "type") !== "model_output")
|
|
238
|
+
continue;
|
|
239
|
+
const content = field(step, "content");
|
|
240
|
+
for (const block of Array.isArray(content) ? content : []) {
|
|
241
|
+
if (!isPlainRecord(block))
|
|
242
|
+
continue;
|
|
243
|
+
if (block.type === type && typeof block.data === "string")
|
|
244
|
+
blocks.push(block);
|
|
245
|
+
else if (block.type === "text")
|
|
246
|
+
text += textOf(block.text);
|
|
247
|
+
}
|
|
248
|
+
}
|
|
249
|
+
return { blocks, text };
|
|
250
|
+
}
|
|
251
|
+
async function googleCall(entry, base, key, body, limitMs, signal) {
|
|
252
|
+
const response = await http(entry, key, `${base}/interactions`, { method: "POST", headers: { "x-goog-api-key": key, "content-type": "application/json" }, body: JSON.stringify(body) }, signal, limitMs);
|
|
253
|
+
return (await ok(entry, key, response)).json().catch(() => undefined);
|
|
254
|
+
}
|
|
255
|
+
async function googleImage(entry, base, key, request, signal) {
|
|
256
|
+
const input = [{ type: "text", text: request.prompt }, ...(request.referenceImages ?? []).map((ref) => ({ type: "image", data: Buffer.from(ref.bytes).toString("base64"), mime_type: ref.mime }))];
|
|
257
|
+
const ratio = ratioOf(request.size);
|
|
258
|
+
const body = { model: entry.id, input, ...(ratio === undefined ? {} : { response_format: { type: "image", aspect_ratio: ratio } }) };
|
|
259
|
+
const images = [];
|
|
260
|
+
let refusal = "";
|
|
261
|
+
for (let at = 0; at < (request.n ?? 1); at++) {
|
|
262
|
+
const { blocks, text } = googleOutputs(await googleCall(entry, base, key, body, timeoutMs, signal), "image");
|
|
263
|
+
for (const block of blocks)
|
|
264
|
+
images.push({ bytes: bytesOf(textOf(block.data)), mime: textOf(block.mime_type) || "image/png" });
|
|
265
|
+
if (blocks.length === 0)
|
|
266
|
+
refusal = text;
|
|
267
|
+
}
|
|
268
|
+
if (images.length === 0)
|
|
269
|
+
throw new MediaError({ failureClass: "refused", message: `google returned no image${refusal === "" ? "" : `: ${refusal.slice(0, 300)}`}`, provider: entry.provider, model: entry.id });
|
|
270
|
+
return { images, model: entry.id, provider: entry.provider, usage: { units: images.length } };
|
|
271
|
+
}
|
|
272
|
+
async function openaiSpeech(entry, base, key, request, signal) {
|
|
273
|
+
const format = request.format ?? "mp3";
|
|
274
|
+
const body = { model: entry.id, input: request.text, voice: request.voice ?? "marin", response_format: format };
|
|
275
|
+
const response = await ok(entry, key, await http(entry, key, `${base}/audio/speech`, { method: "POST", headers: { authorization: `Bearer ${key}`, "content-type": "application/json" }, body: JSON.stringify(body) }, signal, timeoutMs));
|
|
276
|
+
const bytes = new Uint8Array(await response.arrayBuffer());
|
|
277
|
+
if (bytes.length === 0)
|
|
278
|
+
throw transient("openai answered no audio", entry.provider, entry.id);
|
|
279
|
+
return { bytes, mime: response.headers.get("content-type") ?? OPENAI_FORMATS[format] ?? "application/octet-stream", model: entry.id, provider: entry.provider, usage: { characters: request.text.length } };
|
|
280
|
+
}
|
|
281
|
+
async function elevenSpeech(entry, base, key, voice, request, signal) {
|
|
282
|
+
const format = request.format === undefined || request.format === "mp3" ? undefined : request.format;
|
|
283
|
+
const query = format === undefined ? "" : `?output_format=${encodeURIComponent(format)}`;
|
|
284
|
+
const response = await ok(entry, key, await http(entry, key, `${base}/text-to-speech/${encodeURIComponent(voice)}${query}`, { method: "POST", headers: { "xi-api-key": key, "content-type": "application/json" }, body: JSON.stringify({ text: request.text, model_id: entry.id }) }, signal, timeoutMs));
|
|
285
|
+
const bytes = new Uint8Array(await response.arrayBuffer());
|
|
286
|
+
if (bytes.length === 0)
|
|
287
|
+
throw transient("elevenlabs answered no audio", entry.provider, entry.id);
|
|
288
|
+
const fallback = format === undefined || format.startsWith("mp3") ? "audio/mpeg" : format.startsWith("wav") ? "audio/wav" : format.startsWith("opus") ? "audio/opus" : format.startsWith("pcm") ? "audio/pcm" : "application/octet-stream";
|
|
289
|
+
return { bytes, mime: response.headers.get("content-type") ?? fallback, model: entry.id, provider: entry.provider, usage: { characters: request.text.length } };
|
|
290
|
+
}
|
|
291
|
+
function videoBody(entry, request) {
|
|
292
|
+
const format = { type: "video" };
|
|
293
|
+
const ratio = ratioOf(request.size);
|
|
294
|
+
const parts = ratio?.split(":").map(Number) ?? [];
|
|
295
|
+
if (parts.length === 2)
|
|
296
|
+
format.aspect_ratio = (parts[0] ?? 1) >= (parts[1] ?? 1) ? "16:9" : "9:16";
|
|
297
|
+
if (request.seconds !== undefined)
|
|
298
|
+
format.duration = `${String(request.seconds)}s`;
|
|
299
|
+
return { model: entry.id, input: request.prompt, response_format: format };
|
|
300
|
+
}
|
|
301
|
+
function prune() {
|
|
302
|
+
for (const [id, job] of jobs)
|
|
303
|
+
if (now() - job.at > JOB_TTL_MS && job.state.status !== "running")
|
|
304
|
+
jobs.delete(id);
|
|
305
|
+
}
|
|
306
|
+
return {
|
|
307
|
+
async generateImage(request, signal) {
|
|
308
|
+
if (request.prompt.trim() === "")
|
|
309
|
+
throw new MediaError({ failureClass: "invalid_request", message: "an image needs a prompt" });
|
|
310
|
+
if (request.n !== undefined && (!Number.isInteger(request.n) || request.n < 1 || request.n > 10))
|
|
311
|
+
throw new MediaError({ failureClass: "invalid_request", message: "n is 1 to 10" });
|
|
312
|
+
return attempt("image", request, (entry, base, key) => (entry.provider === "openai" ? openaiImage(entry, base, key, request, signal) : googleImage(entry, base, key, request, signal)));
|
|
313
|
+
},
|
|
314
|
+
async synthesizeSpeech(request, signal) {
|
|
315
|
+
if (request.text.trim() === "")
|
|
316
|
+
throw new MediaError({ failureClass: "invalid_request", message: "speech needs text" });
|
|
317
|
+
return attempt("speech", request, (entry, base, key) => {
|
|
318
|
+
if (entry.provider === "openai")
|
|
319
|
+
return request.format !== undefined && !(request.format in OPENAI_FORMATS) ? "skip" : openaiSpeech(entry, base, key, request, signal);
|
|
320
|
+
const voice = request.voice ?? env.ELEVENLABS_VOICE_ID;
|
|
321
|
+
if (voice === undefined || voice.trim() === "")
|
|
322
|
+
return "skip";
|
|
323
|
+
if (request.format !== undefined && request.format !== "mp3" && !/^[a-z0-9]+_\d+/.test(request.format))
|
|
324
|
+
return "skip";
|
|
325
|
+
return elevenSpeech(entry, base, key, voice, request, signal);
|
|
326
|
+
});
|
|
327
|
+
},
|
|
328
|
+
async startVideo(request) {
|
|
329
|
+
if (request.prompt.trim() === "")
|
|
330
|
+
throw new MediaError({ failureClass: "invalid_request", message: "a video needs a prompt" });
|
|
331
|
+
if (request.seconds !== undefined && (request.seconds < 3 || request.seconds > 15))
|
|
332
|
+
throw new MediaError({ failureClass: "invalid_request", message: "seconds is 3 to 15" });
|
|
333
|
+
const [entry] = candidates("video", request.model, request.provider);
|
|
334
|
+
const resolved = entry === undefined ? undefined : providers.get(entry.provider);
|
|
335
|
+
const key = resolved === undefined ? undefined : await resolved.keys[0]?.resolve();
|
|
336
|
+
if (entry === undefined || resolved === undefined || key === undefined || key.trim() === "")
|
|
337
|
+
throw new MediaError({ failureClass: "unavailable", message: "no key resolved for video" });
|
|
338
|
+
prune();
|
|
339
|
+
counter += 1;
|
|
340
|
+
const jobId = `${entry.provider}:${String(now())}-${String(counter)}`;
|
|
341
|
+
const job = { provider: entry.provider, model: entry.id, at: now(), state: { status: "running" } };
|
|
342
|
+
jobs.set(jobId, job);
|
|
343
|
+
void googleCall(entry, resolved.baseUrl, key, videoBody(entry, request), 600_000, undefined).then((json) => {
|
|
344
|
+
const { blocks, text } = googleOutputs(json, "video");
|
|
345
|
+
const block = blocks[0];
|
|
346
|
+
job.state = block === undefined
|
|
347
|
+
? { status: "failed", failureClass: "refused", error: `google returned no video${text === "" ? "" : `: ${text.slice(0, 300)}`}` }
|
|
348
|
+
: { status: "done", bytes: bytesOf(textOf(block.data)), mime: textOf(block.mime_type) || "video/mp4", model: entry.id };
|
|
349
|
+
}, (error) => {
|
|
350
|
+
job.state = error instanceof MediaError ? { status: "failed", failureClass: error.failureClass, error: error.message } : { status: "failed", failureClass: "transient", error: scrub(error instanceof Error ? error.message : String(error), [key]) };
|
|
351
|
+
});
|
|
352
|
+
return { jobId, provider: entry.provider };
|
|
353
|
+
},
|
|
354
|
+
getVideo(jobId) {
|
|
355
|
+
const job = jobs.get(jobId);
|
|
356
|
+
if (job === undefined)
|
|
357
|
+
return Promise.reject(new MediaError({ failureClass: "not_found", message: `no video job ${jobId} in this process` }));
|
|
358
|
+
return Promise.resolve(job.state);
|
|
359
|
+
},
|
|
360
|
+
availableMediaModels: available,
|
|
361
|
+
auto: (kind) => available().find((entry) => entry.kind === kind),
|
|
362
|
+
};
|
|
363
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@starci/ai-router",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.9.0",
|
|
4
4
|
"description": "A router for chat and embeddings calls across many providers (Anthropic, OpenRouter, Qwen, DeepSeek, OpenAI, Gemini, Groq and any OpenAI-compatible endpoint), each with an array of keys, behind one virtual key: a gateway that speaks the OpenAI and the Anthropic Messages protocols with virtual keys, quotas and usage, LangChain clients over it (@starci/ai-router/langchain), a NestJS module (AiRouterModule, AiRouterService, AiRouterGatewayModule), and createRouter for plain Node, with health-aware selection, a persisted key blacklist for spent quotas and refused secrets, circuit breakers, rate-limit awareness, failover across providers and models by profile, prompt-cache breakpoints and cache-priced usage, pinned embeddings, config from env and redacted telemetry.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai",
|
|
@@ -138,6 +138,6 @@
|
|
|
138
138
|
"optional": "true"
|
|
139
139
|
}
|
|
140
140
|
},
|
|
141
|
-
"gitHead": "
|
|
142
|
-
"starciSource": "
|
|
141
|
+
"gitHead": "3bcf7d0e39884f6d68afbadc3f0d6a7fe9ef39ce",
|
|
142
|
+
"starciSource": "fa6c5ecc9f673b39fdd902558dc2145d00e5ac622a44374a9c8942b880ae12d3"
|
|
143
143
|
}
|