@vibedev-si/dsh-vibedev 0.3.0 → 0.3.2

package/README.md

@@ -3,7 +3,7 @@
 在 DeepSeek Harness 里登录 VibeDev 账号，和 DeepSeek 账号并存：
 
 - **VibeDev 的模型**：登录后，模型选择器里会多出「VibeDev」一组（Claude、GPT 等网关提供的模型），和 DeepSeek 的模型并列，选中就能用；
-- **媒体生成**：Agent 可以生成图片、视频、音乐和播客，并把语音转成文字，结果保存在工作区的 `media/` 目录下；
+- **媒体生成**：Agent 可以生成图片、视频、音乐、播客和语音，并把语音转成文字，结果保存在工作区的 `media/` 目录下；
 - **网页搜索**：提供走 VibeDev 网关的网页搜索，见下面的「网页搜索」。
 
 费用都从你的 VibeDev 余额里扣，DeepSeek 账号照常由 DeepSeek Harness 自己管理，两边互不影响。还没有 VibeDev 账号的话，可以在 [VibeDev](https://api.vibedev.studio) 注册。
@@ -65,8 +65,8 @@
 | --- | --- |
 | `image_generate` | 生成图片；带参考图时用来改图、换风格或把产品放进场景。生成的图片 Agent 自己能看到。 |
 | `video_generate` | 生成视频，在后台运行，一般 1 到 10 分钟。完成后会通知 Agent，文件保存在 `media/videos/`。 |
-| `audio_generate` | 生成音乐（可以自带歌词）或一期播客（围绕一个话题或一个网页）。 |
-| `audio_transcribe` | 把语音转成文字（目前只支持普通话）。 |
+| `audio_generate` | 生成音乐（可以自带歌词）、一期播客（围绕一个话题或一个网页），或一段语音（朗读文字，可以用参考音频指定音色）。 |
+| `audio_transcribe` | 把语音转成文字（以文件上传，语言和时长限制以模型声明为准，目前常见为普通话）。 |
 | `media_models` | 列出账号可用的媒体模型和价格；视频模型还会列出每种模式接受哪些输入。 |
 | `media_tasks` | 查看进行中和已完成的视频、音频任务。 |
 | `media_account` | 查看用的是哪个 VibeDev 账号；登录或退出（和侧边栏、设置里的登录是同一个）。 |
@@ -81,11 +81,23 @@
 
 每个视频模型支持的模式、参考素材数量、时长、画面比例和分辨率都不一样。插件会按网关下发的模型能力逐项检查：模型不支持的请求，会在上传和扣费之前就被拒绝，并说明这个模型支持什么。所有参考素材会先上传到 VibeDev 网关的素材库，网关会检查格式，并测量参考视频的时长。
 
+### 音频模型
+
+音频模型是音乐、播客、语音还是转写，只看网关模型目录里 `audio.generation_kind` 的声明，不按模型名称猜；没有声明类型的模型会列出来，但不能用来生成。每个模型只提交它在 `request_fields` 里声明的字段，正文写进 `prompt_field` 指定的字段：
+
+- 音乐：`prompt`、`lyrics`、`genre`、`mood`、`gender`；
+- 播客：`prompt`、`source_url`；
+- 语音：按模型声明，例如 Seed 是 `text_prompt`、`references`（参考音频）、`audio_config`（输出格式和采样率）、`watermark`。
+
+参考音频的数量、大小、时长、不能同时使用的字段，以及 WAV、MP3 等格式各自支持的采样率，都以网关下发的能力为准。音乐和播客不接受参考音频、格式、采样率或时长参数。
+
 ## 费用
 
 - VibeDev 的聊天模型按用量计费，价格以 VibeDev 网关为准。
-- 图片按张计费，视频按秒计费（参考视频的时长也计入），音乐和播客按次计费。`media_models` 会列出当前价格。
-- 视频和音频只在成功时扣费，失败不扣。视频提交后无法取消。
+- 图片按张计费，视频按秒计费（参考视频的时长也计入），音乐和播客按次计费，语音按原始秒计费（按服务商返回的原始时长，可能和播放时长不同），转写按分钟计费。`media_models` 会列出当前价格；目录没有给出价格的模型显示「未定价」，不代表免费。
+- 视频和音频只在成功时扣费，失败和取消的任务不扣。视频提交后无法取消。
+- 同一次操作只会生成一次：提交没有得到确认时，会用同一个请求键向网关确认结果；查询或下载失败只会重新查询、刷新链接，不会重新提交。结果已经生成、但费用还在结算（`settlement_pending`）时，结果照常保存，不会重新生成。
+- 余额在每次付费请求结束后（对话回复、生成、转写）马上重新读取，侧栏和设置页几秒内就能看到扣费后的余额。
 - 插件设置里的「每次付费生成前先询问」默认关闭。打开后，每次付费的媒体请求之前都会先弹出确认，并显示预估金额。
 
 ## 设置
@@ -96,7 +108,7 @@
 | --- | --- | --- |
 | 每次付费生成前先询问 | 关 | 打开后，每次媒体生成前先确认费用。 |
 | 保存目录 | `media` | 相对于工作区。 |
-| 默认生图 / 视频 / 音乐 / 播客 / 转写模型 | 空 | 留空时由 Agent 按模型目录选择。 |
+| 默认生图 / 视频 / 音乐 / 播客 / 语音合成 / 转写模型 | 空 | 留空时由 Agent 按模型目录选择；可以填模型 id 或别名。 |
 
 另有几项只能在配置文件（`cordis.patch.yml` 中 `dsh-vibedev` 一行）里改：`displayName`（模型选择器里显示的组名，默认 `VibeDev`）、`preferredModels`（排在最前的 VibeDev 模型）、`openBrowserOnSignIn`（登录时是否自动打开系统浏览器，默认开）、`catalogRefreshMinutes`（模型目录刷新间隔，默认 30 分钟）、`client`（登录时报给网关的客户端名称，默认 `vibedev-plugin`）、`primary`（作为应用的主账号显示，默认关，VibeDev 应用里开）。
 
@@ -137,7 +149,7 @@ npm run build
 Sign in to VibeDev in DeepSeek Harness, next to the DeepSeek account:
 
 - **VibeDev models**: once signed in, the model picker gains a "VibeDev" group (Claude, GPT and the other models of the VibeDev gateway) next to DeepSeek's;
-- **Media generation**: the agent generates images, videos, music and podcasts and transcribes speech; results are saved in the workspace under `media/`.
+- **Media generation**: the agent generates images, videos, music, podcasts and speech, and transcribes speech; results are saved in the workspace under `media/`.
 
 Everything is paid from your VibeDev balance; the DeepSeek account stays DeepSeek Harness's own, and the two do not affect each other. To create an account, register at [VibeDev](https://api.vibedev.studio). This plugin was renamed from `dsh-media`.
 
@@ -149,13 +161,19 @@ Everything is paid from your VibeDev balance; the DeepSeek account stays DeepSee
 
 **Creator tools withdrawn (0.2.5)**: the Creator tools line 0.2.4 added at the end of Settings → VibeDev account is gone. It was a cross-plugin recommendation slot, and in the settings page it hurt the layout; a redesign can revisit it later. Only that interface change is reverted: the account, the balance, the model list, the sign-in and saved screenplays, boards and film files are untouched.
 
+**Gateway audio (0.3.1)**: audio models are recognized from the catalog's structured `audio` block (music, podcast, speech, transcription), never from their names; speech generation (Seed) joins music and podcasts; prices show their unit (per generation, per second of original audio, per minute) and "not priced" when the catalog states none; the `vibedevMedia` service gains `startAudio` with an operation key and query-only result reading.
+
+**Balance without the wait (0.3.2)**: the balance is read again right after a paid request ends (a chat reply, a generation, a transcription) and the account pages hear it through a long poll, so a charge shows within seconds instead of after up to two minutes. A page that already has a balance is answered at once while a newer one is read behind it; reads after charges are at least five seconds apart.
+
 **Models**: the list comes from the VibeDev gateway for your account, so nothing is listed before you sign in; models appear right after, without a restart. Using a VibeDev model while signed out, or after the sign-in has ended, asks you to sign in; a used-up balance says so with the top-up link. These messages are separate from the DeepSeek account's. An expired access token is renewed and the request sent again, without repeating any output.
 
 **Web search**: the plugin registers the web search provider `vibedev-gateway`, signed in with your VibeDev account. Registering does not select it: DeepSeek Harness keeps its own search unless the profile sets `searchProvider: vibedev-gateway` on the `web` row (the VibeDev app does). Searching is free, but the gateway refuses every request, searches included, while the balance is zero.
 
-**Media tools**: `image_generate`, `video_generate` (runs as a background job), `audio_generate` (music or a podcast), `audio_transcribe` (Mandarin only for now), `media_models`, `media_tasks` and `media_account`. Reference images, videos and audio, and first and last frames, can each be a workspace path, an http(s) link, or `chat:1` for the most recent matching file you attached. A request the model cannot serve is refused before anything is uploaded or charged.
+**Media tools**: `image_generate`, `video_generate` (runs as a background job), `audio_generate` (music, a podcast or speech), `audio_transcribe` (a multipart file upload; languages and limits as each model declares), `media_models`, `media_tasks` and `media_account`. Reference images, videos and audio, and first and last frames, can each be a workspace path, an http(s) link, or `chat:1` for the most recent matching file you attached. A request the model cannot serve is refused before anything is uploaded or charged.
+
+**Audio models**: whether an audio model makes music, a podcast, speech or a transcription comes from its catalog declaration (`audio.generation_kind`), never from its name; a model that declares no kind is listed but cannot generate. Each model is sent only the fields it declares in `request_fields`, with the text in its `prompt_field`: music takes `prompt`, `lyrics`, `genre`, `mood` and `gender`; a podcast `prompt` and `source_url`; speech what it declares (Seed: `text_prompt`, `references`, `audio_config`, `watermark`). Speech reference limits (count, size, length, exclusions) and the sample rates of each output format come from the gateway. Music and podcasts take no references, format, sample rate or duration.
 
-**Cost**: chat models are billed by usage at the gateway's prices; images per image, video per second (reference video length counts), music and podcasts per request. Failed video and audio tasks are not charged, and a submitted video cannot be cancelled. "Ask before each paid generation" is off by default.
+**Cost**: chat models are billed by usage at the gateway's prices; images per image, video per second (reference video length counts), music and podcasts per request, speech per second of original audio (the length the provider reports, which can differ from playback), transcription per minute. A model without a catalog price is shown as "not priced" (未定价), never as free. Failed and cancelled video and audio tasks are not charged, and a submitted video cannot be cancelled. One operation generates once: an unconfirmed submission is settled under the same key, a failed query or download only queries again or renews the link, and a result whose charge is still settling (`settlement_pending`) is kept, never generated again. "Ask before each paid generation" is off by default.
 
 **Settings**: open "VibeDev Account & Models" on the Plugins page for the spending confirmation, the output folder and the default media models. `displayName`, `preferredModels`, `openBrowserOnSignIn`, `catalogRefreshMinutes`, `client` (the client name sent with the sign-in, default `vibedev-plugin`) and `primary` (show the account as the app's main account; off by default, on in the VibeDev app) are set in the profile's `cordis.patch.yml` row `dsh-vibedev`.
 
@@ -163,7 +181,7 @@ Everything is paid from your VibeDev balance; the DeepSeek account stays DeepSee
 
 **Privacy**: prompts and reference media are sent to the VibeDev gateway, which forwards them to the model provider. The plugin's own sign-in is kept in the DeepSeek Harness credential store, and task records under `dsh-media/` in the DeepSeek Harness data folder (the former name, kept so an upgrade loses nothing).
 
-**For other plugins**: the `vibedevMedia` service generates images and videos through the same model choice, reference checks, idempotent submission and background saving; the film workbench ([dsh-film](https://github.com/VibeDev-Si/dsh-film)) storyboard uses it.
+**For other plugins**: the `vibedevMedia` service generates images, videos and audio (`startAudio`: music, podcasts and speech) through the same model choice, reference checks, idempotent submission and background saving; the film workbench ([dsh-film](https://github.com/VibeDev-Si/dsh-film)) storyboard uses it. `audioForms()` describes the form of each audio model (only its declared fields, choices and limits, and its price). Every task is read the same way: `task` (as recorded), `refreshTask` (ask the gateway again — fresh links, settlement; a query, never a submission), `onTask` (each change) and `result` (the files to play or download). Pass the same `operationId` when one click is retried: its task is returned and nothing is submitted twice.
 
 **Third-party code**: the model route in `src/llm/` is ported from the VibeDev app; `context.ts`, `replay.ts` and `stream.ts` are copied from DeepSeek Harness's `@deepseek-ai/dsh-llm-pi-ai` (MIT). The licences of the bundled pi-ai, Anthropic SDK, OpenAI SDK and their dependencies are in `THIRD-PARTY-NOTICES.txt`.
 

package/THIRD-PARTY-NOTICES.txt

@@ -1,4 +1,4 @@
-THIRD-PARTY NOTICES · @vibedev-si/dsh-vibedev 0.3.0
+THIRD-PARTY NOTICES · @vibedev-si/dsh-vibedev 0.3.2
 
 This package bundles code from the projects below. Each is used under the licence that
 follows its name; the licence texts are reproduced as the projects ship them.

package/client/client.js

@@ -9,6 +9,8 @@ window.__ModuleLoader__.load({ id: "@vibedev-si/dsh-vibedev", factory: (require)
 		const ACCOUNT_ROUTE = "/api/dsh-vibedev/account";
 		const PENDING_POLL_MS = 2e3;
 		const IDLE_POLL_MS = 6e4;
+		/** After a failed long poll, the next one waits this long (the Host may be restarting). */
+		const WATCH_RETRY_MS = 5e3;
 		/** Who to show: the nickname, else the email. */
 		function displayName(view) {
 			return view?.user?.nickname ?? view?.user?.email;
@@ -33,6 +35,8 @@ window.__ModuleLoader__.load({ id: "@vibedev-si/dsh-vibedev", factory: (require)
 			reading;
 			stopped = true;
 			unsubscribeFocus;
+			/** The open long poll, when one is. */
+			watching;
 			/**
 			* @param options - fetch, timers, window opening and focus (test seams).
 			*/
@@ -82,17 +86,61 @@ window.__ModuleLoader__.load({ id: "@vibedev-si/dsh-vibedev", factory: (require)
 					globalThis.clearTimeout(handle);
 				}))(this.timer);
 				this.timer = void 0;
+				this.watching?.abort();
+				this.watching = void 0;
 			}
-			schedule() {
-				if (this.stopped) return;
+			later(callback, ms) {
 				if (this.timer !== void 0) (this.options.clearTimeout ?? ((handle) => {
 					globalThis.clearTimeout(handle);
 				}))(this.timer);
-				const delay = this.state.view?.pending !== void 0 ? PENDING_POLL_MS : IDLE_POLL_MS;
-				this.timer = (this.options.setTimeout ?? ((callback, ms) => globalThis.setTimeout(callback, ms)))(() => {
+				this.timer = (this.options.setTimeout ?? ((run, delay) => globalThis.setTimeout(run, delay)))(() => {
+					this.timer = void 0;
+					callback();
+				}, ms);
+			}
+			schedule() {
+				if (this.stopped) return;
+				const view = this.state.view;
+				if (view?.pending === void 0 && typeof view?.revision === "number") {
+					if (this.timer !== void 0) (this.options.clearTimeout ?? ((handle) => {
+						globalThis.clearTimeout(handle);
+					}))(this.timer);
 					this.timer = void 0;
+					this.watch(view.revision);
+					return;
+				}
+				this.watching?.abort();
+				this.watching = void 0;
+				this.later(() => {
 					this.refresh();
-				}, delay);
+				}, view?.pending !== void 0 ? PENDING_POLL_MS : IDLE_POLL_MS);
+			}
+			/** Keep one long poll open for changes past `revision`. */
+			watch(revision) {
+				if (this.watching !== void 0) return;
+				const controller = new AbortController();
+				this.watching = controller;
+				(async () => {
+					let failed = false;
+					try {
+						const response = await this.fetch(`${ACCOUNT_ROUTE}?since=${revision}`, { signal: controller.signal });
+						if (!response.ok) throw new Error(`HTTP ${response.status}`);
+						const view = await response.json();
+						if (!controller.signal.aborted && (view.revision ?? 0) >= (this.state.view?.revision ?? 0)) this.set({
+							view,
+							loadFailed: false
+						});
+					} catch {
+						failed = !controller.signal.aborted;
+					} finally {
+						if (this.watching === controller) this.watching = void 0;
+					}
+					if (this.stopped || controller.signal.aborted) return;
+					if (failed) this.later(() => {
+						this.refresh();
+					}, WATCH_RETRY_MS);
+					else this.schedule();
+				})();
 			}
 			/**
 			* Read the account now.
@@ -263,6 +311,7 @@ window.__ModuleLoader__.load({ id: "@vibedev-si/dsh-vibedev", factory: (require)
 			videoModel: "视频",
 			musicModel: "音乐",
 			podcastModel: "播客",
+			speechModel: "语音合成",
 			transcriptionModel: "语音转写",
 			modelPlaceholder: "留空自动选择",
 			account: "账号",
@@ -338,6 +387,7 @@ window.__ModuleLoader__.load({ id: "@vibedev-si/dsh-vibedev", factory: (require)
 			videoModel: "Video",
 			musicModel: "Music",
 			podcastModel: "Podcasts",
+			speechModel: "Speech",
 			transcriptionModel: "Speech to text",
 			modelPlaceholder: "Empty: chosen automatically",
 			account: "Account",
@@ -680,6 +730,7 @@ window.__ModuleLoader__.load({ id: "@vibedev-si/dsh-vibedev", factory: (require)
 			"videoModel",
 			"musicModel",
 			"podcastModel",
+			"speechModel",
 			"transcriptionModel"
 		];
 		/** Every field the page edits. */

package/lib/account/index.d.ts

@@ -7,6 +7,9 @@
  *   app's account, the plugin's own sign-in, a development key, or none), who
  *   is signed in, the balance, a sign-in waiting for the browser, and how many
  *   VibeDev models the pickers list; `?fresh=1` reads the balance again;
+ *   `?since=<revision>` holds the request until the person or the balance
+ *   changes (a charge is read again right after it ends), so the pages show a
+ *   new balance within seconds instead of on their next minute poll;
  * - `POST /api/dsh-vibedev/account/sign-in` — `{ open?: boolean }`: start the
  *   plugin's own sign-in; answers the authorization link (`open: false` leaves
  *   opening it to the page);
@@ -58,6 +61,11 @@ export interface AccountView {
     };
     /** The VibeDev account is the app's main account (the VibeDev app): its row leads the sidebar foot and Settings. */
     readonly primary: boolean;
+    /**
+     * Grows whenever the person or the balance changes; `GET …/account?since=<revision>`
+     * answers as soon as it grows (or after a while with the same view).
+     */
+    readonly revision: number;
 }
 /** What the account routes need from the plugin. */
 export interface AccountServiceOptions {
@@ -74,16 +82,51 @@ export interface AccountServiceOptions {
     readonly primary?: boolean;
     readonly fetch?: typeof globalThis.fetch;
     readonly now?: () => number;
+    /** Timer seams for the reads after a charge and the long poll. */
+    readonly setTimeout?: (callback: () => void, ms: number) => unknown;
+    readonly clearTimeout?: (handle: unknown) => void;
 }
 /** The account state behind the pages. */
 export declare class AccountService {
     private readonly options;
     private balanceCache;
+    /** A background read of a stale balance, so only one runs at a time. */
+    private revalidating;
     private readonly now;
+    private revision;
+    /** What the last read showed, to tell whether the next one changed anything. */
+    private lastSeen;
+    private readonly waiters;
+    private settleTimers;
+    /** Grows with every credential change; a read started before one is dropped. */
+    private epoch;
+    /** When the last read after a charge started, to space them out. */
+    private lastSpendReadAt;
     /**
      * @param options - origin, credential chain, the plugin's sign-in and the route's model state.
      */
     constructor(options: AccountServiceOptions);
+    private timeout;
+    private cancelTimeout;
+    /** Something the pages show changed: answer every waiting `?since=` request. */
+    changed(): void;
+    /** The account changed hands (sign-in, sign-out, switch): forget the balance and tell the pages. */
+    credentialChanged(): void;
+    /**
+     * A request that may have cost money ended (a chat reply, a generation, a
+     * transcription): read the balance again shortly, and once more later if the
+     * charge had not settled yet. Repeated calls in a burst share the reads.
+     */
+    spent(): void;
+    /** Read the balance of the current credential now (the pages hear about a change through the long poll). */
+    private readNow;
+    /**
+     * Wait until the view changes past `since`, or a while.
+     * @param since - the revision the page shows.
+     * @param signal - the page went away.
+     * @returns once there is something new, or the wait ran out.
+     */
+    wait(since: number, signal?: AbortSignal): Promise<void>;
     /** Links to the gateway's pages. */
     links(): AccountView['links'];
     /**
@@ -92,8 +135,12 @@ export declare class AccountService {
      * @returns the view.
      */
     view(fresh?: boolean): Promise<AccountView>;
-    /** `GET /v1/account/auth/me` for one token, reused for a minute. */
+    /**
+     * `GET /v1/account/auth/me` for one token. A read younger than the TTL is
+     * reused; an older one is answered at once while a new read runs behind it.
+     */
     private me;
+    private read;
     /**
      * Start the plugin's own sign-in.
      * @param open - open the system browser (false: the page opens the link itself).
@@ -111,6 +158,8 @@ export declare class AccountService {
      * @returns false when the credential is the app's account, which is signed out in the app.
      */
     signOut(): Promise<boolean>;
+    /** Stop the pending reads after a charge (the plugin is going away). */
+    dispose(): void;
 }
 /**
  * The account routes.

package/lib/audio/request.d.ts

@@ -0,0 +1,141 @@
+/**
+ * Audio generation requests — music, podcasts and speech — checked against what
+ * the chosen model declares in the catalog's `audio` block, and turned into the
+ * body of `POST /v1/audio/generations`.
+ *
+ * Each kind sends only its own fields:
+ *
+ * | kind    | wire fields                                                     |
+ * | ------- | --------------------------------------------------------------- |
+ * | music   | model, prompt, lyrics, genre, mood, gender                      |
+ * | podcast | model, prompt, source_url                                       |
+ * | speech  | as declared; Seed: model, text_prompt, references, audio_config, watermark |
+ *
+ * The main text goes in the model's `prompt_field`. A field the model does not
+ * declare in `request_fields` is refused before anything is uploaded or charged;
+ * music and podcasts never take speech references, output formats, sample rates
+ * or a duration. Speech references, formats and sample rates come from the
+ * catalog only (count, size, length, mutual exclusions, the sample rates of
+ * each format), never from a fixed list.
+ * @module dsh-vibedev/audio/request
+ */
+import type { AudioCapabilities, AudioGenerationKind, MediaModel } from '../gateway/catalog.js';
+import type { AudioQuote } from '../pricing.js';
+import type { MediaCall, MediaRuntime } from '../runtime.js';
+import type { TaskRecord } from '../tasks/store.js';
+/** The kinds of audio a generation request makes. */
+export type AudioMakeKind = Exclude<AudioGenerationKind, 'transcription'>;
+export declare const AUDIO_MAKE_KINDS: readonly AudioMakeKind[];
+/** What to generate. Unset fields are not sent. */
+export interface AudioRequest {
+    /** What to make; with a named model it must match the model's declared kind. */
+    readonly kind?: AudioMakeKind;
+    /** A model id or alias; otherwise the pinned or first model of the kind. */
+    readonly model?: string;
+    /** The main text: a song description, a podcast topic, or the words to speak. */
+    readonly prompt?: string;
+    /** Music. */
+    readonly lyrics?: string;
+    readonly genre?: string;
+    readonly mood?: string;
+    readonly gender?: string;
+    /** Podcast: an http(s) page for the hosts to discuss. */
+    readonly sourceUrl?: string;
+    /** Speech: reference audio (a voice to imitate) — workspace paths, links or `chat:` attachments. */
+    readonly references?: readonly string[];
+    /** Speech: an output format the model declares (`wav`, `mp3`, …). */
+    readonly format?: string;
+    /** Speech: a sample rate the model declares for that format. */
+    readonly sampleRate?: number;
+    /** Speech: whether the provider adds its watermark, when the model takes the field. */
+    readonly watermark?: boolean;
+    /** Not an audio-generation parameter; refused for music and podcasts, and for speech unless declared. */
+    readonly duration?: number;
+    /**
+     * One user operation's key: the same key always answers with the same task,
+     * so a repeated click or a retried call never creates a second one. Letters,
+     * digits, `-` and `_`, at most 80. Generated when absent.
+     */
+    readonly operationId?: string;
+}
+/**
+ * What an audio model makes, from its declaration only.
+ * @param model - the model.
+ * @returns the kind, or undefined when the catalog does not declare it.
+ */
+export declare function audioKindOf(model: MediaModel): AudioMakeKind | undefined;
+/** The wire fields a model takes. */
+export declare function requestFieldsOf(model: MediaModel): readonly string[];
+/** The field the main text goes in. */
+export declare function promptFieldOf(model: MediaModel): string | undefined;
+/** The output format and sample rate a speech request uses, from the declared formats only. */
+export declare function audioConfigOf(model: MediaModel, format: string | undefined, sampleRate: number | undefined): {
+    format: string;
+    sample_rate?: number;
+} | undefined;
+/**
+ * Check a request against the model and build the body, without references
+ * (they are uploaded first; their URLs are added by {@link prepareAudioTask}).
+ * @param model - the chosen model.
+ * @param request - what to generate.
+ * @returns the body without references, and the wire fields the request uses.
+ * @throws {@link MediaError} for a field the model does not take, a missing text, or a value outside the declared choices.
+ */
+export declare function audioRequestBody(model: MediaModel, request: AudioRequest): Record<string, unknown>;
+/**
+ * Choose the model for a request: the one named (an id or an alias), else the
+ * pinned or first model of the kind. A named model must make the kind asked for.
+ */
+export declare function chooseAudioModel(runtime: MediaRuntime, request: Pick<AudioRequest, 'kind' | 'model'>, signal?: AbortSignal): Promise<MediaModel>;
+/** A checked, priced audio task ready to submit. */
+export interface PreparedAudio {
+    readonly model: MediaModel;
+    readonly kind: AudioMakeKind;
+    readonly quote: AudioQuote;
+    readonly draft: TaskRecord;
+}
+/**
+ * Everything before an audio submission: choose the model, check every field
+ * against its declaration, upload and measure speech references, price it,
+ * confirm the spending, and build the task under the operation's key.
+ * @param runtime - the media runtime.
+ * @param call - the tool call or host call.
+ * @param request - what to generate.
+ * @param save - where the result goes.
+ * @returns the prepared task.
+ */
+export declare function prepareAudioTask(runtime: MediaRuntime, call: MediaCall, request: AudioRequest, save: {
+    readonly folder: string;
+    readonly stem: string;
+}): Promise<PreparedAudio>;
+/** One field of an audio form, for interfaces that build their form from the catalog. */
+export interface AudioFormField {
+    /** The {@link AudioRequest} key. */
+    readonly key: 'prompt' | 'lyrics' | 'genre' | 'mood' | 'gender' | 'sourceUrl' | 'references' | 'format' | 'sampleRate' | 'watermark';
+    /** The wire field it is sent as. */
+    readonly wire: string;
+    readonly required: boolean;
+    /** Declared choices. */
+    readonly choices?: readonly string[];
+    /** For `sampleRate`: the rates of each format. */
+    readonly sampleRates?: Readonly<Record<string, readonly number[]>>;
+    /** For `references`: the declared limits. */
+    readonly limits?: NonNullable<AudioCapabilities['references']>;
+    readonly maxChars?: number;
+}
+/** What a form for one audio model shows. */
+export interface AudioForm {
+    readonly model: string;
+    readonly name: string;
+    readonly kind: AudioMakeKind;
+    readonly fields: readonly AudioFormField[];
+    readonly price: AudioQuote;
+    readonly defaultFormat?: string;
+}
+/**
+ * The form an interface shows for one model: only the fields the model
+ * declares, with their declared choices and limits.
+ * @param model - the model.
+ * @returns the form, or undefined for a model that cannot generate audio.
+ */
+export declare function audioFormOf(model: MediaModel): AudioForm | undefined;

package/lib/gateway/assets.d.ts

@@ -25,8 +25,8 @@ export interface UploadedAsset {
     readonly sha256: string;
     readonly bytes: number;
 }
-/** Purposes the plugin uploads under. */
-export type AssetPurpose = 'video_reference';
+/** Purposes the plugin uploads under: references of a video request, and the reference audio of a speech request. */
+export type AssetPurpose = 'video_reference' | 'audio_reference' | (string & {});
 /**
  * Read an asset answer from `uploads`, `complete` or `refresh`.
  * @param body - the parsed answer.

package/lib/gateway/catalog.d.ts

@@ -8,6 +8,12 @@
  * the versioned per-mode declarations (`modes_schema_version: 1`). A mode the
  * model does not list is unsupported; an unknown schema version makes the model
  * unusable rather than guessed at.
+ *
+ * Audio capabilities follow the `audio` block: `generation_kind` (music,
+ * podcast, speech or transcription), `prompt_field`, `request_fields`,
+ * `billing_unit` and the declared limits (references, output formats and their
+ * sample rates). An audio model that declares no kind is listed but cannot be
+ * chosen for generation: neither its name nor its id decides what it makes.
  * @module dsh-vibedev/gateway/catalog
  */
 /** What a media model produces. */
@@ -81,10 +87,16 @@ export interface VideoCapabilities {
     /** References must be relayed through the gateway media library. */
     readonly gatewayRelayRequired?: boolean;
 }
+/**
+ * What a price is charged per: one generation, one second of output, one
+ * second of ORIGINAL audio (speech is billed on the provider's original length,
+ * not on what plays back after processing), or one minute of input audio.
+ */
+export type PriceUnit = 'generation' | 'second' | 'original_second' | 'minute';
 /** One price tier: video tiers are resolutions (`720p`), image tiers size buckets (`1K`). */
 export interface MediaPriceTier {
     readonly tier: string;
-    readonly unit: 'generation' | 'second';
+    readonly unit: PriceUnit;
     readonly amount: number;
 }
 /** A model's selling price, account multiplier included. */
@@ -92,23 +104,76 @@ export interface MediaPricing {
     readonly currency: string;
     readonly tiers: readonly MediaPriceTier[];
     readonly default?: {
-        readonly unit: 'generation' | 'second';
+        readonly unit: PriceUnit;
         readonly amount: number;
     };
 }
+/** What an audio model does, as the catalog declares it (`audio.generation_kind`). */
+export declare const AUDIO_GENERATION_KINDS: readonly ["music", "podcast", "speech", "transcription"];
+export type AudioGenerationKind = typeof AUDIO_GENERATION_KINDS[number];
+/** How an audio model is billed (`audio.billing_unit`). */
+export type AudioBillingUnit = 'generation' | 'original_second' | 'minute';
+/** The reference audio a speech model takes (a voice to imitate), as the catalog declares it. */
+export interface AudioReferenceLimits {
+    readonly min: number;
+    readonly max: number;
+    readonly maxBytes?: number;
+    readonly minSeconds?: number;
+    readonly maxSeconds?: number;
+    readonly maxTotalSeconds?: number;
+    /** Accepted MIME types; absent states no restriction. */
+    readonly mimes?: readonly string[];
+    /** Request fields that may not be sent together with references. */
+    readonly exclusiveWith?: readonly string[];
+    /** The media-library purpose references are uploaded under, when the catalog names one. */
+    readonly purpose?: string;
+}
+/** One output format a speech model can write, with the sample rates it takes in that format. */
+export interface AudioOutputFormat {
+    readonly format: string;
+    readonly sampleRates: readonly number[];
+    readonly defaultSampleRate?: number;
+}
+/**
+ * An audio model's declared capabilities. Nothing here is assumed: a field the
+ * catalog does not declare is absent, and a model without `generationKind`
+ * cannot be chosen for any kind of generation.
+ */
+export interface AudioCapabilities {
+    readonly generationKind?: AudioGenerationKind;
+    /** The request field the main text goes in (`prompt`, `text_prompt`, …). */
+    readonly promptField?: string;
+    /** The request fields the model accepts, under their wire names. */
+    readonly requestFields: readonly string[];
+    readonly billingUnit?: AudioBillingUnit;
+    readonly references?: AudioReferenceLimits;
+    readonly outputFormats?: readonly AudioOutputFormat[];
+    readonly defaultFormat?: string;
+    /** Declared choices for enumerated fields (`gender`, `genre`, …), under their wire names. */
+    readonly choices?: Readonly<Record<string, readonly string[]>>;
+    readonly maxPromptChars?: number;
+    /** Transcription input limits. */
+    readonly maxInputBytes?: number;
+    readonly maxInputSeconds?: number;
+    readonly inputMimes?: readonly string[];
+    readonly languages?: readonly string[];
+}
 /** One media model of the catalog. */
 export interface MediaModel {
     readonly id: string;
     readonly name: string;
     readonly description?: string;
     readonly kind: MediaKind;
-    /** For audio models: what they produce. */
-    readonly audioKind?: 'music' | 'podcast';
+    /** Other ids the gateway answers to for this model. */
+    readonly aliases?: readonly string[];
+    /** For audio models: what they produce, from `audio.generation_kind` only; absent when undeclared. */
+    readonly audioKind?: Exclude<AudioGenerationKind, 'transcription'>;
     /** Endpoints the catalog lists for the model. */
     readonly endpoints: readonly string[];
     readonly inputModalities: readonly string[];
     readonly pricing?: MediaPricing;
     readonly video?: VideoCapabilities;
+    readonly audio?: AudioCapabilities;
 }
 type Json = Record<string, unknown>;
 /**
@@ -124,6 +189,12 @@ export declare function parseVideoCapabilities(raw: Json): VideoCapabilities;
  * @returns the usable modes and their constraints.
  */
 export declare function effectiveVideoModes(video: VideoCapabilities): Partial<Record<VideoMode, VideoModeConstraint>>;
+/**
+ * Parse an audio capability block.
+ * @param raw - the block.
+ * @returns the declared capabilities; `generationKind` is absent unless declared.
+ */
+export declare function parseAudioCapabilities(raw: Json): AudioCapabilities;
 /**
  * Parse a `/v1/models` body into its media models.
  * @param body - `{data:[...]}` or a bare array.

package/lib/gateway/http.d.ts

@@ -31,6 +31,11 @@ export interface GatewayHttpOptions {
     readonly userAgent: string;
     readonly fetch?: typeof globalThis.fetch;
     readonly sleep?: (ms: number, signal?: AbortSignal) => Promise<void>;
+    /** Told about every request the gateway accepted (2xx), after the fact. */
+    readonly onAccepted?: (request: {
+        readonly method: string;
+        readonly path: string;
+    }) => void;
 }
 /** One request. */
 export interface GatewayRequest {

package/lib/index.d.ts

@@ -24,11 +24,13 @@ import type { Context, Volatile } from '@deepseek-ai/cordis';
 import Schema from '@deepseek-ai/schemastery';
 export { MediaError } from './gateway/errors.js';
 export { MediaHostService } from './service.js';
-export type { HostTarget, MediaTaskView } from './service.js';
+export type { HostTarget, MediaOutputView, MediaTaskView } from './service.js';
+export type { AudioForm, AudioFormField, AudioMakeKind, AudioRequest } from './audio/request.js';
+export type { AudioQuote } from './pricing.js';
 export type { ImageRequest } from './tools/image.js';
 export type { VideoRequest } from './tools/video.js';
 export { parseMediaCatalog } from './gateway/catalog.js';
-export type { MediaModel, VideoCapabilities, VideoMode } from './gateway/catalog.js';
+export type { AudioCapabilities, AudioGenerationKind, MediaModel, PriceUnit, VideoCapabilities, VideoMode } from './gateway/catalog.js';
 export { ACCOUNT_ROUTE_PREFIX, AccountService } from './account/index.js';
 export type { AccountSource, AccountView } from './account/index.js';
 export { GATEWAY_SEARCH_PROVIDER_ID as SEARCH_PROVIDER_ID } from './search/provider.js';
@@ -55,6 +57,7 @@ export interface Config {
     videoModel: Volatile<string>;
     musicModel: Volatile<string>;
     podcastModel: Volatile<string>;
+    speechModel: Volatile<string>;
     transcriptionModel: Volatile<string>;
     /** Gateway origin. */
     gatewayOrigin: string;
@@ -78,6 +81,7 @@ export declare const Config: Schema<Schemastery.ObjectS<NoInfer<{
     videoModel: Schema<string, string, "volatile-defined">;
     musicModel: Schema<string, string, "volatile-defined">;
     podcastModel: Schema<string, string, "volatile-defined">;
+    speechModel: Schema<string, string, "volatile-defined">;
     transcriptionModel: Schema<string, string, "volatile-defined">;
     gatewayOrigin: Schema<string, string, "defined">;
     client: Schema<string, string, "defined">;
@@ -95,6 +99,7 @@ export declare const Config: Schema<Schemastery.ObjectS<NoInfer<{
     videoModel: Schema<string, string, "volatile-defined">;
     musicModel: Schema<string, string, "volatile-defined">;
     podcastModel: Schema<string, string, "volatile-defined">;
+    speechModel: Schema<string, string, "volatile-defined">;
     transcriptionModel: Schema<string, string, "volatile-defined">;
     gatewayOrigin: Schema<string, string, "defined">;
     client: Schema<string, string, "defined">;