@klex/agent-admin-api 0.10.0 → 0.11.0

package/README.md

@@ -34,6 +34,16 @@ returned by the API are redacted and must never contain resolved environment
 secrets. Model selection uses separate `providerId` and opaque `modelId` fields;
 clients must not split model IDs on colons.
 
+## Realtime voice client contract
+
+- `GET /v1/providers/{id}/voices?modelId=...&refresh=false` returns `selectionSupported`, optional `defaultVoiceId`, and compatible entries with opaque `voiceId`, display metadata, provenance and preview availability. Provider capabilities advertise discovery/preview hooks; model-scoped support determines whether a selector is available. Listing does not generate audio. Encode the complete model ID in the query, including slashes/colons.
+- `POST /v1/providers/{id}/voices/preview` takes `{ modelId, voiceId }`. It can trigger billable same-model Live generation and returns binary `audio/wav`, not JSON/base64 or a public URL. `X-Voice-Preview-Model` is URI-encoded and `X-Voice-Preview-Mode` is `live`. On failure parse the JSON error contract; 429 means quota, 502 means upstream generation failure. Arbitrary text/URLs/model overrides are not accepted.
+- `PATCH /v1/settings/model-selection` saves `voice.sts` references with optional `voiceId`. Preserve both `voice.tts` and `voice.stt` because the voice object is replaced. Omit `voiceId` in the replacement STS reference to reset. Other selection purposes reject it. New calls use changes; active calls do not switch voices.
+
+Existing omitted defaults remain: OpenAI Realtime/GPT-Live `marin`, Gemini's provider default. Unknown/custom model profiles remain usable with defaults, but have no selectable catalog. Catalog provenance distinguishes maintained/discovered data, with explicit fallback/stale flags. Samples improvise a short introduction from the current agent name and full soul. Realtime/Gemini generate it directly; GPT-Live uses the first configured chat model to write commentary before speaking it with the selected Live voice. That extra text call is potentially billable and requires a chat selection. GPT-Live may paraphrase commentary. Name/soul changes bypass old cached samples; different voices may use different wording. Browser clients fetch over their existing authorized Cloud path, check `response.ok`, then use `response.blob()` and an object URL for playback (revoke it afterward). Headless/TUI clients use `arrayBuffer()` and an external WAV player; the backend does not implement playback.
+
+Local admin access remains unauthenticated and loopback-only; Cloud access remains authorized through the existing tunnel. Responses use private/no-store headers, while the agent caches successful samples in bounded memory for reuse. Previewing never starts a production call or supplies history, tools, or credentials as prompt context. It intentionally supplies the full soul to the speech provider or, for GPT-Live, the configured chat provider. Prompts instruct models not to speak private details, but generated output still depends on provider behavior. No UI selector is included in this contract change.
+
 ## MCP OAuth authorization
 
 Version 0.2 adds the consolidated MCP authorization contract. Authorization is

package/dist/index.d.ts

@@ -207,6 +207,7 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                                 [x: string]: import("hono/utils/types").JSONValue;
                             };
                         } | undefined;
+                        voiceId?: string | undefined;
                     }[];
                     tts: {
                         providerId: string;
@@ -280,6 +281,7 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                             providerId: string;
                             modelId: string;
                             providerOptions?: Record<string, Record<string, unknown>> | undefined;
+                            voiceId?: string | undefined;
                         }[];
                         tts: {
                             providerId: string;
@@ -339,6 +341,7 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                             providerId: string;
                             modelId: string;
                             providerOptions?: Record<string, Record<string, unknown>> | undefined;
+                            voiceId?: string | undefined;
                         }[];
                         tts: {
                             providerId: string;
@@ -418,6 +421,7 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                                 [x: string]: import("hono/utils/types").JSONValue;
                             };
                         } | undefined;
+                        voiceId?: string | undefined;
                     }[];
                     tts: {
                         providerId: string;
@@ -482,6 +486,7 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                             providerId: string;
                             modelId: string;
                             providerOptions?: Record<string, Record<string, unknown>> | undefined;
+                            voiceId?: string | undefined;
                         }[];
                         tts: {
                             providerId: string;
@@ -1924,6 +1929,8 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                     documentationUrl?: string | undefined;
                     capabilities: {
                         modelDiscovery: boolean;
+                        voiceDiscovery: boolean;
+                        voicePreview: boolean;
                         connectivityTest: boolean;
                         customModels: boolean;
                     };
@@ -1989,6 +1996,8 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                         documentationUrl?: string | undefined;
                         capabilities: {
                             modelDiscovery: boolean;
+                            voiceDiscovery: boolean;
+                            voicePreview: boolean;
                             connectivityTest: boolean;
                             customModels: boolean;
                         };
@@ -2039,6 +2048,8 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                         documentationUrl?: string | undefined;
                         capabilities: {
                             modelDiscovery: boolean;
+                            voiceDiscovery: boolean;
+                            voicePreview: boolean;
                             connectivityTest: boolean;
                             customModels: boolean;
                         };
@@ -2129,6 +2140,8 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                         documentationUrl?: string | undefined;
                         capabilities: {
                             modelDiscovery: boolean;
+                            voiceDiscovery: boolean;
+                            voicePreview: boolean;
                             connectivityTest: boolean;
                             customModels: boolean;
                         };
@@ -2231,6 +2244,8 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                         documentationUrl?: string | undefined;
                         capabilities: {
                             modelDiscovery: boolean;
+                            voiceDiscovery: boolean;
+                            voicePreview: boolean;
                             connectivityTest: boolean;
                             customModels: boolean;
                         };
@@ -2336,6 +2351,8 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
                         documentationUrl?: string | undefined;
                         capabilities: {
                             modelDiscovery: boolean;
+                            voiceDiscovery: boolean;
+                            voicePreview: boolean;
                             connectivityTest: boolean;
                             customModels: boolean;
                         };
@@ -2528,6 +2545,243 @@ export type AdminApi = import("hono/hono-base").HonoBase<any, {
             };
         };
     };
+} & {
+    "/v1/providers/:id/voices": {
+        $get: {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 400;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                query: {
+                    modelId: string;
+                    refresh?: "false" | "true" | undefined;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 404;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                query: {
+                    modelId: string;
+                    refresh?: "false" | "true" | undefined;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 429;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                query: {
+                    modelId: string;
+                    refresh?: "false" | "true" | undefined;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 500;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                query: {
+                    modelId: string;
+                    refresh?: "false" | "true" | undefined;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 502;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                query: {
+                    modelId: string;
+                    refresh?: "false" | "true" | undefined;
+                };
+            };
+        } | {
+            output: {
+                selectionSupported: boolean;
+                defaultVoiceId?: string | undefined;
+                fallback?: boolean | undefined;
+                stale?: boolean | undefined;
+                voices: {
+                    voiceId: string;
+                    displayName: string;
+                    description?: string | undefined;
+                    language?: string | undefined;
+                    region?: string | undefined;
+                    provenance: {
+                        source: "discovered" | "maintained";
+                        documentationUrl: string;
+                        reviewedAt?: string | undefined;
+                    };
+                    preview: {
+                        available: boolean;
+                        mode?: "live" | undefined;
+                    };
+                }[];
+            };
+            outputFormat: "json";
+            status: 200;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                query: {
+                    modelId: string;
+                    refresh?: "false" | "true" | undefined;
+                };
+            };
+        };
+    };
+} & {
+    "/v1/providers/:id/voices/preview": {
+        $post: {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 400;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                json: {
+                    modelId: string;
+                    voiceId: string;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 404;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                json: {
+                    modelId: string;
+                    voiceId: string;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 429;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                json: {
+                    modelId: string;
+                    voiceId: string;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 500;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                json: {
+                    modelId: string;
+                    voiceId: string;
+                };
+            };
+        } | {
+            output: {
+                error: string;
+                code: string;
+                remediation?: string | undefined;
+            };
+            outputFormat: "json";
+            status: 502;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                json: {
+                    modelId: string;
+                    voiceId: string;
+                };
+            };
+        } | {
+            output: Response;
+            outputFormat: "json";
+            status: import("hono/utils/http-status").StatusCode;
+            input: {
+                param: {
+                    id: string;
+                };
+            } & {
+                json: {
+                    modelId: string;
+                    voiceId: string;
+                };
+            };
+        };
+    };
 } & {
     "/v1/providers/:id/models": {
         $post: {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@klex/agent-admin-api",
-  "version": "0.10.0",
+  "version": "0.11.0",
   "type": "module",
   "sideEffects": false,
   "description": "Shared types for the Klex Bot Admin API",