@shop-os/modelkit 0.8.0 → 0.10.0

package/CHANGELOG.md

@@ -9,6 +9,41 @@ header in the release PR.
 
 ## [Unreleased]
 
+## [0.10.0] - Unreleased
+
+ElevenLabs direct: text to speech and instant voice cloning on our own ElevenLabs account (ENG-3400).
+
+### Added
+- Adapter `elevenlabs` for the ElevenLabs REST API (`https://api.elevenlabs.io`, base-url override `endpoints.elevenlabs`). Key `elevenlabs` from `ELEVENLABS_API_KEY`, with the `ENTERPRISE_` / `SLOOSH_` prefixes; a sloosh call without `SLOOSH_ELEVENLABS_API_KEY` fails with `auth`. fal's ElevenLabs rows run on fal's account and cannot see a cloned voice, so custom voices need the direct API.
+- Speech row `elevenlabs-v4-direct` (`model_id` `eleven_v4`): the 18 previewed premade voices under their ElevenLabs voice ids (taken from the preview URLs; default Sarah `EXAVITQu4vr4xnSDxMaL`), the v4 languages, the ElevenLabs MP3 `output_format`s (default `mp3_44100_128`), 10,000 characters, `capabilities.customVoices`. Usage is one `images`-unit call, like the other speech rows.
+- Voice-clone row `elevenlabs-ivc` (Instant Voice Clone, `POST /v1/voices/add`): 1-25 samples, `enhance` sent as `remove_background_noise`, transcripts ignored (IVC takes none), a `designSignature` refused with `unsupported_option` (a Fish design cannot be kept on ElevenLabs). Returns `voiceId` and `state` `trained`, or `created` while ElevenLabs holds the voice for verification. Usage unit `requests`. `deleteVoice` on the same row (`DELETE /v1/voices/{id}`).
+
+### Notes
+- Both rows ship with `exposure: []`: callable by id, listed nowhere, unpriced. fal's ElevenLabs rows keep their exposure.
+- `createVoice` retries only a 429: after a 5xx ElevenLabs may already hold the voice. A full voice library (`voice_limit_reached`) is `provider_rejected` with ElevenLabs' message; a 401 `quota_exceeded` is `provider_billing`, not `auth`.
+- `deleteVoice` follows Fish: a first-attempt "not found" (404, or a 400 `voice_not_found` / `voice_does_not_exist`) is `provider_rejected`, since a voice of another account looks unknown too; after a failed attempt of the same call it counts as deleted.
+- A kit before 0.10.0 refuses the `elevenlabs` adapter id, so these rows stay invisible to consumers that have not upgraded.
+
+## [0.9.0] - Unreleased
+
+Fish Audio: text to speech, voice cloning and voice design (ENG-3400).
+
+### Added
+- Adapter `fish` for the Fish Audio REST API (`https://api.fish.audio`, base-url override `endpoints.fish`). Key `fish` from `FISH_AUDIO_API_KEY`, with the `ENTERPRISE_` / `SLOOSH_` prefixes; a sloosh call without `SLOOSH_FISH_AUDIO_API_KEY` fails with `auth`.
+- Speech rows `fish-s2.1-pro`, `fish-s2.1-pro-free` and `fish-drama-3` (the Fish `model` header `s2.1-pro`, `s2.1-pro-free`, `drama-3`). 24 Fish Official library voices (voice-designed or licensed, none a real-person clone; default Hannah), `mp3` (default) or `wav` output returned as bytes, a 10,000-character kit cap (Fish states none). Fish auto-detects the language, so the rows list none and `language` is dropped with a warning. Usage is one `images`-unit call, like the other speech rows; Fish bills per UTF-8 byte of the script ($15 / M bytes, Drama 3 $50 / M list, the free tier $0).
+- `SpeechRequest.customVoice`: a voice id `createVoice` returned, on rows with the new `capabilities.customVoices` (the Fish speech rows). It replaces `voice`; sending both, or a custom voice to another row, is `unsupported_option`.
+- Row types `voice-clone` and `voice-design`, and kit methods `createVoice` (row `fish-voice-clone`: 1-20 samples, optional transcripts and voice-design signatures on every sample or none, a private fast clone; returns `voiceId` and, when Fish reports a known one, `state`), `designVoice` (row `fish-voice-design-1`: a description, an optional preview line of up to 150 characters (the Fish OpenAPI limit), 1-4 candidates returned as WAV bytes with a `signature`; $0.01 per request) and `deleteVoice` (on the clone row; a 404 is `provider_rejected`, because Fish also answers 404 for a private voice of another account, except after a failed attempt of the same call, which may have deleted it). Usage unit `requests`. `resolveVoiceCloneOptions` / `resolveVoiceDesignOptions` and `resolveOptions` cover the new methods.
+- A cloned voice lives in the Fish account of the key set that made it: create, speak and delete with the same `tier`.
+- `constraints.voiceDesign` (`maxCandidates`, `previewMaxChars`).
+
+### Fixed
+- A 403 whose text says "Top up API credit" (Fish, out of API credit) is `provider_billing`, not `auth`.
+
+### Notes
+- Every Fish row ships with `exposure: []`: callable by id, listed nowhere, unpriced. Price and expose them in pro-backend per env when ENG-3400 is ready.
+- `createVoice` retries only a 429: after a 5xx Fish may already hold the voice, and a retry would make a second one.
+- A kit before 0.9.0 drops `voice-clone` / `voice-design` rows on load with a warning, and refuses the `fish` adapter id, so those rows stay invisible to consumers that have not upgraded.
+
 ## [0.8.0] - Unreleased
 
 The BytePlus ModelArk key has one env name.

package/README.md

@@ -47,6 +47,13 @@ const synced = await kit.lipsync({ model: "lipsync-2", sourceVideo: { url }, aud
 const dubbed = await kit.dub({ model: "elevenlabs-dubbing", sourceVideo: { url }, targetLanguage: "es" });
 const transcript = await kit.transcribe({ model: "elevenlabs-scribe-v2", source: { url }, timestamps: true });
 
+// Custom voices (Fish Audio): clone once per workspace, keep `voiceId`, speak with it on the
+// same tier. `designVoice` returns candidates to keep the same way (pass `signature`).
+const { voiceId } = await kit.createVoice({ model: "fish-voice-clone", title, samples: [{ audio: { url } }] }, { tier });
+const mine = await kit.generateSpeech({ model: "fish-s2.1-pro", text, customVoice: voiceId }, { tier });
+await kit.deleteVoice({ model: "fish-voice-clone", voiceId }, { tier });
+// ElevenLabs works the same way: clone on `elevenlabs-ivc`, speak on `elevenlabs-v4-direct`.
+
 const clip = await kit.prepareMediaPart(
     "gemini-2.5-flash",
     { bytes, mime: "video/mp4" },
@@ -344,7 +351,7 @@ the app's secrets, not a Cloudflare gateway token.
 
 ## Adapters
 
-`ai-sdk-text`, `google-image`, `openai-image`, `google-genai` (Veo, Omni with `delivery: "uri"`, Lyria 3, Files API media parts), `ark` (Seedream, Seedance), `fal`, `replicate` (images, video, Lyria 2, MMAudio), `prodia`, `higgsfield`, `xai-video`, `luma` (Uni images, Ray 3.2 video via the Agents API; `LUMA_API_KEY`, base-url override `endpoints.luma`). See `src/adapters/`.
+`ai-sdk-text`, `google-image`, `openai-image`, `google-genai` (Veo, Omni with `delivery: "uri"`, Lyria 3, Files API media parts), `ark` (Seedream, Seedance), `fal`, `replicate` (images, video, Lyria 2, MMAudio), `prodia`, `higgsfield`, `xai-video`, `luma` (Uni images, Ray 3.2 video via the Agents API; `LUMA_API_KEY`, base-url override `endpoints.luma`), `fish` (Fish Audio speech, `createVoice` / `designVoice` / `deleteVoice`; `FISH_AUDIO_API_KEY`, base-url override `endpoints.fish`), `elevenlabs` (direct ElevenLabs speech, `createVoice` / `deleteVoice`; `ELEVENLABS_API_KEY`, base-url override `endpoints.elevenlabs`). See `src/adapters/`.
 
 ## Not in the package