@mevdragon/vidfarm-devcli 0.19.1 → 0.20.0

package/.agents/skills/text-to-speech/references/voice-settings.md

@@ -0,0 +1,115 @@
+# Voice Settings
+
+Fine-tune voice characteristics for your use case.
+
+## Parameters
+
+| Parameter | Range | Default | Description |
+|-----------|-------|---------|-------------|
+| `stability` | 0.0 - 1.0 | 0.5 | How consistent the voice sounds across the generation. Lower = more emotional variation and expressiveness (but can sound erratic). Higher = steady, predictable tone. |
+| `similarity_boost` | 0.0 - 1.0 | 0.75 | How closely to match the original voice sample. Higher sounds more like the source voice but may amplify audio artifacts or background noise from the original recording. |
+| `style` | 0.0 - 1.0 | 0.0 | Exaggerates the unique characteristics of the voice's speaking style (v2+ and v3 models only). Higher values make the voice more "characterful" but can reduce stability. |
+| `speed` | 0.25 - 4.0 | 1.0 | Speech speed multiplier. 1.0 = normal speed. Range is 0.25-4.0 for the REST API; the Agents Platform restricts to 0.7-1.2. |
+| `use_speaker_boost` | boolean | true | Post-processing that enhances voice clarity and similarity to the original. Generally leave this on unless you're experiencing artifacts. |
+
+## Python Example
+
+```python
+from elevenlabs import ElevenLabs
+from elevenlabs import VoiceSettings
+
+client = ElevenLabs()
+
+audio = client.text_to_speech.convert(
+    text="Testing different voice settings.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    model_id="eleven_v3",
+    voice_settings=VoiceSettings(
+        stability=0.5,
+        similarity_boost=0.75,
+        style=0.0,
+        use_speaker_boost=True
+    )
+)
+```
+
+## JavaScript Example
+
+```javascript
+const audio = await client.textToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", {
+  text: "Testing different voice settings.",
+  modelId: "eleven_v3",
+  voiceSettings: {
+    stability: 0.5,
+    similarityBoost: 0.75,
+    style: 0.0,
+    useSpeakerBoost: true,
+  },
+});
+```
+
+## cURL Example
+
+```bash
+curl -X POST "https://api.elevenlabs.io/v1/text-to-speech/JBFqnCBsd6RMkjVDRZzb" \
+  -H "xi-api-key: $ELEVENLABS_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "text": "Testing different voice settings.",
+    "model_id": "eleven_v3",
+    "voice_settings": {
+      "stability": 0.5,
+      "similarity_boost": 0.75,
+      "style": 0.0,
+      "use_speaker_boost": true
+    }
+  }' \
+  --output output.mp3
+```
+
+## Use Case Recommendations
+
+### Audiobooks / Narration
+```python
+voice_settings=VoiceSettings(
+    stability=0.7,        # Consistent tone
+    similarity_boost=0.5, # Natural variation
+    style=0.0
+)
+```
+
+### Conversational / Chatbots
+```python
+voice_settings=VoiceSettings(
+    stability=0.4,        # More expressive
+    similarity_boost=0.75,
+    style=0.3             # Slight style emphasis
+)
+```
+
+### News / Professional
+```python
+voice_settings=VoiceSettings(
+    stability=0.8,        # Very consistent
+    similarity_boost=0.6,
+    style=0.0
+)
+```
+
+### Character Voices / Drama
+```python
+voice_settings=VoiceSettings(
+    stability=0.3,        # Highly expressive
+    similarity_boost=0.8,
+    style=0.5             # Strong style
+)
+```
+
+## Tips
+
+- **Start with defaults** and adjust incrementally
+- **Lower stability** if voice sounds monotonous
+- **Reduce similarity_boost** if you hear audio artifacts
+- **Style works** with v2+, v3, and multilingual models
+- **Test with representative text** from your actual use case
+- **Flash models** ignore some voice settings for speed

package/.agents/skills/vidfarm-media/SKILL.md

@@ -1,11 +1,11 @@
 ---
 name: vidfarm-media
-description: Media resolution for video workflows — narration TTS, voice-matched speech regeneration ("same speaker, new words"), word-level timings, BGM, SFX, and image/clip sourcing through vidfarm. Use when a video needs audio (voiceover/narration, music, sound effects), rewording of existing narration in the original speaker's voice, caption word timings, or media assets. Ships the shared audio engine scripts/audio.mjs (audio_request.json → audio_meta.json) used by faceless-explainer, product-launch-video, and general-video; documents vidfarm tts/stt BYOK primitives, the keyless local Kokoro + whisper.cpp engines, and My Files / raws library / generation sourcing. No third-party media accounts — everything is user-owned or generated on the user's keys.
+description: Media resolution for video workflows — AI music generation, narration TTS, voice-matched speech regeneration ("same speaker, new words"), transcription, word-level timings, BGM, SFX, and image/clip sourcing through vidfarm. Use when a video needs audio (music, voiceover/narration, sound effects), rewording of existing narration in the original speaker's voice, caption word timings, or media assets. ElevenLabs powers music/TTS/STT via the platform key (use_wallet_credits, wallet-billed) or the customer's own ElevenLabs key. Ships the shared audio engine scripts/audio.mjs (audio_request.json → audio_meta.json) used by faceless-explainer, product-launch-video, and general-video; documents the vidfarm music/tts/stt primitives, the keyless local Kokoro + whisper.cpp engines, and My Files / raws library / generation sourcing.
 ---
 
 # vidfarm-media
 
-One engine and one rule. The engine: `scripts/audio.mjs` turns a neutral `audio_request.json` into narration files, word timings, a music bed, and SFX — written into the project and described by one meta JSON. The rule: **no third-party media accounts, ever** — audio and images are the user's own files, found in the user's My Files / raws library, generated on the user's provider keys, or produced by keyless local models. If a source needs a sign-up, it is not a source.
+One engine and one rule. The engine: `scripts/audio.mjs` turns a neutral `audio_request.json` into narration files, word timings, a music bed, and SFX — written into the project and described by one meta JSON. The rule: **the user never signs up for a third-party media account.** Audio and images are the user's own files (My Files / raws library), generated on the user's provider keys, produced by keyless local models, OR generated through vidfarm's own ElevenLabs integration — music/TTS/STT on the platform key billed to the wallet (`use_wallet_credits`, the default) or on the customer's own ElevenLabs key. If a source would need the user to sign up somewhere else, it is not a source.
 
 ## Preflight (run once, before the brief)
 
@@ -56,27 +56,41 @@ Contract points:
 - BGM is **not** loop-extended by the engine — the workflow assemblers loop-extend short beds to the video length at mount time. `bgm_pending` is never set; `scripts/wait-bgm.mjs` exists as an exit-0 shim.
 - Mount the results per `hyperframes-core`: each `voices[].path` and (`bgm.path`, `sfx[]`) as `<audio>` tracks; `voices[].words` drive captions.
 - **Multi-track mixing is native — keep voice, music, and SFX on SEPARATE `<audio>` layers, each at its own `data-volume`** (do NOT pre-mix them into one file). The runtime and both render paths mix all audio layers together with per-track volume honored (a real ffmpeg `amix` at render). Standard balance: **narration/voice ~1.0, BGM ~0.1–0.2** under the voice, SFX per-item (the engine already emits `sfx[].volume`, e.g. `0.35`). This is the whole reason each stem is a distinct track — the user (or a later edit) can re-level any track independently.
-- **De-combining an original on recreate.** When you're recreating a video whose original had music + narration baked into ONE track, don't try to reproduce that single bed — you cannot stem-separate baked audio. Instead build the mix fresh: a new narration track (`tts` / `regenerate-speech`) at ~1.0 **plus** a real music file (owned / user-provided / found via `vidfarm directory search "<query> music"`) at ~0.1–0.2, as two separate `<audio>` layers, and drop the original combined track. Never fake a music layer or duplicate the voice track to stand in for music.
+- **De-combining an original on recreate.** When you're recreating a video whose original had music + narration baked into ONE track, don't try to reproduce that single bed — you cannot stem-separate baked audio. Instead build the mix fresh: a new narration track (`tts` / `regenerate-speech`) at ~1.0 **plus** a real music file (owned / user-provided / found via `vidfarm directory search "<query> music"` / generated with `vidfarm music "<vibe>"`) at ~0.1–0.2, as two separate `<audio>` layers, and drop the original combined track. Never fake a music layer or duplicate the voice track to stand in for music.
 
-## TTS / STT providers
+## Music, TTS & STT (ElevenLabs + BYOK)
 
-| Need | Command | Notes |
+**Music and speech are core primitives — use them freely.** ElevenLabs powers high-quality
+music, narration, and transcription. The `use_wallet_credits` flag (DEFAULT **true** on the
+music / TTS / STT primitives) runs on vidfarm's platform ElevenLabs key and bills the customer's
+wallet. **Keep it true** for the best voices/music out of the box; only set it **false** to save
+wallet credits or to use the customer's OWN saved ElevenLabs key (Settings → AI keys → ElevenLabs
+is an audio-only key that does NOT count as a qualified text/image/video key). With
+`use_wallet_credits: false` and no ElevenLabs key, TTS/STT fall back to a BYOK openai/gemini/
+openrouter key; music always needs ElevenLabs (own key or platform).
+
+| Need | Command / route | Notes |
 | --- | --- | --- |
-| Narration, promptable voice (BYOK) | `vidfarm tts "…" --style "calm, warm narrator"` | Local-first on your env key; `--voice alloy`/`Kore`…; `--cloud` = saved-key platform job → `POST /api/v1/primitives/audio/speech` |
+| **Music** (bed, beat, jingle, song, score) | `vidfarm music "upbeat lo-fi beat" --length 30` · `POST /api/v1/primitives/music/generate` | ElevenLabs. `use_wallet_credits` default true (platform key + wallet); `--own-key` = your ElevenLabs key. `music_length_ms` ≤ 300000 (5 min). Place as its own `<audio>` layer ~0.1–0.2 under narration. |
+| **Narration** (default) | `vidfarm tts "…" --cloud` · `POST /api/v1/primitives/audio/speech` | Default = ElevenLabs on the platform key (wallet-billed). Pick a voice with `--voice <voice_id>` (browse below). `--own-key` for your ElevenLabs/BYOK key. Local-first `vidfarm tts` (no `--cloud`) still runs on your env openai/gemini key. |
+| **List voices** | `vidfarm voices` · `GET /api/v1/primitives/audio/voices` | ElevenLabs voice catalog (voice_id, name, labels, preview_url). Default = platform account; `--own-key` / `?use_wallet_credits=false` = the customer's ElevenLabs account. **Default a sensible voice AND tell the user they can pick from many.** |
 | Narration, zero keys | `npx hyperframes tts "…" -v af_heart --json` | Kokoro-82M, local, WAV + duration in JSON |
-| Transcript + SRT (BYOK) | `vidfarm stt <file\|url>` | Gemini key labels speakers; OpenAI whisper-1 gives real word timestamps |
-| Reword existing narration in the (approximate) original voice | `POST /api/v1/primitives/audio/regenerate-speech` (platform job on saved keys; no devcli command yet) | One job: listens to a video/audio source, profiles the speaker (needs a Gemini key), rewords the transcript per `rewrite_instruction` (or takes exact `text`), regenerates with the closest preset voice + matched style. Always an approximation — OpenAI/Gemini cannot clone voices. Details: `references/tts.md` |
+| **Transcript + SRT** | `vidfarm stt <file\|url> --cloud` · `POST /api/v1/primitives/audio/transcribe` | Default = ElevenLabs Scribe (native diarization + real word timestamps), wallet-billed. `--own-key`/BYOK: gemini labels speakers, openai/whisper-1 gives real word timings. |
+| Reword existing narration in the (approximate) original voice | `POST /api/v1/primitives/audio/regenerate-speech` | Listens, profiles the speaker (needs a Gemini key), rewords, regenerates with the closest preset voice + matched style. Approximation, never a clone. Details: `references/tts.md` |
 | Word timings, zero keys | `npx hyperframes transcribe <audio> --json` | whisper.cpp; writes word-level `transcript.json` |
 | Captions straight into a composition | `vidfarm captions generate <dir>` | whisper-1 real timings when an OpenAI key exists; estimates otherwise |
 
-Voices, styles, formats, and the exact word-timestamp paths: `references/tts.md`.
+Voices, styles, formats, the ElevenLabs SDK details, and the exact word-timestamp paths:
+`references/tts.md`, plus the loadable `music` and `text-to-speech` skill packs for the raw
+ElevenLabs API.
 
 ## BGM and SFX resolution order
 
 1. **User file / URL** — `bgm.file` / `bgm.url` in the request; copied or downloaded into `assets/bgm/`.
 2. **File directory** — `vidfarm directory search "<query> music"` or the legacy `vidfarm files --search` (needs `$VIDFARM_API_KEY`); the user's own library, searchable by meaning when notes are annotated.
-3. **Local dir** — `$MUSIC_DIR` / `$SFX_DIR` (or request `bgm.dir` / `sfx_dir`), picked by filename keyword match against the query.
-4. **Skip, with a note** — a silent bed or a missing whoosh is fine; a stock-audio account is not. The meta's `notes[]` says exactly what was skipped and why.
+3. **Generate it** — `vidfarm music "<query>" --length <sec>` (ElevenLabs; `use_wallet_credits` default) writes an mp3 you can pass back as `bgm.file`. Great when the user has no fitting track. The engine script itself has no generate mode — run `vidfarm music` yourself and feed the result in.
+4. **Local dir** — `$MUSIC_DIR` / `$SFX_DIR` (or request `bgm.dir` / `sfx_dir`), picked by filename keyword match against the query.
+5. **Skip, with a note** — a silent bed or a missing whoosh is fine. The meta's `notes[]` says exactly what was skipped and why.
 
 ## The file directory (My Files + raws, unified)
 

package/.agents/skills/vidfarm-media/references/tts.md

@@ -1,6 +1,44 @@
-# TTS / STT providers — voices, timings, formats
+# Music / TTS / STT — voices, timings, formats
 
-Two tiers, one preflight. **BYOK cloud** rides the user's own provider key through vidfarm; **keyless local** runs Kokoro-82M and whisper.cpp on the user's machine with no account and no key. There is never a sign-in step.
+Three tiers. **ElevenLabs (default)** powers music, high-quality narration, and Scribe
+transcription — on vidfarm's platform key billed to the wallet (`use_wallet_credits: true`, the
+default) or on the customer's OWN saved ElevenLabs key (`use_wallet_credits: false`). **BYOK
+cloud** rides the user's own openai/gemini/openrouter key through vidfarm (TTS/STT fallback when
+`use_wallet_credits` is false and no ElevenLabs key is present). **Keyless local** runs Kokoro-82M
+and whisper.cpp on the user's machine with no account. There is never a sign-in step for the user.
+
+## use_wallet_credits (default true — keep it on)
+
+`music`, `tts` (`--cloud`), and `stt` (`--cloud`) default to ElevenLabs on the platform key,
+billed to the customer's wallet. Recommend keeping it on for the best voices/music with zero
+setup. Turn it OFF only to (a) save wallet credits or (b) use the customer's own ElevenLabs key.
+With it off and no ElevenLabs key, TTS/STT fall back to a BYOK openai/gemini/openrouter key; music
+always needs ElevenLabs. ElevenLabs in Settings → AI keys is **audio-only** — it does NOT count as
+a qualified text/image/video key, so the user still needs an openai/gemini/openrouter key for the
+core generation features.
+
+## Music — `vidfarm music`
+
+```bash
+vidfarm music "chill lo-fi hip hop beat with jazzy piano" --length 30 --json
+```
+
+- ElevenLabs music generation (`POST /api/v1/primitives/music/generate`). `--length <sec>` or
+  `--length-ms`, default 30s, max 5 min. `--own-key` uses the customer's ElevenLabs key instead of
+  the wallet. Returns a durable mp3; mount it as its own `<audio>` layer ~0.1–0.2 under narration.
+- For fine control, the primitive also accepts an ElevenLabs `composition_plan` (section-by-section
+  lyrics/style) — see the loadable `music` skill pack for the plan schema.
+
+## Voices — `vidfarm voices`
+
+```bash
+vidfarm voices                 # platform-account voices (voice_id, name, labels, preview_url)
+vidfarm voices --own-key       # the customer's own ElevenLabs account voices
+```
+
+- `GET /api/v1/primitives/audio/voices` (`?use_wallet_credits=false` for the user's own key).
+- **Default a sensible voice and TELL the user they can pick from many** — surface a few names +
+  the returned `voice_library_url`, then pass the chosen `voice_id` to `tts --voice <voice_id>`.
 
 ## Preflight
 
@@ -37,7 +75,7 @@ POST /api/v1/primitives/audio/regenerate-speech   # alias: /api/v1/primitives/re
 
 One job: listens to the source, profiles the dominant speaker with an audio-native Gemini call (verbatim transcript + gender/age/accent/pace/pitch/energy profile + a dense style direction + the closest preset in both voice families), rewords the transcript per `rewrite_instruction` (keeping length and rhythm) or takes the exact `text`, then regenerates via TTS with the matched preset + profiled style instructions. Result: durable `audioUrl`, `script`, `original_transcript`, `voice_profile`, `voice_match`, plus a `regenerate-speech.json` artifact.
 
-- **Honesty rule (non-negotiable): this is an approximation, never a clone.** Neither OpenAI nor Gemini exposes voice cloning through their public APIs — both offer preset voices with style steering only (actual cloning is sales-allowlisted at both, e.g. OpenAI "Custom Voices", Google Chirp 3 Instant Custom Voice). `voice_match` says `"approximate"` or `"none"`; always present it that way to the user. True self-serve cloning would need an ElevenLabs/Cartesia-class provider or a local OSS model (F5-TTS, CosyVoice, XTTS-v2, Chatterbox) — none are wired into vidfarm today.
+- **Honesty rule (non-negotiable): regenerate-speech is an approximation, never a clone.** It runs on OpenAI/Gemini, which expose preset voices with style steering only (no self-serve cloning of an arbitrary speaker). `voice_match` says `"approximate"` or `"none"`; always present it that way. (ElevenLabs IS wired into vidfarm for music, preset-voice TTS, and Scribe STT — but regenerate-speech does not use it for speaker cloning; treat any voice-match as an approximation.)
 - **Voice profiling needs a saved Gemini key** (it is the audio-native LLM). With only openai/openrouter keys the job degrades gracefully: STT transcript + reword still work, but the voice is a default or explicitly passed preset and `voice_match` is `"none"`.
 - Sources over ~40 minutes of speech must be trimmed first (same cap as `/audio/transcribe`).
 - To swap the result into a composition: mute the original span (`/videos/mute` or the layer's volume), then place `audioUrl` as a new audio layer.

package/SKILL.director.md

@@ -147,7 +147,7 @@ Name the plan back in these terms ("I'll SWAP the captions and REPLACE the scene
 
 **Fuel a scene REPLACE with raw clips, not expensive AI video.** A heavy scenes-axis replace needs footage; sources in cost order: (1) the director's own library — search `/raws` and `/files` (`vidfarm raws search …`, `vidfarm files --search …` / `browse_files`); (2) **HUNT new raws** out of a long-form source (podcast/VOD/webinar or any YouTube/TikTok/IG/X URL) — `vidfarm raws scan <src> --prompt "<what the new scenes need>" --aspect <canvas> [--duration N --no-text --range …]` (local-first, free compute) or the async `POST /clips/scan` / `/raws/scan`; then reuse the picks (`set_layer_media` / `vidfarm set-media` to swap in place, `add_layer`/`vidfarm place` for net-new scenes); (3) `generate_layer` / `vidfarm generate` AI generation — the **expensive last resort**, only for scenes no real clip can cover. When a big scene re-work is asked for but no footage is given, **ask for a source to hunt (or point at the raws library) before AI-generating** — see [Raws](#raws-long-form--short-form-raws) and [Generate AI media …](#generate-ai-media-and-drop-it-on-the-timeline).
 
-**Audio is natively multi-track — overlay narration + music + SFX, each at its own volume.** A composition mixes UNLIMITED simultaneous `<audio>` layers; each sits on its own `data-track-index` and carries its own `data-volume` (0–2, default 1), and the runtime mixes them with per-track volume honored identically in the preview and the exported MP4 (a real ffmpeg `amix` of every audio layer at render). So you never need a pre-mixed file — lay **narration/voiceover at ~1.0 on one track, a music bed at ~0.1–0.2 on a separate track, and SFX on their own tracks**, each via `add_layer kind=audio` (web) / `vidfarm place --kind audio --volume …` (devcli), tuning levels later with the Inspector's Volume slider or `set_layer_media` (`volume`, `muted`). **The key move when recreating a template whose original baked music + narration into ONE audio track: rebuild it as TWO independent tracks** — a fresh narration track (`/audio/speech`, or same-voice reword via `/audio/regenerate-speech` / `vidfarm speech regenerate`) at ~1.0 and a separate real music track at ~0.1–0.2 — then mute/remove the original combined source-audio layer. This gives the director independent voice and music volume, and works around AI TTS being unable to emit narration+music in one file: you compose the mix on the timeline. Honesty: you can't un-mix / stem-separate the original's baked audio — the two tracks are a fresh narration track **plus** a real music file (owned / user-provided / `browse_files`), never a faked or duplicated voice layer.
+**Audio is natively multi-track — overlay narration + music + SFX, each at its own volume.** A composition mixes UNLIMITED simultaneous `<audio>` layers; each sits on its own `data-track-index` and carries its own `data-volume` (0–2, default 1), and the runtime mixes them with per-track volume honored identically in the preview and the exported MP4 (a real ffmpeg `amix` of every audio layer at render). So you never need a pre-mixed file — lay **narration/voiceover at ~1.0 on one track, a music bed at ~0.1–0.2 on a separate track, and SFX on their own tracks**, each via `add_layer kind=audio` (web) / `vidfarm place --kind audio --volume …` (devcli), tuning levels later with the Inspector's Volume slider or `set_layer_media` (`volume`, `muted`). **The key move when recreating a template whose original baked music + narration into ONE audio track: rebuild it as TWO independent tracks** — a fresh narration track (`/audio/speech`, or same-voice reword via `/audio/regenerate-speech` / `vidfarm speech regenerate`) at ~1.0 and a separate real music track at ~0.1–0.2 — then mute/remove the original combined source-audio layer. This gives the director independent voice and music volume, and works around AI TTS being unable to emit narration+music in one file: you compose the mix on the timeline. Honesty: you can't un-mix / stem-separate the original's baked audio — the two tracks are a fresh narration track **plus** a real music file (owned / user-provided / `browse_files` / generated with `/api/v1/primitives/music/generate` (`vidfarm music`)), never a faked or duplicated voice layer.
 
 ## Edit in the Trackpad Editor
 
@@ -623,8 +623,10 @@ Send a stable `tracer` on export so retries are traceable and filterable in job
 | `vidfarm inpaint <image> --mask <png> --prompt "…" [--region "label=…"] [--ref …] [--out <f>]` | `POST /api/v1/primitives/images/inpaint` (polls job) | masked image EDIT — replace ONLY the transparent-mask region, keep everything else (devcli twin of the /inpaint page) |
 | `vidfarm create-overlay "<subject>" [--key-color #00FF00] [--aspect-ratio 1:1] [--place <dir>] [--out <f>]` | `POST /api/v1/primitives/images/create-overlay` (polls job) | **Vox-style** transparent OVERLAY — AI image on a forced key-color background, chroma-keyed out in one job → ready-to-composite transparent PNG |
 | `vidfarm remove-background-greenscreen <image> [--key-color #00FF00] [--tolerance 0.3] [--out <f>]` | `POST /api/v1/primitives/images/remove-background-greenscreen` (polls job) | cheap chroma-key removal of a FLAT solid background → transparent PNG (cloud primitive) |
-| `vidfarm tts "…" [--style "…"] [--voice <v>] [--out <file>]` | (LOCAL-FIRST: your own OPENAI/GEMINI/OPENROUTER_API_KEY → audio file on disk; `--cloud` = `POST /api/v1/primitives/audio/speech` + poll) | text → narration audio with a promptable voice style |
-| `vidfarm stt <file\|url> [--out <base>] [--no-diarize]` (alias: `transcribe`) | (LOCAL-FIRST: local ffmpeg demux + your own key; `--cloud` = `POST /api/v1/primitives/audio/transcribe` + poll) | video/audio → transcript in BOTH formats: simple subtitles (txt + SRT) and multi-speaker segments (json) |
+| `vidfarm tts "…" [--style "…"] [--voice <v>] [--out <file>]` | (LOCAL-FIRST: your own OPENAI/GEMINI/OPENROUTER_API_KEY → audio file on disk; `--cloud` = `POST /api/v1/primitives/audio/speech` + poll, ElevenLabs on the platform key by default, `--own-key` for yours) | text → narration audio; `--cloud --voice <voice_id>` picks an ElevenLabs voice |
+| `vidfarm music "<prompt>" [--length <sec>] [--out <f>] [--own-key]` | `POST /api/v1/primitives/music/generate` (polls job) | prompt → music track (ElevenLabs; platform key + wallet by default, `--own-key` for yours) |
+| `vidfarm voices [--own-key] [--limit N]` | `GET /api/v1/primitives/audio/voices` | list ElevenLabs voices (voice_id/name/labels) for `tts --voice`; default a voice + tell the user they can choose |
+| `vidfarm stt <file\|url> [--out <base>] [--no-diarize]` (alias: `transcribe`) | (LOCAL-FIRST: local ffmpeg demux + your own key; `--cloud` = `POST /api/v1/primitives/audio/transcribe` + poll, ElevenLabs Scribe on the platform key by default, `--own-key` for yours) | video/audio → transcript in BOTH formats: simple subtitles (txt + SRT) and multi-speaker segments (json) |
 | `vidfarm place <dir> --src <url\|file> [--at\|--replace]` | (edits local composition.html; local files → serve disk store or temp upload) | drop media (URL **or local file**) into a gap / over a scene |
 | `vidfarm captions generate <dir> [--style <preset>] [--audio <f>\|--srt <f>\|--text "…"]` | (LOCAL-FIRST: STT on your own key — OpenAI = real word timestamps — then edits local composition.html) | transcribe narration → animated word-by-word caption cues |
 | `vidfarm captions style <dir> --style <preset>` / `captions list` / `captions clear` | (edits local composition.html) | restyle / inspect / remove animated captions |
@@ -978,19 +980,34 @@ curl -X POST "$VIDFARM_BASE/api/v1/primitives/media/dedupe" \
   }'
 ```
 
+## Primitive: music (text → music)
+
+Generate music (instrumental, songs with lyrics, background beds, jingles, scores) via ElevenLabs. **Default this on freely — music is a core primitive.** `use_wallet_credits` defaults **true**: it runs on vidfarm's platform ElevenLabs key and bills the customer's wallet. Recommend keeping it on; set it false only to save wallet credits or to use the customer's OWN saved ElevenLabs key.
+
+- `POST /api/v1/primitives/music/generate` (alias: `POST /api/v1/primitives/music`)
+- Body: `{ "tracer": "...", "payload": { "prompt": "...", "music_length_ms"?: 30000, "composition_plan"?: {...}, "model"?: "...", "use_wallet_credits"?: true }, "webhook_url"?: "..." }`
+  - `prompt` — describe the vibe/genre/instrumentation. `music_length_ms` ≤ 300000 (5 min), default 30000.
+  - `composition_plan` (optional) — ElevenLabs section-by-section plan for granular lyric/style control (see the loadable `music` skill pack).
+- Result: durable mp3 (`primary_file_url` / `audio.file_url`). Mount as its own `<audio>` layer ~0.1–0.2 under narration.
+- devcli: `vidfarm music "<prompt>" --length 30` (`--own-key` to use your ElevenLabs key).
+
 ## Primitive: tts (text → speech)
 
-Generate spoken narration audio from text on the caller's saved AI provider keys (OpenAI, Gemini, or OpenRouter — BYOK, never wallet-billed). The **voice style is promptable**: `instructions` is a free-form direction for tone, pacing, accent, emotion, and persona, on top of the provider's `voice` preset. Result is a durable Vidfarm audio URL (mp3/wav) ready to drop into a composition as an audio layer.
+Generate spoken narration audio from text. **`use_wallet_credits` defaults true** → high-quality ElevenLabs narration on vidfarm's platform key, billed to the wallet — the recommended default. Set it false to use the customer's OWN ElevenLabs key or a BYOK OpenAI/Gemini/OpenRouter key (BYOK is never wallet-billed). The **voice style is promptable** via `instructions` (tone/pacing/accent/emotion/persona). Result is a durable Vidfarm audio URL (mp3/wav) ready to drop into a composition as an audio layer.
 
 - `POST /api/v1/primitives/audio/speech` (alias: `POST /api/v1/primitives/tts`)
-- Body: `{ "tracer": "...", "payload": { "text": "...", "voice"?: "...", "instructions"?: "...", "provider"?: "openai" | "gemini" | "openrouter", "model"?: "...", "output_format"?: "mp3" | "wav" }, "webhook_url"?: "..." }`
+- Body: `{ "tracer": "...", "payload": { "text": "...", "voice"?: "...", "instructions"?: "...", "use_wallet_credits"?: true, "provider"?: "openai" | "gemini" | "openrouter", "model"?: "...", "output_format"?: "mp3" | "wav" }, "webhook_url"?: "..." }`
   - `text` (required, ≤8000 chars) — the words to speak, verbatim. Aliases: `input`, `script`.
-  - `voice` (optional) — provider voice preset (OpenAI: `alloy`, `ash`, `coral`, …; Gemini: `Kore`, `Puck`, `Charon`, …).
-  - `instructions` (optional) — the voice-STYLE prompt, e.g. `"calm, warm bedtime narrator with a slight British accent"` or `"excited sports announcer, fast paced"`. Aliases: `style`, `style_instructions`, `voice_style`.
-  - `output_format` defaults to `mp3` (note: Gemini answers in wav regardless — read `audio.content_type`).
+  - `voice` (optional) — on the ElevenLabs path, an ElevenLabs **voice_id** (or friendly name george/sarah/…); on the BYOK path a provider preset (OpenAI `alloy`/…; Gemini `Kore`/…). **Default a sensible voice and tell the user they can pick from many** — list them with `GET /api/v1/primitives/audio/voices` (or `vidfarm voices`).
+  - `instructions` (optional) — the voice-STYLE prompt. Aliases: `style`, `style_instructions`, `voice_style`.
+  - `output_format` defaults to `mp3` (Gemini/ElevenLabs may answer in wav — read `audio.content_type`).
 - Response: standard primitive job. Poll to completion, then read `primary_file_url` / `audio.file_url`.
-- Billing: BYOK — the speech call runs on the customer's own provider key and is never wallet-billed.
-- devcli: `vidfarm tts "…" --style "…"` runs LOCAL-FIRST on your own env key (no job, no cloud); add `--cloud` to run this route instead.
+- Billing: wallet-billed on the platform ElevenLabs key (default); free/BYOK when `use_wallet_credits: false` and the caller's own key serves it.
+- devcli: `vidfarm tts "…" --style "…"` runs LOCAL-FIRST on your own env key (no job); add `--cloud` for the ElevenLabs platform job (`--own-key` = your key). `vidfarm voices` lists voices.
+
+## Primitive: audio/voices (list ElevenLabs voices)
+
+`GET /api/v1/primitives/audio/voices` returns the ElevenLabs voice catalog — `{ scope, default_voice_id, voice_library_url, voices: [{ voice_id, name, category, labels, description, preview_url }] }`. Default scope is vidfarm's platform account; add `?use_wallet_credits=false` to list the voices on the customer's OWN saved ElevenLabs key. Pick a `voice_id` and pass it as `voice` to `/audio/speech`. devcli: `vidfarm voices` (`--own-key` for the user's account).
 
 Example:
 
@@ -1016,13 +1033,14 @@ Transcribe speech from **any video or audio URL** on the caller's saved AI provi
 2. **Advanced multi-speaker version** — speaker-attributed, timestamped segments (`output.segments`, each `{ speaker, start_sec, end_sec, text }`, with `output.speakers` listing the cast) for multi-narration sources like podcasts, interviews, and skits — stored durably as `transcript.json`.
 
 - `POST /api/v1/primitives/audio/transcribe` (alias: `POST /api/v1/primitives/stt`)
-- Body: `{ "tracer": "...", "payload": { "source_url": "https://...", "diarize"?: true, "language"?: "en", "prompt"?: "...", "provider"?: "gemini" | "openai" | "openrouter", "model"?: "...", "media_type"?: "auto" | "video" | "audio" }, "webhook_url"?: "..." }`
+- Body: `{ "tracer": "...", "payload": { "source_url": "https://...", "diarize"?: true, "word_timestamps"?: false, "use_wallet_credits"?: true, "language"?: "en", "prompt"?: "...", "provider"?: "gemini" | "openai" | "openrouter", "model"?: "...", "media_type"?: "auto" | "video" | "audio" }, "webhook_url"?: "..." }`
   - `source_url` (required) — video OR audio URL. Aliases: `source_video_url`, `source_audio_url`, `video_url`, `audio_url`, `url`. For a fork's composition, pick the right source via `GET /api/v1/compositions/:forkId/remove-video-captions` first.
-  - `diarize` (default `true`) — attribute segments to speakers. **Speaker labels need a Gemini key** (audio-native model); OpenAI/OpenRouter keys degrade to a plain single-speaker transcript (`diarization: "none"`).
+  - `use_wallet_credits` (default `true`) — ElevenLabs Scribe on the platform key (native diarization + real word timestamps), billed to the wallet. Set false to use the customer's OWN ElevenLabs key or a BYOK gemini/openai/openrouter key.
+  - `diarize` (default `true`) — attribute segments to speakers. On the BYOK path speaker labels need a **Gemini key**; OpenAI/OpenRouter degrade to a single-speaker transcript. ElevenLabs Scribe (the default) diarizes natively.
   - `prompt` (optional) — domain/vocabulary hint; `language` (optional) — expected language code.
 - Response: standard primitive job. Poll to completion; `output` carries both formats inline (very long transcripts set `text_truncated: true` and keep the full copy in the `transcript.json` artifact), and the artifacts `transcript.json` / `transcript.srt` / `transcript.txt` are durable URLs.
 - Limits: ~40 minutes of speech per job (20MB extracted audio) — trim longer sources with `/videos/trim` or `/audio/trim` and transcribe in parts.
-- Billing: the AI transcription is BYOK (never wallet-billed); only the tiny audio-demux compute step bills (`primitive_media_lambda`), like every media primitive.
+- Billing: wallet-billed on the platform ElevenLabs key (default); free/BYOK when `use_wallet_credits: false`. The tiny audio-demux compute step always bills (`primitive_media_lambda`), like every media primitive.
 - For the CURRENT fork, the decompose-time transcript may already exist via `video-context.json` — prefer it before paying for a new transcription; use this primitive for other URLs, multi-speaker attribution, or SRT output.
 - devcli: `vidfarm stt <file|url>` runs LOCAL-FIRST (local ffmpeg + your own key); add `--cloud` to run this route instead.
 

package/SKILL.platform.md

@@ -281,9 +281,9 @@ Primitives are hosted operations that are NOT templates — they're reusable bui
 - `download_media_video` — social/media URL → durable MP4
 - `hyperframes_render` — composition HTML → MP4 (the render primitive publishing uses)
 - `brainstorm` — LLM brainstorm route
-- `tts` / `stt` — speech, BYOK on the caller's saved provider keys. `POST /api/v1/primitives/audio/speech` (`primitive:tts`) turns text into narration audio with a promptable voice style (`instructions`); `POST /api/v1/primitives/audio/transcribe` (`primitive:stt`) transcribes any video or audio URL (video demuxes through the primitive-media ffmpeg seam first, 64kbps mono, ~40min/20MB cap) and returns **two formats**: a simple subtitle transcript (plain text + SRT cues) and an advanced multi-speaker version (speaker-attributed timed segments; diarization is Gemini-only — JSON-mode audio transcription in `src/services/speech.ts`). The raw provider speech calls live in `src/services/speech.ts`, shared by `ProviderService` (leased BYOK keys, usage metering) and the devcli `vidfarm tts`/`vidfarm stt` commands, which run LOCAL-FIRST on the user's own env key (local ffmpeg demux, no job, no wallet) with `--cloud` as the explicit backup route.
+- `music` / `tts` / `stt` — ElevenLabs audio (music, narration, transcription) with a `use_wallet_credits` flag that **defaults true**. `POST /api/v1/primitives/music/generate` (`primitive:music`) generates music from a prompt or composition plan. `POST /api/v1/primitives/audio/speech` (`primitive:tts`) turns text into narration with a promptable style (`instructions`) and an ElevenLabs `voice_id` (browse via `GET /api/v1/primitives/audio/voices` → `providers.listElevenLabsVoices`). `POST /api/v1/primitives/audio/transcribe` (`primitive:stt`) transcribes any video/audio URL (video demuxes through the primitive-media ffmpeg seam first, 64kbps mono, ~40min/20MB cap) into **two formats** — simple subtitles (text + SRT) and speaker-attributed segments. **Routing (`src/services/providers.ts` `resolveAudioAuthPlan`):** `use_wallet_credits: true` (default) → the platform `ELEVENLABS_API_KEY`, billed to the wallet (`elevenlabs_music`/`elevenlabs_tts`/`elevenlabs_stt` cost centers); `false` → the customer's own saved `elevenlabs` key (no charge), else a BYOK gemini/openai/openrouter key (TTS/STT only), else the platform key + wallet. The ElevenLabs calls live in `src/services/elevenlabs.ts`; the openai/gemini/openrouter speech calls in `src/services/speech.ts`. The `elevenlabs` provider key is **audio-only** — it never counts as a qualified text/image/video key. devcli: `vidfarm music`, `vidfarm voices`, and `vidfarm tts`/`vidfarm stt` (LOCAL-FIRST on the user's env key; `--cloud` hits ElevenLabs, `--own-key` uses the user's own key).
 
-Directors call primitives when they want a one-off asset before feeding it into a template. Templates call primitives internally as workflow steps. Each primitive declares one of three billing modes: **WALLET** — a metered platform charge (e.g. `remove_background` / `image_remove_background` RapidAPI pass-through, `video_remove_captions` GhostCut, cloud `hyperframes_render`, the local `image_remove_background_greenscreen` chroma key, and `media_overlay` which layers a small platform fee on top of its BYOK image-gen leg); **BYOK** — the AI spend rides the caller's own provider key and is never wallet-billed (tts/stt, smart decompose, clip tagging/embedding); or **COMPUTE-ONLY** — only the AWS Lambda / Step-Functions compute bills (clip hunts, the stt audio-demux step). So "all primitives are metered against the caller's wallet" holds only for the compute/platform legs — every BYOK AI call is excepted.
+Directors call primitives when they want a one-off asset before feeding it into a template. Templates call primitives internally as workflow steps. Each primitive declares one of three billing modes: **WALLET** — a metered platform charge (e.g. `remove_background` / `image_remove_background` RapidAPI pass-through, `video_remove_captions` GhostCut, cloud `hyperframes_render`, the local `image_remove_background_greenscreen` chroma key, `media_overlay`, and **platform ElevenLabs music/TTS/STT when `use_wallet_credits` is on** — the default); **BYOK** — the AI spend rides the caller's own provider key and is never wallet-billed (smart decompose, clip tagging/embedding, and music/TTS/STT run with `use_wallet_credits: false` on the user's own ElevenLabs/openai/gemini/openrouter key); or **COMPUTE-ONLY** — only the AWS Lambda / Step-Functions compute bills (clip hunts, the stt audio-demux step). So "all primitives are metered against the caller's wallet" holds only for the compute/platform legs — every BYOK AI call is excepted.
 
 ## Billing