@mevdragon/vidfarm-devcli 0.19.1 → 0.20.0

package/.agents/skills/text-to-speech/SKILL.md

@@ -0,0 +1,226 @@
+---
+name: text-to-speech
+description: Convert text to speech using ElevenLabs voice AI. Use when generating audio from text, creating voiceovers, building voice apps, or synthesizing speech in 70+ languages.
+license: MIT
+compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).
+metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}}
+---
+
+# ElevenLabs Text-to-Speech
+
+Generate natural speech from text - supports 70+ languages, multiple models for quality vs latency tradeoffs.
+
+> **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only.
+
+## Quick Start
+
+### Python
+
+```python
+from elevenlabs import ElevenLabs
+
+client = ElevenLabs()
+
+audio = client.text_to_speech.convert(
+    text="Hello, welcome to ElevenLabs!",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",  # George
+    model_id="eleven_multilingual_v2"
+)
+
+with open("output.mp3", "wb") as f:
+    for chunk in audio:
+        f.write(chunk)
+```
+
+### JavaScript
+
+```javascript
+import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
+import { createWriteStream } from "fs";
+
+const client = new ElevenLabsClient();
+const audio = await client.textToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", {
+  text: "Hello, welcome to ElevenLabs!",
+  modelId: "eleven_multilingual_v2",
+});
+audio.pipe(createWriteStream("output.mp3"));
+```
+
+### cURL
+
+```bash
+curl -X POST "https://api.elevenlabs.io/v1/text-to-speech/JBFqnCBsd6RMkjVDRZzb" \
+  -H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
+  -d '{"text": "Hello!", "model_id": "eleven_multilingual_v2"}' --output output.mp3
+```
+
+## Models
+
+| Model ID | Languages | Latency | Best For |
+|----------|-----------|---------|----------|
+| `eleven_v3` | 70+ | Standard | Highest quality, emotional range |
+| `eleven_multilingual_v2` | 29 | Standard | High quality, long-form content |
+| `eleven_flash_v2_5` | 32 | ~75ms | Ultra-low latency, real-time |
+| `eleven_flash_v2` | English | ~75ms | English-only, fastest |
+| `eleven_turbo_v2_5` | 32 | ~250-300ms | Balanced quality/speed |
+| `eleven_turbo_v2` | English | ~250-300ms | English-only, balanced |
+
+## Voice IDs
+
+Use pre-made voices or create custom voices in the dashboard.
+
+**Popular voices:**
+- `JBFqnCBsd6RMkjVDRZzb` - George (male, narrative)
+- `EXAVITQu4vr4xnSDxMaL` - Sarah (female, soft)
+- `onwK4e9ZLuTAKqWW03F9` - Daniel (male, authoritative)
+- `XB0fDUnXU5powFXDhCwa` - Charlotte (female, conversational)
+
+```python
+voices = client.voices.get_all()
+for voice in voices.voices:
+    print(f"{voice.voice_id}: {voice.name}")
+```
+
+## Voice Settings
+
+Fine-tune how the voice sounds:
+
+- **Stability**: How consistent the voice stays. Lower values = more emotional range and variation, but can sound unstable. Higher = steady, predictable delivery.
+- **Similarity boost**: How closely to match the original voice sample. Higher values sound more like the original but may amplify audio artifacts.
+- **Style**: Exaggerates the voice's unique style characteristics (only works with v2+ models).
+- **Speaker boost**: Post-processing that enhances clarity and voice similarity.
+
+```python
+from elevenlabs import VoiceSettings
+
+audio = client.text_to_speech.convert(
+    text="Customize my voice settings.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    voice_settings=VoiceSettings(
+        stability=0.5,
+        similarity_boost=0.75,
+        style=0.5,
+        speed=1.0,             # 0.25 to 4.0 (default 1.0)
+        use_speaker_boost=True
+    )
+)
+```
+
+## Language Selection
+
+Use `language_code` with models that support language enforcement to guide pronunciation and text normalization. Unsupported language codes are ignored, and `language_code` is not supported on `eleven_multilingual_v2`.
+
+```python
+audio = client.text_to_speech.convert(
+    text="Bonjour, comment allez-vous?",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    model_id="eleven_v3",
+    language_code="fr"  # ISO 639-1 code
+)
+```
+
+## Text Normalization
+
+Controls how numbers, dates, and abbreviations are converted to spoken words. For example, "01/15/2026" becomes "January fifteenth, twenty twenty-six":
+
+- `"auto"` (default): Model decides based on context
+- `"on"`: Always normalize (use when you want natural speech)
+- `"off"`: Speak literally (use when you want "zero one slash one five...")
+
+```python
+audio = client.text_to_speech.convert(
+    text="Call 1-800-555-0123 on 01/15/2026",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    apply_text_normalization="on"
+)
+```
+
+## Request Stitching
+
+When generating long audio in multiple requests, the audio can have pops, unnatural pauses, or tone shifts at the boundaries. Request stitching solves this by letting each request know what comes before/after it:
+
+```python
+# First request
+audio1 = client.text_to_speech.convert(
+    text="This is the first part.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    next_text="And this continues the story."
+)
+
+# Second request using previous context
+audio2 = client.text_to_speech.convert(
+    text="And this continues the story.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    previous_text="This is the first part."
+)
+```
+
+## Output Formats
+
+| Format | Description |
+|--------|-------------|
+| `mp3_44100_128` | MP3 44.1kHz 128kbps (default) - compressed, good for web/apps |
+| `mp3_44100_192` | MP3 44.1kHz 192kbps (Creator+) - higher quality compressed |
+| `mp3_44100_64` | MP3 44.1kHz 64kbps - lower quality, smaller files |
+| `mp3_22050_32` | MP3 22.05kHz 32kbps - smallest MP3 files |
+| `pcm_16000` | Raw PCM 16kHz - use for real-time processing |
+| `pcm_22050` | Raw PCM 22.05kHz |
+| `pcm_24000` | Raw PCM 24kHz - good balance for streaming |
+| `pcm_44100` | Raw PCM 44.1kHz (Pro+) - CD quality |
+| `pcm_48000` | Raw PCM 48kHz (Pro+) - highest quality |
+| `ulaw_8000` | μ-law 8kHz - standard for phone systems (Twilio, telephony) |
+| `alaw_8000` | A-law 8kHz - telephony (alternative to μ-law) |
+| `opus_48000_64` | Opus 48kHz 64kbps - efficient streaming codec |
+| `wav_44100` | WAV 44.1kHz - uncompressed with headers |
+
+## Streaming
+
+For real-time applications, use the `stream` method (returns audio chunks as they're generated):
+
+```python
+audio_stream = client.text_to_speech.stream(
+    text="This text will be streamed as audio.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    model_id="eleven_flash_v2_5"  # Ultra-low latency
+)
+
+for chunk in audio_stream:
+    play_audio(chunk)
+```
+
+See [references/streaming.md](references/streaming.md) for WebSocket streaming.
+
+## Error Handling
+
+```python
+try:
+    audio = client.text_to_speech.convert(
+        text="Generate speech",
+        voice_id="invalid-voice-id"
+    )
+except Exception as e:
+    print(f"API error: {e}")
+```
+
+Common errors:
+- **401**: Invalid API key
+- **422**: Invalid parameters (check voice_id, model_id)
+- **429**: Rate limit exceeded
+
+## Tracking Costs
+
+Monitor character usage via response headers (`x-character-count`, `request-id`):
+
+```python
+response = client.text_to_speech.convert.with_raw_response(
+    text="Hello!", voice_id="JBFqnCBsd6RMkjVDRZzb", model_id="eleven_multilingual_v2"
+)
+audio = response.parse()
+print(f"Characters used: {response.headers.get('x-character-count')}")
+```
+
+## References
+
+- [Installation Guide](references/installation.md)
+- [Streaming Audio](references/streaming.md)
+- [Voice Settings](references/voice-settings.md)

package/.agents/skills/text-to-speech/references/installation.md

@@ -0,0 +1,90 @@
+# Installation
+
+## JavaScript / TypeScript
+
+```bash
+npm install @elevenlabs/elevenlabs-js
+```
+
+> **Important:** Always use `@elevenlabs/elevenlabs-js`. The old `elevenlabs` npm package (v1.x) is deprecated and should not be used.
+
+```javascript
+import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
+
+// Option 1: Environment variable (recommended)
+// Set ELEVENLABS_API_KEY in your environment
+const client = new ElevenLabsClient();
+
+// Option 2: Pass directly
+const client = new ElevenLabsClient({ apiKey: "your-api-key" });
+```
+
+### Migrating from deprecated packages
+
+If you have old packages installed, remove them:
+
+```bash
+# Remove deprecated packages
+npm uninstall elevenlabs
+
+# Install the current packages
+npm install @elevenlabs/elevenlabs-js
+```
+
+**Import changes:**
+```javascript
+// OLD (deprecated)
+import { ElevenLabsClient } from "elevenlabs";
+
+// NEW (current)
+import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
+```
+
+## Python
+
+```bash
+pip install elevenlabs
+```
+
+```python
+from elevenlabs import ElevenLabs
+
+# Option 1: Environment variable (recommended)
+# Set ELEVENLABS_API_KEY in your environment
+client = ElevenLabs()
+
+# Option 2: Pass directly
+client = ElevenLabs(api_key="your-api-key")
+```
+
+## cURL / REST API
+
+Set your API key as an environment variable:
+
+```bash
+export ELEVENLABS_API_KEY="your-api-key"
+```
+
+Include in requests via the `xi-api-key` header:
+
+```bash
+curl -X POST "https://api.elevenlabs.io/v1/text-to-speech/{voice_id}" \
+  -H "xi-api-key: $ELEVENLABS_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"text": "Hello world", "model_id": "eleven_multilingual_v2"}'
+```
+
+## Getting an API Key
+
+1. Sign up at [elevenlabs.io](https://elevenlabs.io)
+2. Go to [API Keys](https://elevenlabs.io/app/settings/api-keys)
+3. Click **Create API Key**
+4. Copy and store securely
+
+Or use the `setup-api-key` skill for guided setup.
+
+## Environment Variables
+
+| Variable | Description |
+|----------|-------------|
+| `ELEVENLABS_API_KEY` | Your ElevenLabs API key (required) |

package/.agents/skills/text-to-speech/references/streaming.md

@@ -0,0 +1,307 @@
+# Streaming Audio
+
+Stream audio chunks as they're generated for lower latency.
+
+## Model Selection for Streaming
+
+| Model | Latency | Use Case |
+|-------|---------|----------|
+| `eleven_flash_v2_5` | ~75ms | Lowest latency, 32 languages |
+| `eleven_flash_v2` | ~75ms | Lowest latency, English only |
+| `eleven_turbo_v2_5` | Low | Balanced quality/speed |
+
+## Python Streaming
+
+```python
+from elevenlabs import ElevenLabs
+
+client = ElevenLabs()
+
+audio_stream = client.text_to_speech.stream(
+    text="This is a streaming example with ultra-low latency.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    model_id="eleven_flash_v2_5"
+)
+
+with open("output.mp3", "wb") as f:
+    for chunk in audio_stream:
+        f.write(chunk)
+```
+
+### Real-Time Playback
+
+```python
+import subprocess
+
+def play_stream(audio_stream):
+    process = subprocess.Popen(
+        ["ffplay", "-nodisp", "-autoexit", "-"],
+        stdin=subprocess.PIPE
+    )
+    for chunk in audio_stream:
+        process.stdin.write(chunk)
+    process.stdin.close()
+    process.wait()
+
+audio_stream = client.text_to_speech.stream(
+    text="Playing this audio in real-time.",
+    voice_id="JBFqnCBsd6RMkjVDRZzb",
+    model_id="eleven_flash_v2_5"
+)
+play_stream(audio_stream)
+```
+
+## JavaScript Streaming
+
+```javascript
+import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
+import { createWriteStream } from "fs";
+
+const client = new ElevenLabsClient();
+
+const audioStream = await client.textToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", {
+  text: "Streaming audio in JavaScript.",
+  modelId: "eleven_flash_v2_5",
+});
+
+// Write to file
+const writeStream = createWriteStream("output.mp3");
+audioStream.pipe(writeStream);
+
+// Or process chunks
+for await (const chunk of audioStream) {
+  console.log(`Received ${chunk.length} bytes`);
+}
+```
+
+## WebSocket Streaming
+
+For text-streaming input where you send text chunks as they arrive (e.g., from an LLM).
+
+### Connection
+
+```
+wss://api.elevenlabs.io/v1/text-to-speech/{voiceId}/stream-input?model_id={modelId}
+```
+
+**Note:** WebSockets are unavailable for the `eleven_v3` model. Use `eleven_flash_v2_5` for lowest latency.
+
+### Message Flow
+
+1. **Initialize** - Send voice settings and configuration
+2. **Send text** - Stream text chunks as they arrive
+3. **Close** - Send empty string to signal completion
+4. **Receive** - Process audio chunks as they're generated
+
+### Python WebSocket
+
+```python
+import asyncio
+import json
+import base64
+import os
+import websockets
+from dotenv import load_dotenv
+
+load_dotenv()
+
+ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY")
+
+async def text_to_speech_ws_streaming(voice_id: str, model_id: str):
+    uri = f"wss://api.elevenlabs.io/v1/text-to-speech/{voice_id}/stream-input?model_id={model_id}"
+
+    async with websockets.connect(uri) as websocket:
+        # Initialize connection
+        await websocket.send(json.dumps({
+            "text": " ",
+            "voice_settings": {
+                "stability": 0.5,
+                "similarity_boost": 0.8
+            },
+            "generation_config": {
+                "chunk_length_schedule": [120, 160, 250, 290]
+            },
+            "xi_api_key": ELEVENLABS_API_KEY
+        }))
+
+        # Send text chunks
+        await websocket.send(json.dumps({"text": "Hello, "}))
+        await websocket.send(json.dumps({"text": "this is streaming text "}))
+        await websocket.send(json.dumps({"text": "from a WebSocket connection."}))
+
+        # Close stream (empty text signals completion)
+        await websocket.send(json.dumps({"text": ""}))
+
+        # Receive and process audio chunks
+        audio_chunks = []
+        while True:
+            message = await websocket.recv()
+            data = json.loads(message)
+            if data.get("audio"):
+                audio_chunks.append(base64.b64decode(data["audio"]))
+            elif data.get("isFinal"):
+                break
+
+        return b"".join(audio_chunks)
+
+async def main():
+    audio = await text_to_speech_ws_streaming(
+        voice_id="JBFqnCBsd6RMkjVDRZzb",
+        model_id="eleven_flash_v2_5"
+    )
+    with open("output.mp3", "wb") as f:
+        f.write(audio)
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+### JavaScript WebSocket
+
+```javascript
+import "dotenv/config";
+import WebSocket from "ws";
+import * as fs from "node:fs";
+
+const ELEVENLABS_API_KEY = process.env.ELEVENLABS_API_KEY;
+
+async function textToSpeechWsStreaming(voiceId, modelId) {
+  const uri = `wss://api.elevenlabs.io/v1/text-to-speech/${voiceId}/stream-input?model_id=${modelId}`;
+
+  return new Promise((resolve, reject) => {
+    const websocket = new WebSocket(uri, {
+      headers: { "xi-api-key": ELEVENLABS_API_KEY },
+    });
+
+    const audioChunks = [];
+
+    websocket.on("open", () => {
+      // Initialize connection
+      websocket.send(
+        JSON.stringify({
+          text: " ",
+          voice_settings: {
+            stability: 0.5,
+            similarity_boost: 0.8,
+          },
+          generation_config: {
+            chunk_length_schedule: [120, 160, 250, 290],
+          },
+        })
+      );
+
+      // Send text chunks
+      websocket.send(JSON.stringify({ text: "Hello, " }));
+      websocket.send(JSON.stringify({ text: "this is streaming text " }));
+      websocket.send(JSON.stringify({ text: "from a WebSocket connection." }));
+
+      // Close stream
+      websocket.send(JSON.stringify({ text: "" }));
+    });
+
+    websocket.on("message", (event) => {
+      const data = JSON.parse(event.toString());
+      if (data.audio) {
+        audioChunks.push(Buffer.from(data.audio, "base64"));
+      } else if (data.isFinal) {
+        websocket.close();
+        resolve(Buffer.concat(audioChunks));
+      }
+    });
+
+    websocket.on("error", reject);
+  });
+}
+
+const audio = await textToSpeechWsStreaming(
+  "JBFqnCBsd6RMkjVDRZzb",
+  "eleven_flash_v2_5"
+);
+fs.writeFileSync("output.mp3", audio);
+```
+
+### Input Messages
+
+**Initialization (first message):**
+
+```json
+{
+  "text": " ",
+  "voice_settings": {
+    "stability": 0.5,
+    "similarity_boost": 0.8,
+    "use_speaker_boost": false
+  },
+  "generation_config": {
+    "chunk_length_schedule": [120, 160, 250, 290]
+  },
+  "xi_api_key": "your_api_key"
+}
+```
+
+**Text chunks:**
+
+```json
+{ "text": "Your text content here" }
+```
+
+**Force flush (generate audio immediately):**
+
+```json
+{ "text": "End of sentence.", "flush": true }
+```
+
+**Close connection:**
+
+```json
+{ "text": "" }
+```
+
+### Output Messages
+
+**Audio chunk:**
+
+```json
+{
+  "audio": "base64_encoded_audio_data"
+}
+```
+
+**Stream complete:**
+
+```json
+{
+  "isFinal": true
+}
+```
+
+### Key Parameters
+
+| Parameter | Description |
+|-----------|-------------|
+| `chunk_length_schedule` | Array of character counts that trigger audio generation. The model waits until it has this many characters before generating audio, which improves quality but adds latency. Lower values = faster response, higher values = better prosody. Example: `[120, 160, 250, 290]` means generate after 120 chars, then after 160 more, etc. |
+| `flush` | Set `true` to force immediate audio generation without waiting for the character threshold. Use at the end of sentences or when you need audio NOW. |
+| `voice_settings` | Adjustable per-message: `stability`, `similarity_boost`, `use_speaker_boost` |
+
+### Important Notes
+
+- **Inactivity timeout**: Connection closes after 20 seconds without activity. Send a space `" "` to keep alive.
+- **TTFB (Time to First Byte)**: How long until audio starts playing. Affected by `chunk_length_schedule` - the model waits for enough text before generating.
+- **Model limitation**: WebSockets are unavailable for `eleven_v3`.
+- **Best practice**: Use `flush: true` at conversation turn endings to ensure the buffered text gets spoken.
+- **Alignment data**: Word-level timestamps available via `alignment` field for lip-sync or captions.
+
+## Best Practices
+
+1. **Use Flash models** for real-time:
+   - `eleven_flash_v2_5` for multilingual (~75ms)
+   - `eleven_flash_v2` for English-only (~75ms)
+
+2. **Buffer audio** before playback to prevent choppy output
+
+3. **Handle disconnections** gracefully in WebSocket streams
+
+4. **Choose output format based on use case**:
+   - `pcm_24000` - lowest latency processing
+   - `mp3_44100_128` - direct playback
+   - `ulaw_8000` - telephony/Twilio integration