smart_prompt 0.5.0 → 0.5.1

data/docs/STT_README.md

@@ -0,0 +1,302 @@
+# SmartPrompt STT Guide
+
+This guide explains how to use the new Speech-to-Text (STT) capabilities in SmartPrompt.
+
+## Overview
+
+The STT feature adds support for:
+- **Speech-to-Text Transcription**: Convert audio files to text
+- **URL-based Transcription**: Transcribe audio from URLs
+- **Multi-language Support**: Chinese, English, Japanese, Korean
+- **Batch Processing**: Process multiple audio files efficiently
+- **Language Detection**: Automatically detect language from audio
+- **Multiple Formats**: JSON, text, SRT, VTT output formats
+
+## Installation
+
+Make sure you have the required dependencies:
+
+```bash
+gem install openai
+```
+
+## Configuration
+
+Add the STT adapter to your configuration:
+
+```yaml
+# config.yml
+adapters:
+  multimodal: "MultimodalAdapter"
+  image_generation: "ImageGenerationAdapter"
+  video_generation: "VideoGenerationAdapter"
+  tts: "TTSAdapter"
+  stt: "STTAdapter"
+
+llms:
+  stt_service:
+    adapter: "stt"
+    url: "https://api.siliconflow.cn/v1/"
+    api_key: "ENV[SILICONFLOW_API_KEY]"
+    model: "FunAudioLLM/CosyVoice2-0.5B"
+
+default_llm: "qwen_vl"
+template_path: "./templates"
+worker_path: "./workers"
+logger_file: "./logs/smart_prompt.log"
+```
+
+## Available Workers
+
+### 1. STT Transcriber Worker
+Basic speech-to-text transcription.
+
+```ruby
+result = engine.call_worker(:stt_transcriber, {
+  audio_file: "./audio.wav",
+  language: "zh",              # Optional: "zh", "en", "ja", "ko"
+  prompt: "专业术语",           # Optional: Context prompt
+  temperature: 0.0,            # Optional: 0.0 to 1.0
+  response_format: "json"      # Optional: "json", "text", "srt", "vtt"
+})
+```
+
+### 2. STT URL Transcriber Worker
+Transcribe audio from URL.
+
+```ruby
+result = engine.call_worker(:stt_url_transcriber, {
+  audio_url: "https://example.com/audio.wav",
+  language: "en",
+  response_format: "text"
+})
+```
+
+### 3. Batch STT Worker
+Process multiple audio files.
+
+```ruby
+result = engine.call_worker(:batch_stt, {
+  audio_files: [
+    "./audio1.wav",
+    "./audio2.mp3",
+    "./audio3.webm"
+  ],
+  language: "zh",
+  temperature: 0.0
+})
+```
+
+### 4. Audio Info Worker
+Get audio file information.
+
+```ruby
+result = engine.call_worker(:audio_info, {
+  audio_file: "./audio.wav"
+})
+```
+
+### 5. Language Detector Worker
+Detect language from audio or text.
+
+```ruby
+# From audio file
+result = engine.call_worker(:language_detector, {
+  audio_file: "./audio.wav"
+})
+
+# From text
+result = engine.call_worker(:language_detector, {
+  text: "这是一个中文文本"
+})
+```
+
+### 6. Multi-language STT Worker
+Automatic language detection and transcription.
+
+```ruby
+result = engine.call_worker(:multilingual_stt, {
+  audio_file: "./audio.wav"
+})
+```
+
+### 7. STT Format Converter Worker
+Generate transcriptions in multiple formats.
+
+```ruby
+result = engine.call_worker(:stt_format_converter, {
+  audio_file: "./audio.wav",
+  formats: ["json", "text", "srt", "vtt"]
+})
+```
+
+## Direct Adapter Usage
+
+You can also use the adapter directly without workers:
+
+```ruby
+# Get the adapter
+adapter = engine.llms["stt_service"]
+
+# Transcribe audio file
+transcription_data = adapter.transcribe_audio(
+  "./audio.wav",
+  language: "zh",
+  temperature: 0.0,
+  response_format: "json"
+)
+
+# Transcribe from URL
+transcription_data = adapter.transcribe_audio_url(
+  "https://example.com/audio.wav",
+  language: "en",
+  response_format: "text"
+)
+
+# Batch transcription
+batch_result = adapter.transcribe_batch(
+  ["./audio1.wav", "./audio2.mp3"],
+  language: "zh"
+)
+
+# Get audio information
+audio_info = adapter.get_audio_info("./audio.wav")
+
+# Detect language
+detected_language = adapter.detect_language("这是一个中文文本")
+```
+
+## Response Formats
+
+### Transcription Response
+```ruby
+{
+  text: "转录的文本内容",           # Transcribed text
+  language: "zh",                  # Language used
+  duration: 120,                   # Audio duration in seconds
+  file_size: 1024000,              # File size in bytes
+  format: "wav"                    # Audio format
+}
+```
+
+### Batch Response
+```ruby
+{
+  total_files: 3,                  # Total files processed
+  successful: 2,                   # Successful transcriptions
+  failed: 1,                       # Failed transcriptions
+  results: [                       # Individual results
+    {
+      file: "./audio1.wav",
+      index: 0,
+      transcription: { ... },
+      success: true
+    },
+    {
+      file: "./audio2.wav",
+      index: 1,
+      error: "File not found",
+      success: false
+    }
+  ]
+}
+```
+
+### Audio Information Response
+```ruby
+{
+  file_path: "./audio.wav",        # File path
+  file_name: "audio.wav",          # File name
+  file_size: 1024000,              # File size in bytes
+  format: "wav",                   # Audio format
+  estimated_duration: 120,         # Estimated duration in seconds
+  supported: true                  # Whether format is supported
+}
+```
+
+## Supported Models
+
+SiliconFlow supports various STT models:
+- `FunAudioLLM/CosyVoice2-0.5B` - Multi-language speech recognition
+- `fnlp/MOSS-TTSD-v0.5` - High accuracy speech recognition
+
+## Supported Audio Formats
+
+- `mp3` - MP3 format
+- `mp4` - MP4 format
+- `mpeg` - MPEG format
+- `mpga` - MPGA format
+- `m4a` - M4A format
+- `wav` - WAV format
+- `webm` - WebM format
+
+## Language Support
+
+- `zh` - Chinese
+- `en` - English
+- `ja` - Japanese
+- `ko` - Korean
+
+## Response Formats
+
+- `json` - JSON format (default)
+- `text` - Plain text format
+- `srt` - SubRip subtitle format
+- `vtt` - WebVTT subtitle format
+
+## File Size Limits
+
+- **Maximum file size**: 25MB
+- **Recommended duration**: Under 30 minutes
+- **Bitrate**: Standard audio bitrates
+
+## Error Handling
+
+```ruby
+begin
+  result = engine.call_worker(:stt_transcriber, params)
+rescue SmartPrompt::LLMAPIError => e
+  puts "API Error: #{e.message}"
+rescue SmartPrompt::Error => e
+  puts "General Error: #{e.message}"
+rescue => e
+  puts "Unexpected Error: #{e.message}"
+end
+```
+
+## Best Practices
+
+1. **Audio Quality**: Use clear audio with minimal background noise
+2. **Language Specification**: Specify language for better accuracy
+3. **File Size**: Keep files under 25MB for optimal performance
+4. **Batch Processing**: Use batch workers for multiple files
+5. **Format Selection**: Choose appropriate response format for your use case
+6. **Temperature**: Use lower temperature (0.0-0.2) for more accurate transcriptions
+
+## Example
+
+See `examples/stt_example.rb` for complete working examples.
+
+## Troubleshooting
+
+**Common Issues:**
+- **API Key Error**: Ensure `SILICONFLOW_API_KEY` environment variable is set
+- **File Not Found**: Check file path and permissions
+- **Unsupported Format**: Use only supported audio formats
+- **File Too Large**: Maximum file size is 25MB
+- **Network Error**: Check internet connection and API endpoint
+
+**Error Messages:**
+- `Audio file not found` - Check file path
+- `Unsupported audio format` - Use supported formats only
+- `Audio file too large` - Reduce file size to under 25MB
+- `Unsupported response format` - Use only supported response formats
+- `Network error: Unable to connect to STT API` - Check network connectivity
+
+## Performance Tips
+
+1. **Preprocessing**: Normalize audio levels before transcription
+2. **Language Detection**: Use automatic detection for mixed-language content
+3. **Batch Processing**: Process multiple files together for efficiency
+4. **Format Selection**: Use JSON for structured data, text for simple output
+5. **Error Recovery**: Implement retry logic for network failures

data/docs/TTS_README.md

@@ -0,0 +1,303 @@
+# SmartPrompt TTS Guide
+
+This guide explains how to use the new Text-to-Speech (TTS) capabilities in SmartPrompt.
+
+## Overview
+
+The TTS feature adds support for:
+- **Text-to-Speech Synthesis**: Convert text to natural-sounding speech
+- **Multi-language Support**: Chinese, English, Japanese, Korean
+- **Voice Selection**: Multiple predefined voices and custom voices
+- **Speed Control**: Adjust speech speed from 0.25x to 4.0x
+- **Multiple Formats**: MP3, WAV, Opus, PCM output formats
+- **Custom Voices**: Create and manage custom voices from reference audio
+
+## Installation
+
+Make sure you have the required dependencies:
+
+```bash
+gem install openai
+```
+
+## Configuration
+
+Add the TTS adapter to your configuration:
+
+```yaml
+# config.yml
+adapters:
+  multimodal: "MultimodalAdapter"
+  image_generation: "ImageGenerationAdapter"
+  video_generation: "VideoGenerationAdapter"
+  tts: "TTSAdapter"
+
+llms:
+  tts_service:
+    adapter: "tts"
+    url: "https://api.siliconflow.cn/v1/"
+    api_key: "ENV[SILICONFLOW_API_KEY]"
+    model: "FunAudioLLM/CosyVoice2-0.5B"
+
+default_llm: "tts_service"
+template_path: "./templates"
+worker_path: "./workers"
+logger_file: "./logs/smart_prompt.log"
+```
+
+## Available Workers
+
+### 1. TTS Synthesizer Worker
+Basic text-to-speech synthesis.
+
+```ruby
+result = engine.call_worker(:tts_synthesizer, {
+  text: "欢迎使用智能提示系统",
+  voice: "alloy",              # Optional: "alloy", "echo", "fable", "onyx", "nova", "shimmer"
+  speed: 1.0,                  # Optional: 0.25 to 4.0
+  response_format: "mp3",      # Optional: "mp3", "wav", "opus", "pcm"
+  language: "zh",              # Optional: "zh", "en", "ja", "ko"
+  save_to_file: true,          # Optional: Save audio to file
+  output_dir: "./audio",       # Optional: Output directory
+  filename_prefix: "tts"       # Optional: Filename prefix
+})
+```
+
+### 2. Multilingual TTS Worker
+Automatic language detection and synthesis.
+
+```ruby
+result = engine.call_worker(:multilingual_tts, {
+  text: "Hello, this is a demonstration",
+  voice: "echo",
+  save_to_file: true
+})
+```
+
+### 3. Voice Selector Worker
+List available voices and test different voices.
+
+```ruby
+result = engine.call_worker(:voice_selector, {
+  text: "测试不同音色的效果",
+  voice: "nova",
+  save_to_file: true
+})
+```
+
+### 4. Speed Variation Worker
+Generate audio at different speeds.
+
+```ruby
+result = engine.call_worker(:speed_variation_tts, {
+  text: "这是一个语速变化的演示",
+  speeds: [0.5, 0.75, 1.0, 1.5, 2.0],
+  save_to_file: true
+})
+```
+
+### 5. Custom Voice Manager Worker
+Manage custom voices (create, list, delete, synthesize).
+
+```ruby
+# List voices
+result = engine.call_worker(:custom_voice_manager, {
+  action: "list"
+})
+
+# Create custom voice
+result = engine.call_worker(:custom_voice_manager, {
+  action: "create",
+  name: "my_voice",
+  reference_audio_file: "./reference.wav",
+  description: "My custom voice"
+})
+
+# Delete custom voice
+result = engine.call_worker(:custom_voice_manager, {
+  action: "delete",
+  voice_id: "voice_123"
+})
+
+# Synthesize with custom voice
+result = engine.call_worker(:custom_voice_manager, {
+  action: "synthesize",
+  voice_id: "voice_123",
+  text: "使用自定义音色朗读",
+  save_to_file: true
+})
+```
+
+### 6. Batch TTS Worker
+Process multiple texts in batch.
+
+```ruby
+result = engine.call_worker(:batch_tts, {
+  texts: [
+    "第一条文本",
+    "第二条文本",
+    "第三条文本"
+  ],
+  voice: "alloy",
+  save_to_file: true
+})
+```
+
+## Direct Adapter Usage
+
+You can also use the adapter directly without workers:
+
+```ruby
+# Get the adapter
+adapter = engine.llms["tts_service"]
+
+# Synthesize speech
+audio_data = adapter.synthesize_speech(
+  "这是一个直接合成的演示",
+  voice: "echo",
+  speed: 1.2,
+  response_format: "mp3"
+)
+
+# Synthesize and save to file
+result = adapter.synthesize_to_file(
+  "保存到文件的演示",
+  "./audio/demo.mp3",
+  voice: "nova",
+  speed: 1.0
+)
+
+# Get available voices
+voices = adapter.available_voices
+
+# Create custom voice
+voice_data = adapter.create_custom_voice(
+  "my_voice",
+  "./reference.wav",
+  description: "My custom voice"
+)
+
+# List custom voices
+custom_voices = adapter.list_custom_voices
+
+# Delete custom voice
+result = adapter.delete_custom_voice("voice_123")
+```
+
+## Response Formats
+
+### Audio Data Response
+```ruby
+{
+  audio_data: "data:audio/mp3;base64,...",  # Base64 encoded audio
+  format: "mp3",                           # Audio format
+  text_length: 25,                         # Input text length
+  voice: "alloy"                           # Voice used
+}
+```
+
+### File Response
+```ruby
+{
+  file_path: "./audio/demo.mp3",           # Saved file path
+  text_length: 25,                         # Input text length
+  voice: "alloy",                          # Voice used
+  format: "mp3"                            # Audio format
+}
+```
+
+### Voice Management Response
+```ruby
+{
+  voice_id: "voice_123",                   # Voice identifier
+  name: "my_voice",                        # Voice name
+  status: "active",                        # Voice status
+  created_at: "2024-01-01..."              # Creation timestamp
+}
+```
+
+## Supported Models
+
+SiliconFlow supports various TTS models:
+- `FunAudioLLM/CosyVoice2-0.5B` - Multi-language support with emotion control
+- `fnlp/MOSS-TTSD-v0.5` - High expressiveness, dual voice cloning
+
+## Predefined Voices
+
+- `alloy` - 沉稳男声alex
+- `echo` - 温柔女声claire
+- `fable` - 活泼女声fable
+- `onyx` - 磁性男声onyx
+- `nova` - 甜美女声nova
+- `shimmer` - 优雅女声shimmer
+
+## Language Support
+
+- `zh` - Chinese
+- `en` - English
+- `ja` - Japanese
+- `ko` - Korean
+
+## Audio Formats
+
+- `mp3` - MP3 format (default)
+- `wav` - WAV format
+- `opus` - Opus format
+- `pcm` - PCM format
+
+## Speed Control
+
+- **Range**: 0.25 to 4.0
+- **Default**: 1.0 (normal speed)
+- **Slow**: 0.25 - 0.75
+- **Fast**: 1.25 - 4.0
+
+## Custom Voice Requirements
+
+- **Reference Audio**: 8-10 seconds recommended
+- **Audio Quality**: Clear speech, no background noise
+- **File Size**: Maximum 5MB
+- **Formats**: Common audio formats supported
+
+## Error Handling
+
+```ruby
+begin
+  result = engine.call_worker(:tts_synthesizer, params)
+rescue SmartPrompt::LLMAPIError => e
+  puts "API Error: #{e.message}"
+rescue SmartPrompt::Error => e
+  puts "General Error: #{e.message}"
+rescue => e
+  puts "Unexpected Error: #{e.message}"
+end
+```
+
+## Best Practices
+
+1. **Text Preparation**: Remove unnecessary spaces, use proper punctuation
+2. **Language Selection**: Specify language for better pronunciation
+3. **Speed Adjustment**: Use 0.8-1.2 for natural speech
+4. **Voice Selection**: Test different voices for your use case
+5. **Batch Processing**: Use batch workers for multiple texts
+6. **File Management**: Use `save_to_file: true` for persistent storage
+
+## Example
+
+See `examples/tts_example.rb` for complete working examples.
+
+## Troubleshooting
+
+**Common Issues:**
+- **API Key Error**: Ensure `SILICONFLOW_API_KEY` environment variable is set
+- **Text Too Long**: Maximum 4096 characters per request
+- **Invalid Voice**: Use only predefined voice names or valid custom voice IDs
+- **Speed Out of Range**: Speed must be between 0.25 and 4.0
+- **File Permissions**: Ensure write permissions for output directories
+- **Reference Audio**: For custom voices, use clear 8-10 second audio files
+
+**Error Messages:**
+- `Text cannot be empty` - Provide non-empty text
+- `Text too long` - Reduce text length to under 4096 characters
+- `Unsupported response format` - Use only supported audio formats
+- `Reference audio file not found` - Check file path for custom voice creation