lipdub 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,955 @@
1
+ # Lipdub Ruby Client
2
+
3
+ A comprehensive Ruby client library for the [Lipdub.ai API](https://lipdub.ai), providing easy access to AI-powered lip-dubbing functionality.
4
+
5
+ [![Gem Version](https://badge.fury.io/rb/lipdub.svg)](https://badge.fury.io/rb/lipdub)
6
+ [![Build Status](https://github.com/upriser/lipdub-ruby/workflows/CI/badge.svg)](https://github.com/upriser/lipdub-ruby/actions)
7
+ [![Security](https://img.shields.io/badge/security-bundler--audit-blue.svg)](https://github.com/rubysec/bundler-audit)
8
+
9
+ ## Table of Contents
10
+
11
+ - [Features](#features)
12
+ - [Installation](#installation)
13
+ - [Configuration](#configuration)
14
+ - [Quick Start](#quick-start)
15
+ - [API Reference](#api-reference)
16
+ - [Videos](#videos)
17
+ - [Audios](#audios)
18
+ - [Shots](#shots)
19
+ - [Projects](#projects)
20
+ - [Usage Examples](#usage-examples)
21
+ - [Video Upload](#video-upload)
22
+ - [Audio Upload](#audio-upload)
23
+ - [Shot Management](#shot-management)
24
+ - [Shot Generation](#shot-generation)
25
+ - [Project Management](#project-management)
26
+ - [Complete Workflow Examples](#complete-workflow-examples)
27
+ - [Basic Lip-dubbing Workflow](#basic-lip-dubbing-workflow)
28
+ - [Selective Lip-dubbing Workflow](#selective-lip-dubbing-workflow)
29
+ - [Supported File Formats](#supported-file-formats)
30
+ - [Error Handling](#error-handling)
31
+ - [Rate Limits and Best Practices](#rate-limits-and-best-practices)
32
+ - [Troubleshooting](#troubleshooting)
33
+ - [Development](#development)
34
+ - [Contributing](#contributing)
35
+ - [License](#license)
36
+
37
+ ## Features
38
+
39
+ - **Video Upload**: Upload and process videos for lip-dubbing
40
+ - **Audio Upload**: Upload audio files for voice replacement
41
+ - **Generation**: Generate lip-dubbed videos with AI
42
+ - **Status Monitoring**: Track processing and generation progress
43
+ - **File Management**: Handle file uploads and downloads seamlessly
44
+ - **Error Handling**: Comprehensive error handling with custom exceptions
45
+ - **Test Coverage**: Full test suite with RSpec
46
+
47
+ ## Installation
48
+
49
+ Add this line to your application's Gemfile:
50
+
51
+ ```ruby
52
+ gem 'lipdub'
53
+ ```
54
+
55
+ And then execute:
56
+
57
+ $ bundle install
58
+
59
+ Or install it yourself as:
60
+
61
+ $ gem install lipdub
62
+
63
+ ## Configuration
64
+
65
+ Configure the client with your API key:
66
+
67
+ ```ruby
68
+ require 'lipdub'
69
+
70
+ # Global configuration
71
+ Lipdub.configure do |config|
72
+ config.api_key = "your_api_key_here"
73
+ config.base_url = "https://api.lipdub.ai" # Optional, this is the default
74
+ config.timeout = 30 # Optional, default is 30 seconds
75
+ config.open_timeout = 10 # Optional, default is 10 seconds
76
+ end
77
+
78
+ # Or create a client with specific configuration
79
+ config = Lipdub::Configuration.new
80
+ config.api_key = "your_api_key_here"
81
+ client = Lipdub::Client.new(config)
82
+ ```
83
+
84
+ ## Quick Start
85
+
86
+ Here's a minimal example to get you started with lip-dubbing:
87
+
88
+ ```ruby
89
+ require 'lipdub'
90
+
91
+ # Configure the client
92
+ Lipdub.configure do |config|
93
+ config.api_key = "your_api_key_here"
94
+ end
95
+
96
+ client = Lipdub.client
97
+
98
+ # Upload video and audio, then generate lip-dubbed video
99
+ video_response = client.videos.upload_complete("input/video.mp4")
100
+ shot_id = video_response.dig("data", "shot_id")
101
+
102
+ audio_response = client.audios.upload_complete("input/audio.mp3")
103
+ audio_id = audio_response.dig("data", "audio_id")
104
+
105
+ # Generate and download the result
106
+ result = client.shots.generate_and_wait(
107
+ shot_id: shot_id,
108
+ audio_id: audio_id,
109
+ output_filename: "dubbed_video.mp4"
110
+ )
111
+
112
+ generate_id = result.dig("data", "generate_id")
113
+ client.shots.download_file(shot_id, generate_id, "output/result.mp4")
114
+ ```
115
+
116
+ ## API Reference
117
+
118
+ The Lipdub Ruby client provides four main resource classes for interacting with the API:
119
+
120
+ ### Videos
121
+
122
+ The Videos resource handles video file uploads and processing.
123
+
124
+ #### Methods
125
+
126
+ | Method | Description | Parameters | Returns |
127
+ |--------|-------------|------------|---------|
128
+ | `upload(size_bytes:, file_name:, content_type:, video_source_url: nil)` | Initiate video upload process | `size_bytes` (Integer), `file_name` (String), `content_type` (String), `video_source_url` (String, optional) | Hash with `video_id`, `upload_url`, `success_url`, `failure_url` |
129
+ | `upload_file(upload_url, file_content, content_type)` | Upload video file to provided URL | `upload_url` (String), `file_content` (String/IO), `content_type` (String) | Hash |
130
+ | `upload_complete(file_path, content_type: nil)` | Complete upload workflow from file path | `file_path` (String), `content_type` (String, optional) | Hash with `shot_id` and `asset_type` |
131
+ | `success(video_id)` | Mark video upload as successful | `video_id` (String) | Hash with `shot_id` and `asset_type` |
132
+ | `failure(video_id)` | Mark video upload as failed | `video_id` (String) | Hash |
133
+ | `status(video_id)` | Get video processing status | `video_id` (String) | Hash with status information |
134
+
135
+ #### Endpoints
136
+
137
+ - `POST /v1/video` - Initiate video upload
138
+ - `POST /v1/video/success/{video_id}` - Mark upload successful
139
+ - `POST /v1/video/failure/{video_id}` - Mark upload failed
140
+ - `GET /v1/video/status/{video_id}` - Get processing status
141
+
142
+ ### Audios
143
+
144
+ The Audios resource handles audio file uploads and management.
145
+
146
+ #### Methods
147
+
148
+ | Method | Description | Parameters | Returns |
149
+ |--------|-------------|------------|---------|
150
+ | `upload(size_bytes:, file_name:, content_type:, audio_source_url: nil)` | Initiate audio upload process | `size_bytes` (Integer, 1-104857600), `file_name` (String), `content_type` (String), `audio_source_url` (String, optional) | Hash with `audio_id`, `upload_url`, `success_url`, `failure_url` |
151
+ | `upload_file(upload_url, file_content, content_type)` | Upload audio file to provided URL | `upload_url` (String), `file_content` (String/IO), `content_type` (String) | Hash |
152
+ | `upload_complete(file_path, content_type: nil)` | Complete upload workflow from file path | `file_path` (String), `content_type` (String, optional) | Hash with upload result |
153
+ | `success(audio_id)` | Mark audio upload as successful | `audio_id` (String) | Hash |
154
+ | `failure(audio_id)` | Mark audio upload as failed | `audio_id` (String) | Hash |
155
+ | `status(audio_id)` | Get audio processing status | `audio_id` (String) | Hash with status information |
156
+ | `list(page: 1, page_size: 10)` | List all audio files | `page` (Integer), `page_size` (Integer) | Hash with audio list and pagination |
157
+
158
+ #### Endpoints
159
+
160
+ - `POST /v1/audio` - Initiate audio upload
161
+ - `POST /v1/audio/success/{audio_id}` - Mark upload successful
162
+ - `POST /v1/audio/failure/{audio_id}` - Mark upload failed
163
+ - `GET /v1/audio/status/{audio_id}` - Get processing status
164
+ - `GET /v1/audio` - List audio files
165
+
166
+ #### Supported Audio Formats
167
+
168
+ - `audio/mpeg` (MP3)
169
+ - `audio/wav` (WAV)
170
+ - `audio/mp4` (MP4/M4A)
171
+
172
+ ### Shots
173
+
174
+ The Shots resource handles lip-dubbing generation, translation, and shot management.
175
+
176
+ #### Methods
177
+
178
+ | Method | Description | Parameters | Returns |
179
+ |--------|-------------|------------|---------|
180
+ | `list(page: 1, per_page: 20)` | List available shots | `page` (Integer), `per_page` (Integer, max 100) | Hash with shots list and count |
181
+ | `status(shot_id)` | Get shot processing status | `shot_id` (String/Integer) | Hash with status information |
182
+ | `generate(shot_id:, audio_id:, output_filename:, **options)` | Generate lip-dubbed video | See generation options below | Hash with `generate_id` |
183
+ | `generation_status(shot_id, generate_id)` | Get generation progress | `shot_id` (String/Integer), `generate_id` (String) | Hash with progress and status |
184
+ | `download(shot_id, generate_id)` | Get download URL for generated video | `shot_id` (String/Integer), `generate_id` (String) | Hash with `download_url` |
185
+ | `download_file(shot_id, generate_id, file_path)` | Download generated video to local path | `shot_id` (String/Integer), `generate_id` (String), `file_path` (String) | String (file path) |
186
+ | `generate_and_wait(shot_id:, audio_id:, output_filename:, **options)` | Generate and wait for completion | Same as generate + `polling_interval` (Integer), `max_wait_time` (Integer) | Hash with final status |
187
+ | `actors(shot_id)` | Get actors for a shot | `shot_id` (String/Integer) | Hash with actors information |
188
+ | `translate(shot_id:, source_language:, target_language:, full_resolution: nil)` | Translate a shot | `shot_id` (String/Integer), `source_language` (String), `target_language` (String), `full_resolution` (Boolean, optional) | Hash with translation details |
189
+ | `generate_multi_actor(shot_id:, **params)` | Generate multi-actor lip-dub | `shot_id` (String/Integer), `params` (Hash) | Hash with generation details |
190
+
191
+ #### Generation Options
192
+
193
+ | Parameter | Type | Description | Default |
194
+ |-----------|------|-------------|---------|
195
+ | `shot_id` | String/Integer | **Required.** Unique identifier of the shot | - |
196
+ | `audio_id` | String | **Required.** Unique identifier of the audio file | - |
197
+ | `output_filename` | String | **Required.** Name for the output file | - |
198
+ | `language` | String | Language specification (ISO 639-1) | `nil` |
199
+ | `start_frame` | Integer | Frame number to start lip-sync from | `0` |
200
+ | `loop_video` | Boolean | Whether to loop video during rendering | `false` |
201
+ | `full_resolution` | Boolean | Whether to use full resolution | `true` |
202
+ | `callback_url` | String | HTTPS URL for completion callback | `nil` |
203
+ | `timecode_ranges` | Array | List of `[start, end]` timecode pairs for selective lip-dubbing | `nil` |
204
+
205
+ #### Timecode Ranges for Selective Lip-dubbing
206
+
207
+ Timecode ranges allow you to lip-dub only specific parts of a video:
208
+
209
+ ```ruby
210
+ # Using seconds (float)
211
+ timecode_ranges: [[2.5, 4.2], [10.0, 12.5]]
212
+
213
+ # Using SMPTE format (HH:MM:SS:FF)
214
+ timecode_ranges: [["00:00:02:15", "00:00:04:06"], ["00:00:10:00", "00:00:12:15"]]
215
+ ```
216
+
217
+ #### Helper Methods
218
+
219
+ | Method | Description | Parameters | Returns |
220
+ |--------|-------------|------------|---------|
221
+ | `validate_timecode_ranges(ranges, video_duration: nil)` | Validate timecode ranges | `ranges` (Array), `video_duration` (Numeric, optional) | Boolean or raises ArgumentError |
222
+ | `add_frame_buffer(ranges, buffer_frames: 10, fps: 30, video_duration: nil)` | Add frame buffer to ranges | `ranges` (Array), `buffer_frames` (Integer), `fps` (Integer), `video_duration` (Numeric, optional) | Array of buffered ranges |
223
+ | `parse_timecode_to_seconds(timecode, fps: 30)` | Convert timecode to seconds | `timecode` (String/Numeric), `fps` (Integer) | Float |
224
+
225
+ #### Endpoints
226
+
227
+ - `GET /v1/shots` - List shots
228
+ - `GET /v1/shots/{shot_id}/status` - Get shot status
229
+ - `POST /v1/shots/{shot_id}/generate` - Generate lip-dubbed video
230
+ - `GET /v1/shots/{shot_id}/generate/{generate_id}` - Get generation status
231
+ - `GET /v1/shots/{shot_id}/generate/{generate_id}/download` - Get download URL
232
+ - `GET /v1/shots/{shot_id}/actors` - Get shot actors
233
+ - `POST /v1/shots/{shot_id}/translate` - Translate shot
234
+ - `POST /v1/shots/{shot_id}/generate-multi-actor` - Multi-actor generation
235
+
236
+ ### Projects
237
+
238
+ The Projects resource handles project management and listing.
239
+
240
+ #### Methods
241
+
242
+ | Method | Description | Parameters | Returns |
243
+ |--------|-------------|------------|---------|
244
+ | `list(page: 1, per_page: 20)` | List all projects | `page` (Integer), `per_page` (Integer, max 100) | Hash with projects list and count |
245
+
246
+ #### Endpoints
247
+
248
+ - `GET /v1/projects` - List projects
249
+
250
+ ## Usage Examples
251
+
252
+ ### Video Upload
253
+
254
+ #### Simple Video Upload
255
+
256
+ ```ruby
257
+ client = Lipdub.client
258
+
259
+ # Upload a video file (handles the entire workflow)
260
+ response = client.videos.upload_complete("path/to/your/video.mp4")
261
+ # => {
262
+ # "data" => {
263
+ # "shot_id" => 123,
264
+ # "asset_type" => "dubbing-video"
265
+ # }
266
+ # }
267
+
268
+ shot_id = response.dig("data", "shot_id")
269
+ ```
270
+
271
+ #### Manual Video Upload Process
272
+
273
+ ```ruby
274
+ # Step 1: Initiate upload
275
+ upload_response = client.videos.upload(
276
+ size_bytes: File.size("video.mp4"),
277
+ file_name: "my_video.mp4",
278
+ content_type: "video/mp4"
279
+ )
280
+
281
+ video_id = upload_response.dig("data", "video_id")
282
+ upload_url = upload_response.dig("data", "upload_url")
283
+
284
+ # Step 2: Upload the file
285
+ file_content = File.read("video.mp4")
286
+ client.videos.upload_file(upload_url, file_content, "video/mp4")
287
+
288
+ # Step 3: Mark as successful
289
+ success_response = client.videos.success(video_id)
290
+ shot_id = success_response.dig("data", "shot_id")
291
+ ```
292
+
293
+ #### Check Video Status
294
+
295
+ ```ruby
296
+ status = client.videos.status(video_id)
297
+ # => {
298
+ # "data" => {
299
+ # "status" => "processing",
300
+ # "progress" => 50
301
+ # }
302
+ # }
303
+ ```
304
+
305
+ ### Audio Upload
306
+
307
+ #### Simple Audio Upload
308
+
309
+ ```ruby
310
+ # Upload an audio file (handles the entire workflow)
311
+ response = client.audios.upload_complete("path/to/your/audio.mp3")
312
+
313
+ # Get the audio_id for generation
314
+ audio_id = response.dig("data", "audio_id")
315
+ ```
316
+
317
+ #### Manual Audio Upload Process
318
+
319
+ ```ruby
320
+ # Step 1: Initiate upload
321
+ upload_response = client.audios.upload(
322
+ size_bytes: File.size("audio.mp3"),
323
+ file_name: "voiceover.mp3",
324
+ content_type: "audio/mpeg"
325
+ )
326
+
327
+ audio_id = upload_response.dig("data", "audio_id")
328
+ upload_url = upload_response.dig("data", "upload_url")
329
+
330
+ # Step 2: Upload the file
331
+ file_content = File.read("audio.mp3")
332
+ client.audios.upload_file(upload_url, file_content, "audio/mpeg")
333
+
334
+ # Step 3: Mark as successful
335
+ client.audios.success(audio_id)
336
+ ```
337
+
338
+ #### List Audio Files
339
+
340
+ ```ruby
341
+ # List all audio files
342
+ audios = client.audios.list(page: 1, page_size: 20)
343
+ # => {
344
+ # "data" => [
345
+ # { "audio_id" => "audio_1", "file_name" => "voice1.mp3" },
346
+ # { "audio_id" => "audio_2", "file_name" => "voice2.wav" }
347
+ # ],
348
+ # "pagination" => { "page" => 1, "page_size" => 20, "total" => 50 }
349
+ # }
350
+ ```
351
+
352
+ ### Shot Management
353
+
354
+ #### List Available Shots
355
+
356
+ ```ruby
357
+ # List all shots
358
+ shots = client.shots.list(page: 1, per_page: 50)
359
+ # => {
360
+ # "data" => [
361
+ # {
362
+ # "shot_id" => 99,
363
+ # "shot_label" => "api-full-test-new.mp4",
364
+ # "shot_project_id" => 37,
365
+ # "shot_scene_id" => 37,
366
+ # "shot_project_name" => "Lee Studios",
367
+ # "shot_scene_name" => "Under the tent"
368
+ # }
369
+ # ],
370
+ # "count" => 1
371
+ # }
372
+ ```
373
+
374
+ #### Get Shot Actors
375
+
376
+ ```ruby
377
+ # Get actors for a specific shot
378
+ actors = client.shots.actors(shot_id)
379
+ ```
380
+
381
+ ### Shot Generation
382
+
383
+ #### Generate Lip-Dubbed Video
384
+
385
+ ```ruby
386
+ # Start generation with all available options
387
+ generate_response = client.shots.generate(
388
+ shot_id: shot_id,
389
+ audio_id: audio_id,
390
+ output_filename: "final_dubbed_video.mp4",
391
+ language: "en-US", # Optional (ISO 639-1)
392
+ start_frame: 0, # Optional
393
+ loop_video: false, # Optional
394
+ full_resolution: true, # Optional
395
+ callback_url: "https://example.com/webhook", # Optional
396
+ timecode_ranges: [[0, 100], [200, 300]] # Optional
397
+ )
398
+
399
+ generate_id = generate_response["generate_id"]
400
+ ```
401
+
402
+ #### Selective Lip-dubbing for Single Actors
403
+
404
+ For scenarios where you only want to lip-dub specific parts of a video (e.g., personalization where only a name needs to be replaced), you can use selective lip-dubbing with `timecode_ranges`:
405
+
406
+ ```ruby
407
+ # Basic selective lip-dubbing with time ranges in seconds
408
+ response = client.shots.generate(
409
+ shot_id: 123,
410
+ audio_id: "audio_abc123",
411
+ output_filename: "personalized_video.mp4",
412
+ timecode_ranges: [[0, 10], [20, 30]] # Replace seconds 0-10 and 20-30
413
+ )
414
+
415
+ # With SMPTE timecode format (be consistent with format)
416
+ response = client.shots.generate(
417
+ shot_id: 123,
418
+ audio_id: "audio_abc123",
419
+ output_filename: "personalized_video.mp4",
420
+ timecode_ranges: [["00:00:00:00", "00:00:10:00"], ["00:00:20:00", "00:00:30:00"]]
421
+ )
422
+
423
+ # Example: Replace a name greeting with proper buffering
424
+ # Calculate 10-frame buffer (assuming 30fps: 10/30 = 0.33 seconds)
425
+ name_start = 2.5 - 0.33 # Start 10 frames before
426
+ name_end = 4.2 + 0.33 # End 10 frames after
427
+
428
+ response = client.shots.generate(
429
+ shot_id: 123,
430
+ audio_id: "audio_with_new_name",
431
+ output_filename: "personalized_greeting.mp4",
432
+ timecode_ranges: [[name_start, name_end]],
433
+ language: "en-US"
434
+ )
435
+ ```
436
+
437
+ ##### Best Practices for Selective Lip-dubbing
438
+
439
+ 1. **Match Original Region Length**: Ensure replaced audio regions match the original region length to maintain sync
440
+ 2. **Add Frame Buffer**: Include a 10-frame buffer around start/end timecodes for seamless blending
441
+ 3. **Normalize Audio**: Normalize audio levels and isolate vocals from background noise for best results
442
+ 4. **Audio Duration**: The total audio duration must match the video duration
443
+ 5. **Consistent Timecode Format**: Use either seconds (float) or SMPTE format consistently
444
+ 6. **Non-overlapping Ranges**: Ensure timecode ranges don't overlap each other
445
+
446
+ #### Translation
447
+
448
+ ```ruby
449
+ # Translate a shot to different language
450
+ translation = client.shots.translate(
451
+ shot_id: shot_id,
452
+ source_language: "English",
453
+ target_language: "Spanish",
454
+ full_resolution: true # Optional
455
+ )
456
+ ```
457
+
458
+ #### Multi-Actor Generation
459
+
460
+ ```ruby
461
+ # Generate with multiple actors
462
+ multi_result = client.shots.generate_multi_actor(
463
+ shot_id: shot_id,
464
+ actors: [
465
+ { actor_id: 1, voice_id: "voice_1" },
466
+ { actor_id: 2, voice_id: "voice_2" }
467
+ ]
468
+ )
469
+ ```
470
+
471
+ #### Monitor Generation Progress
472
+
473
+ ```ruby
474
+ # Check generation status
475
+ status = client.shots.generation_status(shot_id, generate_id)
476
+ # => {
477
+ # "data" => {
478
+ # "generate_id" => "gen_789",
479
+ # "status" => "processing",
480
+ # "progress" => 75
481
+ # }
482
+ # }
483
+ ```
484
+
485
+ #### Generate and Wait for Completion
486
+
487
+ ```ruby
488
+ # Generate and automatically wait for completion
489
+ result = client.shots.generate_and_wait(
490
+ shot_id: shot_id,
491
+ audio_id: audio_id,
492
+ output_filename: "dubbed_video.mp4",
493
+ polling_interval: 10, # Check every 10 seconds
494
+ max_wait_time: 1800 # Wait up to 30 minutes
495
+ )
496
+ # => Returns when generation is complete or raises an error
497
+ ```
498
+
499
+ #### Download Generated Video
500
+
501
+ ```ruby
502
+ # Get download URL
503
+ download_info = client.shots.download(shot_id, generate_id)
504
+ download_url = download_info.dig("data", "download_url")
505
+
506
+ # Or download directly to a file
507
+ local_path = client.shots.download_file(
508
+ shot_id,
509
+ generate_id,
510
+ "output/my_dubbed_video.mp4"
511
+ )
512
+ ```
513
+
514
+ ### Project Management
515
+
516
+ #### List Projects
517
+
518
+ ```ruby
519
+ # List all projects
520
+ projects = client.projects.list(page: 1, per_page: 20)
521
+ # => {
522
+ # "data" => [
523
+ # {
524
+ # "project_id" => 123,
525
+ # "projects_tenant_id" => 1,
526
+ # "projects_user_id" => 47,
527
+ # "project_name" => "My Sample Project",
528
+ # "user_email" => "user@example.com",
529
+ # "created_at" => "2024-01-15T10:30:00Z",
530
+ # "updated_at" => nil,
531
+ # "source_language" => {
532
+ # "language_id" => 1,
533
+ # "name" => "English",
534
+ # "supported" => true
535
+ # },
536
+ # "project_identity_type" => "single_identity",
537
+ # "language_project_links" => []
538
+ # }
539
+ # ],
540
+ # "count" => 1
541
+ # }
542
+ ```
543
+
544
+ ### Complete Workflow Examples
545
+
546
+ #### Basic Lip-dubbing Workflow
547
+
548
+ Here's a complete example that uploads a video and audio, generates a lip-dubbed video, and downloads the result:
549
+
550
+ ```ruby
551
+ require 'lipdub'
552
+
553
+ # Configure the client
554
+ Lipdub.configure do |config|
555
+ config.api_key = "your_api_key_here"
556
+ end
557
+
558
+ client = Lipdub.client
559
+
560
+ begin
561
+ # 0. List existing projects and shots (optional)
562
+ puts "Listing projects..."
563
+ projects = client.projects.list
564
+ puts "Found #{projects['count']} projects"
565
+
566
+ puts "Listing available shots..."
567
+ shots = client.shots.list
568
+ puts "Found #{shots['count']} shots"
569
+
570
+ # 1. Upload video
571
+ puts "Uploading video..."
572
+ video_response = client.videos.upload_complete("input/original_video.mp4")
573
+ shot_id = video_response.dig("data", "shot_id")
574
+ puts "Video uploaded, shot_id: #{shot_id}"
575
+
576
+ # 2. Upload audio
577
+ puts "Uploading audio..."
578
+ audio_response = client.audios.upload_complete("input/new_voice.mp3")
579
+ audio_id = audio_response.dig("data", "audio_id") || "audio_from_upload"
580
+ puts "Audio uploaded, audio_id: #{audio_id}"
581
+
582
+ # 3. Generate lip-dubbed video
583
+ puts "Starting generation..."
584
+ result = client.shots.generate_and_wait(
585
+ shot_id: shot_id,
586
+ audio_id: audio_id,
587
+ output_filename: "dubbed_output.mp4",
588
+ language: "en",
589
+ maintain_expression: true,
590
+ polling_interval: 15,
591
+ max_wait_time: 3600 # 1 hour
592
+ )
593
+
594
+ generate_id = result.dig("data", "generate_id")
595
+ puts "Generation complete, generate_id: #{generate_id}"
596
+
597
+ # 4. Download the result
598
+ puts "Downloading result..."
599
+ output_path = client.shots.download_file(
600
+ shot_id,
601
+ generate_id,
602
+ "output/final_dubbed_video.mp4"
603
+ )
604
+ puts "Video downloaded to: #{output_path}"
605
+ ```
606
+
607
+ #### Selective Lip-dubbing Workflow
608
+
609
+ Here's an example showing how to use selective lip-dubbing for personalization (e.g., replacing just a name in a greeting):
610
+
611
+ ```ruby
612
+ require 'lipdub'
613
+
614
+ # Configure the client
615
+ Lipdub.configure do |config|
616
+ config.api_key = "your_api_key_here"
617
+ end
618
+
619
+ client = Lipdub.client
620
+
621
+ begin
622
+ # 1. Upload original video (done once)
623
+ video_response = client.videos.upload_complete(
624
+ file_path: "./original_greeting.mp4",
625
+ content_type: "video/mp4"
626
+ )
627
+ video_id = video_response.dig("data", "video_id")
628
+
629
+ # 2. Upload personalized audio (replace original audio with new name)
630
+ # NOTE: Audio duration must match the video duration exactly
631
+ personalized_audio = client.audios.upload_complete(
632
+ file_path: "./personalized_greeting_audio.mp3", # Contains new name
633
+ content_type: "audio/mp3"
634
+ )
635
+ audio_id = personalized_audio.dig("data", "audio_id")
636
+
637
+ # 3. Wait for video processing and get shot_id
638
+ loop do
639
+ status = client.videos.status(video_id: video_id)
640
+ if status.dig("data", "status") == "success"
641
+ shot_id = status.dig("data", "shot_id")
642
+ break
643
+ elsif status.dig("data", "status") == "failed"
644
+ raise "Video processing failed"
645
+ end
646
+ sleep 5
647
+ end
648
+
649
+ # 4. Define timecode ranges for selective replacement
650
+ # Example: Replace name at 2.5-4.2 seconds with 10-frame buffer
651
+ name_start = 2.5
652
+ name_end = 4.2
653
+
654
+ # Use helper method to add frame buffer (recommended)
655
+ timecode_ranges = client.shots.add_frame_buffer(
656
+ [[name_start, name_end]],
657
+ buffer_frames: 10,
658
+ fps: 30
659
+ )
660
+
661
+ # Validate ranges (optional but recommended)
662
+ client.shots.validate_timecode_ranges(
663
+ timecode_ranges,
664
+ video_duration: 30.0 # Your video duration
665
+ )
666
+
667
+ # 5. Generate selective lip-dub
668
+ generation = client.shots.generate(
669
+ shot_id: shot_id,
670
+ audio_id: audio_id,
671
+ output_filename: "personalized_greeting.mp4",
672
+ timecode_ranges: timecode_ranges, # Only lip-dub the name part
673
+ language: "en-US"
674
+ )
675
+
676
+ generate_id = generation["generate_id"]
677
+
678
+ # 6. Wait for generation to complete and download
679
+ client.shots.download_file(
680
+ shot_id,
681
+ generate_id,
682
+ "output/personalized_greeting.mp4"
683
+ )
684
+
685
+ puts "Personalized video with selective lip-dubbing saved!"
686
+
687
+ # Alternative: Multiple selective ranges (e.g., name + closing)
688
+ multiple_ranges = [
689
+ [2.5, 4.2], # Name replacement
690
+ [25.0, 27.5] # Closing replacement
691
+ ]
692
+
693
+ buffered_ranges = client.shots.add_frame_buffer(
694
+ multiple_ranges,
695
+ buffer_frames: 10,
696
+ fps: 30,
697
+ video_duration: 30.0
698
+ )
699
+
700
+ # Generate with multiple selective ranges
701
+ multi_selective = client.shots.generate(
702
+ shot_id: shot_id,
703
+ audio_id: audio_id,
704
+ output_filename: "multi_personalized.mp4",
705
+ timecode_ranges: buffered_ranges
706
+ )
707
+
708
+ rescue Lipdub::AuthenticationError => e
709
+ puts "Authentication failed: #{e.message}"
710
+ rescue Lipdub::ValidationError => e
711
+ puts "Validation error: #{e.message}"
712
+ rescue Lipdub::TimeoutError => e
713
+ puts "Request timed out: #{e.message}"
714
+ rescue Lipdub::APIError => e
715
+ puts "API error (#{e.status_code}): #{e.message}"
716
+ rescue => e
717
+ puts "Unexpected error: #{e.message}"
718
+ end
719
+ ```
720
+
721
+ ## Supported File Formats
722
+
723
+ ### Video Formats
724
+ - **MP4** (recommended: 1080p HD, 23.976 fps, H.264 codec) - `video/mp4`
725
+ - **MOV** (QuickTime) - `video/quicktime`
726
+ - **AVI** (Audio Video Interleave) - `video/x-msvideo`
727
+ - **WebM** (Web Media) - `video/webm`
728
+ - **MKV** (Matroska Video) - `video/x-matroska`
729
+
730
+ ### Audio Formats
731
+ - **MP3** (MPEG Audio Layer III) - `audio/mpeg` (1 byte to 100MB)
732
+ - **WAV** (Waveform Audio File Format) - `audio/wav` (1 byte to 100MB)
733
+ - **MP4/M4A** (MPEG-4 Audio) - `audio/mp4` (1 byte to 100MB)
734
+
735
+ ### Recommendations
736
+ - **Video**: Use MP4 with H.264 codec for best compatibility and processing speed
737
+ - **Audio**: Use MP3 or WAV for optimal lip-sync results
738
+ - **Resolution**: 1080p HD recommended for best quality output
739
+ - **Frame Rate**: 23.976 fps or 30 fps for smooth lip-sync
740
+ - **Audio Quality**: 44.1kHz sample rate, 16-bit depth minimum
741
+
742
+ ## Error Handling
743
+
744
+ The gem provides comprehensive error handling with specific exception types:
745
+
746
+ ```ruby
747
+ begin
748
+ client.videos.upload_complete("video.mp4")
749
+ rescue Lipdub::AuthenticationError => e
750
+ # API key is invalid or missing
751
+ puts "Authentication failed: #{e.message}"
752
+ rescue Lipdub::ValidationError => e
753
+ # Request parameters are invalid
754
+ puts "Validation error: #{e.message}"
755
+ puts "Status code: #{e.status_code}"
756
+ puts "Response body: #{e.response_body}"
757
+ rescue Lipdub::NotFoundError => e
758
+ # Resource not found
759
+ puts "Resource not found: #{e.message}"
760
+ rescue Lipdub::RateLimitError => e
761
+ # Rate limit exceeded
762
+ puts "Rate limit exceeded: #{e.message}"
763
+ rescue Lipdub::ServerError => e
764
+ # Server error (5xx)
765
+ puts "Server error: #{e.message}"
766
+ rescue Lipdub::TimeoutError => e
767
+ # Request timed out
768
+ puts "Request timed out: #{e.message}"
769
+ rescue Lipdub::ConnectionError => e
770
+ # Connection failed
771
+ puts "Connection failed: #{e.message}"
772
+ rescue Lipdub::APIError => e
773
+ # Generic API error
774
+ puts "API error: #{e.message}"
775
+ rescue Lipdub::ConfigurationError => e
776
+ # Configuration is invalid
777
+ puts "Configuration error: #{e.message}"
778
+ end
779
+ ```
780
+
781
+ ## Rate Limits and Best Practices
782
+
783
+ ### Rate Limits
784
+ The Lipdub API implements rate limiting to ensure fair usage:
785
+ - **Upload endpoints**: 10 requests per minute
786
+ - **Generation endpoints**: 5 requests per minute
787
+ - **Status/List endpoints**: 100 requests per minute
788
+
789
+ ### Best Practices
790
+
791
+ #### Performance Optimization
792
+ - **Batch operations**: Upload multiple files before starting generation
793
+ - **Polling intervals**: Use appropriate intervals (10-30 seconds) when polling for status
794
+ - **File optimization**: Compress videos and normalize audio before upload
795
+ - **Concurrent uploads**: Upload video and audio files in parallel when possible
796
+
797
+ #### Error Handling
798
+ - **Retry logic**: Implement exponential backoff for transient errors
799
+ - **Validation**: Validate file formats and sizes before upload
800
+ - **Monitoring**: Log API responses for debugging and monitoring
801
+ - **Graceful degradation**: Handle API failures gracefully in production
802
+
803
+ #### Security
804
+ - **API key protection**: Store API keys securely (environment variables, secrets management)
805
+ - **HTTPS only**: All API communications use HTTPS
806
+ - **File validation**: Validate uploaded files on your end before sending to API
807
+ - **Webhook security**: Verify webhook signatures if using callback URLs
808
+
809
+ #### Resource Management
810
+ - **Cleanup**: Remove temporary files after processing
811
+ - **Storage**: Monitor storage usage for large video files
812
+ - **Timeouts**: Set appropriate timeouts for long-running operations
813
+ - **Memory**: Stream large files instead of loading entirely into memory
814
+
815
+ ## Troubleshooting
816
+
817
+ ### Common Issues
818
+
819
+ #### Upload Failures
820
+ ```ruby
821
+ # Issue: File upload fails with timeout
822
+ # Solution: Increase timeout settings
823
+ Lipdub.configure do |config|
824
+ config.timeout = 120 # 2 minutes for large files
825
+ config.open_timeout = 30 # 30 seconds to establish connection
826
+ end
827
+ ```
828
+
829
+ #### Generation Errors
830
+ ```ruby
831
+ # Issue: Generation fails with validation error
832
+ # Solution: Validate inputs before generation
833
+ begin
834
+ # Ensure audio duration matches video duration
835
+ client.shots.validate_timecode_ranges(ranges, video_duration: 30.0)
836
+
837
+ result = client.shots.generate(
838
+ shot_id: shot_id,
839
+ audio_id: audio_id,
840
+ output_filename: "output.mp4"
841
+ )
842
+ rescue Lipdub::ValidationError => e
843
+ puts "Validation failed: #{e.message}"
844
+ # Handle validation error
845
+ end
846
+ ```
847
+
848
+ #### Network Issues
849
+ ```ruby
850
+ # Issue: Connection timeouts or failures
851
+ # Solution: Implement retry logic with exponential backoff
852
+ def upload_with_retry(file_path, max_retries: 3)
853
+ retries = 0
854
+ begin
855
+ client.videos.upload_complete(file_path)
856
+ rescue Lipdub::TimeoutError, Lipdub::ConnectionError => e
857
+ retries += 1
858
+ if retries <= max_retries
859
+ sleep(2 ** retries) # Exponential backoff
860
+ retry
861
+ else
862
+ raise e
863
+ end
864
+ end
865
+ end
866
+ ```
867
+
868
+ ### Debug Mode
869
+
870
+ Enable debug logging to troubleshoot issues:
871
+
872
+ ```ruby
873
+ # Enable debug logging (if supported by your HTTP client)
874
+ Lipdub.configure do |config|
875
+ config.api_key = "your_api_key"
876
+ config.debug = true # Enable debug mode
877
+ end
878
+
879
+ # Or use a custom logger
880
+ require 'logger'
881
+ logger = Logger.new(STDOUT)
882
+ logger.level = Logger::DEBUG
883
+
884
+ # Log API requests and responses
885
+ client = Lipdub.client
886
+ # Add logging middleware to your HTTP client if needed
887
+ ```
888
+
889
+ ### Getting Help
890
+
891
+ - **Documentation**: Check this README and inline code documentation
892
+ - **API Status**: Monitor [Lipdub API status page](https://status.lipdub.ai) for service issues
893
+ - **Support**: Contact support@lipdub.ai for API-related issues
894
+ - **Issues**: Report bugs on [GitHub Issues](https://github.com/upriser/lipdub-ruby/issues)
895
+
896
+ ## Development
897
+
898
+ After checking out the repo, run:
899
+
900
+ ```bash
901
+ bin/setup
902
+ ```
903
+
904
+ To install dependencies. Then, run:
905
+
906
+ ```bash
907
+ rake spec
908
+ ```
909
+
910
+ To run the tests. You can also run:
911
+
912
+ ```bash
913
+ bin/console
914
+ ```
915
+
916
+ For an interactive prompt that will allow you to experiment.
917
+
918
+ ### Running Tests
919
+
920
+ ```bash
921
+ # Run all tests
922
+ bundle exec rspec
923
+
924
+ # Run specific test file
925
+ bundle exec rspec spec/lipdub/client_spec.rb
926
+
927
+ # Run tests with coverage
928
+ bundle exec rspec --format documentation
929
+
930
+ # Run security audit
931
+ bundle exec rake audit
932
+
933
+ # Run all CI checks (tests + security audit)
934
+ bundle exec rake ci
935
+
936
+ # Optional: Run rubocop for linting (not included in CI)
937
+ bundle exec rubocop
938
+ ```
939
+
940
+ ## Contributing
941
+
942
+ Bug reports and pull requests are welcome on GitHub at https://github.com/upriser/lipdub-ruby.
943
+
944
+ 1. Fork it
945
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
946
+ 3. Make your changes and add tests
947
+ 4. Run the test suite (`bundle exec rake ci`)
948
+ 5. Ensure security audit passes (`bundle exec rake audit`)
949
+ 6. Commit your changes (`git commit -am 'Add some feature'`)
950
+ 7. Push to the branch (`git push origin my-new-feature`)
951
+ 8. Create new Pull Request
952
+
953
+ ## License
954
+
955
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).