runway-ruby 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.tool-versions +1 -0
- data/LICENSE.txt +21 -0
- data/OPENAPI_VALIDATION.md +100 -0
- data/README.md +789 -0
- data/Rakefile +11 -0
- data/lib/runway_ml/character_performance.rb +106 -0
- data/lib/runway_ml/client.rb +84 -0
- data/lib/runway_ml/contract_validator.rb +151 -0
- data/lib/runway_ml/errors.rb +179 -0
- data/lib/runway_ml/image_processor.rb +128 -0
- data/lib/runway_ml/image_to_video.rb +158 -0
- data/lib/runway_ml/media_processor.rb +343 -0
- data/lib/runway_ml/openapi_spec_loader.rb +120 -0
- data/lib/runway_ml/organization.rb +89 -0
- data/lib/runway_ml/sound_effect.rb +57 -0
- data/lib/runway_ml/spec_coverage_analyzer.rb +134 -0
- data/lib/runway_ml/speech_to_speech.rb +90 -0
- data/lib/runway_ml/task.rb +116 -0
- data/lib/runway_ml/test_client.rb +62 -0
- data/lib/runway_ml/text_to_speech.rb +52 -0
- data/lib/runway_ml/text_to_video.rb +103 -0
- data/lib/runway_ml/uploads.rb +74 -0
- data/lib/runway_ml/validator_builder.rb +95 -0
- data/lib/runway_ml/validators/base_validators.rb +338 -0
- data/lib/runway_ml/validators/character_performance_validator.rb +41 -0
- data/lib/runway_ml/validators/gen3a_turbo_validator.rb +48 -0
- data/lib/runway_ml/validators/gen4_turbo_validator.rb +50 -0
- data/lib/runway_ml/validators/prompt_image_validator.rb +97 -0
- data/lib/runway_ml/validators/sound_effect_validator.rb +27 -0
- data/lib/runway_ml/validators/speech_to_speech_validator.rb +57 -0
- data/lib/runway_ml/validators/text_to_speech_validator.rb +28 -0
- data/lib/runway_ml/validators/text_veo3_stable_validator.rb +35 -0
- data/lib/runway_ml/validators/text_veo3_validator.rb +33 -0
- data/lib/runway_ml/validators/uploads_validator.rb +44 -0
- data/lib/runway_ml/validators/veo3_stable_validator.rb +43 -0
- data/lib/runway_ml/validators/veo3_validator.rb +46 -0
- data/lib/runway_ml/validators/voice_dubbing_validator.rb +128 -0
- data/lib/runway_ml/validators/voice_isolation_validator.rb +35 -0
- data/lib/runway_ml/validators/voice_validator.rb +32 -0
- data/lib/runway_ml/version.rb +5 -0
- data/lib/runway_ml/voice_dubbing.rb +74 -0
- data/lib/runway_ml/voice_isolation.rb +57 -0
- data/lib/runway_ml.rb +81 -0
- data/lib/tasks/openapi.rake +33 -0
- metadata +90 -0
data/README.md
ADDED
|
@@ -0,0 +1,789 @@
|
|
|
1
|
+
# RunwayML Ruby SDK
|
|
2
|
+
|
|
3
|
+
Ruby client for the [RunwayML API](https://docs.dev.runwayml.com/). Generate videos from images using state-of-the-art AI models like Gen-4 Turbo and Veo 3.1.
|
|
4
|
+
|
|
5
|
+
## Installation
|
|
6
|
+
|
|
7
|
+
Install the gem and add to the application's Gemfile by executing:
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
bundle add runway-ruby
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
If bundler is not being used to manage dependencies, install the gem by executing:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
gem install runway-ruby
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Setup
|
|
20
|
+
|
|
21
|
+
1. Create a developer account following [Runway's guide](https://docs.dev.runwayml.com/guides/setup/)
|
|
22
|
+
2. Create an API key
|
|
23
|
+
3. Set your API key as an environment variable:
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
export RUNWAY_API_SECRET='your-api-key-here'
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## Table of Contents
|
|
30
|
+
|
|
31
|
+
- [Usage](#usage)
|
|
32
|
+
- [Basic Image-to-Video Generation](#basic-image-to-video-generation)
|
|
33
|
+
- [Text-to-Video Generation](#text-to-video-generation)
|
|
34
|
+
- [Character Performance](#character-performance)
|
|
35
|
+
- [Sound Effect Generation](#sound-effect-generation)
|
|
36
|
+
- [Speech-to-Speech Conversion](#speech-to-speech-conversion)
|
|
37
|
+
- [Text-to-Speech Generation](#text-to-speech-generation)
|
|
38
|
+
- [Voice Dubbing](#voice-dubbing)
|
|
39
|
+
- [Voice Isolation](#voice-isolation)
|
|
40
|
+
- [Uploading Media Files](#uploading-media-files)
|
|
41
|
+
- [Organization Information](#organization-information)
|
|
42
|
+
- [Task Object](#task-object)
|
|
43
|
+
- [Using Local Image Files](#using-local-image-files)
|
|
44
|
+
- [Using File Objects or StringIO](#using-file-objects-or-stringio)
|
|
45
|
+
- [Automatic File Uploads](#automatic-file-uploads)
|
|
46
|
+
- [Using Data URIs](#using-data-uris)
|
|
47
|
+
- [Supported Models](#supported-models)
|
|
48
|
+
- [Supported Image Formats](#supported-image-formats)
|
|
49
|
+
- [Validation](#validation)
|
|
50
|
+
- [Error Handling](#error-handling)
|
|
51
|
+
- [Development](#development)
|
|
52
|
+
- [Contributing](#contributing)
|
|
53
|
+
- [License](#license)
|
|
54
|
+
|
|
55
|
+
## Usage
|
|
56
|
+
|
|
57
|
+
### Basic Image-to-Video Generation
|
|
58
|
+
|
|
59
|
+
Generate a video from an image URL:
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
require 'runway_ml'
|
|
63
|
+
|
|
64
|
+
# Create a new image-to-video task using the "gen4_turbo" model
|
|
65
|
+
begin
|
|
66
|
+
task = RunwayML.image_to_video(
|
|
67
|
+
model: 'gen4_turbo',
|
|
68
|
+
prompt_image: 'https://upload.wikimedia.org/wikipedia/commons/8/85/Tour_Eiffel_Wikimedia_Commons_(cropped).jpg',
|
|
69
|
+
prompt_text: 'A timelapse on a sunny day with clouds flying by',
|
|
70
|
+
ratio: '1280:720',
|
|
71
|
+
duration: 5
|
|
72
|
+
)
|
|
73
|
+
|
|
74
|
+
puts "Task created with ID: #{task.id}"
|
|
75
|
+
# => Task created with ID: 497f6eca-6276-4993-bfeb-53cbbbba6f08
|
|
76
|
+
|
|
77
|
+
# Wait for the task to finish (polls the API and updates the task attributes)
|
|
78
|
+
task.wait_for_output
|
|
79
|
+
task.status # => "SUCCEEDED"
|
|
80
|
+
task.output # => ["https://..."]
|
|
81
|
+
rescue RunwayML::ValidationError => e
|
|
82
|
+
puts "Validation failed: #{e.message}"
|
|
83
|
+
rescue RunwayML::Error => e
|
|
84
|
+
puts "Error: #{e.message}"
|
|
85
|
+
end
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
The method returns a `RunwayML::Task` object with the task ID. You can use this ID to check the status of your video generation.
|
|
89
|
+
|
|
90
|
+
### Text-to-Video Generation
|
|
91
|
+
|
|
92
|
+
Generate a video from a text prompt without requiring an image:
|
|
93
|
+
|
|
94
|
+
```ruby
|
|
95
|
+
# Create a new text-to-video task using the "veo3.1" model
|
|
96
|
+
task = RunwayML.text_to_video(
|
|
97
|
+
model: 'veo3.1',
|
|
98
|
+
prompt_text: 'A cute bunny hopping in a meadow',
|
|
99
|
+
ratio: '1280:720',
|
|
100
|
+
duration: 8
|
|
101
|
+
).wait_for_output
|
|
102
|
+
|
|
103
|
+
task.status # => "SUCCEEDED"
|
|
104
|
+
task.output # => ["https://..."]
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
The text-to-video method works similarly to image-to-video, but generates videos directly from text descriptions without requiring an input image. This is useful for creating videos from scratch based on creative prompts.
|
|
108
|
+
|
|
109
|
+
### Character Performance
|
|
110
|
+
|
|
111
|
+
Control a character's facial expressions and body movements using a reference video. Apply a performer's movements to a character image or video:
|
|
112
|
+
|
|
113
|
+
```ruby
|
|
114
|
+
require 'runway_ml'
|
|
115
|
+
|
|
116
|
+
# Create a character performance task using an image character
|
|
117
|
+
begin
|
|
118
|
+
task = RunwayML.character_performance(
|
|
119
|
+
model: 'act_two',
|
|
120
|
+
character: {
|
|
121
|
+
type: 'image',
|
|
122
|
+
uri: 'https://example.com/character.jpg'
|
|
123
|
+
},
|
|
124
|
+
reference: {
|
|
125
|
+
type: 'video',
|
|
126
|
+
uri: 'https://example.com/performance.mp4'
|
|
127
|
+
},
|
|
128
|
+
ratio: '1280:720'
|
|
129
|
+
)
|
|
130
|
+
|
|
131
|
+
puts "Task created with ID: #{task.id}"
|
|
132
|
+
# => Task created with ID: 497f6eca-6276-4993-bfeb-53cbbbba6f08
|
|
133
|
+
|
|
134
|
+
# Wait for the task to finish
|
|
135
|
+
task.wait_for_output
|
|
136
|
+
task.status # => "SUCCEEDED"
|
|
137
|
+
task.output # => ["https://..."]
|
|
138
|
+
rescue RunwayML::ValidationError => e
|
|
139
|
+
puts "Validation failed: #{e.message}"
|
|
140
|
+
rescue RunwayML::Error => e
|
|
141
|
+
puts "Error: #{e.message}"
|
|
142
|
+
end
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
You can also use a video as the character input instead of an image:
|
|
146
|
+
|
|
147
|
+
```ruby
|
|
148
|
+
task = RunwayML.character_performance(
|
|
149
|
+
model: 'act_two',
|
|
150
|
+
character: { type: 'video', uri: 'https://example.com/character_video.mp4' },
|
|
151
|
+
reference: { type: 'video', uri: 'https://example.com/performance.mp4' },
|
|
152
|
+
ratio: '1280:720',
|
|
153
|
+
body_control: true,
|
|
154
|
+
expression_intensity: 3,
|
|
155
|
+
seed: 12345
|
|
156
|
+
).wait_for_output
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**Character Performance Parameters:**
|
|
160
|
+
|
|
161
|
+
- `model` - Required. Must be `'act_two'`
|
|
162
|
+
- `character` - Required. An image or video of your character. Must contain a visually recognizable face
|
|
163
|
+
- `type`: Either `'image'` or `'video'`
|
|
164
|
+
- `uri`: HTTPS URL, Runway URI, or data URI
|
|
165
|
+
- `reference` - Required. A video containing the performance to apply to the character (3-30 seconds)
|
|
166
|
+
- `type`: Must be `'video'`
|
|
167
|
+
- `uri`: HTTPS URL, Runway URI, or data URI
|
|
168
|
+
- `ratio` - Required. Output resolution: `'1280:720'`, `'720:1280'`, `'960:960'`, `'1104:832'`, `'832:1104'`, or `'1584:672'`
|
|
169
|
+
- `body_control` - Optional. Boolean to enable body movement (default: false)
|
|
170
|
+
- `expression_intensity` - Optional. 1-5 scale for expression intensity (default: 3)
|
|
171
|
+
- `seed` - Optional. Random seed for reproducibility (0-4294967295)
|
|
172
|
+
- `public_figure_threshold` - Optional. Content moderation threshold: `'auto'` or `'low'`
|
|
173
|
+
|
|
174
|
+
### Sound Effect Generation
|
|
175
|
+
|
|
176
|
+
Generate sound effects from text descriptions:
|
|
177
|
+
|
|
178
|
+
```ruby
|
|
179
|
+
# Create a new sound effect task
|
|
180
|
+
task = RunwayML.sound_effect(
|
|
181
|
+
model: 'eleven_text_to_sound_v2',
|
|
182
|
+
prompt_text: 'A thunderstorm with heavy rain',
|
|
183
|
+
duration: 10,
|
|
184
|
+
loop: true
|
|
185
|
+
).wait_for_output
|
|
186
|
+
|
|
187
|
+
task.status # => "SUCCEEDED"
|
|
188
|
+
task.output # => ["https://..."]
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
**Sound Effect Parameters:**
|
|
192
|
+
|
|
193
|
+
- `model` - Required. Must be `'eleven_text_to_sound_v2'`
|
|
194
|
+
- `prompt_text` - Required. A text description of the sound effect (1-3000 characters)
|
|
195
|
+
- `duration` - Optional. The duration of the sound effect in seconds (0.5-30). If not provided, the duration will be determined automatically based on the text description
|
|
196
|
+
- `loop` - Optional. Whether the output sound effect should be designed to loop seamlessly (default: false)
|
|
197
|
+
|
|
198
|
+
### Speech-to-Speech Conversion
|
|
199
|
+
|
|
200
|
+
Convert speech from one voice to another in audio or video files:
|
|
201
|
+
|
|
202
|
+
```ruby
|
|
203
|
+
# Convert speech in an audio file to a different voice
|
|
204
|
+
audio_task = RunwayML.speech_to_speech(
|
|
205
|
+
model: 'eleven_multilingual_sts_v2',
|
|
206
|
+
media: { type: 'audio', uri: 'https://example.com/audio.mp3' },
|
|
207
|
+
voice: { type: 'runway-preset', presetId: 'Maggie' }
|
|
208
|
+
).wait_for_output
|
|
209
|
+
|
|
210
|
+
audio_task.status # => "SUCCEEDED"
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
You can also convert speech in video files:
|
|
214
|
+
|
|
215
|
+
```ruby
|
|
216
|
+
# Convert speech in a video file to a different voice
|
|
217
|
+
video_task = RunwayML.speech_to_speech(
|
|
218
|
+
model: 'eleven_multilingual_sts_v2',
|
|
219
|
+
media: { type: 'video', uri: 'https://example.com/video.mp4' },
|
|
220
|
+
voice: { type: 'runway-preset', presetId: 'Noah' },
|
|
221
|
+
remove_background_noise: true
|
|
222
|
+
).wait_for_output
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
**Available Voice Presets:**
|
|
226
|
+
|
|
227
|
+
The following preset voices are available: Maya, Arjun, Serene, Bernard, Billy, Mark, Clint, Mabel, Chad, Leslie, Eleanor, Elias, Elliot, Grungle, Brodie, Sandra, Kirk, Kylie, Lara, Lisa, Malachi, Marlene, Martin, Miriam, Monster, Paula, Pip, Rusty, Ragnar, Xylar, Maggie, Jack, Katie, Noah, James, Rina, Ella, Mariah, Frank, Claudia, Niki, Vincent, Kendrick, Myrna, Tom, Wanda, Benjamin, Kiana, Rachel
|
|
228
|
+
|
|
229
|
+
**Supported Audio Formats:**
|
|
230
|
+
|
|
231
|
+
The gem supports the following audio formats for speech-to-speech conversion:
|
|
232
|
+
|
|
233
|
+
- **MP3** (`.mp3`) - MPEG-1/2 Layer 3 codec
|
|
234
|
+
- **WAV** (`.wav`) - PCM (uncompressed) codec
|
|
235
|
+
- **FLAC** (`.flac`) - FLAC (lossless) codec
|
|
236
|
+
- **M4A** (`.m4a`) - AAC or ALAC codec
|
|
237
|
+
- **AAC** (`.aac`) - AAC (raw) codec
|
|
238
|
+
|
|
239
|
+
### Text-to-Speech Generation
|
|
240
|
+
|
|
241
|
+
Generate speech from text descriptions using various voice presets:
|
|
242
|
+
|
|
243
|
+
```ruby
|
|
244
|
+
require 'runway_ml'
|
|
245
|
+
|
|
246
|
+
# Generate speech from text
|
|
247
|
+
task = RunwayML.text_to_speech(
|
|
248
|
+
model: 'eleven_multilingual_v2',
|
|
249
|
+
prompt_text: 'The quick brown fox jumps over the lazy dog',
|
|
250
|
+
voice: {
|
|
251
|
+
type: 'runway-preset',
|
|
252
|
+
presetId: 'Leslie'
|
|
253
|
+
}
|
|
254
|
+
).wait_for_output
|
|
255
|
+
|
|
256
|
+
task.status # => "SUCCEEDED"
|
|
257
|
+
task.output # => ["https://..."]
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
You can also generate speech with different voices:
|
|
261
|
+
|
|
262
|
+
```ruby
|
|
263
|
+
# Generate speech with a different voice
|
|
264
|
+
task = RunwayML.text_to_speech(
|
|
265
|
+
model: 'eleven_multilingual_v2',
|
|
266
|
+
prompt_text: 'Hello, this is a test of the text-to-speech system',
|
|
267
|
+
voice: {
|
|
268
|
+
type: 'runway-preset',
|
|
269
|
+
presetId: 'Noah'
|
|
270
|
+
}
|
|
271
|
+
).wait_for_output
|
|
272
|
+
|
|
273
|
+
task.status # => "SUCCEEDED"
|
|
274
|
+
task.output # => ["https://..."]
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
**Text-to-Speech Parameters:**
|
|
278
|
+
|
|
279
|
+
- `model` - Required. Must be `'eleven_multilingual_v2'`
|
|
280
|
+
- `prompt_text` - Required. A text description of the speech to generate (1-1000 characters)
|
|
281
|
+
- `voice` - Required. The voice preset to use for the generated speech
|
|
282
|
+
- `type`: Must be `'runway-preset'`
|
|
283
|
+
- `presetId`: One of the available voice IDs (see list below)
|
|
284
|
+
|
|
285
|
+
**Available Voice Presets:**
|
|
286
|
+
|
|
287
|
+
The following preset voices are available: Maya, Arjun, Serene, Bernard, Billy, Mark, Clint, Mabel, Chad, Leslie, Eleanor, Elias, Elliot, Grungle, Brodie, Sandra, Kirk, Kylie, Lara, Lisa, Malachi, Marlene, Martin, Miriam, Monster, Paula, Pip, Rusty, Ragnar, Xylar, Maggie, Jack, Katie, Noah, James, Rina, Ella, Mariah, Frank, Claudia, Niki, Vincent, Kendrick, Myrna, Tom, Wanda, Benjamin, Kiana, Rachel
|
|
288
|
+
|
|
289
|
+
### Voice Dubbing
|
|
290
|
+
|
|
291
|
+
Dub audio content to a target language while optionally preserving the original voice characteristics:
|
|
292
|
+
|
|
293
|
+
```ruby
|
|
294
|
+
require 'runway_ml'
|
|
295
|
+
|
|
296
|
+
# Dub audio to Spanish
|
|
297
|
+
task = RunwayML.voice_dubbing(
|
|
298
|
+
model: 'eleven_voice_dubbing',
|
|
299
|
+
audio_uri: 'https://example.com/audio.mp3',
|
|
300
|
+
target_lang: 'es'
|
|
301
|
+
).wait_for_output
|
|
302
|
+
|
|
303
|
+
task.status # => "SUCCEEDED"
|
|
304
|
+
task.output # => ["https://..."]
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
You can customize the dubbing process with additional options:
|
|
308
|
+
|
|
309
|
+
```ruby
|
|
310
|
+
# Dub audio with custom options
|
|
311
|
+
task = RunwayML.voice_dubbing(
|
|
312
|
+
model: 'eleven_voice_dubbing',
|
|
313
|
+
audio_uri: 'https://example.com/audio.mp3',
|
|
314
|
+
target_lang: 'fr',
|
|
315
|
+
disable_voice_cloning: true, # Use generic voice instead of cloning
|
|
316
|
+
drop_background_audio: true, # Remove background audio
|
|
317
|
+
num_speakers: 2 # Specify number of speakers
|
|
318
|
+
).wait_for_output
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
**Voice Dubbing Parameters:**
|
|
322
|
+
|
|
323
|
+
- `model` - Required. Must be `'eleven_voice_dubbing'`
|
|
324
|
+
- `audio_uri` - Required. HTTPS URL, Runway URI, or data URI containing the audio to dub
|
|
325
|
+
- `target_lang` - Required. The target language code (e.g., "es" for Spanish, "fr" for French). Supported languages: en, hi, pt, zh, es, fr, de, ja, ar, ru, ko, id, it, nl, tr, pl, sv, fil, ms, ro, uk, el, cs, da, fi, bg, hr, sk, ta
|
|
326
|
+
- `disable_voice_cloning` - Optional. Boolean. Set to true to use a generic voice instead of cloning the original voice
|
|
327
|
+
- `drop_background_audio` - Optional. Boolean. Set to true to remove background audio from the dubbed output
|
|
328
|
+
- `num_speakers` - Optional. Integer (0-9007199254740991). The number of speakers in the audio. If not provided, it will be detected automatically
|
|
329
|
+
|
|
330
|
+
### Voice Isolation
|
|
331
|
+
|
|
332
|
+
Isolate the voice from background audio. Audio duration must be greater than 4.6 seconds and less than 3600 seconds:
|
|
333
|
+
|
|
334
|
+
```ruby
|
|
335
|
+
require 'runway_ml'
|
|
336
|
+
|
|
337
|
+
# Isolate voice from audio
|
|
338
|
+
task = RunwayML.voice_isolation(
|
|
339
|
+
model: 'eleven_voice_isolation',
|
|
340
|
+
audio_uri: 'https://example.com/audio.mp3'
|
|
341
|
+
).wait_for_output
|
|
342
|
+
|
|
343
|
+
task.status # => "SUCCEEDED"
|
|
344
|
+
task.output # => ["https://..."]
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
You can also use Runway URIs or data URIs:
|
|
348
|
+
|
|
349
|
+
```ruby
|
|
350
|
+
# Using a Runway URI
|
|
351
|
+
task = RunwayML.voice_isolation(
|
|
352
|
+
model: 'eleven_voice_isolation',
|
|
353
|
+
audio_uri: 'runway://audio123'
|
|
354
|
+
).wait_for_output
|
|
355
|
+
|
|
356
|
+
# Using a data URI
|
|
357
|
+
require 'base64'
|
|
358
|
+
audio_data = File.binread('audio.mp3')
|
|
359
|
+
data_uri = "data:audio/mpeg;base64,#{Base64.strict_encode64(audio_data)}"
|
|
360
|
+
|
|
361
|
+
task = RunwayML.voice_isolation(
|
|
362
|
+
model: 'eleven_voice_isolation',
|
|
363
|
+
audio_uri: data_uri
|
|
364
|
+
).wait_for_output
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
**Voice Isolation Parameters:**
|
|
368
|
+
|
|
369
|
+
- `model` - Required. Must be `'eleven_voice_isolation'`
|
|
370
|
+
- `audio_uri` - Required. HTTPS URL, Runway URI, or data URI containing the audio. Duration must be > 4.6 seconds and < 3600 seconds
|
|
371
|
+
|
|
372
|
+
### Uploading Media Files
|
|
373
|
+
|
|
374
|
+
Upload media files to RunwayML for use in generation requests. Uploaded files are temporary and will be automatically expired after a period of time:
|
|
375
|
+
|
|
376
|
+
```ruby
|
|
377
|
+
require 'runway_ml'
|
|
378
|
+
|
|
379
|
+
# Upload a video file
|
|
380
|
+
uploads = RunwayML.uploads
|
|
381
|
+
runway_uri = uploads.create_ephemeral('path/to/video.mp4')
|
|
382
|
+
|
|
383
|
+
puts "Upload successful!"
|
|
384
|
+
puts "Runway URI: #{runway_uri}"
|
|
385
|
+
# => Runway URI: runway://uploads/abc123...
|
|
386
|
+
|
|
387
|
+
# Use the runway_uri in your generation requests
|
|
388
|
+
task = RunwayML.image_to_video(
|
|
389
|
+
model: 'gen4_turbo',
|
|
390
|
+
prompt_image: runway_uri,
|
|
391
|
+
prompt_text: 'Add motion effects to the video',
|
|
392
|
+
ratio: '1280:720',
|
|
393
|
+
duration: 5
|
|
394
|
+
).wait_for_output
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
You can also upload with a custom filename:
|
|
398
|
+
|
|
399
|
+
```ruby
|
|
400
|
+
# Upload with a custom filename
|
|
401
|
+
runway_uri = uploads.create_ephemeral('path/to/video.mp4', filename: 'my-custom-video.mp4')
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
And upload from File objects or StringIO:
|
|
405
|
+
|
|
406
|
+
```ruby
|
|
407
|
+
# Upload from a File object
|
|
408
|
+
File.open('video.mp4', 'rb') do |file|
|
|
409
|
+
runway_uri = uploads.create_ephemeral(file, filename: 'video.mp4')
|
|
410
|
+
end
|
|
411
|
+
|
|
412
|
+
# Upload from StringIO
|
|
413
|
+
require 'stringio'
|
|
414
|
+
|
|
415
|
+
video_data = StringIO.new(File.binread('video.mp4'))
|
|
416
|
+
runway_uri = uploads.create_ephemeral(video_data, filename: 'video.mp4')
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
**Supported file types for uploads:**
|
|
420
|
+
|
|
421
|
+
- **Images:** JPG, JPEG, PNG, WebP
|
|
422
|
+
- **Videos:** MP4, MOV, MKV, WebM, 3GP, OGV, AVI, FLV, MPG, MPEG
|
|
423
|
+
- **Audio:** MP3, WAV, FLAC, M4A, AAC, OGG, WebA
|
|
424
|
+
|
|
425
|
+
### Organization Information
|
|
426
|
+
|
|
427
|
+
Get information about your organization's usage and credit balance:
|
|
428
|
+
|
|
429
|
+
```ruby
|
|
430
|
+
require 'runway_ml'
|
|
431
|
+
|
|
432
|
+
# Get organization details
|
|
433
|
+
organization = RunwayML.organization
|
|
434
|
+
info = organization.retrieve
|
|
435
|
+
|
|
436
|
+
puts "Credit balance: #{info.credit_balance}"
|
|
437
|
+
puts "Monthly spend limit: #{info.max_monthly_credit_spend}"
|
|
438
|
+
puts "Tier: #{info.tier}"
|
|
439
|
+
```
|
|
440
|
+
|
|
441
|
+
Query usage data for your organization by model and day (up to 90 days of data):
|
|
442
|
+
|
|
443
|
+
```ruby
|
|
444
|
+
# Get usage data for the last 30 days
|
|
445
|
+
usage = organization.usage
|
|
446
|
+
|
|
447
|
+
# Access usage by model
|
|
448
|
+
puts usage.models # => { "gen4_turbo" => [...], "veo3.1" => [...] }
|
|
449
|
+
|
|
450
|
+
# Query specific date range
|
|
451
|
+
usage = organization.usage(
|
|
452
|
+
start_date: '2026-01-01',
|
|
453
|
+
before_date: '2026-02-08'
|
|
454
|
+
)
|
|
455
|
+
|
|
456
|
+
puts usage.by_model # => { ... }
|
|
457
|
+
puts usage.to_h # => Returns full usage data as hash
|
|
458
|
+
```
|
|
459
|
+
|
|
460
|
+
**Organization Methods:**
|
|
461
|
+
|
|
462
|
+
- `retrieve()` - Returns `OrganizationInfo` with tier, credit balance, and current usage
|
|
463
|
+
- `credit_balance` - Current credit balance for the organization
|
|
464
|
+
- `tier` - Tier information including max monthly credit spend
|
|
465
|
+
- `usage` - Current usage data broken down by models
|
|
466
|
+
- `max_monthly_credit_spend()` - Maximum monthly credits available for this tier
|
|
467
|
+
- `tier_models()` - Models included in this tier
|
|
468
|
+
- `usage_models()` - Current usage by model
|
|
469
|
+
|
|
470
|
+
- `usage(start_date:, before_date:)` - Returns `UsageInfo` with usage breakdown
|
|
471
|
+
- `start_date` - Optional. ISO-8601 date (YYYY-MM-DD) for query start. Defaults to 30 days before current date
|
|
472
|
+
- `before_date` - Optional. ISO-8601 date (YYYY-MM-DD) for query end (not inclusive). Defaults to 30 days after start date
|
|
473
|
+
- `models()` - Returns usage data by model
|
|
474
|
+
- `by_model()` - Alias for models()
|
|
475
|
+
- `to_h()` - Returns full usage data as hash
|
|
476
|
+
|
|
477
|
+
### Task Object
|
|
478
|
+
|
|
479
|
+
The API returns a `RunwayML::Task` object which provides convenient methods to manage your task:
|
|
480
|
+
|
|
481
|
+
```ruby
|
|
482
|
+
# Create a new task
|
|
483
|
+
task = RunwayML.image_to_video(...)
|
|
484
|
+
|
|
485
|
+
# Get task information
|
|
486
|
+
task.id # => "497f6eca-6276-4993-bfeb-53cbbbba6f08"
|
|
487
|
+
task.status # => "IN_PROGRESS" (IN_PROGRESS, QUEUED, FAILED, SUCCEEDED)
|
|
488
|
+
|
|
489
|
+
# Wait for the task to complete (polls API by default every 2 seconds)
|
|
490
|
+
task.wait_for_output
|
|
491
|
+
|
|
492
|
+
# Get the results
|
|
493
|
+
task.status # => "SUCCEEDED"
|
|
494
|
+
task.output # => ["https://..."]
|
|
495
|
+
task.failure_code # => "SOME_ERROR" (FAILED)
|
|
496
|
+
task.output # => ["https://..."] (SUCCEEDED)
|
|
497
|
+
|
|
498
|
+
# Delete a task
|
|
499
|
+
task.delete # => true/false
|
|
500
|
+
```
|
|
501
|
+
|
|
502
|
+
### Using Local Image Files
|
|
503
|
+
|
|
504
|
+
The gem makes it easy to work with local files - just pass a file path and it will automatically be converted to a data URI:
|
|
505
|
+
|
|
506
|
+
```ruby
|
|
507
|
+
require 'runway_ml'
|
|
508
|
+
|
|
509
|
+
# Simply pass the file path - the gem handles the conversion
|
|
510
|
+
task = RunwayML.image_to_video(
|
|
511
|
+
model: 'gen4_turbo',
|
|
512
|
+
prompt_image: 'path/to/your/image.jpg', # Local file path
|
|
513
|
+
prompt_text: 'A timelapse on a sunny day with clouds flying by',
|
|
514
|
+
ratio: '1280:720',
|
|
515
|
+
duration: 5
|
|
516
|
+
)
|
|
517
|
+
```
|
|
518
|
+
|
|
519
|
+
### Using File Objects or StringIO
|
|
520
|
+
|
|
521
|
+
You can also pass File objects or StringIO directly:
|
|
522
|
+
|
|
523
|
+
```ruby
|
|
524
|
+
# Using a File object
|
|
525
|
+
File.open('image.jpg', 'rb') do |file|
|
|
526
|
+
task = RunwayML.image_to_video(
|
|
527
|
+
model: 'gen4_turbo',
|
|
528
|
+
prompt_image: file,
|
|
529
|
+
prompt_text: 'A timelapse on a sunny day with clouds flying by',
|
|
530
|
+
ratio: '1280:720',
|
|
531
|
+
duration: 5
|
|
532
|
+
)
|
|
533
|
+
end
|
|
534
|
+
|
|
535
|
+
# Using StringIO
|
|
536
|
+
require 'stringio'
|
|
537
|
+
|
|
538
|
+
image_data = StringIO.new(File.binread('image.jpg'))
|
|
539
|
+
task = RunwayML.image_to_video(
|
|
540
|
+
model: 'gen4_turbo',
|
|
541
|
+
prompt_image: image_data,
|
|
542
|
+
prompt_text: 'A timelapse on a sunny day with clouds flying by',
|
|
543
|
+
ratio: '1280:720',
|
|
544
|
+
duration: 5
|
|
545
|
+
)
|
|
546
|
+
```
|
|
547
|
+
|
|
548
|
+
### Automatic File Uploads
|
|
549
|
+
|
|
550
|
+
For generation methods that accept media parameters (audio, images, or videos), the SDK automatically uploads local files, File objects, and StringIO to Runway when you pass them as parameters. This happens transparently without requiring manual upload steps:
|
|
551
|
+
|
|
552
|
+
**Supported for all media-accepting methods:**
|
|
553
|
+
|
|
554
|
+
- Voice Dubbing, Voice Isolation, Speech-to-Speech
|
|
555
|
+
- Image-to-Video, Text-to-Video, Character Performance
|
|
556
|
+
- Any method that accepts `audio_uri` or media parameters
|
|
557
|
+
|
|
558
|
+
```ruby
|
|
559
|
+
require 'runway_ml'
|
|
560
|
+
|
|
561
|
+
# Automatically uploads the local audio file internally
|
|
562
|
+
task = RunwayML.voice_dubbing(
|
|
563
|
+
model: 'eleven_voice_dubbing',
|
|
564
|
+
audio_uri: 'path/to/audio.mp3', # Local file - automatically uploaded!
|
|
565
|
+
target_lang: 'es'
|
|
566
|
+
).wait_for_output
|
|
567
|
+
|
|
568
|
+
# Works with File objects too
|
|
569
|
+
File.open('audio.mp3', 'rb') do |file|
|
|
570
|
+
task = RunwayML.voice_isolation(
|
|
571
|
+
model: 'eleven_voice_isolation',
|
|
572
|
+
audio_uri: file # Automatically uploaded!
|
|
573
|
+
).wait_for_output
|
|
574
|
+
end
|
|
575
|
+
|
|
576
|
+
# And with StringIO
|
|
577
|
+
require 'stringio'
|
|
578
|
+
|
|
579
|
+
audio_data = StringIO.new(File.binread('audio.mp3'))
|
|
580
|
+
task = RunwayML.speech_to_speech(
|
|
581
|
+
model: 'eleven_multilingual_sts_v2',
|
|
582
|
+
media: { type: 'audio', uri: audio_data }, # Automatically uploaded!
|
|
583
|
+
voice: { type: 'runway-preset', presetId: 'Noah' }
|
|
584
|
+
).wait_for_output
|
|
585
|
+
|
|
586
|
+
# Works with any supported file type (images, videos, audio)
|
|
587
|
+
video_data = StringIO.new(File.binread('video.mp4'))
|
|
588
|
+
task = RunwayML.speech_to_speech(
|
|
589
|
+
model: 'eleven_multilingual_sts_v2',
|
|
590
|
+
media: { type: 'video', uri: video_data }, # Also auto-uploaded!
|
|
591
|
+
voice: { type: 'runway-preset', presetId: 'Noah' }
|
|
592
|
+
).wait_for_output
|
|
593
|
+
|
|
594
|
+
# Image-to-Video with automatic image upload
|
|
595
|
+
task = RunwayML.image_to_video(
|
|
596
|
+
model: 'gen4_turbo',
|
|
597
|
+
prompt_image: 'path/to/image.jpg', # Local file - automatically uploaded!
|
|
598
|
+
prompt_text: 'A timelapse on a sunny day',
|
|
599
|
+
ratio: '1280:720',
|
|
600
|
+
duration: 5
|
|
601
|
+
).wait_for_output
|
|
602
|
+
|
|
603
|
+
# Image-to-Video with optional audio file
|
|
604
|
+
task = RunwayML.image_to_video(
|
|
605
|
+
model: 'veo3.1',
|
|
606
|
+
prompt_image: 'path/to/image.jpg',
|
|
607
|
+
prompt_text: 'A walk through the forest',
|
|
608
|
+
ratio: '16:9',
|
|
609
|
+
duration: 10,
|
|
610
|
+
audio: 'path/to/music.mp3' # Also automatically uploaded!
|
|
611
|
+
).wait_for_output
|
|
612
|
+
|
|
613
|
+
# Text-to-Video with automatic audio upload
|
|
614
|
+
task = RunwayML.text_to_video(
|
|
615
|
+
model: 'veo3.1',
|
|
616
|
+
prompt_text: 'A cute bunny hopping in a meadow',
|
|
617
|
+
ratio: '1280:720',
|
|
618
|
+
duration: 8,
|
|
619
|
+
audio: 'path/to/background.wav' # Automatically uploaded!
|
|
620
|
+
).wait_for_output
|
|
621
|
+
|
|
622
|
+
# Character Performance with automatic media upload
|
|
623
|
+
task = RunwayML.character_performance(
|
|
624
|
+
model: 'act_two',
|
|
625
|
+
character: { type: 'image', uri: 'path/to/character.jpg' }, # Auto-uploaded!
|
|
626
|
+
reference: { type: 'video', uri: 'path/to/reference.mp4' }, # Auto-uploaded!
|
|
627
|
+
ratio: '1280:720'
|
|
628
|
+
).wait_for_output
|
|
629
|
+
```
|
|
630
|
+
|
|
631
|
+
**Disabling Automatic Uploads:**
|
|
632
|
+
|
|
633
|
+
If you want to disable automatic uploads and use local files as data URIs instead, pass `auto_upload: false`:
|
|
634
|
+
|
|
635
|
+
```ruby
|
|
636
|
+
# Convert to data URI instead of uploading
|
|
637
|
+
task = RunwayML.voice_dubbing(
|
|
638
|
+
model: 'eleven_voice_dubbing',
|
|
639
|
+
audio_uri: 'path/to/audio.mp3',
|
|
640
|
+
target_lang: 'es',
|
|
641
|
+
auto_upload: false # Uses data URI instead
|
|
642
|
+
).wait_for_output
|
|
643
|
+
|
|
644
|
+
# Also works for image/video methods
|
|
645
|
+
task = RunwayML.image_to_video(
|
|
646
|
+
model: 'gen4_turbo',
|
|
647
|
+
prompt_image: 'path/to/image.jpg',
|
|
648
|
+
prompt_text: 'A timelapse',
|
|
649
|
+
ratio: '1280:720',
|
|
650
|
+
duration: 5,
|
|
651
|
+
auto_upload: false # Uses data URI instead
|
|
652
|
+
).wait_for_output
|
|
653
|
+
```
|
|
654
|
+
|
|
655
|
+
Note: Automatic uploads are enabled by default for all media parameters in Voice Dubbing, Voice Isolation, Speech-to-Speech, Image-to-Video, Text-to-Video, Character Performance, and related methods.
|
|
656
|
+
|
|
657
|
+
### Using Data URIs
|
|
658
|
+
|
|
659
|
+
You can manually create base64-encoded data URIs if needed:
|
|
660
|
+
|
|
661
|
+
```ruby
|
|
662
|
+
require 'runway_ml'
|
|
663
|
+
require 'base64'
|
|
664
|
+
|
|
665
|
+
# Read and encode the image file
|
|
666
|
+
image_buffer = File.binread('example.png')
|
|
667
|
+
data_uri = "data:image/png;base64,#{Base64.strict_encode64(image_buffer)}"
|
|
668
|
+
|
|
669
|
+
# Create a new image-to-video task
|
|
670
|
+
task = RunwayML.image_to_video(
|
|
671
|
+
model: 'gen4_turbo',
|
|
672
|
+
prompt_image: data_uri,
|
|
673
|
+
prompt_text: 'A timelapse on a sunny day with clouds flying by',
|
|
674
|
+
ratio: '1280:720',
|
|
675
|
+
duration: 5
|
|
676
|
+
)
|
|
677
|
+
```
|
|
678
|
+
|
|
679
|
+
### Supported Models
|
|
680
|
+
|
|
681
|
+
The gem supports the following AI models for video generation, character control, sound effect generation, and speech conversion:
|
|
682
|
+
|
|
683
|
+
- `gen4_turbo` - Fast generation with flexible parameters (Image-to-Video)
|
|
684
|
+
- `veo3.1` - High-quality with audio support and optional end frames (Image/Text-to-Video)
|
|
685
|
+
- `veo3.1_fast` - Faster variant of Veo 3.1 (Image/Text-to-Video)
|
|
686
|
+
- `gen3a_turbo` - Alternative model with different aspect ratios (Image-to-Video)
|
|
687
|
+
- `veo3` - Stable model with 8-second duration (Image/Text-to-Video)
|
|
688
|
+
- `act_two` - Character performance control (Character Performance)
|
|
689
|
+
- `eleven_text_to_sound_v2` - Sound effect generation from text (Sound Effects)
|
|
690
|
+
- `eleven_multilingual_sts_v2` - Speech-to-speech conversion (Speech-to-Speech)
|
|
691
|
+
- `eleven_multilingual_v2` - Text-to-speech generation (Text-to-Speech)
|
|
692
|
+
- `eleven_voice_dubbing` - Voice dubbing to different languages (Voice Dubbing)
|
|
693
|
+
- `eleven_voice_isolation` - Voice isolation from background audio (Voice Isolation)
|
|
694
|
+
|
|
695
|
+
Each model has different capabilities, supported ratios, and parameters. Refer to the [RunwayML API documentation](https://docs.dev.runwayml.com/api) for model-specific requirements.
|
|
696
|
+
|
|
697
|
+
### Supported Image Formats
|
|
698
|
+
|
|
699
|
+
The gem supports the following image formats:
|
|
700
|
+
|
|
701
|
+
- **JPEG** (`.jpg`, `.jpeg`)
|
|
702
|
+
- **PNG** (`.png`)
|
|
703
|
+
- **WebP** (`.webp`)
|
|
704
|
+
|
|
705
|
+
GIF images are not supported.
|
|
706
|
+
|
|
707
|
+
### Validation
|
|
708
|
+
|
|
709
|
+
The gem performs comprehensive client-side validation before making API calls:
|
|
710
|
+
|
|
711
|
+
```ruby
|
|
712
|
+
# Example with Image-to-Video
|
|
713
|
+
begin
|
|
714
|
+
task = RunwayML.image_to_video(
|
|
715
|
+
model: 'gen4_turbo',
|
|
716
|
+
prompt_image: 'image.jpg',
|
|
717
|
+
prompt_text: '', # Invalid: empty text
|
|
718
|
+
ratio: '999:999', # Invalid: unsupported ratio
|
|
719
|
+
duration: 100 # Invalid: out of range
|
|
720
|
+
)
|
|
721
|
+
rescue RunwayML::ValidationError => e
|
|
722
|
+
puts e.message
|
|
723
|
+
# => Validation failed:
|
|
724
|
+
# - prompt_text: cannot be empty
|
|
725
|
+
# - ratio: must be one of: 1280:720, 720:1280, 1104:832, 832:1104, 960:960, 1584:672
|
|
726
|
+
# - duration: must be between 2 and 10 seconds
|
|
727
|
+
|
|
728
|
+
# Access individual errors
|
|
729
|
+
puts e.errors
|
|
730
|
+
# => { prompt_text: "cannot be empty", ratio: "must be one of: ...", ... }
|
|
731
|
+
end
|
|
732
|
+
|
|
733
|
+
# Example with Character Performance
|
|
734
|
+
begin
|
|
735
|
+
task = RunwayML.character_performance(
|
|
736
|
+
model: 'act_two',
|
|
737
|
+
character: { type: 'invalid', uri: 'image.jpg' }, # Invalid type
|
|
738
|
+
reference: { type: 'video', uri: 'https://example.com/performance.mp4' },
|
|
739
|
+
ratio: '999:999' # Invalid: unsupported ratio
|
|
740
|
+
)
|
|
741
|
+
rescue RunwayML::ValidationError => e
|
|
742
|
+
puts e.errors
|
|
743
|
+
# => { character: "...", ratio: "must be one of: ..." }
|
|
744
|
+
end
|
|
745
|
+
```
|
|
746
|
+
|
|
747
|
+
### Error Handling
|
|
748
|
+
|
|
749
|
+
The gem provides several error classes for handling different failure scenarios. All errors inherit from `RunwayML::Error`:
|
|
750
|
+
|
|
751
|
+
```ruby
|
|
752
|
+
begin
|
|
753
|
+
task = RunwayML.image_to_video(
|
|
754
|
+
model: 'gen4_turbo',
|
|
755
|
+
prompt_image: 'image.jpg',
|
|
756
|
+
prompt_text: 'A beautiful scene',
|
|
757
|
+
ratio: '1280:720',
|
|
758
|
+
duration: 5
|
|
759
|
+
)
|
|
760
|
+
rescue RunwayML::Error => e
|
|
761
|
+
puts "Error: #{e.message}"
|
|
762
|
+
end
|
|
763
|
+
```
|
|
764
|
+
|
|
765
|
+
**Available Error Classes:**
|
|
766
|
+
|
|
767
|
+
- `RunwayML::ValidationError` - Client-side parameter validation failed
|
|
768
|
+
- `RunwayML::BadRequestError` - API validation errors (includes formatted validation issues)
|
|
769
|
+
- `RunwayML::AuthenticationError` - Invalid or missing API key
|
|
770
|
+
- `RunwayML::RateLimitError` - Too many requests (includes `retry_after`)
|
|
771
|
+
- `RunwayML::SSLError` - SSL certificate verification failed
|
|
772
|
+
- `RunwayML::APIConnectionError` - Network connectivity issues
|
|
773
|
+
- `RunwayML::APIError` - Other API errors
|
|
774
|
+
|
|
775
|
+
The gem performs client-side validation before making API calls (raises `ValidationError`), but the API may also perform additional server-side validation (raises `BadRequestError` with detailed error information).
|
|
776
|
+
|
|
777
|
+
## Development
|
|
778
|
+
|
|
779
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
780
|
+
|
|
781
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
782
|
+
|
|
783
|
+
## Contributing
|
|
784
|
+
|
|
785
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/oriolgual/runway-ruby.
|
|
786
|
+
|
|
787
|
+
## License
|
|
788
|
+
|
|
789
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|