videodraft 0.25.0 → 0.25.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "videodraft",
|
|
3
|
-
"version": "0.25.
|
|
3
|
+
"version": "0.25.2",
|
|
4
4
|
"description": "Official VideoDraft CLI for AI videos, images, audio and 3D assets. Agent-friendly: --json everywhere, stable exit codes, async job polling.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
package/skills/index.json
CHANGED
|
@@ -12,18 +12,18 @@
|
|
|
12
12
|
},
|
|
13
13
|
{
|
|
14
14
|
"path": "references/editor.md",
|
|
15
|
-
"sha256": "
|
|
16
|
-
"bytes":
|
|
15
|
+
"sha256": "f5f3280c4d6d2cbf6f4849ebeeb0857052aa7d605fa1c1ff69a6f46d122e8f3e",
|
|
16
|
+
"bytes": 23734
|
|
17
17
|
},
|
|
18
18
|
{
|
|
19
19
|
"path": "references/examples.md",
|
|
20
|
-
"sha256": "
|
|
21
|
-
"bytes":
|
|
20
|
+
"sha256": "a899b9cbd890cd4645bbb4ffe23edcf18cd188175f1dbc0a47820643a1ce2112",
|
|
21
|
+
"bytes": 10103
|
|
22
22
|
},
|
|
23
23
|
{
|
|
24
24
|
"path": "references/models.md",
|
|
25
|
-
"sha256": "
|
|
26
|
-
"bytes":
|
|
25
|
+
"sha256": "1a2f7feb34e5af588c180e64bb19ddf169c1aedc0bd0511c4293378a7fb7cf26",
|
|
26
|
+
"bytes": 45577
|
|
27
27
|
},
|
|
28
28
|
{
|
|
29
29
|
"path": "references/pipeline.md",
|
|
@@ -32,8 +32,8 @@
|
|
|
32
32
|
},
|
|
33
33
|
{
|
|
34
34
|
"path": "SKILL.md",
|
|
35
|
-
"sha256": "
|
|
36
|
-
"bytes":
|
|
35
|
+
"sha256": "c9ce31ecb4cb216108d33e62b2786a9e1cfedef89a399292fc1c3966e3b26903",
|
|
36
|
+
"bytes": 43425
|
|
37
37
|
}
|
|
38
38
|
]
|
|
39
39
|
}
|
|
@@ -27,13 +27,13 @@ Cloud generation has two equivalent surfaces (same backend, credits, and hosted
|
|
|
27
27
|
- Asset lane: `videodraft generate ...`, `videodraft edit video|motion`, `videodraft avatar ...`, `videodraft upscale ...`, `videodraft interpolate ...`, `videodraft upload`, and `videodraft download`.
|
|
28
28
|
- Full API access: `videodraft tools schema <name>`, `videodraft call <tool> --args '<json>'`.
|
|
29
29
|
2. **MCP connector**: if VideoDraft MCP tools (e.g. `generate_storyboard_from_idea`) are available, call them directly — the CLI's curated commands map 1:1 onto these tools.
|
|
30
|
-
3. **Native editor MCP** (`videodraft_editor`): prefer this for project production, timeline assembly, cutting, layouts, transitions, captions, audio placement, and final export. Inside VideoDraft ADE on a supported Mac, Claude and
|
|
30
|
+
3. **Native editor MCP** (`videodraft_editor`): prefer this for project production, timeline assembly, cutting, layouts, transitions, captions, audio placement, and final export. Inside VideoDraft ADE on a supported Mac, Claude, Codex, OpenCode and Grok receive it automatically in both Code and VideoDraft modes. It runs headlessly, so an Open Editor click is not required. Start with `project_manage` (`operation: "project"`, action `list`, `open`, or `create`); standalone asset generation remains in the cloud CLI or MCP.
|
|
31
31
|
|
|
32
|
-
|
|
32
|
+
Send native editor edits serially, batching everything a request needs into one `edit_apply` recipe. Pass the latest `context` when a person may be editing at the same time, so a stale write is refused. See [references/editor.md](references/editor.md) for project selection, media import, timing units, edit recipes, verification, export, and the `videodraft-editor` terminal bridge.
|
|
33
33
|
|
|
34
34
|
If you are reading this skill through `videodraft skills show skill`, run `videodraft skills show editor` before native editor work to load that reference.
|
|
35
35
|
|
|
36
|
-
**VideoDraft ADE routing rule:** the presence of `videodraft_editor` means the native editor is ready, even when no editor window is visible. Use cloud tools to generate or source assets and, when helpful, scripts or storyboards. Do not call hosted `produce_project` / `videodraft produce` or `export_video` / `videodraft export` by default. Import the assets into the native project, assemble there, and
|
|
36
|
+
**VideoDraft ADE routing rule:** the presence of `videodraft_editor` means the native editor is ready, even when no editor window is visible. Use cloud tools to generate or source assets and, when helpful, scripts or storyboards. Do not call hosted `produce_project` / `videodraft produce` or `export_video` / `videodraft export` by default. Import the assets into the native project, assemble there, and export with native `delivery_manage` `submit`. Use hosted production/export only when the user explicitly asks for the web workflow or the native editor tools are unavailable. Do not silently fall back to hosted production after a native tool error.
|
|
37
37
|
|
|
38
38
|
## First decision: asset, hosted project, or native edit?
|
|
39
39
|
|
|
@@ -45,7 +45,7 @@ If you are reading this skill through `videodraft skills show skill`, run `video
|
|
|
45
45
|
- **A generated multi-scene video / ad / explainer**: when the editor is available, use hosted tools only for any needed script, storyboard, shot planning, or generated assets; stop before hosted production, import the assets, and build/export the native timeline. A hosted project is optional unless the user wants the web project or its storyboard workflow.
|
|
46
46
|
- **A hosted web project or hosted export**: use the hosted pipeline only when the user explicitly asks for it or the native editor is unavailable.
|
|
47
47
|
- **Just a script** (no video asked for): A script-only request creates a script-stage project but stops at the script. Use `videodraft create "..." --script-only`; do not build a storyboard the user didn't ask for.
|
|
48
|
-
- **Iterating on existing work**: identify the surface first. Use `
|
|
48
|
+
- **Iterating on existing work**: identify the surface first. Use `project_manage` `project` with `action: "list"` for native projects and `videodraft projects list` only for hosted work. Never create a replacement project just to change an existing one.
|
|
49
49
|
|
|
50
50
|
## Choose the model from the task
|
|
51
51
|
|
|
@@ -101,7 +101,7 @@ Every `videodraft models image|video|audio --json` entry carries `tier` (1 or 2)
|
|
|
101
101
|
**Audio and utilities:**
|
|
102
102
|
|
|
103
103
|
- Use Seed Audio 1.0 for open-ended text-to-audio, speech/music/sound synthesis, voice conditioning, or prompt-driven editing with up to three audio references or one image. Use `videodraft generate audio`. Reference clips are `@Audio1`, `@Audio2`, and `@Audio3` in array order. There is no duration input. Output is up to two minutes and settles at 19 credits per actual minute, with up to 38 credits reserved during generation. The CLI automatically retries transient responses with one operation key. To recover after the CLI process itself is interrupted, set `--idempotency-key <uuid>` on the original command and reuse it.
|
|
104
|
-
- Prefer ElevenLabs for voiceover, dialogue, voice changing, dubbing, and sound effects. Honor an explicitly selected supported TTS voice/provider. Use `lyria-3.5` for all music, short or long (up to ~3 minutes, 10 credits flat), with vocals/lyrics or instrumental arrangements
|
|
104
|
+
- Prefer ElevenLabs for voiceover, dialogue, voice changing, dubbing, and sound effects. Honor an explicitly selected supported TTS voice/provider. Use `lyria-3.5` for all music, short or long (up to ~3 minutes, 10 credits flat), with vocals/lyrics or instrumental arrangements. When `--model` is omitted the CLI sends no model and the server default applies, which is `lyria-3.5` wherever the backend supports it. If the server answers that `lyria-3.5` is an unknown model, that backend predates it: rerun without `--model` (the server default then applies), or use `lyria-3-pro-preview` for a track longer than 30 seconds. Ask for the length in the prompt. Keep `lyria-3-clip-preview` (fixed ~30s, 4 credits) and `lyria-3-pro-preview` (8 credits) for explicit requests. For Lyria, put desired length, lyrics and "instrumental only, no vocals" in the prompt; `--length` and `--instrumental` are ElevenLabs-only. Use ElevenLabs Music for exact timing, composition plans, or a style reference track. Lyria allows 10 reference images on Google, 1 on Fal BYOK; Fal 3.5 prompts are limited to 5000 characters. BYOK uses zero VideoDraft credits. ElevenLabs Music means `elevenlabs-music-v2.5` (`elevenlabs-music` is an alias for it); use `elevenlabs-music-v1` only when the user asks for v1 by name.
|
|
105
105
|
- For ElevenLabs voiceover, dialogue, and voice changing, accept a supplied raw voice ID (16-64 alphanumeric characters) or `elevenlabs-<id>`. The voice catalog is for discovery, not an allowlist. Do not reject or substitute a supplied ID because it is absent from `videodraft models voices` / `list_available_voices`. Use `--voice <id>` for voiceover and voice changing, or repeat `--line "<id>:Text"` for dialogue; for example, `--voice kPzsL2i3teMYv0FxEYQ6` and `--line "elevenlabs-kPzsL2i3teMYv0FxEYQ6:Hello."` use the same voice. The voice must be accessible to the provider account used for generation. Private or cloned voices may require the user's connected ElevenLabs key. Kling video-control IDs and MiniMax `custom-*` IDs are separate voice systems.
|
|
106
106
|
- A character who needs to TALK:
|
|
107
107
|
- **Speaking inside a scene**, or any ordinary clip with dialogue: `generate video` with the line in the prompt, in quotes, with who says it and how. `gemini-omni-1.1-flash`, `seedance-2.5`, `seedance-2`, `kling-3.0` and `kling-o3` all voice it natively. For a specific voice use a Seedance `--ref-audio` clip or a Kling voice bound per element. This is the default path; do not generate speech separately and lip-sync it on.
|
|
@@ -192,7 +192,7 @@ videodraft stock search "city skyline night" --min-duration 5 --orientation land
|
|
|
192
192
|
URL=$(videodraft stock import pexels:video:35379336 --quality hd --json | jq -r .url)
|
|
193
193
|
```
|
|
194
194
|
|
|
195
|
-
- Always two steps. A search result's `preview_url` is a thumbnail for judging the shot; never place it on a timeline, attach it to a shot or send it to a model. `import_stock_media` copies the file onto the VideoDraft CDN and returns the URL every other surface accepts, including the native editor's `
|
|
195
|
+
- Always two steps. A search result's `preview_url` is a thumbnail for judging the shot; never place it on a timeline, attach it to a shot or send it to a model. `import_stock_media` copies the file onto the VideoDraft CDN and returns the URL every other surface accepts, including the native editor's `library_manage` import.
|
|
196
196
|
- `--quality hd` (default) caps video at 1080p; `4k` caps at 2160p, `best` takes the largest the provider has, `sd` suits rough cuts. Stills ignore quality and import at full size. Use `--orientation portrait` for 9:16, and `--min-resolution 1920` to drop anything below HD on its long edge.
|
|
197
197
|
- Photos come from Pexels. Pixabay contributes video only, because its full-size image host refuses server-side downloads.
|
|
198
198
|
- Credit the creator and link the provider page when you show results or deliver the finished work; both come back on every result. Skip clips that imply a person or brand endorses the product, and avoid recognisable logos in ads.
|
|
@@ -202,7 +202,7 @@ URL=$(videodraft stock import pexels:video:35379336 --quality hd --json | jq -r
|
|
|
202
202
|
|
|
203
203
|
VideoDraft ships no downloader. When the user asks to pull a video from YouTube, Instagram, TikTok, X, LinkedIn or anywhere else, `yt-dlp` on the user's own machine does the work. Check for it with `command -v yt-dlp` before promising anything.
|
|
204
204
|
|
|
205
|
-
**Present:** pin the format. On its defaults yt-dlp takes the best stream, usually VP9 or AV1 in WebM, which
|
|
205
|
+
**Present:** pin the format. On its defaults yt-dlp takes the best stream, usually VP9 or AV1 in WebM, which the native editor's import rejects (mp4, mov and m4v only). This returns one pre-muxed H.264 + AAC MP4 and needs no ffmpeg:
|
|
206
206
|
|
|
207
207
|
```bash
|
|
208
208
|
yt-dlp -f "b[ext=mp4]" -o "media/%(title)s.%(ext)s" "<url>"
|
|
@@ -246,9 +246,9 @@ Use the path you saved to: a **workspace-relative** path (`./media/clip.mp4`, or
|
|
|
246
246
|
When `videodraft_editor` is present:
|
|
247
247
|
|
|
248
248
|
1. Generate or source the script, storyboard, shot images, clips, voiceovers, music, and other assets through the cloud CLI/MCP as needed.
|
|
249
|
-
2. Call native `
|
|
250
|
-
3. Call native `
|
|
251
|
-
4. Call native `
|
|
249
|
+
2. Call native `project_manage` (`project`, action `open` or `create`) to open or create the `.vdproject`.
|
|
250
|
+
3. Call native `library_manage` `import`, wait for imports to become ready, then assemble and refine the timeline with `edit_apply` recipes.
|
|
251
|
+
4. Call native `delivery_manage` `submit` and use its `jobs` operation for progress and results.
|
|
252
252
|
|
|
253
253
|
Do not run the hosted production or export steps in this path unless the user explicitly asks for a web production.
|
|
254
254
|
|
|
@@ -12,40 +12,135 @@ Use cloud tools for asset generation and optional script/storyboard work, then i
|
|
|
12
12
|
|
|
13
13
|
- `videodraft` and the hosted VideoDraft MCP generate assets and can manage hosted web projects. They use the user's VideoDraft account and credits. In VideoDraft ADE, use them mainly as the source of generated media and optional storyboards for the native production.
|
|
14
14
|
- `videodraft_editor` edits local `.vdproject` packages. It has no generation, account, model, or credit tools.
|
|
15
|
-
- Inside VideoDraft ADE on a supported Mac, the editor MCP is injected automatically for Claude and
|
|
15
|
+
- Inside VideoDraft ADE on a supported Mac, the editor MCP is injected automatically for Claude, Codex, OpenCode and Grok in both Code and VideoDraft modes. It starts headlessly before the chat opens. The user does not need to click Open Editor, and closing or hiding the editor window does not stop headless editing.
|
|
16
16
|
- Outside that environment, use the editor only if `videodraft_editor` MCP tools are already exposed or the `videodraft-editor` executable is on PATH. Do not confuse the public `videodraft` cloud CLI with the separate native editor executable.
|
|
17
17
|
|
|
18
18
|
Prefer the direct MCP tools when they are available. The terminal bridge is useful for scripts, diagnostics, or an agent session where the MCP was not injected.
|
|
19
19
|
|
|
20
|
+
## Which editor tools you have
|
|
21
|
+
|
|
22
|
+
Current editors expose eight workflow tools: `project_manage`, `edit_snapshot`, `edit_apply`, `edit_undo`, `library_manage`, `inspect`, `speech_apply`, and `delivery_manage`. Use them whenever `edit_apply` is available. Everything below describes them.
|
|
23
|
+
|
|
24
|
+
Earlier editors expose a different set of tools. If `edit_apply` is not in your catalog, follow the live tool descriptions; the working rules below still apply.
|
|
25
|
+
|
|
26
|
+
The live tool descriptions and schemas are authoritative. When they differ from this reference, follow them.
|
|
27
|
+
|
|
28
|
+
Service tools (`project_manage`, `library_manage`, `inspect`, `speech_apply`, `delivery_manage`) take `operation` and `parameters`, plus an optional `context` (see [Keep a reliable editing model](#keep-a-reliable-editing-model)):
|
|
29
|
+
|
|
30
|
+
```json
|
|
31
|
+
{"operation": "import", "parameters": {"source": {"path": "/Users/me/clips"}}}
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
### Parameters at a glance
|
|
35
|
+
|
|
36
|
+
Each operation's schema has the full list. These are the fields that matter most, and the ones most often confused:
|
|
37
|
+
|
|
38
|
+
| Tool and operation | Key `parameters` |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| `project_manage` `project` | `action` (`list`, `open`, `create`, `close`); `id`, `name` or `path` to open; `name`, `fps`, `aspectRatio`, `quality` to create |
|
|
41
|
+
| `edit_snapshot` (no `parameters`) | `includeLibrary`, a `startFrame`/`endFrame` window, `offset`/`limit` paging, `context` for later pages |
|
|
42
|
+
| `edit_apply` (no `parameters`) | `actions`; optional `context`, `requestId`, `previewOnly` |
|
|
43
|
+
| `library_manage` `import` | `source` with one of `path`, `url`, `bytes` + `mimeType`, or `matte`; optional `name`, `folder`. A `.srt` or `.vtt` file becomes captions |
|
|
44
|
+
| `library_manage` `list` | `ids` to poll imports, `pending`, `folder` |
|
|
45
|
+
| `inspect` `timeline` | a `startFrame`/`endFrame` window; there is no clip filter |
|
|
46
|
+
| `inspect` `frame` | `startFrame` for one frame; add `endFrame` and `maxFrames` to sample a range |
|
|
47
|
+
| `inspect` `media` | `mediaRef` (a library asset, not a file path), `startSeconds`/`endSeconds` in source seconds, `wordTimestamps`, `overview` |
|
|
48
|
+
| `inspect` `transcript` | a `startFrame`/`endFrame` window, `granularity` (`words` or `segments`), `clipId` |
|
|
49
|
+
| `inspect` `color` | `clipId` with `atFrame`, or `mediaRef`; optional `reference` |
|
|
50
|
+
| `speech_apply` `words` | `words`: transcript word indices, each one index or a `[first, last]` pair; or `matches`: exact words to remove everywhere; `pacing` |
|
|
51
|
+
| `speech_apply` `silence` | none |
|
|
52
|
+
| `speech_apply` `captions` | caption `style`, `position` and look fields; it finds the speech itself. `style` `plain` (sentences without a preset) takes `positionY` or `transform.centerY`, not `position` or `punctuation` |
|
|
53
|
+
| `delivery_manage` `submit` | `mode`, `codec`, `resolution`, `outputPath` (its folder must already exist); `captionGroupId` and `wordTiming` for `srt` and `vtt`; `check: false` skips a video's export check |
|
|
54
|
+
| `delivery_manage` `jobs` | `action` `list` (every job with its progress, warnings, output path and check findings; with a `jobId`, that job with all its findings), `cancel` with a `jobId`, or `dismiss` with a `jobId` and optional `findingIds` |
|
|
55
|
+
|
|
56
|
+
`previewOnly` exists only on `edit_apply`. Service operations apply immediately.
|
|
57
|
+
|
|
58
|
+
### `edit_apply` actions at a glance
|
|
59
|
+
|
|
60
|
+
Every action puts its fields beside `kind`, for example `{"kind": "adjust", "clipId": "c1", "speed": 2}`. `remove`, `adjust`, `replace`, `transition`, `mask`, `text`, `grade` and `effects` take `clipId` for one clip or `clipIds` for several; `animate`, `extract`, and the rows of `move` and `split` take a single `clipId`. Advanced actions cannot take `as`, but their clip ids accept `@name`. The one exception is `transition`, whose own `kind` field names the style, so its fields go inside `parameters`: `{"kind": "transition", "parameters": {"clipId": "c2", "kind": "dipToBlack"}}`. These `actions` place a clip, warm it and add a vignette:
|
|
61
|
+
|
|
62
|
+
```json
|
|
63
|
+
[{"kind": "place", "assetId": "…", "atFrame": 0, "as": "shot"},
|
|
64
|
+
{"kind": "grade", "clipId": "@shot", "adjustments": {"temperature": 7500}},
|
|
65
|
+
{"kind": "effects", "clipId": "@shot", "effects": [{"type": "finish.vignette", "params": {"strength": 35}}]}]
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
| Advanced `kind` | Key fields |
|
|
69
|
+
| --- | --- |
|
|
70
|
+
| `place_batch` | `entries: [{mediaRef, startFrame, endFrame or source, trackIndex}]`, not the concise `assetId`, `atFrame`, `durationFrames`; leave `trackIndex` off every entry for a new track |
|
|
71
|
+
| `insert_batch` | `trackIndex`, `atFrame`, `entries: [{mediaRef, durationFrames or source}]`; later clips move right |
|
|
72
|
+
| `move` | `moves: [{clipId, toFrame, toTrack}]` |
|
|
73
|
+
| `remove` | `clipId` or `clipIds`; leaves a gap |
|
|
74
|
+
| `split` | `splits: [{clipId, atFrame}]`, or `trackIndex` with `frames` |
|
|
75
|
+
| `extract` | `trackIndex` with `ranges: [[start, end]]` in frames, or `clipId` with `ranges` and `units` (`frames`, or source `seconds`); closes the gap |
|
|
76
|
+
| `adjust` | `clipId` or `clipIds` plus any of `durationFrames`, `trimStartFrame`, `trimEndFrame`, `speed`, `speedCurve` (`{preset}`, or `{points: [{t, rate}]}` with `t` from 0 to 1 along the source), `preservesPitch`, `volume`, `opacity`, `transform` (`centerX`, `centerY`, `width`, `height`, `flipHorizontal`, `flipVertical`), `blendMode` |
|
|
77
|
+
| `animate` | `clipId`, `property` (`volume`, `opacity`, `rotation`, `position`, `scale`, `crop`), `keyframes: [[frame, ...values]]` with frames counted from the clip's start; `position` is the top-left corner |
|
|
78
|
+
| `arrange` | `layout`, `slots: [{slot, clipIds or mediaRef, anchor}]`, `fit` (`fill` or `fit`); `mediaRef` slots also need `endFrame` |
|
|
79
|
+
| `transition` (fields inside `parameters`) | `clipId` or `clipIds` (the clip after each cut), `kind` (`crossDissolve`, `dipToBlack`, `dipToWhite`, `blurDissolve`, `push`, `linearWipe`, `whipPan`, `crossZoom`, or `none` to remove), `durationFrames`, `params` (`direction` 0 to 3 for left, right, up, down; `feather`; `blur`; `intensity`) |
|
|
80
|
+
| `mask` | `clipId` or `clipIds`, `shape` (`rectangle`, `ellipse`, `none`), `centerX`, `centerY`, `width`, `height` (0 to 1 of the clip's own frame), `feather`, `strength`, `inverted` |
|
|
81
|
+
| `replace` | `clipId` or `clipIds`, `mediaRef` (a library asset of the same kind), `trim` (`keep`, the default, or `reset`), `linkedAudio` (`follow`, the default, or `keep`) |
|
|
82
|
+
| `tracks` | `reorder: [{trackId, to}]`, `set: [{trackId, muted, hidden, syncLocked}]`, `remove: [{trackId}]`; there is no add, since placing without a track makes one |
|
|
83
|
+
| `titles` | `entries: [{startFrame, endFrame, content}]`, optionally with `trackIndex` (on every entry or none), `style`, `animation`, `transform` and typography (`fontName`, `fontSize`, `color`) |
|
|
84
|
+
| `text` | `clipId`, `clipIds` or `captionGroupId`, with `content`, `style`, `position`, `animation` or typography |
|
|
85
|
+
| `grade` | `clipId` or `clipIds`, `adjustments`, `wheels`, `curves`, `hueCurves`, `lut`, `reset` |
|
|
86
|
+
| `effects` | `clipId` or `clipIds`, `effects: [{type, params, enabled}]`, `remove: [type]` |
|
|
87
|
+
|
|
88
|
+
- `grade`: `adjustments` holds `exposure` (EV, -4 to 4), `temperature` (kelvin, 1800 to 15000, 6500 neutral, higher is warmer), `tint` (-150 magenta to 150 green); `contrast`, `highlights`, `shadows`, `whites`, `blacks`, `vibrance` and `saturation` are signed percents (-100 to 100, 0 neutral), not factors like 1.2. `wheels`: `shadows`, `midtones`, `highlights`, each `{hue, strength, brightness}`. `curves`: `luma`, `red`, `green`, `blue` as `[x, y]` points from 0 to 1. `hueCurves`: `targets: [{hue, rotate, saturation, lightness}]`. `lut`: `{path, mix}`, with the `storedPath` from `prepare_look`. Out-of-range values are refused.
|
|
89
|
+
- Effect `type` ids and their knobs, which go inside `params` and run 0 to 100 unless noted. Defaults leave the picture unchanged, so send the knob you want to see:
|
|
90
|
+
- `finish.vignette`: `strength` and `curvature` (-100 to 100; positive `strength` darkens the edges), `size`, `falloff`
|
|
91
|
+
- `finish.grain`: `strength`, `grainSize` (0.5 to 6 px); `finish.glow`: `strength`, `haloRadius` (px), `cutoff`, `halation`
|
|
92
|
+
- `defocus.gaussian`: `blurRadius` (px); `defocus.motion`: `streakLength` (px), `streakAngle` (-180 to 180 degrees)
|
|
93
|
+
- `texture.clarity`: `localContrast`, `hazeRemoval` (-100 to 100); `texture.sharpen`: `strength` (0 to 200); `texture.denoise`: `strength`
|
|
94
|
+
- `matte.chroma`: `screenHue` (0 to 360 degrees, 120 is green), `range`, `edgeSoftness`, `spillSuppression`
|
|
95
|
+
- `arrange` layouts and their slots (fill every slot): `fullscreen` (`stage`); `split` (`left`, `right`); `stack` (`top`, `bottom`); `corner_top_left`, `corner_top_right`, `corner_bottom_left`, `corner_bottom_right` (`stage`, `corner`); `quad` (`top_left`, `top_right`, `bottom_left`, `bottom_right`); `side_panel` (`stage`, `panel`); `thirds` (`left`, `middle`, `right`). The layout picks the corner; `anchor` only biases the crop.
|
|
96
|
+
- `trackIndex` and `toTrack` are an existing track's `order` from `edit_snapshot` (0 draws on top), counted at that point in the recipe: a track an earlier action adds shifts them. `tracks` takes `trackId` handles instead.
|
|
97
|
+
|
|
20
98
|
## Start with the intended project
|
|
21
99
|
|
|
22
100
|
An MCP session can begin without a project selected. Project selection belongs to the session, not to whichever editor window happens to be frontmost.
|
|
23
101
|
|
|
24
|
-
1. If the user named an existing project but its identity is unclear, call `
|
|
25
|
-
2. Open the exact project
|
|
26
|
-
3. Create only when the user wants a new local edit. `action:
|
|
27
|
-
4. Treat `isActive` as this
|
|
28
|
-
5. Use `action:
|
|
102
|
+
1. If the user named an existing project but its identity is unclear, call `project_manage` with `operation: "project"` and `parameters.action: "list"`.
|
|
103
|
+
2. Open the exact project with `action: "open"` and the returned `id`, an unambiguous `name`, or the `.vdproject` `path`.
|
|
104
|
+
3. Create only when the user wants a new local edit. `action: "create"` accepts optional `name`, `fps`, `aspectRatio`, and `quality`.
|
|
105
|
+
4. Treat `isActive` as this session's target and `isVisible` as the project shown in the UI. Headless editing only needs the session target.
|
|
106
|
+
5. Use `action: "close"` only when closing is part of the task. It saves first and never deletes the project. Afterwards other calls answer `no_project` until you open or create a project again.
|
|
107
|
+
|
|
108
|
+
Other `project_manage` operations: `create_timeline` and `select_timeline` for additional timelines, and `configure` for project settings (a frame-rate change applies to every timeline).
|
|
29
109
|
|
|
30
110
|
Do not substitute a hosted project ID for a native project. A hosted project can supply scripts, storyboards, and generated media, but the native edit is a separate `.vdproject` package.
|
|
31
111
|
|
|
32
112
|
## Keep a reliable editing model
|
|
33
113
|
|
|
34
|
-
-
|
|
35
|
-
-
|
|
36
|
-
- Timeline
|
|
37
|
-
- IDs are short
|
|
38
|
-
-
|
|
39
|
-
-
|
|
40
|
-
-
|
|
41
|
-
- Use `
|
|
42
|
-
- Volume
|
|
114
|
+
- Opening or creating a project, and creating or selecting a timeline, returns `data.snapshot` (the `edit_snapshot` view with the library: format, tracks, clips and assets) and a `context`, so you can edit right away. Call `edit_snapshot` only after an out-of-band user edit, to page a long timeline with `offset` and `limit`, or for handles you lack.
|
|
115
|
+
- `context` (`epoch`, `timelineId`, `revision`) is optional on every write except `edit_undo` and `project_manage` `project`, which take none. Without it, a write applies to the current state. With it, the editor refuses the write if the project changed since that context, which is what you want when a person may be editing at the same time. Every reply returns a fresh `context`. After `context_expired` (a reopen or restart), take a new snapshot.
|
|
116
|
+
- Timeline positions and durations are frames at `format.fps`. `sourceSeconds` values are seconds in the source file. Pass values as returned; do not multiply or divide by fps yourself.
|
|
117
|
+
- IDs are short handles. Pass them back exactly as returned; do not derive them from UUIDs. Tracks keep stable handles; indexes can change.
|
|
118
|
+
- When the user says "this clip", "these captions", or "here", take a fresh `edit_snapshot` (a one-frame window is enough). Its `selection` names what they selected in the project's window, in the snapshot's handles: `clipIds`, `captionGroupIds`, a `gap`, the marked `range`, and library `mediaIds`; no `selection` means nothing is selected. `currentFrame` is the playhead. `visible: false` means the user is not looking at this project.
|
|
119
|
+
- Send writes serially. Parallel writes against one project race each other's context.
|
|
120
|
+
- A refused call answers `status: "rejected"`: nothing changed, so fix what the message names and send it again. A service write that answers `failed` may have applied before a save or connection failed; inspect before repeating it.
|
|
121
|
+
- Use `inspect` for detail and verification: `timeline` for exact clip and track properties, `frame` for rendered frames of the composited result, `media` before describing source content, `color` for scopes, and `transcript` to locate spoken words.
|
|
122
|
+
- Volume values, including volume keyframes, are linear from `0` to `1`.
|
|
123
|
+
|
|
124
|
+
## Edit with recipes
|
|
125
|
+
|
|
126
|
+
`edit_apply` takes an ordered list of `actions`, plus an optional `context` and `requestId`. The editor validates the whole recipe before changing anything, then applies it as one undo step. If any action fails, nothing changes and the error names the action. An applied reply lists the resulting `clips` (position, length, source window, text); use it to confirm the edit instead of re-reading.
|
|
127
|
+
|
|
128
|
+
- Concise actions: `place` (an `assetId` at `atFrame`, optionally on a `trackId`, with `durationFrames` or `sourceSeconds`, `mode` `overwrite` or `insert`), `trim` (a `clipId` to a `sourceSeconds` window), and `title` (`text` at `atFrame` for `durationFrames`, optional `look` and `motion`). Name an action with `as` and address its clip in later actions as `@name`.
|
|
129
|
+
- Advanced actions put their fields beside `kind` too: `place_batch`, `insert_batch`, `move`, `remove`, `split`, `extract`, `adjust`, `animate`, `arrange`, `transition` (fields inside `parameters`), `mask`, `replace`, `tracks`, `titles`, `text`, `grade`, and `effects`. Their field help lists every supported effect, grading control, mask, layout, speed curve, and caption control.
|
|
130
|
+
- Use `arrange` for split screens, picture-in-picture, grids, and canvas placement, and `tracks` to fix stacking. Do not synthesize layouts from generic transforms or keyframes.
|
|
131
|
+
- Use `replace` to swap a clip's media and keep its timing, effects, keyframes, and links, for example a re-render of the same shot. `trim: "keep"` holds the clip's source offsets; `trim: "reset"` plays the new media from its first frame and keeps linked audio in sync.
|
|
132
|
+
- Put every edit a request needs into one recipe: placing, trimming, titling, grading and effects together are one call and one undo step. Use `previewOnly: true` to validate a large or risky recipe without editing.
|
|
133
|
+
- Send a `requestId` when a retry must not edit twice: repeating the identical request with the same `requestId` returns the original receipt, even after a dropped connection. Without one, a repeated request applies again. Use a new `requestId` for a different edit.
|
|
134
|
+
- `applied_unsaved` means the edit applied but saving failed. If you sent your own `requestId`, repeat the identical request to retry the save, not the edit. Without one, do not resend: a repeat would apply the edit again.
|
|
135
|
+
- `edit_undo` reverses this connection's latest edit while it is still the top undo step. It never undoes the user's own edits. Take a new snapshot afterward.
|
|
136
|
+
|
|
137
|
+
Edits are undoable. Do not ask for confirmation before each ordinary edit. Ask one focused question only when the user's creative direction is materially ambiguous.
|
|
43
138
|
|
|
44
139
|
## Bring generated or local media into the editor
|
|
45
140
|
|
|
46
|
-
For b-roll and establishing shots, search free stock first (`videodraft stock search "<query>"`), import the pick with `videodraft stock import <ref>`, and feed the returned CDN URL to `
|
|
141
|
+
For b-roll and establishing shots, search free stock first (`videodraft stock search "<query>"`), import the pick with `videodraft stock import <ref>`, and feed the returned CDN URL to `library_manage` `import` as `source.url`. It costs no credits.
|
|
47
142
|
|
|
48
|
-
Otherwise use cloud generation for new assets, save or download the outputs, then call
|
|
143
|
+
Otherwise use cloud generation for new assets, save or download the outputs, then call `library_manage` with `operation: "import"`:
|
|
49
144
|
|
|
50
145
|
- `source.path`: absolute local file or directory. A directory imports recursively and preserves its folder structure.
|
|
51
146
|
- `source.url`: HTTPS asset URL. Set `mimeType` when a signed URL has no usable extension.
|
|
@@ -54,55 +149,61 @@ Otherwise use cloud generation for new assets, save or download the outputs, the
|
|
|
54
149
|
|
|
55
150
|
Readiness differs by source, and so does the poll that detects it:
|
|
56
151
|
|
|
57
|
-
- **URL and single-file path** imports return `status:
|
|
58
|
-
- **Directory** imports also return `status:
|
|
59
|
-
- **Inline bytes and matte** imports finish inline and come back `status:
|
|
152
|
+
- **URL and single-file path** imports return `status: "downloading"` with one `mediaRef`. Poll `library_manage` `list` with `parameters.ids: [mediaRef]` until `generationStatus` is absent.
|
|
153
|
+
- **Directory** imports also return `status: "downloading"` with one placeholder `mediaRef` for the batch. Poll it the same way; the folder's assets appear when `generationStatus` clears. (`pending: true` remains a fallback that lists every unresolved import.)
|
|
154
|
+
- **Inline bytes and matte** imports finish inline and come back `status: "ready"`; no polling is needed.
|
|
60
155
|
|
|
61
|
-
Never place a pending asset on the timeline. `generationStatus` is the signal: `preparing` and
|
|
62
|
-
`downloading` mean keep polling, absent means usable, and **`failed` is terminal** — report it or
|
|
63
|
-
retry the import explicitly, never poll on. Do not treat "not downloading" as ready.
|
|
156
|
+
An import's `mediaRef` is the `assetId` that recipe actions take. Never place a pending asset on the timeline. `generationStatus` is the signal: `preparing`, `generating`, `downloading`, and `rendering` mean keep polling, absent means usable, and **`failed` is terminal**. Report it or retry the import explicitly; never poll on. Do not treat "not downloading" as ready.
|
|
64
157
|
|
|
65
|
-
|
|
158
|
+
A `.srt` or `.vtt` caption file (by `path`, `url`, or `bytes` with `mimeType` `application/x-subrip`, `text/srt`, or `text/vtt`) is not added to the library. Its captions go onto a new caption track at the times the file gives, as one undo step, and the reply names the new `captionGroupId` to restyle with a `text` action.
|
|
66
159
|
|
|
67
|
-
|
|
160
|
+
For a batch of local outputs, download them into one workspace directory and import that directory once when practical. This is safer and faster than racing many import calls.
|
|
68
161
|
|
|
69
|
-
|
|
162
|
+
To apply a LUT, first store it with `library_manage` `prepare_look` (`parameters.path` to a `.cube` file), then use the returned `storedPath` in a `grade` action.
|
|
70
163
|
|
|
71
|
-
|
|
72
|
-
2. `timeline_read` and `media_list` to establish current state.
|
|
73
|
-
3. `media_view` when content selection matters.
|
|
74
|
-
4. Serialized clip, track, layout, text, caption, audio, color, effect, or cut mutations using the current revision.
|
|
75
|
-
5. `timeline_view` when visual composition or layer order matters.
|
|
76
|
-
6. `undo` if the requested result is wrong and the next mutation would not cleanly correct it.
|
|
164
|
+
## Speech edits
|
|
77
165
|
|
|
78
|
-
|
|
166
|
+
`speech_apply` runs speech work as separate operations, not recipe actions: `words` removes transcript words, `silence` trims dead air, and `captions` generates styled captions. Read a fresh transcript with `inspect` `transcript` after any speech edit, because word positions change.
|
|
167
|
+
|
|
168
|
+
## Service operations and timeouts
|
|
169
|
+
|
|
170
|
+
`project_manage`, `library_manage`, `speech_apply`, and `delivery_manage` writes have no retry receipts. After a timeout, inspect the outcome (a snapshot, a library `list`, or delivery `jobs`) before repeating a write. A failed service write may have applied an edit before saving failed, so read its `data` and the current state.
|
|
171
|
+
|
|
172
|
+
## Edit and verify
|
|
173
|
+
|
|
174
|
+
A dependable sequence:
|
|
175
|
+
|
|
176
|
+
1. `project_manage` to select or create the local project; its reply carries the snapshot.
|
|
177
|
+
2. `inspect` `media` when content selection matters.
|
|
178
|
+
3. One `edit_apply` recipe with every clip, track, layout, text, audio, color, and effect change the request needs; `speech_apply` for word cuts, silence, and captions.
|
|
179
|
+
4. Check the reply's `clips`; use `inspect` `frame` only when visual composition or layer order matters.
|
|
180
|
+
5. `edit_undo` if the result is wrong and the next recipe would not cleanly correct it.
|
|
79
181
|
|
|
80
182
|
## Export
|
|
81
183
|
|
|
82
|
-
`
|
|
184
|
+
Call `delivery_manage` with `operation: "submit"`. Submission is not completion: it returns a job, destination, and `started` or `queued` status.
|
|
83
185
|
|
|
84
|
-
- Use `video` for H.264, H.265, or ProRes.
|
|
85
|
-
- Use `xml` for Premiere Pro.
|
|
86
|
-
- Use `
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
- Use `
|
|
90
|
-
-
|
|
91
|
-
- Use `export_status` to list progress, warnings, and results. Cancel only when the user asks or the just-queued settings were wrong. Do not infer that an export is stuck from elapsed time alone.
|
|
186
|
+
- Use `mode: "video"` for H.264, H.265, or ProRes.
|
|
187
|
+
- Use `mode: "xml"` (XMEML) for Premiere Pro **and DaVinci Resolve**. Resolve reads XMEML natively. Use `fcpxml` only for Final Cut Pro; sending Resolve an FCPXML produces a package it cannot open cleanly.
|
|
188
|
+
- Use `mode: "videodraft"` for a self-contained project package.
|
|
189
|
+
- Use `mode: "srt"` or `mode: "vtt"` for a subtitle file of the captions: words and timing only, without styling. `captionGroupId` picks a caption group; `wordTiming: true` adds each word's start to a `vtt`.
|
|
190
|
+
- Omit `outputPath` unless the user named a destination; the default is `~/Downloads`. The editor does not create folders, so a named destination's folder must already exist.
|
|
191
|
+
- Use `operation: "jobs"` with `parameters.action` `list` to follow progress, warnings, and results. Cancel only when the user asks or the just-queued settings were wrong. Do not infer that an export is stuck from elapsed time alone.
|
|
192
|
+
- A finished video export is then checked in the background for black picture, gaps, sound that drops out or clips, and missing media. `list` shows each job's `findingCount` and its first three findings; `list` with that `jobId` returns every finding. Findings are warnings on a completed export: tell the user what was found and where, rather than treating the export as failed.
|
|
92
193
|
|
|
93
194
|
## Terminal bridge
|
|
94
195
|
|
|
95
|
-
VideoDraft desktop terminals expose `videodraft-editor`, which controls the same process and
|
|
196
|
+
VideoDraft desktop terminals expose `videodraft-editor`, which controls the same process and tools:
|
|
96
197
|
|
|
97
198
|
```bash
|
|
98
199
|
videodraft-editor status
|
|
99
200
|
videodraft-editor list-tools
|
|
100
|
-
videodraft-editor tool
|
|
101
|
-
videodraft-editor tool
|
|
201
|
+
videodraft-editor tool project_manage --json '{"operation":"project","parameters":{"action":"list"}}'
|
|
202
|
+
videodraft-editor tool edit_snapshot --project "/path/to/My Video.vdproject" --json '{"includeLibrary":true}'
|
|
102
203
|
videodraft-editor show
|
|
103
204
|
videodraft-editor hide
|
|
104
205
|
```
|
|
105
206
|
|
|
106
|
-
Each `videodraft-editor` invocation is a separate connection, so a project
|
|
207
|
+
Each `videodraft-editor` invocation is a separate connection, so a project opened by one call is NOT remembered by the next. Pass `--project <path>` on every `tool` call that operates on a project; without it, a follow-up call answers `no_project` (no project is open for this session) even though the open succeeded. For the same reason, `edit_undo` has nothing to undo from the terminal: it only reverses an edit made on its own connection.
|
|
107
208
|
|
|
108
209
|
Control and tool commands auto-start a headless editor if none is running. `show` only reveals the already-running UI. Use `videodraft-editor tool <name> --json -` to read a JSON object from stdin when shell quoting would be fragile. Never read, copy, or expose the editor's rotating local authentication secret.
|
|
@@ -69,7 +69,7 @@ videodraft generate audio "Extend @Audio1 into a 20-second transition" --ref-aud
|
|
|
69
69
|
videodraft export "$PROJECT" --download solace-launch.mp4
|
|
70
70
|
```
|
|
71
71
|
|
|
72
|
-
Use this complete hosted path only when the user requested a web project or the native editor is unavailable. Otherwise stop after the storyboard/assets, import them into the native `.vdproject`, and export with `
|
|
72
|
+
Use this complete hosted path only when the user requested a web project or the native editor is unavailable. Otherwise stop after the storyboard/assets, import them into the native `.vdproject`, and export with `delivery_manage`. The hosted project stays editable at the URL in `project.json` (`.urls`).
|
|
73
73
|
|
|
74
74
|
## 3. Talking-head (avatar) video
|
|
75
75
|
|
|
@@ -122,7 +122,7 @@ Kling O3 is also exposed for reference generation. `videodraft generate video --
|
|
|
122
122
|
- **Voiceover/TTS**: prefer ElevenLabs. Brittney is the platform default voice; under ElevenLabs BYOK, use a compatible voice from the user's account. Honor another supported voice/provider when the user explicitly selects it.
|
|
123
123
|
- **Dialogue, voice changing, and dubbing**: ElevenLabs only.
|
|
124
124
|
- **Sound effects**: ElevenLabs Sound Effects only.
|
|
125
|
-
- **Music**: use `lyria-3.5` for all music, short or long (up to ~3 minutes), with vocals/lyrics or instrumental music
|
|
125
|
+
- **Music**: use `lyria-3.5` for all music, short or long (up to ~3 minutes), with vocals/lyrics or instrumental music. When `--model` is omitted the CLI sends no model and the server default applies, which is `lyria-3.5` wherever the backend supports it. If the server answers that `lyria-3.5` is an unknown model, that backend predates it: rerun without `--model` (the server default then applies), or use `lyria-3-pro-preview` for a track longer than 30 seconds. `lyria-3-clip-preview` (fixed ~30s) and `lyria-3-pro-preview` remain available for explicit requests. Lyria length and structure are prompt-guided, not exact; for an instrumental, include "instrumental only, no vocals" in the prompt. `--length` and `--instrumental` only apply to ElevenLabs. Use `elevenlabs-music-v2.5` when a specified 3-300 second length, composition plan, or style reference track matters. Lyria references: up to 10 images on Google, 1 on Fal BYOK; 3.5 Fal prompts are 1-5000 characters (an image-only request gets a neutral prompt). `elevenlabs-music` is an alias for v2.5. Use `elevenlabs-music-v1` only when the user asks for v1 by name; it takes a prompt, `--length` and `--instrumental` only.
|
|
126
126
|
- **ElevenLabs Music v2.5 inputs**: either a prompt (`--length`, `--instrumental`) or a composition plan, never both. A plan is 1-30 sections of 3-120 seconds each, up to 300 seconds in total, and `--plan`/`--section` pick v2.5 when `--model` is omitted. Empty section parts default to 20 seconds and an instrumental part. Build one with repeatable `--section "<seconds>|<style, style>|<text>"` (use `\n` for line breaks) or pass `--plan plan.json` with `{"chunks":[{"text","duration_ms","positive_styles","negative_styles","context_adherence","audio_reference"}]}`. Section text is an optional `[Section name]`, lyric lines, and `{inline directions}`. Put 6-7 specific English styles on the first section; it sets the genre. `--ref-audio <url|file>` adds a style reference clip to the first section (window up to 30 seconds via `--ref-start`/`--ref-end` in ms, `--ref-strength low|medium|high|xhigh`); local files are uploaded, including relative `audio_reference.audio_url` paths in a plan file. `--seed` works only with a plan. `--format` picks the output (`mp3_48000_192` default on v2.5; `pcm_*`, `ulaw_8000` and `alaw_8000` arrive as stereo WAV files). Style references don't run on the user's own ElevenLabs key; with that key connected, drop the reference or ask the user to turn the key off. The CLI retries transient responses with one idempotency key; set `--idempotency-key <uuid>` to recover after an interruption.
|
|
127
127
|
- Voice Changer and Dubbing require the source media duration and currently accept source media up to 300 seconds.
|
|
128
128
|
|