videodraft 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/dist/client.d.ts +6 -0
- package/dist/client.js +130 -1
- package/dist/index.js +157 -19
- package/package.json +1 -1
- package/skills/index.json +14 -9
- package/skills/videodraft/SKILL.md +46 -15
- package/skills/videodraft/references/editor.md +104 -0
- package/skills/videodraft/references/examples.md +10 -4
- package/skills/videodraft/references/models.md +1 -1
- package/skills/videodraft/references/pipeline.md +3 -2
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "videodraft",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"description": "Official VideoDraft CLI — create AI videos, images and audio from your terminal. Agent-friendly: --json everywhere, stable exit codes, async job polling.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
package/skills/index.json
CHANGED
|
@@ -3,27 +3,32 @@
|
|
|
3
3
|
"skills": [
|
|
4
4
|
{
|
|
5
5
|
"name": "videodraft",
|
|
6
|
-
"description": "Create AI videos, images, Seed Audio, voiceovers, music, sound effects, dialogue, dubbing, storyboards, avatar videos, media upscales and product/ad videos with VideoDraft. Use
|
|
6
|
+
"description": "Create and edit AI videos, images, Seed Audio, voiceovers, music, sound effects, dialogue, dubbing, storyboards, avatar videos, media upscales, and product/ad videos with VideoDraft. Use whenever the user mentions VideoDraft; asks to generate a video, image, audio asset, ad, explainer, storyboard, avatar, upscale, or batch/CI workflow; or wants to assemble, cut, caption, mix, lay out, inspect, or export a native VideoDraft Editor timeline. Covers the cloud `videodraft` CLI/MCP and local headless `videodraft_editor` MCP. When the editor MCP is exposed, prefer it for production, timeline assembly, and export; use cloud production/export only when explicitly requested or the editor is unavailable.",
|
|
7
7
|
"files": [
|
|
8
|
+
{
|
|
9
|
+
"path": "references/editor.md",
|
|
10
|
+
"sha256": "9d29259e6b0cb1709a4d2277a53c9079da8d89c9e19757a342d63dbb345b0a23",
|
|
11
|
+
"bytes": 8659
|
|
12
|
+
},
|
|
8
13
|
{
|
|
9
14
|
"path": "references/examples.md",
|
|
10
|
-
"sha256": "
|
|
11
|
-
"bytes":
|
|
15
|
+
"sha256": "cfbf840eb862ba62dea08b0a8b660a29b6beadc7ea6d135d75d3bdc1ff93e4cb",
|
|
16
|
+
"bytes": 7408
|
|
12
17
|
},
|
|
13
18
|
{
|
|
14
19
|
"path": "references/models.md",
|
|
15
|
-
"sha256": "
|
|
16
|
-
"bytes":
|
|
20
|
+
"sha256": "51c61783ed264ddf9df67d730c161d8de11c3aa2f68e995f60201c144bc698df",
|
|
21
|
+
"bytes": 18320
|
|
17
22
|
},
|
|
18
23
|
{
|
|
19
24
|
"path": "references/pipeline.md",
|
|
20
|
-
"sha256": "
|
|
21
|
-
"bytes":
|
|
25
|
+
"sha256": "7544a253cc634bba935d091440410aef4205d0bb63cc259705a683ced7b40d42",
|
|
26
|
+
"bytes": 11613
|
|
22
27
|
},
|
|
23
28
|
{
|
|
24
29
|
"path": "SKILL.md",
|
|
25
|
-
"sha256": "
|
|
26
|
-
"bytes":
|
|
30
|
+
"sha256": "35277796315322ab074982ce3dc940d6c02c0fbf73104d374ebb5f99e70e5801",
|
|
31
|
+
"bytes": 20691
|
|
27
32
|
}
|
|
28
33
|
]
|
|
29
34
|
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: videodraft
|
|
3
|
-
description: Create AI videos, images, Seed Audio, voiceovers, music, sound effects, dialogue, dubbing, storyboards, avatar videos, media upscales and product/ad videos with VideoDraft. Use
|
|
3
|
+
description: Create and edit AI videos, images, Seed Audio, voiceovers, music, sound effects, dialogue, dubbing, storyboards, avatar videos, media upscales, and product/ad videos with VideoDraft. Use whenever the user mentions VideoDraft; asks to generate a video, image, audio asset, ad, explainer, storyboard, avatar, upscale, or batch/CI workflow; or wants to assemble, cut, caption, mix, lay out, inspect, or export a native VideoDraft Editor timeline. Covers the cloud `videodraft` CLI/MCP and local headless `videodraft_editor` MCP. When the editor MCP is exposed, prefer it for production, timeline assembly, and export; use cloud production/export only when explicitly requested or the editor is unavailable.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# VideoDraft
|
|
@@ -9,11 +9,12 @@ VideoDraft is an AI video creation platform where asset generation is the priori
|
|
|
9
9
|
|
|
10
10
|
- **Asset generation**: standalone images, video clips, Seed Audio, voiceovers, music, sound effects, dialogue, voice-changed audio, dubbed media, upscales, and image descriptions. This is the fastest and most important lane. Treat these as complete deliverables when the user asks for assets.
|
|
11
11
|
- **Asset I/O**: upload local files, download outputs, auto-upload local references, and save generated media where the user can see it.
|
|
12
|
-
- **
|
|
12
|
+
- **Native editing**: local `.vdproject` timelines, cuts, layouts, captions, effects, audio, and exports through the headless VideoDraft Editor. Inside VideoDraft ADE, this is the default production and export lane whenever `videodraft_editor` is available.
|
|
13
|
+
- **Hosted project production**: idea → script → storyboard → hosted production timeline → exported MP4. Use the early stages for scripts, storyboards, and generated assets when useful. Treat hosted production and export as a fallback when the native editor is unavailable, or as an explicit destination when the user asks for an editable web project or hosted workflow.
|
|
13
14
|
|
|
14
15
|
## How to connect
|
|
15
16
|
|
|
16
|
-
|
|
17
|
+
Cloud generation has two equivalent surfaces (same backend, credits, and hosted projects). Native timeline editing is a separate local surface:
|
|
17
18
|
|
|
18
19
|
1. **CLI** (preferred when you have a shell): run `videodraft` if it's on PATH; otherwise `npx -y videodraft@latest` runs it with no install (needs Node ≥20; the `-y` skips npx's install prompt so it runs non-interactively; the package is fetched on first use and cached). For heavy use, `npm install -g videodraft`. If there's no Node/shell here but the MCP connector below is available, use that instead; if neither works, tell the user how to install (https://videodraft.ai/cli).
|
|
19
20
|
- Auth — pick by context, don't guess:
|
|
@@ -25,17 +26,25 @@ Two equivalent surfaces (same backend, same credits, same projects):
|
|
|
25
26
|
- Asset lane: `videodraft generate ...`, `videodraft edit video|motion`, `videodraft avatar ...`, `videodraft upscale ...`, `videodraft upload`, and `videodraft download`.
|
|
26
27
|
- Full API access: `videodraft tools schema <name>`, `videodraft call <tool> --args '<json>'`.
|
|
27
28
|
2. **MCP connector**: if VideoDraft MCP tools (e.g. `generate_storyboard_from_idea`) are available, call them directly — the CLI's curated commands map 1:1 onto these tools.
|
|
29
|
+
3. **Native editor MCP** (`videodraft_editor`): prefer this for project production, timeline assembly, cutting, layouts, transitions, captions, audio placement, and final export. Inside VideoDraft ADE on a supported Mac, Claude and Codex receive it automatically in both Code and VideoDraft modes. It runs headlessly, so an Open Editor click is not required. Start with `manage_project` (`list`, `open`, or `create`); standalone asset generation remains in the cloud CLI or MCP.
|
|
28
30
|
|
|
29
|
-
|
|
31
|
+
Native editor mutations are revision-guarded. Send them serially and carry forward each result's fresh revision. See [references/editor.md](references/editor.md) for project selection, media import, timing units, mutation deltas, verification, export, and the `videodraft-editor` terminal bridge.
|
|
32
|
+
|
|
33
|
+
If you are reading this skill through `videodraft skills show skill`, run `videodraft skills show editor` before native editor work to load that reference.
|
|
34
|
+
|
|
35
|
+
**VideoDraft ADE routing rule:** the presence of `videodraft_editor` means the native editor is ready, even when no editor window is visible. Use cloud tools to generate or source assets and, when helpful, scripts or storyboards. Do not call hosted `produce_project` / `videodraft produce` or `export_video` / `videodraft export` by default. Import the assets into the native project, assemble there, and call native `export_project`. Use hosted production/export only when the user explicitly asks for the web workflow or the native editor tools are unavailable. Do not silently fall back to hosted production after a native tool error.
|
|
36
|
+
|
|
37
|
+
## First decision: asset, hosted project, or native edit?
|
|
30
38
|
|
|
31
39
|
- **One standalone asset** (image, clip, voiceover, music track, sound effect, dialogue track, voice-changed file, dubbed media file, upscale, or description): generate it directly. Do NOT create a project.
|
|
32
40
|
- `videodraft generate image "a red fox in snow, cinematic" --ar 16:9 --download ./out/`
|
|
33
41
|
- `videodraft generate video "slow dolly over a misty lake" --model gemini-omni-flash --duration 6 --download ./out/`
|
|
42
|
+
- **Any final video, production timeline, existing footage, local `.vdproject`, or hands-on edit**: use `videodraft_editor` when available. List or open the intended local project, or create a native project for a new production. The editor can work without showing its UI.
|
|
34
43
|
- **A small set of related assets**: still stay in the asset lane. Use an AI Studio session if you need to group related generations. Switch to a project only when the deliverable matches the project criteria below or the user asks to attach the assets to one.
|
|
35
|
-
- **A multi-scene video / ad / explainer, storyboard,
|
|
36
|
-
|
|
37
|
-
- **Just a script** (no video asked for): `videodraft create "..." --script-only
|
|
38
|
-
- **Iterating on existing work**:
|
|
44
|
+
- **A generated multi-scene video / ad / explainer**: when the editor is available, use hosted tools only for any needed script, storyboard, shot planning, or generated assets; stop before hosted production, import the assets, and build/export the native timeline. A hosted project is optional unless the user wants the web project or its storyboard workflow.
|
|
45
|
+
- **A hosted web project or hosted export**: use the hosted pipeline only when the user explicitly asks for it or the native editor is unavailable.
|
|
46
|
+
- **Just a script** (no video asked for): A script-only request creates a script-stage project but stops at the script. Use `videodraft create "..." --script-only`; do not build a storyboard the user didn't ask for.
|
|
47
|
+
- **Iterating on existing work**: identify the surface first. Use `manage_project` with `action:'list'` for native projects and `videodraft projects list` only for hosted work. Never create a replacement project just to change an existing one.
|
|
39
48
|
|
|
40
49
|
## Choose the model from the task
|
|
41
50
|
|
|
@@ -72,7 +81,7 @@ Pure text-to-image or text-to-video is fine for a generic one-off asset. When a
|
|
|
72
81
|
|
|
73
82
|
- If the user supplies reference media, preserve and pass it. Never reduce the request to text alone.
|
|
74
83
|
- When continuity matters, generate/select a strong still first with the selected image model (`nano-banana-2` by default), wait for its URL, then animate it as a start frame/reference. Confirm the combined image and video cost.
|
|
75
|
-
-
|
|
84
|
+
- When using a hosted storyboard stage for multiple shots, use `videodraft shots <project_id> --model <selected-image-model> --grid`, then animate the decoded shots. Preserve explicit models. In VideoDraft ADE, import the resulting assets into the native editor instead of continuing into hosted production. A requested non-Seedance video model must use manual per-shot generation instead of Seedance full-video mode.
|
|
76
85
|
|
|
77
86
|
## Cost and credits
|
|
78
87
|
|
|
@@ -84,7 +93,7 @@ For expensive work, estimate with `--estimate` or `videodraft costs`, state the
|
|
|
84
93
|
|
|
85
94
|
## Async jobs
|
|
86
95
|
|
|
87
|
-
Image/video generation is asynchronous: commands submit a job and **wait by default**, printing output URLs (and saving files with `--download`). In scripts/CI prefer explicit control:
|
|
96
|
+
Image/video generation is asynchronous: commands submit a job and **wait by default**, printing output URLs (and saving files with `--download`). Large downloaded images also get a downscaled copy in `previews/` next to them (the `preview` field / "inspect via preview" line in the output) — **look at the preview, deliver the original**; viewing full-resolution images bloats the chat permanently. In scripts/CI prefer explicit control:
|
|
88
97
|
|
|
89
98
|
```bash
|
|
90
99
|
JOB=$(videodraft generate image "..." --no-wait --json | jq -r .job_id)
|
|
@@ -105,7 +114,7 @@ URL=$(videodraft upload ./product.png --json | jq -r .url)
|
|
|
105
114
|
|
|
106
115
|
Never silently drop a reference you couldn't upload — stop and tell the user. Never upload a user's file to a third-party host.
|
|
107
116
|
|
|
108
|
-
When the user attaches media, classify each item before acting: a recurring **visual asset** (character/product/location/style), actual **footage to place as shots**, or **inspiration only**. See [references/pipeline.md](references/pipeline.md) for
|
|
117
|
+
When the user attaches media for a native production, import actual footage into the editor by default. For hosted generation/storyboarding, classify each item before acting: a recurring **visual asset** (character/product/location/style), actual **footage to place as shots**, or **inspiration only**. See [references/pipeline.md](references/pipeline.md) for the hosted role mapping.
|
|
109
118
|
|
|
110
119
|
## Showing media to the user
|
|
111
120
|
|
|
@@ -121,7 +130,24 @@ Put the Markdown link **in your message text** — video and audio embed exactly
|
|
|
121
130
|
|
|
122
131
|
Use the path you saved to: a **workspace-relative** path (`./media/clip.mp4`, or `./<any-folder>/clip.mp4` — any folder in the workspace works), or the **absolute** path for a file outside the workspace (e.g. `/Users/you/Desktop/clip.mp4` or another workspace's path). Both render. Show the finished results worth showing (and only those — not every intermediate job). A bare CDN URL or a JSON dump of output URLs does **not** render; the local-path Markdown link is what produces an inline card.
|
|
123
132
|
|
|
124
|
-
##
|
|
133
|
+
## Native-first VideoDraft ADE pipeline (idea → MP4)
|
|
134
|
+
|
|
135
|
+
When `videodraft_editor` is present:
|
|
136
|
+
|
|
137
|
+
1. Generate or source the script, storyboard, shot images, clips, voiceovers, music, and other assets through the cloud CLI/MCP as needed.
|
|
138
|
+
2. Call native `manage_project` to open or create the `.vdproject`.
|
|
139
|
+
3. Call native `import_media`, wait for imports to become ready, then assemble and refine the timeline with editor tools.
|
|
140
|
+
4. Call native `export_project` and use `manage_exports` for progress and results.
|
|
141
|
+
|
|
142
|
+
Do not run the hosted production or export steps in this path unless the user explicitly asks for a web production.
|
|
143
|
+
|
|
144
|
+
## Hosted fallback pipeline (idea → MP4)
|
|
145
|
+
|
|
146
|
+
Use this only when there is NO native editor at all, or the user explicitly requests the hosted web
|
|
147
|
+
workflow. The native surface is not only the injected `videodraft_editor` MCP: a `videodraft-editor`
|
|
148
|
+
executable on PATH is the same editor reached through its terminal bridge, and
|
|
149
|
+
[references/editor.md](references/editor.md) covers driving it that way. Treating a missing MCP as
|
|
150
|
+
"no editor" sends sessions that have the binary into hosted production for no reason.
|
|
125
151
|
|
|
126
152
|
```bash
|
|
127
153
|
videodraft create "<idea>" --ar 9:16 # project: script → visual assets → storyboard
|
|
@@ -133,14 +159,19 @@ videodraft export <project_id> --download final.mp4
|
|
|
133
159
|
|
|
134
160
|
Optional between produce and export: per-shot motion clips (`videodraft generate video ... --project <id>` then place it with `videodraft attach <project> --scene N --shot M --media <url|file> --type video --duration <s>`), music (`videodraft generate music "..." --attach <project_id>`), and standalone audio assets (`generate audio`, `generate sound-effect`, `generate dialogue`, `generate voice-changer`, `generate dub`). Details, per-step tools and editing rules: [references/pipeline.md](references/pipeline.md).
|
|
135
161
|
|
|
162
|
+
## Avatar and talking-head videos (both surfaces)
|
|
163
|
+
|
|
164
|
+
Avatar generation is cloud-only — the native editor has no avatar or lipsync tools — so this applies whether or not `videodraft_editor` is present. Generate the avatar in the cloud; in VideoDraft ADE, import the rendered clip and cut it on the native timeline like any other footage.
|
|
165
|
+
|
|
136
166
|
Avatar/talking-head videos use dedicated commands. For a reusable managed avatar, obtain or generate a clear portrait → `videodraft avatar script` when needed → `videodraft avatar create` → `videodraft avatar render --resolution 720p`. For a one-off portrait, use `videodraft avatar fabric <portrait> --text "..."` or `--audio <file>`. For an existing video plus replacement audio, use `videodraft avatar lipsync <video> --audio <file>`. Managed script/creation is bundled/free; direct Fabric, Sync, the managed Fabric render, and optional portrait generation/upscaling are paid. Confirm expensive steps first.
|
|
137
167
|
|
|
138
|
-
## Working with project data
|
|
168
|
+
## Working with hosted project data
|
|
139
169
|
|
|
140
|
-
A project is one JSON blob (script, storyboard scenes, shot cards, visual assets, production timeline). To inspect: `videodraft projects get <id>`. To edit: fetch `--raw`, modify, then `videodraft call update_project` — objects deep-merge, **arrays replace wholesale** (send the complete `storyboard.scenes` array to change one scene). Snapshot first with `videodraft checkpoint create <id>` before risky edits. Schema reference: `videodraft call get_project_schema`.
|
|
170
|
+
A hosted project is one JSON blob (script, storyboard scenes, shot cards, visual assets, production timeline). To inspect: `videodraft projects get <id>`. To edit: fetch `--raw`, modify, then `videodraft call update_project` — objects deep-merge, **arrays replace wholesale** (send the complete `storyboard.scenes` array to change one scene). Snapshot first with `videodraft checkpoint create <id>` before risky edits. Schema reference: `videodraft call get_project_schema`. This does not replace native editor tools when `videodraft_editor` is available for the production itself.
|
|
141
171
|
|
|
142
172
|
## More
|
|
143
173
|
|
|
144
|
-
- [references/pipeline.md](references/pipeline.md) —
|
|
174
|
+
- [references/pipeline.md](references/pipeline.md) — hosted fallback data model and production workflow
|
|
175
|
+
- [references/editor.md](references/editor.md) — native headless editor routing, project selection, import, timeline edits, verification, and export
|
|
145
176
|
- [references/models.md](references/models.md) — choosing image/video models, pricing patterns, voices and styles
|
|
146
177
|
- [references/examples.md](references/examples.md) — recipes: batch product videos from a CSV, talking-head from a script, changelog video in CI
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# Native VideoDraft Editor reference
|
|
2
|
+
|
|
3
|
+
Use this reference when the user wants to assemble, cut, caption, mix, lay out, inspect, or export a local VideoDraft Editor project. The native editor is deterministic and local. Cloud generation remains in the `videodraft` CLI or hosted MCP.
|
|
4
|
+
|
|
5
|
+
## VideoDraft ADE preference rule
|
|
6
|
+
|
|
7
|
+
When `videodraft_editor` tools are exposed, treat the native editor as available and make it the default surface for production, timeline assembly, and final export. It is headless by design, so a hidden window or an untouched Open Editor button does not justify using hosted production instead.
|
|
8
|
+
|
|
9
|
+
Use cloud tools for asset generation and optional script/storyboard work, then import the results. Do not call hosted `produce_project` / `videodraft produce` or `export_video` / `videodraft export` unless the user explicitly requests an editable web production or the native editor tools are unavailable. If a native tool call fails after the editor was available, report or recover that native failure rather than silently switching surfaces.
|
|
10
|
+
|
|
11
|
+
## Choose the correct surface
|
|
12
|
+
|
|
13
|
+
- `videodraft` and the hosted VideoDraft MCP generate assets and can manage hosted web projects. They use the user's VideoDraft account and credits. In VideoDraft ADE, use them mainly as the source of generated media and optional storyboards for the native production.
|
|
14
|
+
- `videodraft_editor` edits local `.vdproject` packages. It has no generation, account, model, or credit tools.
|
|
15
|
+
- Inside VideoDraft ADE on a supported Mac, the editor MCP is injected automatically for Claude and Codex in both Code and VideoDraft modes. It starts headlessly before the chat opens. The user does not need to click Open Editor, and closing or hiding the editor window does not stop headless editing.
|
|
16
|
+
- Outside that environment, use the editor only if `videodraft_editor` MCP tools are already exposed or the `videodraft-editor` executable is on PATH. Do not confuse the public `videodraft` cloud CLI with the separate native editor executable.
|
|
17
|
+
|
|
18
|
+
Prefer the direct MCP tools when they are available. The terminal bridge is useful for scripts, diagnostics, or an agent session where the MCP was not injected.
|
|
19
|
+
|
|
20
|
+
## Start with the intended project
|
|
21
|
+
|
|
22
|
+
An MCP session can begin without a project selected. Project selection belongs to the session, not to whichever editor window happens to be frontmost.
|
|
23
|
+
|
|
24
|
+
1. If the user named an existing project but its identity is unclear, call `manage_project` with `action:'list'`.
|
|
25
|
+
2. Open the exact project by the returned `id`, unambiguous `name`, or `.vdproject` `path`.
|
|
26
|
+
3. Create only when the user wants a new local edit. `action:'create'` accepts optional `name`, `fps`, `aspectRatio`, and `quality`.
|
|
27
|
+
4. Treat `isActive` as this MCP session's target and `isVisible` as the project shown in the UI. Headless editing only needs the session target.
|
|
28
|
+
5. Use `action:'close'` only when closing is part of the task. It saves first and never deletes the project.
|
|
29
|
+
|
|
30
|
+
Do not substitute a hosted project ID for a native project. A hosted project can supply scripts, storyboards, and generated media, but the native edit is a separate `.vdproject` package.
|
|
31
|
+
|
|
32
|
+
## Keep a reliable editing model
|
|
33
|
+
|
|
34
|
+
- Call `get_timeline` once after opening or creating a project, after switching timelines, or after an out-of-band user edit. It returns the revision and current clip/track state.
|
|
35
|
+
- Call `get_media` before using a `mediaRef`. Poll imports with a filtered read (`ids` for a known asset, `pending:true` for a batch) instead of repeatedly loading the full library.
|
|
36
|
+
- Timeline placement uses project frames. Source spans, media durations, transcript segments, search hits, and beat times use seconds. Pass those values to the relevant tools as returned; do not multiply by fps yourself.
|
|
37
|
+
- IDs are short stable prefixes. Pass them back exactly as returned. Tracks use stable `trackId` values; indexes can change.
|
|
38
|
+
- Send project mutations serially. Pass `expectedRevision` from the latest read or mutation when available, then replace it with the fresh revision from the next result. Parallel edits against one project can race or invalidate each other's revision.
|
|
39
|
+
- Every mutation returns a delta in `get_timeline` vocabulary. Patch your working model from that delta instead of re-reading after every successful call. Re-read after a stale-state failure or an out-of-band change.
|
|
40
|
+
- Use `apply_layout` for split screens, picture-in-picture, grids, and canvas placement. Use `manage_tracks` to fix stacking. Do not synthesize layouts from generic transforms or keyframes.
|
|
41
|
+
- Use `inspect_media` before describing source content. Use `search_media` to locate a visual or spoken moment. Use `inspect_timeline` to verify the composited result the viewer will actually see.
|
|
42
|
+
- Volume inputs, including volume keyframes, are linear values from `0` to `1`. Timeline reads return the same linear scale.
|
|
43
|
+
|
|
44
|
+
## Bring generated or local media into the editor
|
|
45
|
+
|
|
46
|
+
Use cloud generation for new assets, save or download the outputs, then call native `import_media`:
|
|
47
|
+
|
|
48
|
+
- `source.path`: absolute local file or directory. A directory imports recursively and preserves its folder structure.
|
|
49
|
+
- `source.url`: HTTPS asset URL. Set `mimeType` when a signed URL has no usable extension.
|
|
50
|
+
- `source.bytes`: small base64 media with a required `mimeType`.
|
|
51
|
+
- `source.matte`: generated solid-color image.
|
|
52
|
+
|
|
53
|
+
Readiness differs by source, and so does the poll that detects it:
|
|
54
|
+
|
|
55
|
+
- **URL and single-file path** imports return `status:'downloading'` with one `mediaRef`. Poll `get_media` with `ids:[mediaRef]` until `generationStatus` is absent.
|
|
56
|
+
- **Directory** imports return `status:'preparing'` once the batch is registered — not ready. A batch has no single `mediaRef` to poll by, so poll `get_media` with `pending:true` until it reports no unresolved imports.
|
|
57
|
+
- **Inline bytes and matte** imports finish inline and come back `status:'ready'`; no polling needed.
|
|
58
|
+
|
|
59
|
+
Never place a pending asset on the timeline. `generationStatus` is the signal: `preparing` and
|
|
60
|
+
`downloading` mean keep polling, absent means usable, and **`failed` is terminal** — report it or
|
|
61
|
+
retry the import explicitly, never poll on. Do not treat "not downloading" as ready.
|
|
62
|
+
|
|
63
|
+
For a batch of local outputs, download them into one workspace directory and import that directory once when practical. This is safer and faster than racing many import calls; just remember it is the `pending:true` poll that tells you when the batch is usable.
|
|
64
|
+
|
|
65
|
+
## Edit and verify
|
|
66
|
+
|
|
67
|
+
Use the tool descriptions as the exact schema. A dependable sequence is:
|
|
68
|
+
|
|
69
|
+
1. `manage_project` to select or create the local project.
|
|
70
|
+
2. `get_timeline` and `get_media` to establish current state.
|
|
71
|
+
3. `inspect_media` or `search_media` when content selection matters.
|
|
72
|
+
4. Serialized clip, track, layout, text, caption, audio, color, effect, or cut mutations using the current revision.
|
|
73
|
+
5. `inspect_timeline` when visual composition or layer order matters.
|
|
74
|
+
6. `undo` if the requested result is wrong and the next mutation would not cleanly correct it.
|
|
75
|
+
|
|
76
|
+
Edits are undoable. Do not ask for confirmation before each ordinary edit. Ask one focused question only when the user's creative direction is materially ambiguous.
|
|
77
|
+
|
|
78
|
+
## Export
|
|
79
|
+
|
|
80
|
+
`export_project` queues work in the background and returns a `jobId`, destination, and `started` or `queued` status.
|
|
81
|
+
|
|
82
|
+
- Use `video` for H.264, H.265, or ProRes.
|
|
83
|
+
- Use `xml` for Premiere Pro.
|
|
84
|
+
- Use `xml` (XMEML) for Premiere Pro **and DaVinci Resolve** — Resolve reads XMEML natively.
|
|
85
|
+
Use `fcpxml` only for Final Cut Pro. Sending Resolve an FCPXML produces a package it cannot
|
|
86
|
+
open cleanly, so the target matters more than the file extension suggests.
|
|
87
|
+
- Use `videodraft` for a self-contained project package.
|
|
88
|
+
- Omit `outputPath` unless the user named a destination; the default is `~/Downloads`.
|
|
89
|
+
- Use `manage_exports` to list progress, warnings, and results. Cancel only when the user asks or the just-queued settings were wrong. Do not infer that an export is stuck from elapsed time alone.
|
|
90
|
+
|
|
91
|
+
## Terminal bridge
|
|
92
|
+
|
|
93
|
+
VideoDraft desktop terminals expose `videodraft-editor`, which controls the same process and MCP surface:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
videodraft-editor status
|
|
97
|
+
videodraft-editor list-tools
|
|
98
|
+
videodraft-editor tool manage_project --json '{"action":"list"}'
|
|
99
|
+
videodraft-editor tool get_timeline --json '{}'
|
|
100
|
+
videodraft-editor show
|
|
101
|
+
videodraft-editor hide
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Control and tool commands auto-start a headless editor if none is running. `show` only reveals the already-running UI. Use `videodraft-editor tool <name> --json -` to read a JSON object from stdin when shell quoting would be fragile. Never read, copy, or expose the editor's rotating local authentication secret.
|
|
@@ -2,6 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
Working patterns for common asks. All assume auth (`videodraft login` once, or `VIDEODRAFT_API_KEY` in the environment) and use `--json` for parsing.
|
|
4
4
|
|
|
5
|
+
Inside VideoDraft ADE, if a native editor is available, use these recipes for asset generation and
|
|
6
|
+
optional script/storyboard stages. Hand the results to the editor for production and export ONLY
|
|
7
|
+
when the deliverable the user asked for is a composed production. A standalone output — the batch
|
|
8
|
+
product clips in recipe 1, the upscale in recipe 7 — is finished when it is generated; importing it
|
|
9
|
+
into a project and exporting a timeline builds an edit nobody asked for. Recipes below that call `videodraft produce` or `videodraft export` are hosted fallbacks only. Do not choose them over the available native editor unless the user explicitly asks for the hosted web workflow.
|
|
10
|
+
|
|
5
11
|
## 1. Batch product videos from a CSV
|
|
6
12
|
|
|
7
13
|
One 9:16 product clip per row of `products.csv` (`name,image_url,tagline`):
|
|
@@ -28,7 +34,7 @@ videodraft wait $(cut -d, -f2 outputs/jobs.csv) \
|
|
|
28
34
|
|
|
29
35
|
Submit-then-collect parallelizes server-side generation; the single multi-id `wait` keeps it to one local process and one batched poll request per tick no matter how many jobs. Gemini Omni Flash is selected because these are six-second first-frame product clips. Estimate first: `videodraft costs gemini-omni-flash --type video --duration 6 --resolution 720p --audio` × rows, and confirm with the user.
|
|
30
36
|
|
|
31
|
-
## 2.
|
|
37
|
+
## 2. Hosted full marketing video from one idea (fallback)
|
|
32
38
|
|
|
33
39
|
```bash
|
|
34
40
|
videodraft create "30-second launch video for Solace, a sleep-tracking ring. Calm, premium, dark palette." \
|
|
@@ -43,7 +49,7 @@ videodraft generate audio "Extend @Audio1 into a 20-second transition" --ref-aud
|
|
|
43
49
|
videodraft export "$PROJECT" --download solace-launch.mp4
|
|
44
50
|
```
|
|
45
51
|
|
|
46
|
-
The project stays editable at the URL in `project.json` (`.urls`)
|
|
52
|
+
Use this complete hosted path only when the user requested a web project or the native editor is unavailable. Otherwise stop after the storyboard/assets, import them into the native `.vdproject`, and export with `export_project`. The hosted project stays editable at the URL in `project.json` (`.urls`).
|
|
47
53
|
|
|
48
54
|
## 3. Talking-head (avatar) video
|
|
49
55
|
|
|
@@ -104,7 +110,7 @@ videodraft edit motion ./character.png \
|
|
|
104
110
|
--download ./media/character-dance.mp4
|
|
105
111
|
```
|
|
106
112
|
|
|
107
|
-
## 4.
|
|
113
|
+
## 4. Hosted changelog video in CI
|
|
108
114
|
|
|
109
115
|
In a GitHub Action with `VIDEODRAFT_API_KEY` set as a secret:
|
|
110
116
|
|
|
@@ -132,7 +138,7 @@ videodraft tools schema attach_media_to_shot --json
|
|
|
132
138
|
videodraft call attach_media_to_shot --args '{"project_id":"...","scene_index":0,"shot_index":1,"media_url":"https://...","media_type":"video","duration_seconds":6}'
|
|
133
139
|
```
|
|
134
140
|
|
|
135
|
-
Anything the VideoDraft MCP exposes
|
|
141
|
+
Anything the hosted VideoDraft MCP exposes, including character studio, product studio, and hosted project data, is reachable this way even before it gets a curated command. Native `.vdproject` editing uses the separate `videodraft_editor` MCP described in SKILL.md.
|
|
136
142
|
|
|
137
143
|
## 7. Enhance an existing asset without changing it
|
|
138
144
|
|
|
@@ -121,7 +121,7 @@ Direct Fabric text/audio and Sync Labs do not use the managed avatar record. The
|
|
|
121
121
|
- `seedream-v5-pro` supports unified text-to-image and reference-image editing with up to 10 image references. Use `--resolution 1K` for 7 credits/image or `--resolution 2K` for 14 credits/image.
|
|
122
122
|
- Reference inputs: `--ref <img>` (images), `--ref-video <v>` (Gemini Omni Flash, Seedance 2, Wan 2.7), `--ref-audio <a>` (Seedance 2). The CLI uploads local files for all of these, so you can pass a path or a URL. `--segment "<prompt>:<seconds>"` (repeatable) drives multi-prompt models (Kling 3.0 / 3.0 Turbo / O3); total 3-15s. `generate image --video-ref` is the nano-banana-2 video reference.
|
|
123
123
|
- The top-level prompt is OPTIONAL for `generate video` with multi-prompt models and for Kling 3.0 Turbo (`--model kling-v3-turbo`) image-to-video — a `--segment`-only or `--start-image`-only call is valid. Every other model still needs a prompt; the server enforces per-model rules.
|
|
124
|
-
- AI Production: `videodraft produce <project> --mode full_video` generates one Seedance 2 video per scene; poll with `videodraft generations`, then `videodraft finalize <project>` swaps them into the timeline before `export`. If the user explicitly requests another compatible video model, do not use this fixed Seedance path
|
|
124
|
+
- Hosted AI Production fallback: `videodraft produce <project> --mode full_video` generates one Seedance 2 video per scene; poll with `videodraft generations`, then `videodraft finalize <project>` swaps them into the hosted timeline before `export`. In VideoDraft ADE, do not choose this path while `videodraft_editor` is available unless the user explicitly requests hosted production. Generate or download the scene assets, import them, and assemble/export with the native editor instead. If the user explicitly requests another compatible video model for a hosted production, do not use this fixed Seedance path; generate the project shots manually with the requested model and attach them to the hosted timeline.
|
|
125
125
|
|
|
126
126
|
## Cost model
|
|
127
127
|
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# VideoDraft pipeline reference
|
|
2
2
|
|
|
3
|
-
Everything here
|
|
3
|
+
Everything here describes the hosted fallback pipeline through the CLI (`videodraft <command>` / `videodraft call <tool>`) or hosted MCP connector (tool names in backticks). When the local `videodraft_editor` MCP is available, do not use hosted production or export by default. Use hosted tools only for asset generation and optional script/storyboard stages, then import the results and finish with the native editor reference linked from SKILL.md. Continue through `produce_project` and `export_video` only when the user explicitly requests a hosted web production or the native editor is unavailable.
|
|
4
4
|
|
|
5
|
-
Use direct asset tools for standalone images, clips, audio, upscales, and descriptions. Use a project
|
|
5
|
+
Use direct asset tools for standalone images, clips, audio, upscales, and descriptions. Use a hosted project when the user explicitly wants the editable web project, when a hosted storyboard stage is useful, or when the native editor is unavailable. Script-only uses a script-stage project and stops at the script. In VideoDraft ADE with editor tools present, stop before hosted production, import the generated assets, and build/export the native project.
|
|
6
6
|
|
|
7
7
|
## Stages and their tools
|
|
8
8
|
|
|
@@ -42,6 +42,7 @@ Use direct asset tools for standalone images, clips, audio, upscales, and descri
|
|
|
42
42
|
- **Reference-first video**: when identity, styling, or composition matters, do not generate each motion clip from text alone. Generate or select the shot still first, then pass the decoded shot image as `--start-image` or `--ref` to the selected video model. AI Production already composes scene grids and sends them to Seedance as references. If the user explicitly requests another compatible video model, bypass fixed Seedance full-video mode and generate the per-shot clips with the requested model, using the individual decoded shot images as anchors.
|
|
43
43
|
- **Hold off generating shot images while the user is still iterating** on storyboard structure.
|
|
44
44
|
- **produce → export ordering**: `export` requires a produced project where every production scene has timeline media. If `produce` returns `generating_shot_images`, poll the job ids it returns, then re-run produce.
|
|
45
|
+
- **Do not attach motion clips before production exists**: run `produce` successfully first, then attach finished motion clips to the production timeline. Attaching before `production_data` exists cannot place them in the final timeline.
|
|
45
46
|
- **Generated motion clips do not auto-attach**: after `generate video` completes, attach the clip with `attach_media_to_shot` (`media_type:"video"`, include `duration_seconds`) — it replaces the production timeline clip while keeping the storyboard still.
|
|
46
47
|
- **Talking heads use dedicated avatar tools**: do not use `generate video`. Use managed `avatar create` and `avatar render` for reusable avatars, direct `avatar fabric` for a portrait plus text/audio, and `avatar lipsync` for an existing video plus replacement audio. Reuse a supplied person image or generate a clear front-facing portrait with the explicitly requested compatible image model, otherwise Nano Banana 2. Managed avatar creation and speech are bundled/free; direct Fabric, Sync, and render are paid.
|
|
47
48
|
- **Existing-video edits use their own category**: call `edit_video` or `videodraft edit video` with a `video_edit` model when transforming the source itself. Kling O3 and Wan 2.7 Ref/Edit are dual-mode: their reference-generation modes may use generic `generate_video` to create a new guided clip. Motion transfer similarly uses `generate_motion_control_video` or `videodraft edit motion` with a `motion_control` model.
|