@slatesvideo/shared 0.4.8 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/clients/cloud.d.ts +7 -7
- package/dist/operations/index.d.ts +43 -6
- package/dist/operations/index.js +456 -113
- package/dist/prompts/model-facts.js +34 -2
- package/dist/prompts/reference-composer.d.ts +35 -0
- package/dist/prompts/reference-composer.js +81 -0
- package/dist/skills/content.js +13 -12
- package/package.json +1 -1
- package/skills/slates-content-policy.md +8 -0
- package/skills/slates-cost-discipline.md +13 -11
- package/skills/slates-direct-response-ad.md +2 -2
- package/skills/slates-edit-and-iterate.md +1 -1
- package/skills/slates-model-selection.md +31 -1
- package/skills/slates-prompting-gpt-image-2.md +40 -0
- package/skills/slates-prompting-kling-v3.md +26 -0
- package/skills/slates-prompting-lip-sync.md +31 -17
- package/skills/slates-prompting-motion-transfer.md +29 -15
- package/skills/slates-prompting-nano-banana-2.md +9 -0
- package/skills/slates-prompting-seedance.md +9 -0
- package/skills/slates-storyboard-from-script.md +1 -1
- package/skills/slates-vision-feedback-loop.md +1 -1
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@slatesvideo/shared",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.1",
|
|
4
4
|
"description": "Shared operations layer for the Slates MCP server and CLI: auth, cloud/desktop clients, and the single tool surface both consume. Most users want @slatesvideo/mcp-server or @slatesvideo/cli instead.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -53,3 +53,11 @@ Never write romantic, sexual, or suggestive content involving or directed at min
|
|
|
53
53
|
- [ ] Anything with children is wholesome and age-appropriate.
|
|
54
54
|
|
|
55
55
|
If a box fails, apply the substitution table before writing the prompt.
|
|
56
|
+
|
|
57
|
+
## Editing real footage (Kling O3 edit) — real people in the SOURCE
|
|
58
|
+
|
|
59
|
+
Video edit takes the user's own footage, which often contains real people. Rules:
|
|
60
|
+
|
|
61
|
+
- The user must hold rights/consent for any real person's likeness in footage they edit — ask once when it's clearly someone other than the user, then proceed.
|
|
62
|
+
- Kling's video-to-video filter behavior on real faces is **not yet verified** (unlike Seedance, where the consent-gated real-face route is confirmed). If an edit of real-person footage is rejected by the provider, do NOT retry-spam variations — tell the user the filter blocked it and offer a no-face crop/segment or an AI-character swap instead.
|
|
63
|
+
- Never use edit to put a real, named public figure into a scene, or to make someone appear to say/do something they didn't. Faceless b-roll (hands, products, landscapes, crowds-from-behind) edits freely.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: slates-cost-discipline
|
|
3
|
-
description: Mandatory pre-flight discipline before ANY generation call (image or video) — estimate cost, announce in
|
|
3
|
+
description: Mandatory pre-flight discipline before ANY generation call (image or video) — estimate cost, announce in credits, get confirmation, aggregate batches. Read this every time before calling slates_generate_image or any future slates_generate_* op. Skipping this risks burning the user's credits on guesses.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Slates cost discipline — read before every generation
|
|
@@ -20,23 +20,25 @@ Before ANY `slates_generate_*` call, run `slates_estimate_generation_cost` first
|
|
|
20
20
|
|
|
21
21
|
If aspect ratio or resolution isn't obvious from the user's request, **ask before estimating**. Don't guess.
|
|
22
22
|
|
|
23
|
-
### 2. Announce in
|
|
23
|
+
### 2. Announce in credits, plainly, before spending
|
|
24
24
|
|
|
25
|
-
|
|
25
|
+
Slates bills abstract **credits** (they never expire). Announce the credit total the estimate returns — never dollars.
|
|
26
|
+
|
|
27
|
+
Format: `About to spend N credits on M image(s) at [resolution] [aspect ratio]. Proceed?`
|
|
26
28
|
|
|
27
29
|
Examples:
|
|
28
|
-
- `About to spend
|
|
29
|
-
- `About to spend
|
|
30
|
+
- `About to spend 4 credits on 1 image at 1k 16:9. Proceed?`
|
|
31
|
+
- `About to spend 24 credits on 4 images at 2k 9:16 (variants). Proceed?`
|
|
30
32
|
|
|
31
|
-
Below
|
|
33
|
+
Below ~7 credits you can proceed silently after announcing once. Above ~7 credits wait for explicit confirmation. Above ~17 credits the server itself will gate with `requires_confirm` — pass `confirm: true` only after the user explicitly OKs.
|
|
32
34
|
|
|
33
35
|
### 3. Aggregate batches into ONE upfront announcement
|
|
34
36
|
|
|
35
|
-
If you're planning a multi-call workflow (5 storyboard frames, 3 character variants, a grid of options), **announce the total before the first call**, not five
|
|
37
|
+
If you're planning a multi-call workflow (5 storyboard frames, 3 character variants, a grid of options), **announce the total before the first call**, not five small announcements after the fact.
|
|
36
38
|
|
|
37
|
-
Format: `Plan: N generations totaling
|
|
39
|
+
Format: `Plan: N generations totaling C credits. [Brief description of the sequence.] Proceed with the batch?`
|
|
38
40
|
|
|
39
|
-
Example: `Plan: 6 frame generations at 1k 16:9 totaling
|
|
41
|
+
Example: `Plan: 6 frame generations at 1k 16:9 totaling 24 credits — establishing wide, push-in, two-shot, reverse, OTS, insert. Proceed?`
|
|
40
42
|
|
|
41
43
|
### 3b. Batch authorization — one approval covers the enumerated batch
|
|
42
44
|
|
|
@@ -52,7 +54,7 @@ One approval = that plan, as enumerated, at those prices. Nothing else.
|
|
|
52
54
|
|
|
53
55
|
### 4. Track the running total
|
|
54
56
|
|
|
55
|
-
After each generation completes, the response includes `
|
|
57
|
+
After each generation completes, the response includes `cost_credits` (when available). Keep a running tally in your context. Surface it every 3 generations or whenever the user asks "how much have we spent?"
|
|
56
58
|
|
|
57
59
|
## Resolution decision rules
|
|
58
60
|
|
|
@@ -83,7 +85,7 @@ If the user prompt mixes signals (e.g. "cinematic Instagram post"), ask. Don't g
|
|
|
83
85
|
|
|
84
86
|
## When the gate fires
|
|
85
87
|
|
|
86
|
-
The server returns `requires_clarification` when aspect ratio or resolution is missing. The server returns `requires_confirm` when total spend exceeds
|
|
88
|
+
The server returns `requires_clarification` when aspect ratio or resolution is missing. The server returns `requires_confirm` when total spend exceeds ~17 credits. In both cases:
|
|
87
89
|
|
|
88
90
|
1. Surface the gate response to the user
|
|
89
91
|
2. Get a clean answer
|
|
@@ -10,7 +10,7 @@ You are building a 30-second hyper-motion direct-response ad. The user has hande
|
|
|
10
10
|
**Hard rules**
|
|
11
11
|
|
|
12
12
|
- Always estimate cost before generating. Use `slates_estimate_generation_cost` and surface the total.
|
|
13
|
-
- All
|
|
13
|
+
- All Slates generation routes through Slates Credits, period (BYOK is retired) — don't suggest "use your own keys" workarounds.
|
|
14
14
|
- Default model: `nano-banana-2-2k`. For close-up product hero frames step up to `4k` only if the user asks.
|
|
15
15
|
- Hyper-motion = punchy cuts, 4 frames in 30 seconds, ~7s each. Don't over-storyboard.
|
|
16
16
|
|
|
@@ -58,7 +58,7 @@ For each frame:
|
|
|
58
58
|
|
|
59
59
|
- **Don't** generate text overlays in the image. Slates renders captions/CTAs at the editor stage.
|
|
60
60
|
- **Don't** burn credits on slot-machine prompting. If the first generation is off, refine the prompt; don't just regenerate.
|
|
61
|
-
- **Don't** skip the cost estimate. Confirm with the user above
|
|
61
|
+
- **Don't** skip the cost estimate. Confirm with the user above ~17 credits.
|
|
62
62
|
- **Don't** invent visual specifics about the product (colors, textures, angles) that aren't in the reference image. Reference-anchored prompts only.
|
|
63
63
|
|
|
64
64
|
## Voice
|
|
@@ -28,7 +28,7 @@ The user's request is one of:
|
|
|
28
28
|
| Aesthetic / compositional | `slates_generate_image` with the original in `referenceAssetIds` + a refined prompt. Don't re-roll from scratch. |
|
|
29
29
|
| Wholesale | New prompt, no reference, fresh generation. Treat as a new brief. |
|
|
30
30
|
|
|
31
|
-
**`slates_edit_image` shape:** `projectId` + `sourceAssetId` + `prompt` (the edit instruction). Default model `nano-banana-2` — the only edit model that also takes extra `referenceAssetIds`; `flux-2-max` / `seedream-5-lite` use their own edit endpoints and ignore references. The result lands as a NEW asset (prompt prefixed `[Edit]`); the source is untouched. Cost
|
|
31
|
+
**`slates_edit_image` shape:** `projectId` + `sourceAssetId` + `prompt` (the edit instruction). Default model `nano-banana-2` — the only edit model that also takes extra `referenceAssetIds`; `flux-2-max` / `seedream-5-lite` use their own edit endpoints and ignore references. The result lands as a NEW asset (prompt prefixed `[Edit]`); the source is untouched. Cost above ~17 credits gates on `confirm=true`.
|
|
32
32
|
|
|
33
33
|
### 4. Generate, evaluate, decide
|
|
34
34
|
- Estimate cost first.
|
|
@@ -18,6 +18,31 @@ Pick the model FIRST, deliberately, before writing a prompt or quoting a plan. M
|
|
|
18
18
|
| The premium hero shot a piece hangs on | Seedance 2.0 | Spend where it shows. |
|
|
19
19
|
| Native synchronized audio (dialogue + SFX generated WITH the video in one gen), 16:9, ≤8s | Veo 3.1 | The only job Veo wins. |
|
|
20
20
|
|
|
21
|
+
## Video EDIT routing (changing an existing clip)
|
|
22
|
+
|
|
23
|
+
| Job | Tool | Why |
|
|
24
|
+
|---|---|---|
|
|
25
|
+
| **Fix a 90%-right clip** — wrong shirt, one artifact, swap the subject, change the environment | **Kling O3 Edit** (`slates_edit_video`) | The default edit tool. Element lock (frontal + angles) holds identity; original motion, camera, and AUDIO preserved. One pass, no masking, ~19¢/s. |
|
|
26
|
+
| Style-transfer-heavy re-imagining, full relocate of the scene | Seedance edit/relocate (`videoReferenceAssetId` on `slates_generate_video`) | Seedance's strength is transfer intensity; it re-generates rather than surgically edits. |
|
|
27
|
+
| AI-edit the user's OWN footage (3–15s clips) | Kling O3 Edit | Takes any MP4/MOV 3–15s, 720–3840px — not just Slates gens. |
|
|
28
|
+
|
|
29
|
+
- **Edit before re-roll.** A re-roll gambles away the parts the user already likes; an edit changes only what the prompt names. Quote the edit first when a clip is mostly right.
|
|
30
|
+
- Edited clips are themselves editable ≤15s clips — chain passes; lineage links each output to its parent.
|
|
31
|
+
|
|
32
|
+
## Motion Transfer & Lip Sync routing (two engines per tool)
|
|
33
|
+
|
|
34
|
+
Both tools have a cheap Kling utility lane and a premium Seedance lane. The capability is the same; the execution model differs: Kling bolts motion/lip onto the source as a dedicated post-process; Seedance generates in a single pass with the driving clip / dialogue as native conditioning signals — better motion fidelity, natural speech delivery, audio included.
|
|
35
|
+
|
|
36
|
+
| Job | Engine | Why |
|
|
37
|
+
|---|---|---|
|
|
38
|
+
| Quick motion retarget, budget lane, or driving clip >15s | Kling MC std/pro (`slates_generate_motion_transfer`) | Structured skeleton/depth retarget, ~32–42 credits / 5s, takes up to 30s driving clips. |
|
|
39
|
+
| **Motion transfer where fidelity or audio matters** — dance, choreography, cinematic action | **Seedance 2.0** (`motionModel=seedance-2`) | Single-pass conditioning beats post-hoc retargeting; prompt-driven; native audio. Driving clip 2–15s; bills input+output seconds (vref keys). |
|
|
40
|
+
| Cheap lip-sync utility (re-voice a clip, simple avatar) | Kling lip-sync / avatar (`slates_generate_lip_sync`) | ~4–29 credits / 5s blocks. |
|
|
41
|
+
| **Natural speech, voice cloned from the source clip, premium delivery** | **Seedance 2.0** (`engine=seedance-2`) | The line is spoken IN the generation (no TTS layer); a video source keeps its own voice; uploaded ≤15s audio can drive it. |
|
|
42
|
+
|
|
43
|
+
- Faces: Seedance tool gens default `seedanceFace=true` (sources are people). A REAL person triggers the consent cascade (`[REAL_FACE_DETECTED]` → `seedanceRealFace` + `realFaceConsent`, premium realface pricing).
|
|
44
|
+
- Billing: any Seedance gen with a video reference bills COMBINED input+output seconds (`seedance-2*-vref-*` keys) — always pass the clip duration and quote before confirming.
|
|
45
|
+
|
|
21
46
|
**Rules:**
|
|
22
47
|
|
|
23
48
|
- **Default video = Kling 3.0 std.** Escalate to Seedance the moment the shot has physics/effects weight or is the hero moment — and say why in the plan ("physics-heavy, routing to Seedance").
|
|
@@ -31,8 +56,13 @@ Pick the model FIRST, deliberately, before writing a prompt or quoting a plan. M
|
|
|
31
56
|
**Video models (Kling, Seedance, Veo) cannot generate standalone images — ever.** A "premium hero reference image" is still an image job: it routes to an image model below, never to Seedance.
|
|
32
57
|
|
|
33
58
|
- **Default: Nano Banana 2** — best reference handling (14 refs), best legible text, the standard start-frame generator.
|
|
59
|
+
- **NB2 Lite** — the fast/draft seat: ~half NB2's price, ~2.7× faster, 1K only. Route iteration volume and drafts here; finals go back to NB2 full (2K/4K).
|
|
60
|
+
- **Nano Banana Pro** — the hero-frame/typography ceiling (~2× NB2). NB2 ≈ 95% of Pro; escalate only when spatial composition, cinematic lighting/skin, fine typography-in-scene, or deep multi-element frames must be perfect. Up to 14 refs — feed it a full subject library.
|
|
61
|
+
- **GPT Image 2** — readable text / panels / UI king: character sheets, shot grids, diagrams, text-bearing panels. Medium quality is the default (half NB2's price at 1080p); high (~4×) only when text precision is the whole job. 4K at both tiers is API-only — even paid ChatGPT can't render it.
|
|
34
62
|
- **FLUX.2 Max** — photoreal texture, hex-color binding, typography, less censored.
|
|
35
|
-
- **Seedream 5 Lite** —
|
|
63
|
+
- **Seedream 5 Lite** — uncensored + any-resolution flat price; volume exploration when the Gemini filter is in the way.
|
|
64
|
+
|
|
65
|
+
**Split rule of thumb:** readable text / panels / UI → GPT Image 2; photoreal, character-locked, widescreen, or edit-heavy → the Banana line; drafts → NB2 Lite; uncensored or odd resolutions → Seedream/FLUX.
|
|
36
66
|
|
|
37
67
|
## Cost is a tiebreaker, not the router
|
|
38
68
|
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: slates-prompting-gpt-image-2
|
|
3
|
+
description: Prompting GPT Image 2 — the readable-text / character-sheet / shot-grid engine. Read before calling slates_generate_image with model gpt-image-2. Covers the quality tiers (medium default, high for max text precision), resolution classes (1k/2k=1080p/3k=1440p/4k), text-accuracy prompting, panel/grid layout direction, and when to route to the Banana line instead.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# GPT Image 2 — sheets, grids, and text that actually reads
|
|
7
|
+
|
|
8
|
+
GPT Image 2's edge is **character-level text accuracy** (~99% on English), ordered panels, and exact element placement — the jobs where every other model garbles a word or shuffles a layout. It is NOT the photoreal or character-locked pick: route those to the Banana line (`slates-model-selection` has the split).
|
|
9
|
+
|
|
10
|
+
## Quality tiers — always set explicitly
|
|
11
|
+
|
|
12
|
+
- **medium** (default) — sharp text, fast, the value seat: half NB2's price at the 1080p class. Blind benchmarks put it within a hair of high at a quarter of the cost. Start here.
|
|
13
|
+
- **high** — ~4× the price; max text precision + reasoning. A deliberate premium pick when tiny type, dense diagrams, or many labeled elements ARE the job.
|
|
14
|
+
|
|
15
|
+
Never rely on the provider default (it's high — the priciest tier). The Slates ops send medium unless you say otherwise.
|
|
16
|
+
|
|
17
|
+
## Resolution classes
|
|
18
|
+
|
|
19
|
+
`1k` = 1024²-class · `2k` = 1920×1080-class · `3k` = 2560×1440-class · `4k` = 3840×2160-class. Pick 2k for most sheets/panels; 4k for print-density grids. 4K exists at BOTH tiers and is API-only — even paid ChatGPT can't render it.
|
|
20
|
+
|
|
21
|
+
## Prompting for text accuracy
|
|
22
|
+
|
|
23
|
+
- **Quote every string that must render verbatim**: `the sign reads "OPEN 24 HOURS"` — quoted strings render most reliably.
|
|
24
|
+
- Specify font *feel*, not font names: "clean geometric sans, high contrast", "hand-painted brush lettering".
|
|
25
|
+
- For dense text (posters, UI mocks), list the copy as ordered lines: `Line 1: "..." Line 2: "..."` — GPT Image 2 respects ordering.
|
|
26
|
+
- Keep total on-image text under ~30 words for perfect accuracy; beyond that, accuracy degrades gracefully but degrades.
|
|
27
|
+
|
|
28
|
+
## Panels, sheets, and grids
|
|
29
|
+
|
|
30
|
+
- State the grid explicitly and number the cells: "a 2×3 grid of panels, numbered 1–6, reading left-to-right, top-to-bottom".
|
|
31
|
+
- Give each cell ONE content clause: "Panel 3: the character mid-jump, side view".
|
|
32
|
+
- Character sheets: "character turnaround sheet: front, 3/4 left, profile, back — same character, same outfit, flat even lighting, plain background". GPT Image 2 holds the layout; the Banana line holds the *face* better — for identity-critical turnarounds prefer NB2/NB Pro and use GPT Image 2 when labels/annotations matter.
|
|
33
|
+
|
|
34
|
+
## References & editing
|
|
35
|
+
|
|
36
|
+
Reference images route through the edit endpoint (up to ~10). The composed "image N" naming applies as everywhere else. Mask-based inpainting exists at the API level but isn't surfaced — describe the change instead.
|
|
37
|
+
|
|
38
|
+
## Filter regime
|
|
39
|
+
|
|
40
|
+
OpenAI moderate — a third regime distinct from Gemini (NB family) and ByteDance (Seedream). Real-face references pass more readily than Gemini; violence/brand rules are similar. `slates-content-policy` applies unchanged.
|
|
@@ -159,6 +159,32 @@ When talking to the user about the gen, refer to each reference by its short cod
|
|
|
159
159
|
- ✅ "I'm anchoring on **IMG-A12** as the detective and **IMG-A18** as the alleyway environment — Omni will handle the line delivery in EN."
|
|
160
160
|
- ❌ "I'm using the detective image and the alley one..." (which alley? Three exist.)
|
|
161
161
|
|
|
162
|
+
## Video-to-video EDIT (`slates_edit_video`) — @Video1 / @ElementN / @ImageN
|
|
163
|
+
|
|
164
|
+
Kling O3 edit takes an EXISTING 3-15s clip and changes only what the prompt names — character swap, environment change, style transfer — in one pass, no masking. Original motion, camera, and audio are preserved by default. Its notation is Kling's own, different from the "image N" naming used everywhere else:
|
|
165
|
+
|
|
166
|
+
- **`@Video1`** — the source clip (always; the transport anchors the instruction to it).
|
|
167
|
+
- **`@Element1..`** — subjects to swap IN. Each element = one frontal image + up to 3 angle images (pass as `characterAssetIds`; @mention names in the prompt compile to @ElementN automatically).
|
|
168
|
+
- **`@Image1..`** — style/appearance references (pass as `styleAssetIds`).
|
|
169
|
+
- Max **4 combined** element + image refs per edit.
|
|
170
|
+
|
|
171
|
+
**Prompt shape — the change, not the whole scene:**
|
|
172
|
+
|
|
173
|
+
```
|
|
174
|
+
Replace the man in @Video1 with @Element1, keeping his walk cycle, the camera move, and the rain unchanged.
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
```
|
|
178
|
+
Edit @Video1: turn the daytime street into a neon-lit Tokyo alley at night, wet asphalt reflections. Apply the visual style of @Image1. Keep the subject and camera motion exactly as they are.
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
Rules:
|
|
182
|
+
- Name what CHANGES; explicitly state what stays ("keep the motion / camera / everything else unchanged") — the model preserves better when told to.
|
|
183
|
+
- One edit intent per pass. Chain passes for compound changes (each output is itself an editable clip, linked to its parent).
|
|
184
|
+
- Billing is per second of OUTPUT ≈ the clip length, rounded UP to the next second. A 7.3s clip bills as 8s.
|
|
185
|
+
- Clip constraints: 3-15s, 720-3840px, MP4/MOV. Agents can pre-trim on the timeline when a clip runs long.
|
|
186
|
+
- Routing: Kling edit is the default edit tool (element lock + audio intact); Seedance edit/relocate wins style-transfer-heavy re-imaginings — see `slates-model-selection`.
|
|
187
|
+
|
|
162
188
|
## Sources
|
|
163
189
|
|
|
164
190
|
- [fal.ai — Kling 3.0 Prompting Guide](https://blog.fal.ai/kling-3-0-prompting-guide/)
|
|
@@ -1,26 +1,40 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: slates-prompting-lip-sync
|
|
3
|
-
description: How to set up
|
|
3
|
+
description: How to set up lip-sync — Kling (cheap utility lane) or Seedance 2.0 (premium single-pass lane). Read before calling slates_generate_lip_sync. Flows — video→video re-dub, image→video avatar, and Seedance native speech — with different inputs, pricing, and gotchas. Voice catalog, framing rules, audio file constraints, and which engine/tier to pick.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
#
|
|
6
|
+
# Lip-sync — setup guide
|
|
7
7
|
|
|
8
|
-
Two
|
|
8
|
+
Two engines. Kling is the cheap utility lane (dedicated lip-sync endpoints, 5-second outputs); Seedance is the premium lane (speech generated IN the video itself, single pass):
|
|
9
9
|
|
|
10
|
-
| Flow | Source | Model | Cost
|
|
10
|
+
| Flow | Source | Engine/Model | Cost | Use case |
|
|
11
11
|
|------|--------|-------|-----------|----------|
|
|
12
|
-
| Re-dub | video clip | kling-lip-sync-video |
|
|
13
|
-
| Avatar standard | still image | ai-avatar/v2/standard |
|
|
14
|
-
| Avatar pro | still image | ai-avatar/v2/pro |
|
|
12
|
+
| Re-dub | video clip | kling-lip-sync-video | ~4 credits / 5s | Replace dialogue on an existing talking head |
|
|
13
|
+
| Avatar standard | still image | ai-avatar/v2/standard | ~14 credits / 5s | Animate a portrait into a talking avatar |
|
|
14
|
+
| Avatar pro | still image | ai-avatar/v2/pro | ~29 credits / 5s | Higher facial fidelity for hero shots |
|
|
15
|
+
| **Seedance native** | image or video | `engine=seedance-2` | per second (`seedance-2-face-*`; video sources bill input+output seconds) | **Premium**: natural delivery, whole-body performance, voice cloned from a video source, audio included |
|
|
15
16
|
|
|
16
|
-
Pick `sourceType` deliberately —
|
|
17
|
+
Pick engine + `sourceType` deliberately — they decide the pricing tier and the underlying endpoint.
|
|
18
|
+
|
|
19
|
+
## Seedance engine (premium single-pass)
|
|
20
|
+
|
|
21
|
+
Kling lip-sync moves the mouth on finished pixels; Seedance *generates* the performance — head movement, gesture, delivery energy — with the dialogue as a native conditioning signal. Key facts:
|
|
22
|
+
|
|
23
|
+
- **`ttsText` becomes the spoken line, natively.** No TTS voice/speed params — a VIDEO source keeps its **own voice** (the model clones it from the clip's audio track); for an image source the voice follows the character's look, or describe it in the line's context.
|
|
24
|
+
- **`audioMethod=upload`** drives the speech from a ≤15s audio file instead (a reference-audio input, no billing surcharge).
|
|
25
|
+
- **Sources:** image (any style; same framing rules as the avatar flow below) or a 2–15s video clip. Output duration follows the source/audio/line length (4–15s), not a fixed 5s.
|
|
26
|
+
- **Billing:** image sources bill the normal `seedance-2-face-{res}-{N}s` keys; video sources bill combined input+output seconds (`-vref-` keys) — pass `sourceSeconds` and quote via the confirm gate.
|
|
27
|
+
- **Faces:** `seedanceFace` defaults true. A REAL person → `[REAL_FACE_DETECTED]` → confirm consent → retry with `seedanceRealFace=true, realFaceConsent=true` (premium realface pricing).
|
|
28
|
+
- When to pick it: hero dialogue shots, natural delivery, "make this clip's person say X in their own voice". Stay on Kling for cheap utility re-dubs and long clips.
|
|
29
|
+
|
|
30
|
+
Everything below applies to the **Kling** engine.
|
|
17
31
|
|
|
18
32
|
## Choosing video vs avatar
|
|
19
33
|
|
|
20
34
|
Use **video** (re-dub) when:
|
|
21
35
|
- A talking-head clip already exists (Slates-generated, recorded, or imported)
|
|
22
36
|
- The mouth/face is already moving and only the audio needs to change
|
|
23
|
-
-
|
|
37
|
+
- ~4 credits is hard to beat for short dialogue replacement
|
|
24
38
|
|
|
25
39
|
Use **avatar** when:
|
|
26
40
|
- Only a still portrait exists
|
|
@@ -113,9 +127,9 @@ Default `"."` is fine if you have nothing useful to add.
|
|
|
113
127
|
**Use pro** when:
|
|
114
128
|
- Final ads where the avatar's face fills the screen
|
|
115
129
|
- The character is named / branded — identity drift kills the take
|
|
116
|
-
- You're already paying
|
|
130
|
+
- You're already paying tens of credits for the surrounding video pipeline
|
|
117
131
|
|
|
118
|
-
Don't default to pro. The
|
|
132
|
+
Don't default to pro. The ~15-credit delta per take adds up across iteration.
|
|
119
133
|
|
|
120
134
|
## Common failure modes
|
|
121
135
|
|
|
@@ -131,17 +145,17 @@ Don't default to pro. The $0.44 delta per take adds up across iteration.
|
|
|
131
145
|
|
|
132
146
|
## Cost discipline
|
|
133
147
|
|
|
134
|
-
- Video re-dub at
|
|
135
|
-
- Avatar standard at
|
|
136
|
-
- Avatar pro at
|
|
148
|
+
- Video re-dub at ~4 credits is the cheapest dialogue iteration in the entire Slates stack — use it for voice A/B testing
|
|
149
|
+
- Avatar standard at ~14 credits is fine for medium use
|
|
150
|
+
- Avatar pro at ~29 credits trips the confirm gate — explicit user OK required every time
|
|
137
151
|
- All 5s. There is no shorter option.
|
|
138
152
|
|
|
139
153
|
## Workflow patterns
|
|
140
154
|
|
|
141
155
|
**Voice A/B test (cheap):**
|
|
142
|
-
1. Generate one base talking-head video clip with Veo or Seedance (
|
|
156
|
+
1. Generate one base talking-head video clip with Veo or Seedance (~40 credits)
|
|
143
157
|
2. Run `slates_generate_lip_sync` with `sourceType: 'video'` against 3–5 different `ttsVoice` values
|
|
144
|
-
3. Total cost:
|
|
158
|
+
3. Total cost: ~40 + (5 × ~4) ≈ 60 credits to compare voices
|
|
145
159
|
|
|
146
160
|
**Brand avatar from a single portrait:**
|
|
147
161
|
1. Generate or upload the hero portrait (face fills frame, eyes open, neutral mouth)
|
|
@@ -157,7 +171,7 @@ Don't default to pro. The $0.44 delta per take adds up across iteration.
|
|
|
157
171
|
|
|
158
172
|
Lip-sync is mechanical — the model re-syncs the chosen source to the chosen audio. The confirm response carries the source asset's code so you can announce it in chat.
|
|
159
173
|
|
|
160
|
-
- ✅ "Lip-syncing **IMG-A12 — Founder Portrait** to the new line.
|
|
174
|
+
- ✅ "Lip-syncing **IMG-A12 — Founder Portrait** to the new line. ~29 credits on avatar-pro. Confirm?"
|
|
161
175
|
- ❌ "Using the founder image..." (which? Three exist.)
|
|
162
176
|
|
|
163
177
|
Don't second-guess the source. If the output is wrong, iterate on source choice or audio, not on a refinement prompt (there isn't one).
|
|
@@ -1,18 +1,32 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: slates-prompting-motion-transfer
|
|
3
|
-
description: How to set up Kling Motion Control (
|
|
3
|
+
description: How to set up motion transfer — Kling Motion Control (cheap utility lane) or Seedance 2.0 (premium single-pass lane). Read before calling slates_generate_motion_transfer. Reference image (character) + driving video (motion source) → new video of the character performing the motion. Asset selection rules, engine choice, character_orientation, tiers, and prompt usage.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
#
|
|
6
|
+
# Motion transfer — setup guide
|
|
7
7
|
|
|
8
|
-
Take a still **target image** (your character) and a **source video** (the motion you want), produce a new
|
|
8
|
+
Take a still **target image** (your character) and a **source video** (the motion you want), produce a new video of your character performing the source video's motion. Two engines:
|
|
9
9
|
|
|
10
|
-
|
|
|
10
|
+
| Engine | Cost | Use case |
|
|
11
11
|
|------|-----------|----------|
|
|
12
|
-
| std (`kling-mc-std-5s`) |
|
|
13
|
-
| pro (`kling-mc-pro-5s`) |
|
|
12
|
+
| Kling std (`kling-mc-std-5s`) | ~32 credits / 5s | General motion transfer, budget lane |
|
|
13
|
+
| Kling pro (`kling-mc-pro-5s`) | ~42 credits / 5s | Cleaner anatomy, better identity preservation |
|
|
14
|
+
| **Seedance 2.0** (`motionModel=seedance-2`) | per second of input+output (`seedance-2-face-vref-*`) | **Premium lane** — single-pass generation with the driving clip as a native conditioning signal: better motion fidelity, native audio, prompt-directed |
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
All tiers trip the confirm gate. User OK required every time. (Prices are approximate — `slates_estimate_generation_cost` returns the exact credit total.)
|
|
17
|
+
|
|
18
|
+
## Seedance engine (premium single-pass)
|
|
19
|
+
|
|
20
|
+
Kling MC retargets a skeleton onto a finished image; Seedance *generates* the shot with the motion as a conditioning input — the difference shows on fast choreography, physical contact, cloth/hair, and camera motion. Key differences from Kling:
|
|
21
|
+
|
|
22
|
+
- **Prompt-driven.** The prompt is the primary control, using ordinal references: `The character from image 1 performs the exact motion, choreography, and camera movement from video 1. Preserve the character's identity, appearance, and outfit.` Add style/setting/camera direction freely — Seedance re-generates the whole shot.
|
|
23
|
+
- **Driving clip must be 2–15s** (all providers cap reference video at 15s). Longer clips: trim first, or use Kling MC (`character_orientation: 'video'` takes up to 30s).
|
|
24
|
+
- **Billing = combined input+output seconds** (the vref keys). Pass `sourceVideoSeconds`; output `duration` defaults to the clip length (4–15s). The server probes the clip and corrects an understated key — quote via the confirm gate before spending.
|
|
25
|
+
- **Faces route through the face cascade**: `seedanceFace` defaults true (driving clips contain people). A REAL person → `[REAL_FACE_DETECTED]` → confirm consent with the user → retry `seedanceRealFace=true, realFaceConsent=true` (premium realface vref pricing).
|
|
26
|
+
- **Native audio included** — the output can carry sound from the prompt (or the driving clip's vibe); no audio surcharge.
|
|
27
|
+
- `characterOrientation` is Kling-only; Seedance framing follows the prompt + `aspectRatio`.
|
|
28
|
+
|
|
29
|
+
Everything below applies to the **Kling** engine.
|
|
16
30
|
|
|
17
31
|
## Inputs
|
|
18
32
|
|
|
@@ -66,18 +80,18 @@ Switch to `image` when the target image's composition is the brand asset and the
|
|
|
66
80
|
|
|
67
81
|
## Tier choice — std vs pro
|
|
68
82
|
|
|
69
|
-
**std (
|
|
83
|
+
**std (~32 credits)** for:
|
|
70
84
|
- Drafts, motion exploration, blocking
|
|
71
85
|
- Group scenes where the character isn't a hero shot
|
|
72
86
|
- When the budget is tight and the motion is the focus
|
|
73
87
|
|
|
74
|
-
**pro (
|
|
88
|
+
**pro (~42 credits)** for:
|
|
75
89
|
- Final hero takes
|
|
76
90
|
- Branded characters where identity drift = unacceptable
|
|
77
91
|
- Anatomically complex motion (limbs crossing, fast direction changes)
|
|
78
92
|
- Anime / cartoon target images — pro handles non-realistic styles better
|
|
79
93
|
|
|
80
|
-
Don't default to pro. The
|
|
94
|
+
Don't default to pro. The ~10-credit delta compounds fast across iteration.
|
|
81
95
|
|
|
82
96
|
## Prompt usage (optional)
|
|
83
97
|
|
|
@@ -112,7 +126,7 @@ Leave it empty if you don't have a specific atmospheric note.
|
|
|
112
126
|
2. Find driving footage — a clean reference video of the dance you want
|
|
113
127
|
3. Upload both as project assets
|
|
114
128
|
4. Run motion transfer with `motionModel: 'kling-mc-pro'`, `characterOrientation: 'video'`
|
|
115
|
-
5. Total cost:
|
|
129
|
+
5. Total cost: ~42 credits per 5s take
|
|
116
130
|
|
|
117
131
|
**Subtle motion on a hero portrait:**
|
|
118
132
|
1. Use the locked hero portrait as the target image
|
|
@@ -129,15 +143,15 @@ Leave it empty if you don't have a specific atmospheric note.
|
|
|
129
143
|
## Cost discipline
|
|
130
144
|
|
|
131
145
|
- 5 seconds, no shorter option
|
|
132
|
-
- Both tiers trip the
|
|
133
|
-
- Iteration is expensive: 4 takes at pro
|
|
146
|
+
- Both tiers trip the confirm gate — every call needs explicit user OK
|
|
147
|
+
- Iteration is expensive: 4 takes at pro ≈ 168 credits. Lock framing + driving video before tier-up to pro.
|
|
134
148
|
- Always run a single std take first to validate the motion + framing combo before committing to pro
|
|
135
149
|
|
|
136
150
|
## Confirm gate: cost + codes, no inline preview
|
|
137
151
|
|
|
138
|
-
Motion transfer is mechanical — the model deterministically applies source motion to target image. Both tiers trip the
|
|
152
|
+
Motion transfer is mechanical — the model deterministically applies source motion to target image. Both tiers trip the confirm gate; the response includes the asset codes for source and target so you can announce them in chat.
|
|
139
153
|
|
|
140
|
-
- ✅ "Transferring motion from **VID-V3** onto **IMG-A12 — Detective Closeup**.
|
|
154
|
+
- ✅ "Transferring motion from **VID-V3** onto **IMG-A12 — Detective Closeup**. ~42 credits, confirm?"
|
|
141
155
|
- ❌ "Using the walk video and the detective image..." (multiple of each in the project.)
|
|
142
156
|
|
|
143
157
|
Don't second-guess the assets the user picked — the model executes the transfer. If the output is wrong, iterate on motion source or target choice, not on a refinement prompt.
|
|
@@ -144,3 +144,12 @@ Default to #1. Reach for #2 only when positive framing can't suppress the unwant
|
|
|
144
144
|
## The 3-strike rule
|
|
145
145
|
|
|
146
146
|
If three iterations on the same prompt haven't produced what the user wants, stop. Hand back to the user with what you tried and what isn't working. The slot machine doesn't converge — the prompt structure is wrong, not the seed.
|
|
147
|
+
|
|
148
|
+
## Family variants — Lite and Pro
|
|
149
|
+
|
|
150
|
+
Everything in this skill applies to the whole Nano Banana family; two variants trade speed/ceiling around NB2 full:
|
|
151
|
+
|
|
152
|
+
- **nano-banana-2-lite** — ~half the price, ~2.7× faster, **1K output only**, max 4 refs. The draft/iteration seat: explore compositions here, then re-run the winner on NB2 full at 2K/4K. Same Gemini filter.
|
|
153
|
+
- **nano-banana-pro** — the hero-frame/typography ceiling (~2× NB2, 4K native). NB2 ≈ 95% of Pro; escalate only when spatial composition, cinematic lighting/skin, fine typography-in-scene, or deep multi-element frames must be perfect. Up to 14 refs — it takes a full subject library in one call.
|
|
154
|
+
|
|
155
|
+
Routing between them (and vs GPT Image 2 / FLUX / Seedream): `slates-model-selection`.
|
|
@@ -83,6 +83,15 @@ Reference-to-video endpoint accepts up to **9 reference images, 3 reference vide
|
|
|
83
83
|
|
|
84
84
|
**Mutually exclusive:** First-frame/last-frame mode CANNOT be combined with reference images. The error reads `"first/last frame content cannot be mixed with reference media content."` Pick one or the other.
|
|
85
85
|
|
|
86
|
+
### Motion transfer & lip-sync recipes (reference video / audio)
|
|
87
|
+
|
|
88
|
+
These aren't separate Seedance features — they're prompting strategies over reference media, and the Slates tools (`slates_generate_motion_transfer` / `slates_generate_lip_sync` with the seedance engine) compose them for you. When driving them by hand through `slates_generate_video`:
|
|
89
|
+
|
|
90
|
+
- **Motion transfer:** subject image as a reference + the driving clip via `videoReferenceAssetId` (2–15s) + `The character from image 1 performs the exact motion, choreography, and camera movement from video 1. Preserve the character's identity, appearance, and outfit.`
|
|
91
|
+
- **Lip-sync / dialogue:** write the line in the prompt — `The person in video 1 says: "…"` — with `generate_audio` on (always on in Slates). A **video** source's own voice is cloned natively; an **audio** reference (`audioReferenceAssetId`, ≤15s) drives speech from an existing recording: `…speaks the dialogue from audio 1 with accurate lip sync.`
|
|
92
|
+
- **Voice + face from one clip (the talking-head recipe):** ONE unedited 2–15s clip of the person speaking (clear voice, no music, no cuts) as the video reference + prompt with the new script → their likeness AND voice deliver the new line.
|
|
93
|
+
- **Billing:** a reference VIDEO switches the cost key to `seedance-2*-vref-{res}-{T}s` where T = clip seconds + output seconds — quote before confirming. Audio references are free (audio is included on every route).
|
|
94
|
+
|
|
86
95
|
## Faces — set `seedanceFace` for AI-character faces
|
|
87
96
|
|
|
88
97
|
Seedance routes through **three tiers** depending on the face in the reference, exposed as the "Face in Reference" toggle plus the real-face params on `slates_generate_video`:
|
|
@@ -33,7 +33,7 @@ Ask: **"Generate frame images now? (y/N)"**
|
|
|
33
33
|
|
|
34
34
|
### 3. Generate frames if requested
|
|
35
35
|
For each frame:
|
|
36
|
-
- Estimate cost (`slates_estimate_generation_cost`, `count = total frames`). Confirm with user if total >
|
|
36
|
+
- Estimate cost (`slates_estimate_generation_cost`, `count = total frames`). Confirm with user if total > ~17 credits.
|
|
37
37
|
- Generate sequentially, with character/environment/style references attached when present in the project (`slates_list_characters`, `slates_list_environments`).
|
|
38
38
|
- Each result returns inline. Evaluate. If wrong, refine prompt + regenerate (charge once, not multiple).
|
|
39
39
|
- Bind to the frame via `slates_add_frame`.
|
|
@@ -47,7 +47,7 @@ The code is the FORMAL reference. The label is human texture. Use both: `IMG-A12
|
|
|
47
47
|
|
|
48
48
|
- Track total credits spent across the loop. Surface to the user every 3 iterations.
|
|
49
49
|
- Stop after 3 failed iterations on the same prompt — escalate to the user with what you tried and what's not working. The slot machine never converges.
|
|
50
|
-
- For high-cost generations (
|
|
50
|
+
- For high-cost generations (above ~17 credits), confirm before *every* attempt, not just the first.
|
|
51
51
|
|
|
52
52
|
## When to break the loop
|
|
53
53
|
|