@slatesvideo/shared 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/README.md +5 -0
  2. package/dist/clients/cloud.js +32 -1
  3. package/dist/clients/desktop.d.ts +10 -4
  4. package/dist/clients/desktop.js +95 -7
  5. package/dist/index.d.ts +2 -1
  6. package/dist/index.js +1 -0
  7. package/dist/operations/index.d.ts +207 -1
  8. package/dist/operations/index.js +1513 -32
  9. package/dist/skills/content.d.ts +2 -0
  10. package/dist/skills/content.js +20 -0
  11. package/package.json +31 -4
  12. package/skills/slates-character-turnaround.md +53 -0
  13. package/skills/slates-cost-discipline.md +96 -0
  14. package/skills/slates-direct-response-ad.md +66 -0
  15. package/skills/slates-edit-and-iterate.md +48 -0
  16. package/skills/slates-one-prompt-film.md +67 -0
  17. package/skills/slates-prompting-flux-2-max.md +106 -0
  18. package/skills/slates-prompting-kling-v3.md +160 -0
  19. package/skills/slates-prompting-lip-sync.md +169 -0
  20. package/skills/slates-prompting-motion-transfer.md +148 -0
  21. package/skills/slates-prompting-nano-banana-2.md +147 -0
  22. package/skills/slates-prompting-seedance.md +144 -0
  23. package/skills/slates-prompting-seedream-5-lite.md +74 -0
  24. package/skills/slates-prompting-veo-3.md +149 -0
  25. package/skills/slates-storyboard-from-script.md +49 -0
  26. package/skills/slates-vision-feedback-loop.md +62 -0
  27. package/dist/auth.d.ts.map +0 -1
  28. package/dist/auth.js.map +0 -1
  29. package/dist/clients/cloud.d.ts.map +0 -1
  30. package/dist/clients/cloud.js.map +0 -1
  31. package/dist/clients/desktop.d.ts.map +0 -1
  32. package/dist/clients/desktop.js.map +0 -1
  33. package/dist/index.d.ts.map +0 -1
  34. package/dist/index.js.map +0 -1
  35. package/dist/operations/index.d.ts.map +0 -1
  36. package/dist/operations/index.js.map +0 -1
@@ -0,0 +1,106 @@
1
+ ---
2
+ name: slates-prompting-flux-2-max
3
+ description: How to prompt FLUX.2 Max (Black Forest Labs image model). Read before calling slates_generate_image with model flux-2-max, or slates_edit_image with editModel flux-2-max. FLUX.2 wants front-loaded structure, real camera vocabulary, and positive-only phrasing — no negative prompts, no tag soup.
4
+ ---
5
+
6
+ # FLUX.2 Max — prompting
7
+
8
+ Black Forest Labs' top image model, routed via fal.ai. In Slates: `slates_generate_image` with `model: flux-2-max` (REQUIRES projectId — no headless path), priced per resolution (1k/2k/4k — call `slates_estimate_generation_cost` for current numbers, never quote from memory). Strengths vs Nano Banana 2: photoreal texture, less censored, precise hex-color control, strong typography. Reference images route through FLUX's edit endpoint and carry a lower per-model cap than NB2's 14.
9
+
10
+ ## Core structure — front-load what matters
11
+
12
+ ```
13
+ Subject + Action + Style + Context
14
+ ```
15
+
16
+ Word order is weight. FLUX.2 attends hardest to the start of the prompt: main subject → key action → critical style → essential context → secondary details.
17
+
18
+ **Length:** 10-30 words for concept tests, 30-80 words for most work, 80+ only for genuinely complex scenes.
19
+
20
+ ## Photorealism: name real gear, not "professional photo"
21
+
22
+ The single biggest realism lever is concrete camera vocabulary:
23
+
24
+ ```
25
+ Shot on Hasselblad X2D, 80mm lens, f/2.8, natural lighting
26
+ Shot on Sony A7IV, 35mm, golden hour, shallow depth of field
27
+ Kodak Portra 400, natural grain, organic colors
28
+ ```
29
+
30
+ Era cues work the same way: "early digital camera, slight noise, flash photography, candid" reads 2000s digicam; "film grain, warm color cast, soft focus" reads 80s.
31
+
32
+ For portraits add: natural skin texture, realistic pores, subtle imperfections, soft diffused lighting.
33
+
34
+ ## No negative prompts — reframe positively
35
+
36
+ FLUX.2 has no negative prompt support. Describe the presence you want, not the absence:
37
+
38
+ - ❌ "no blur" → ✅ "sharp focus throughout"
39
+ - ❌ "no people" → ✅ "empty scene"
40
+ - ❌ "no harsh shadows" → ✅ "soft, diffused lighting"
41
+
42
+ ## Hex colors — bind them to objects
43
+
44
+ FLUX.2 matches hex codes, but only when each code is attached to a specific object:
45
+
46
+ ```
47
+ walls in hex #C4725A, sofa in #1B6B6F, accent pillows #E8A847
48
+ gradient starting with color #02eb3c and finishing with color #edfa3c
49
+ ```
50
+
51
+ ❌ "use #FF0000 somewhere" — unbound colors land inconsistently.
52
+
53
+ ## Text rendering
54
+
55
+ Quote the exact text, then place and style it:
56
+
57
+ ```
58
+ The text 'OPEN' appears in red neon letters above the door
59
+ Logo text 'ACME' in color #FF5733, ultra-bold decorative serif, centered
60
+ ```
61
+
62
+ Specify placement relative to other elements, font family feel (serif / sans / script), and relative size ("large headline," "small body copy").
63
+
64
+ ## JSON prompting for production work
65
+
66
+ For multi-element scenes that must come out exactly right (product shots, infographics, brand work), FLUX.2 parses structured JSON prompts:
67
+
68
+ ```json
69
+ {
70
+ "scene": "Professional studio product photography on polished concrete",
71
+ "subjects": [{ "description": "matte black ceramic mug with steam", "position": "center foreground" }],
72
+ "style": "commercial product photography",
73
+ "color_palette": ["#1B1B1B", "#E8A847"],
74
+ "lighting": "three-point softbox, soft diffused highlights",
75
+ "camera": { "lens-mm": 85, "f-number": "f/5.6" }
76
+ }
77
+ ```
78
+
79
+ Use natural language for exploration, JSON when the layout is locked and you're matching a spec.
80
+
81
+ ## Reference images (edit path)
82
+
83
+ In Slates, pass `referenceAssetIds` on `slates_generate_image` — FLUX routes them through its edit endpoint. Label every reference's role in the prompt text ("subject from image 1, style from image 2, background from image 3"); unlabeled references get blended unpredictably. For surgical changes to one existing image use `slates_edit_image` with `editModel: flux-2-max` (note: FLUX edits ignore extra referenceAssetIds — that's NB2-only).
84
+
85
+ ## Character consistency across a series
86
+
87
+ Define the character exhaustively once, then repeat those exact descriptors verbatim in every subsequent prompt. FLUX has no memory between generations — the repeated description IS the consistency mechanism.
88
+
89
+ ## Common failure modes + fixes
90
+
91
+ | Failure | Fix |
92
+ |---|---|
93
+ | Generic "AI look" on photoreal | Name a camera body + lens + f-stop instead of "professional photo" |
94
+ | Colors drift from brand spec | Bind each hex code to a named object |
95
+ | Text garbled | Quote the exact string, specify font feel + placement + size |
96
+ | Multi-reference blend chaos | Label each reference's role explicitly in the prompt |
97
+ | Wanted element missing | Move it earlier in the prompt — order is weight |
98
+
99
+ ## Pre-flight: references arrive inline, refer by code
100
+
101
+ When you pass `referenceAssetIds`, the first call returns the references **inline as image content blocks** with a cost estimate and `requires_confirm: true`. Look at them — revise the prompt if they suggest a different composition or style — then re-call with `confirm=true`. Refer to each asset by its short code (`IMG-A12 — Beach Sunset`) when talking to the user; it matches the badge on their gallery thumbnail.
102
+
103
+ ## Sources
104
+
105
+ - [Black Forest Labs — FLUX.2 Prompting Guide](https://docs.bfl.ml/guides/prompting_guide_flux2)
106
+ - [fal.ai — FLUX.2 [max] Prompt Guide](https://fal.ai/learn/devs/flux-2-max-prompt-guide)
@@ -0,0 +1,160 @@
1
+ ---
2
+ name: slates-prompting-kling-v3
3
+ description: How to prompt Kling V3.0 (Kuaishou). Read before calling slates_generate_video with kling-v3.0-std, kling-v3.0-pro, or kling-v3.0-omni. Kling has dialogue + SFX + ambient native syntax (Omni adds multi-character dialogue and language codes). Multi-shot rules differ from Seedance/Veo — don't cross syntaxes.
4
+ ---
5
+
6
+ # Kling V3.0 — prompting
7
+
8
+ Kuaishou's video model. Three tiers: `kling-v3.0-std` (general use, no audio), `kling-v3.0-pro` (higher visual quality, no audio), `kling-v3.0-omni` (multi-character dialogue + audio-visual co-generation).
9
+
10
+ Up to 15s. Multi-shot supported (up to 6 cuts in 15s total). Strong on image-to-video — preserves identity, layout, and text from the input image well.
11
+
12
+ ## Subject definition rule (verbatim, fal blog)
13
+
14
+ > "Define your core subjects clearly at the beginning of the prompt and keep descriptions consistent across shots."
15
+
16
+ ## Dialogue syntax
17
+
18
+ ```
19
+ Character says, "exact words here"
20
+ ```
21
+
22
+ Use quotation marks for precise speech. Languages (Omni only): EN, ZH, JA, KO, ES.
23
+
24
+ ## Voice direction formula (Omni)
25
+
26
+ ```
27
+ Gender + Age Range + Voice Quality + Speech Rate + Emotional Tone + Language
28
+ ```
29
+
30
+ Example:
31
+ ```
32
+ [Character A: Detective, mid-40s, raspy voice, slow cadence, weary]: "I've seen this before."
33
+ ```
34
+
35
+ Tone phrases that fire:
36
+ - `speaking in a hushed, trembling whisper`
37
+ - `shouting with commanding authority`
38
+ - `clear, fearful voice`
39
+ - `with a trembling voice, "I'm scared"`
40
+
41
+ ## The `Immediately` keyword (Omni only)
42
+
43
+ Without `Immediately`, Kling adds a natural conversational beat between speakers. With it, dialogue is back-to-back. Use when timing matters.
44
+
45
+ ```
46
+ [Alice]: "Get down!" Immediately, [Bob]: "Where?"
47
+ ```
48
+
49
+ ## Speaker label discipline
50
+
51
+ Unique labels per character. **No pronouns or synonyms after first introduction** — they cause voice drift.
52
+
53
+ ✅ `[Character A: Black-suited Agent]` ... `[Character A: Black-suited Agent]: "Stop."`
54
+ ❌ `[Agent]... then he says...`
55
+
56
+ ## Multi-character dialogue (Omni)
57
+
58
+ ```
59
+ Alice says in English, "Hello!" Then Bob replies in Spanish, "¡Hola!"
60
+ ```
61
+
62
+ ## Sound effects, ambient noise, music
63
+
64
+ ```
65
+ SFX: thunder cracks, footsteps approaching
66
+ Ambient noise: city traffic, birds chirping, ocean waves
67
+ Background music: tense orchestral strings, low cello
68
+ ```
69
+
70
+ SFX accepts physical-cause specificity:
71
+ - ✅ `SFX: heavy boots on wet pavement, distant siren wailing`
72
+ - ❌ `SFX: footsteps`
73
+
74
+ ## Image-to-video guidance
75
+
76
+ **Verbatim (fal blog):**
77
+ > "Treat the input image as an anchor. Kling 3.0 excels at preserving the identity, layout, and text details. Focus prompts on how the scene evolves *from* the image: subtle movements, camera motion, or environmental changes."
78
+
79
+ **Don't re-describe what's already in the image.** Focus on motion, changes, evolution.
80
+
81
+ ## Multi-shot — what makes them hit
82
+
83
+ **Hard cap: total duration ≤ 15s across all shots. Max 6 cuts.**
84
+
85
+ Hit conditions:
86
+ - Shot labels are explicit: `Shot 1:`, `Shot 2:`
87
+ - One primary action per shot
88
+ - Subject described identically in each shot block
89
+ - Camera move per shot is **one verb**, not a chain
90
+ - Per-shot blocks: 30-60 words
91
+
92
+ Miss conditions:
93
+ - Compressing narrative into one paragraph
94
+ - Pronoun-only references after the first shot
95
+ - Mixing camera moves within a shot ("pan then orbit then push in")
96
+ - Extreme wide → extreme close in adjacent shots without reference images
97
+
98
+ ## Element references (Omni)
99
+
100
+ Upload 2-4 multi-angle reference photos per character/object. Tag inline:
101
+
102
+ ```
103
+ @element1 is the protagonist (refs: front, side, back angles).
104
+ @element2 is the antagonist.
105
+ ```
106
+
107
+ ## Negative prompting — has a real field
108
+
109
+ Kling exposes `negative_prompt` on the fal endpoint (different from Seedance which has none). Default block to start from:
110
+
111
+ ```
112
+ blurry, low quality, watermark, text overlay, distorted hands, extra fingers,
113
+ duplicate limbs, unnatural skin texture, overly saturated colors, lens flare,
114
+ floating objects, inconsistent shadows, jittery, flickering, morphing face
115
+ ```
116
+
117
+ Layer scene-specific suppressions on top.
118
+
119
+ ## Cinematic tactics
120
+
121
+ - **Motion adverb precision** modulates motion energy directly: `slowly`, `rapidly`, `gently`, `explosively`
122
+ - **Camera vocabulary that registers as instructions:** profile shot, tracking, following, freezing, panning, "moving in sync with the subject"
123
+ - **One primary camera move per shot** — never stack
124
+
125
+ ## Tier choice
126
+
127
+ - **Standard**: general use, no audio
128
+ - **Pro**: higher visual quality, no audio
129
+ - **Omni**: multi-character dialogue, audio-visual co-gen, language codes, `@elementN` references
130
+
131
+ Pick by capability: need dialogue/audio → Omni; need maximum visual quality silent → Pro; everything else → Standard. Prices change — call `slates_estimate_generation_cost` or `slates_list_available_models` for current numbers before choosing a tier.
132
+
133
+ ## Benchmark prompt structure
134
+
135
+ ```
136
+ [Character A: <role>, <voice quality>]: "<line>." Immediately, [Character B: <role>, <voice quality>]: "<reply>."
137
+ Ambient noise: <soundscape>.
138
+ Camera <single move>.
139
+ ```
140
+
141
+ Cinematic example (paraphrasing fal blog patterns):
142
+ > "Shot 1: Wide establishing shot of a neon-lit alleyway in heavy rain, steam rising from grates. Camera slowly tracks forward.
143
+ > Shot 2: Medium shot of a detective in a trench coat ducking under an awning, water dripping from his hat brim. [Detective: weary, raspy]: 'I knew she'd come back.' Ambient noise: distant traffic, rain on metal.
144
+ > Shot 3: Close-up on his eyes, narrowing as headlights flash across his face."
145
+
146
+ ## Pre-flight: references arrive inline, refer by code
147
+
148
+ When you call `slates_generate_video` with `firstFrameAssetId` or `ingredientAssetIds`, the first call returns those references **inline as image content blocks** alongside cost + `requires_confirm: true`. Look at them, revise prompt if needed, then re-call with `confirm=true`. Kling Omni multi-character with several ingredient images especially benefits — confirm each character image lands cleanly before spending.
149
+
150
+ When talking to the user about the gen, refer to each reference by its short code: `IMG-A12 — Detective Closeup`. The user sees that code as a gallery badge.
151
+
152
+ - ✅ "I'm anchoring on **IMG-A12** as the detective and **IMG-A18** as the alleyway environment — Omni will handle the line delivery in EN."
153
+ - ❌ "I'm using the detective image and the alley one..." (which alley? Three exist.)
154
+
155
+ ## Sources
156
+
157
+ - [fal.ai — Kling 3.0 Prompting Guide](https://blog.fal.ai/kling-3-0-prompting-guide/)
158
+ - [Vidguru — Kling 3.0 Omni Guide](https://www.vidguru.ai/blog/kling-3.0-omni-guide.html)
159
+ - [AcceptPrompt — Kling 3 Prompt Guide](https://www.acceptprompt.com/blog/kling-3-prompt-guide)
160
+ - [DataCamp — Kling 3.0 Tutorial](https://www.datacamp.com/tutorial/kling-3-0)
@@ -0,0 +1,169 @@
1
+ ---
2
+ name: slates-prompting-lip-sync
3
+ description: How to set up Kling lip-sync. Read before calling slates_generate_lip_sync. Two flows — video→video re-dub and image→video avatar — with different inputs, pricing, and gotchas. Voice catalog, framing rules, audio file constraints, and which tier to pick.
4
+ ---
5
+
6
+ # Kling lip-sync — setup guide
7
+
8
+ Two distinct flows, both 5-second outputs:
9
+
10
+ | Flow | Source | Model | Cost (5s) | Use case |
11
+ |------|--------|-------|-----------|----------|
12
+ | Re-dub | video clip | kling-lip-sync-video | $0.11 | Replace dialogue on an existing talking head |
13
+ | Avatar standard | still image | ai-avatar/v2/standard | $0.42 | Animate a portrait into a talking avatar |
14
+ | Avatar pro | still image | ai-avatar/v2/pro | $0.86 | Higher facial fidelity for hero shots |
15
+
16
+ Pick `sourceType` deliberately — it decides the pricing tier and the underlying endpoint.
17
+
18
+ ## Choosing video vs avatar
19
+
20
+ Use **video** (re-dub) when:
21
+ - A talking-head clip already exists (Slates-generated, recorded, or imported)
22
+ - The mouth/face is already moving and only the audio needs to change
23
+ - $0.11 is hard to beat for short dialogue replacement
24
+
25
+ Use **avatar** when:
26
+ - Only a still portrait exists
27
+ - The character needs to come alive from a single image
28
+ - Identity + face fidelity matter (avatar-pro for hero shots, standard for everything else)
29
+
30
+ ## Source asset constraints
31
+
32
+ ### Video flow (`sourceType: 'video'`)
33
+ - Format: mp4 or mov
34
+ - Duration: 2–10s (lip-sync output is always 5s — long videos get trimmed)
35
+ - Resolution: 720p or 1080p (480p will be rejected)
36
+ - Max file size: 100MB
37
+ - Face must be visible and roughly facing camera. Profile shots fail.
38
+ - Existing audio is replaced.
39
+
40
+ ### Avatar flow (`sourceType: 'image'`)
41
+ - Min 512×512, PNG/JPG/WebP
42
+ - **Face occupies 60–70% of frame.** This is the single biggest avatar quality lever.
43
+ - Eyes open, mouth neutral, looking near-camera. Side profile = bad output.
44
+ - Single subject, clean background. Group photos confuse the face anchor.
45
+
46
+ ## Audio source
47
+
48
+ Two ways to drive the lips:
49
+
50
+ ### TTS (`audioMethod: 'tts'`)
51
+ - Pass `ttsText` (the words spoken)
52
+ - Optional: `ttsVoice` (default `oversea_male1`), `ttsLanguage` (default EN), `ttsSpeed` (default 1.0)
53
+ - **Hard cap: 120 characters of text.** Longer = silently truncated.
54
+ - Languages: EN, ZH, JA, KO, ES
55
+
56
+ ### Upload (`audioMethod: 'upload'`)
57
+ - Pass `audioFilePath` — absolute path to an audio file on the user's machine
58
+ - Format: mp3, wav, m4a, ogg, aac
59
+ - Max 5MB
60
+ - Duration: 2–60s (output is 5s — longer audio gets trimmed)
61
+ - Single clean voice. Music underneath, multiple speakers, or noisy mics produce garbage lips.
62
+
63
+ Prefer upload for production-quality voice. TTS for fast iteration / placeholder dialogue.
64
+
65
+ ## Voice catalog (TTS)
66
+
67
+ Reliable English voices (verified working on the fal endpoint as of 2026):
68
+
69
+ | Voice ID | Description |
70
+ |----------|-------------|
71
+ | `oversea_male1` | Male, English — default, stable |
72
+ | `commercial_lady_en_f-v1` | Female commercial English |
73
+ | `uk_boy1` | Young man, UK accent |
74
+ | `uk_man2` | Man, UK accent |
75
+ | `uk_oldman3` | Older man, UK accent |
76
+ | `calm_story1` | Storyteller / narrator |
77
+
78
+ Avoid `reader_en_m-v1` — listed in fal.ai docs but returns "Voice id not found" in production.
79
+
80
+ Full 48-voice list (ZH, JA, KO included): https://fal.ai/models/fal-ai/kling-video/lipsync/text-to-video/api
81
+
82
+ ## Speech-rate notes
83
+
84
+ `ttsSpeed` range 0.5–2.0:
85
+ - 0.8–1.0: natural conversational
86
+ - 1.1–1.3: punchy ad delivery
87
+ - 1.4+: rushed, clips consonants
88
+ - 0.6–0.7: slow, weighty (good for dramatic lines)
89
+
90
+ Default 1.0 unless the line specifically calls for slower or faster cadence.
91
+
92
+ ## Avatar prompt usage
93
+
94
+ The `prompt` parameter on avatar-v2 (standard + pro) is **scene context**, not motion direction. The mouth animation comes from the audio — the prompt sets ambiance, lighting, micro-expression.
95
+
96
+ Good:
97
+ - `Soft rim light, warm office, gentle confident smile between sentences.`
98
+ - `Cool blue evening light through a window, focused intent expression.`
99
+
100
+ Bad (the model ignores motion verbs):
101
+ - ❌ `She turns her head, raises an eyebrow, then speaks.`
102
+ - ❌ `Hand gestures while talking.`
103
+
104
+ Default `"."` is fine if you have nothing useful to add.
105
+
106
+ ## Tier selection — avatar standard vs pro
107
+
108
+ **Use standard** when:
109
+ - Drafts, A/B testing voices, internal review reels
110
+ - Wide / medium shots where face isn't the focal point
111
+ - Cost matters more than micro-expression fidelity
112
+
113
+ **Use pro** when:
114
+ - Final ads where the avatar's face fills the screen
115
+ - The character is named / branded — identity drift kills the take
116
+ - You're already paying $1+ for the surrounding video pipeline
117
+
118
+ Don't default to pro. The $0.44 delta per take adds up across iteration.
119
+
120
+ ## Common failure modes
121
+
122
+ | Symptom | Likely cause | Fix |
123
+ |---------|--------------|-----|
124
+ | Lip movement looks "rubber" / disconnected | Source face <60% of frame | Re-crop the still tighter |
125
+ | Voice doesn't match character age/gender | Default voice id used | Pick from voice catalog |
126
+ | Output truncated mid-word | TTS text >120 chars | Shorten or chain two takes |
127
+ | Garbled mouth on uploaded audio | Background music / multi-voice | Use clean dialogue-only audio |
128
+ | "Voice id not found" 422 | Hit `reader_en_m-v1` | Switch to `oversea_male1` |
129
+ | Avatar eyes drift / cross | Source had closed/angled eyes | Pick a frame with neutral open eyes |
130
+ | Generation completes but lips don't move | Profile shot / face >70° off-axis | Use a near-frontal portrait |
131
+
132
+ ## Cost discipline
133
+
134
+ - Video re-dub at $0.11 is the cheapest dialogue iteration in the entire Slates stack — use it for voice A/B testing
135
+ - Avatar standard at $0.42 is fine for medium use
136
+ - Avatar pro at $0.86 trips the >$0.50 confirm gate — explicit user OK required every time
137
+ - All 5s. There is no shorter option.
138
+
139
+ ## Workflow patterns
140
+
141
+ **Voice A/B test (cheap):**
142
+ 1. Generate one base talking-head video clip with Veo or Seedance (~$1.20)
143
+ 2. Run `slates_generate_lip_sync` with `sourceType: 'video'` against 3–5 different `ttsVoice` values
144
+ 3. Total cost: $1.20 + (5 × $0.11) = $1.75 to compare voices
145
+
146
+ **Brand avatar from a single portrait:**
147
+ 1. Generate or upload the hero portrait (face fills frame, eyes open, neutral mouth)
148
+ 2. Avatar standard for first-pass dialogue takes
149
+ 3. Avatar pro only on the final selected take
150
+
151
+ **Avoid:**
152
+ - Avatar pro on first iteration (waste — facial fidelity isn't visible until you've locked the line)
153
+ - TTS for final ads (production should use real voice or cloned voice — the upload flow)
154
+ - Uploading raw recordings — clean noise + level the file first, lip detection is sensitive
155
+
156
+ ## Confirm gate: cost + codes, no inline preview
157
+
158
+ Lip-sync is mechanical — the model re-syncs the chosen source to the chosen audio. The confirm response carries the source asset's code so you can announce it in chat.
159
+
160
+ - ✅ "Lip-syncing **IMG-A12 — Founder Portrait** to the new line. $0.86 on avatar-pro. Confirm?"
161
+ - ❌ "Using the founder image..." (which? Three exist.)
162
+
163
+ Don't second-guess the source. If the output is wrong, iterate on source choice or audio, not on a refinement prompt (there isn't one).
164
+
165
+ ## Sources
166
+
167
+ - [fal.ai — Kling LipSync API](https://fal.ai/models/fal-ai/kling-video/lipsync/text-to-video/api)
168
+ - [fal.ai — AI Avatar v2 Standard](https://fal.ai/models/fal-ai/kling-video/ai-avatar/v2/standard/api)
169
+ - [fal.ai — AI Avatar v2 Pro](https://fal.ai/models/fal-ai/kling-video/ai-avatar/v2/pro/api)
@@ -0,0 +1,148 @@
1
+ ---
2
+ name: slates-prompting-motion-transfer
3
+ description: How to set up Kling Motion Control (motion transfer). Read before calling slates_generate_motion_transfer. Reference image (character) + driving video (motion source) → new video of the character performing the motion. Asset selection rules, character_orientation choice, std vs pro tier, and prompt usage.
4
+ ---
5
+
6
+ # Kling Motion Control — setup guide
7
+
8
+ Take a still **target image** (your character) and a **source video** (the motion you want), produce a new 5s video of your character performing the source video's motion.
9
+
10
+ | Tier | Cost (5s) | Use case |
11
+ |------|-----------|----------|
12
+ | std (`kling-mc-std-5s`) | $0.95 | General motion transfer |
13
+ | pro (`kling-mc-pro-5s`) | $1.26 | Cleaner anatomy, better identity preservation |
14
+
15
+ Both tiers trip the >$0.50 confirm gate. User OK required every time.
16
+
17
+ ## Inputs
18
+
19
+ - `sourceVideoAssetId` — driving video. **Must be a realistic human** with clear proportions. Anime/cartoon/CG driving videos fail.
20
+ - `targetImageAssetId` — character to be animated. Can be any style (cartoon, anime, realistic, painted).
21
+ - Both must already exist as assets in the project. Use `slates_list_assets` to find them or upload first.
22
+
23
+ ## Source video constraints
24
+
25
+ - Realistic human (not animated, not CG)
26
+ - Entire body OR upper body visible — head must not be obstructed
27
+ - Subject occupies a clear share of the frame
28
+ - Single primary subject. Multi-person driving videos confuse the motion anchor.
29
+ - Clean motion — choppy / cut-edited driving videos produce jittery output
30
+
31
+ Good driving video sources:
32
+ - Reference dance footage with one subject
33
+ - Walking / gesture / posing clips
34
+ - Talking-head footage when paired with character_orientation: 'video'
35
+
36
+ Bad driving video sources:
37
+ - Music videos with multi-shot edits
38
+ - Anime / animation clips
39
+ - Heavily stylized footage with smoke / particles obscuring the body
40
+ - Footage where the subject's head leaves frame mid-clip
41
+
42
+ ## Target image constraints
43
+
44
+ - Character body proportions clearly visible
45
+ - Character occupies >5% of image area (not a tiny figure in a wide shot)
46
+ - Single character. Group images break the identity anchor.
47
+ - Any artistic style works — cartoon, anime, painted, realistic, 3D render
48
+
49
+ Avoid:
50
+ - Extreme close-up of just the face (no body to drive)
51
+ - Character partially cropped at the waist when the driving video is full-body
52
+ - Multiple characters
53
+
54
+ ## character_orientation — the most-missed choice
55
+
56
+ This single parameter changes the output dramatically. Pick deliberately.
57
+
58
+ | Value | Output framing | Max source duration | Best for |
59
+ |-------|----------------|---------------------|----------|
60
+ | `video` | Matches driving video framing | Up to 30s source | Complex full-body motion (dance, action, athletics) |
61
+ | `image` | Matches target image framing | Up to 10s source | Camera moves, simpler motion, preserving original composition |
62
+
63
+ **Default `video`** when the driving video has the look you want (most cases).
64
+
65
+ Switch to `image` when the target image's composition is the brand asset and the motion is secondary (e.g., a hero shot of a character that needs subtle gesture, not a full performance).
66
+
67
+ ## Tier choice — std vs pro
68
+
69
+ **std ($0.95)** for:
70
+ - Drafts, motion exploration, blocking
71
+ - Group scenes where the character isn't a hero shot
72
+ - When the budget is tight and the motion is the focus
73
+
74
+ **pro ($1.26)** for:
75
+ - Final hero takes
76
+ - Branded characters where identity drift = unacceptable
77
+ - Anatomically complex motion (limbs crossing, fast direction changes)
78
+ - Anime / cartoon target images — pro handles non-realistic styles better
79
+
80
+ Don't default to pro. The $0.31 delta compounds fast across iteration.
81
+
82
+ ## Prompt usage (optional)
83
+
84
+ The `prompt` field is **scene/style refinement**, not motion direction. The motion comes from the driving video — the prompt sets ambiance, lighting, additional detail.
85
+
86
+ Good:
87
+ - `Soft afternoon sunlight, dust motes in the air, vintage warm color grade.`
88
+ - `Clean studio backdrop, sharp focus on the character.`
89
+
90
+ Bad (model ignores motion verbs — they're already in the driving video):
91
+ - ❌ `She spins faster and jumps higher.`
92
+ - ❌ `Add more energy to the dance.`
93
+
94
+ Leave it empty if you don't have a specific atmospheric note.
95
+
96
+ ## Common failure modes
97
+
98
+ | Symptom | Likely cause | Fix |
99
+ |---------|--------------|-----|
100
+ | Limbs distort / extra fingers | std tier, complex motion | Switch to pro |
101
+ | Character identity drifts | Target image cropped too tight | Use a fuller-body target |
102
+ | Output looks "stuck" / minimal motion | Driving video subject too small in frame | Pick a driving video where the subject fills more of the frame |
103
+ | Cartoon target turns realistic | std tier on stylized art | Switch to pro — handles non-realistic styles better |
104
+ | Garbled output entirely | Anime / CG driving video | Use realistic human driving footage |
105
+ | Wrong framing on output | character_orientation set wrong | Try the other value |
106
+ | Background bleeds through character | Target image had complex background | Use a target with cleaner background separation |
107
+
108
+ ## Workflow patterns
109
+
110
+ **Reference dance to brand character:**
111
+ 1. Generate or upload the brand character as a still image (clean background, full body, single subject)
112
+ 2. Find driving footage — a clean reference video of the dance you want
113
+ 3. Upload both as project assets
114
+ 4. Run motion transfer with `motionModel: 'kling-mc-pro'`, `characterOrientation: 'video'`
115
+ 5. Total cost: $1.26 per 5s take
116
+
117
+ **Subtle motion on a hero portrait:**
118
+ 1. Use the locked hero portrait as the target image
119
+ 2. Pick a driving video with subtle gesture (head turn, slight posture shift)
120
+ 3. `characterOrientation: 'image'` to preserve the portrait's framing
121
+ 4. std tier is fine for this case — motion isn't dramatic
122
+
123
+ **Avoid:**
124
+ - Pro tier on first iteration — waste, switch to it once the motion + framing combo is locked
125
+ - Cartoon driving videos — guaranteed failure
126
+ - Cropped or partial target characters — identity will drift
127
+ - Long driving videos when output is 5s — pick the best 5s of the source upfront
128
+
129
+ ## Cost discipline
130
+
131
+ - 5 seconds, no shorter option
132
+ - Both tiers trip the >$0.50 confirm gate — every call needs explicit user OK
133
+ - Iteration is expensive: 4 takes at pro = $5.04. Lock framing + driving video before tier-up to pro.
134
+ - Always run a single std take first to validate the motion + framing combo before committing to pro
135
+
136
+ ## Confirm gate: cost + codes, no inline preview
137
+
138
+ Motion transfer is mechanical — the model deterministically applies source motion to target image. Both tiers trip the >$0.50 confirm gate; the response includes the asset codes for source and target so you can announce them in chat.
139
+
140
+ - ✅ "Transferring motion from **VID-V3** onto **IMG-A12 — Detective Closeup**. $1.26, confirm?"
141
+ - ❌ "Using the walk video and the detective image..." (multiple of each in the project.)
142
+
143
+ Don't second-guess the assets the user picked — the model executes the transfer. If the output is wrong, iterate on motion source or target choice, not on a refinement prompt.
144
+
145
+ ## Sources
146
+
147
+ - [fal.ai — Kling Motion Control V3 Standard](https://fal.ai/models/fal-ai/kling-video/v3/standard/motion-control)
148
+ - [fal.ai — Kling Motion Control V3 Pro](https://fal.ai/models/fal-ai/kling-video/v3/pro/motion-control)