@slatesvideo/shared 0.4.7 → 0.5.0

package/skills/slates-model-selection.md

@@ -18,6 +18,31 @@ Pick the model FIRST, deliberately, before writing a prompt or quoting a plan. M
 | The premium hero shot a piece hangs on | Seedance 2.0 | Spend where it shows. |
 | Native synchronized audio (dialogue + SFX generated WITH the video in one gen), 16:9, ≤8s | Veo 3.1 | The only job Veo wins. |
 
+## Video EDIT routing (changing an existing clip)
+
+| Job | Tool | Why |
+|---|---|---|
+| **Fix a 90%-right clip** — wrong shirt, one artifact, swap the subject, change the environment | **Kling O3 Edit** (`slates_edit_video`) | The default edit tool. Element lock (frontal + angles) holds identity; original motion, camera, and AUDIO preserved. One pass, no masking, ~19¢/s. |
+| Style-transfer-heavy re-imagining, full relocate of the scene | Seedance edit/relocate (`videoReferenceAssetId` on `slates_generate_video`) | Seedance's strength is transfer intensity; it re-generates rather than surgically edits. |
+| AI-edit the user's OWN footage (3–15s clips) | Kling O3 Edit | Takes any MP4/MOV 3–15s, 720–3840px — not just Slates gens. |
+
+- **Edit before re-roll.** A re-roll gambles away the parts the user already likes; an edit changes only what the prompt names. Quote the edit first when a clip is mostly right.
+- Edited clips are themselves editable ≤15s clips — chain passes; lineage links each output to its parent.
+
+## Motion Transfer & Lip Sync routing (two engines per tool)
+
+Both tools have a cheap Kling utility lane and a premium Seedance lane. The capability is the same; the execution model differs: Kling bolts motion/lip onto the source as a dedicated post-process; Seedance generates in a single pass with the driving clip / dialogue as native conditioning signals — better motion fidelity, natural speech delivery, audio included.
+
+| Job | Engine | Why |
+|---|---|---|
+| Quick motion retarget, budget lane, or driving clip >15s | Kling MC std/pro (`slates_generate_motion_transfer`) | Structured skeleton/depth retarget, $0.95–1.26 / 5s, takes up to 30s driving clips. |
+| **Motion transfer where fidelity or audio matters** — dance, choreography, cinematic action | **Seedance 2.0** (`motionModel=seedance-2`) | Single-pass conditioning beats post-hoc retargeting; prompt-driven; native audio. Driving clip 2–15s; bills input+output seconds (vref keys). |
+| Cheap lip-sync utility (re-voice a clip, simple avatar) | Kling lip-sync / avatar (`slates_generate_lip_sync`) | $0.11–0.86 / 5s blocks. |
+| **Natural speech, voice cloned from the source clip, premium delivery** | **Seedance 2.0** (`engine=seedance-2`) | The line is spoken IN the generation (no TTS layer); a video source keeps its own voice; uploaded ≤15s audio can drive it. |
+
+- Faces: Seedance tool gens default `seedanceFace=true` (sources are people). A REAL person triggers the consent cascade (`[REAL_FACE_DETECTED]` → `seedanceRealFace` + `realFaceConsent`, premium realface pricing).
+- Billing: any Seedance gen with a video reference bills COMBINED input+output seconds (`seedance-2*-vref-*` keys) — always pass the clip duration and quote before confirming.
+
 **Rules:**
 
 - **Default video = Kling 3.0 std.** Escalate to Seedance the moment the shot has physics/effects weight or is the hero moment — and say why in the plan ("physics-heavy, routing to Seedance").
@@ -28,9 +53,16 @@ Pick the model FIRST, deliberately, before writing a prompt or quoting a plan. M
 
 ## Image routing
 
+**Video models (Kling, Seedance, Veo) cannot generate standalone images — ever.** A "premium hero reference image" is still an image job: it routes to an image model below, never to Seedance.
+
 - **Default: Nano Banana 2** — best reference handling (14 refs), best legible text, the standard start-frame generator.
+- **NB2 Lite** — the fast/draft seat: ~half NB2's price, ~2.7× faster, 1K only. Route iteration volume and drafts here; finals go back to NB2 full (2K/4K).
+- **Nano Banana Pro** — the hero-frame/typography ceiling (~2× NB2). NB2 ≈ 95% of Pro; escalate only when spatial composition, cinematic lighting/skin, fine typography-in-scene, or deep multi-element frames must be perfect. Up to 14 refs — feed it a full subject library.
+- **GPT Image 2** — readable text / panels / UI king: character sheets, shot grids, diagrams, text-bearing panels. Medium quality is the default (half NB2's price at 1080p); high (~4×) only when text precision is the whole job. 4K at both tiers is API-only — even paid ChatGPT can't render it.
 - **FLUX.2 Max** — photoreal texture, hex-color binding, typography, less censored.
-- **Seedream 5 Lite** — cheapest; flat-priced drafts and volume exploration.
+- **Seedream 5 Lite** — uncensored + any-resolution flat price; volume exploration when the Gemini filter is in the way.
+
+**Split rule of thumb:** readable text / panels / UI → GPT Image 2; photoreal, character-locked, widescreen, or edit-heavy → the Banana line; drafts → NB2 Lite; uncensored or odd resolutions → Seedream/FLUX.
 
 ## Cost is a tiebreaker, not the router
 

package/skills/slates-prompting-gpt-image-2.md

@@ -0,0 +1,40 @@
+---
+name: slates-prompting-gpt-image-2
+description: Prompting GPT Image 2 — the readable-text / character-sheet / shot-grid engine. Read before calling slates_generate_image with model gpt-image-2. Covers the quality tiers (medium default, high for max text precision), resolution classes (1k/2k=1080p/3k=1440p/4k), text-accuracy prompting, panel/grid layout direction, and when to route to the Banana line instead.
+---
+
+# GPT Image 2 — sheets, grids, and text that actually reads
+
+GPT Image 2's edge is **character-level text accuracy** (~99% on English), ordered panels, and exact element placement — the jobs where every other model garbles a word or shuffles a layout. It is NOT the photoreal or character-locked pick: route those to the Banana line (`slates-model-selection` has the split).
+
+## Quality tiers — always set explicitly
+
+- **medium** (default) — sharp text, fast, the value seat: half NB2's price at the 1080p class. Blind benchmarks put it within a hair of high at a quarter of the cost. Start here.
+- **high** — ~4× the price; max text precision + reasoning. A deliberate premium pick when tiny type, dense diagrams, or many labeled elements ARE the job.
+
+Never rely on the provider default (it's high — the priciest tier). The Slates ops send medium unless you say otherwise.
+
+## Resolution classes
+
+`1k` = 1024²-class · `2k` = 1920×1080-class · `3k` = 2560×1440-class · `4k` = 3840×2160-class. Pick 2k for most sheets/panels; 4k for print-density grids. 4K exists at BOTH tiers and is API-only — even paid ChatGPT can't render it.
+
+## Prompting for text accuracy
+
+- **Quote every string that must render verbatim**: `the sign reads "OPEN 24 HOURS"` — quoted strings render most reliably.
+- Specify font *feel*, not font names: "clean geometric sans, high contrast", "hand-painted brush lettering".
+- For dense text (posters, UI mocks), list the copy as ordered lines: `Line 1: "..." Line 2: "..."` — GPT Image 2 respects ordering.
+- Keep total on-image text under ~30 words for perfect accuracy; beyond that, accuracy degrades gracefully but degrades.
+
+## Panels, sheets, and grids
+
+- State the grid explicitly and number the cells: "a 2×3 grid of panels, numbered 1–6, reading left-to-right, top-to-bottom".
+- Give each cell ONE content clause: "Panel 3: the character mid-jump, side view".
+- Character sheets: "character turnaround sheet: front, 3/4 left, profile, back — same character, same outfit, flat even lighting, plain background". GPT Image 2 holds the layout; the Banana line holds the *face* better — for identity-critical turnarounds prefer NB2/NB Pro and use GPT Image 2 when labels/annotations matter.
+
+## References & editing
+
+Reference images route through the edit endpoint (up to ~10). The composed "image N" naming applies as everywhere else. Mask-based inpainting exists at the API level but isn't surfaced — describe the change instead.
+
+## Filter regime
+
+OpenAI moderate — a third regime distinct from Gemini (NB family) and ByteDance (Seedream). Real-face references pass more readily than Gemini; violence/brand rules are similar. `slates-content-policy` applies unchanged.

package/skills/slates-prompting-kling-v3.md

@@ -159,6 +159,32 @@ When talking to the user about the gen, refer to each reference by its short cod
 - ✅ "I'm anchoring on **IMG-A12** as the detective and **IMG-A18** as the alleyway environment — Omni will handle the line delivery in EN."
 - ❌ "I'm using the detective image and the alley one..." (which alley? Three exist.)
 
+## Video-to-video EDIT (`slates_edit_video`) — @Video1 / @ElementN / @ImageN
+
+Kling O3 edit takes an EXISTING 3-15s clip and changes only what the prompt names — character swap, environment change, style transfer — in one pass, no masking. Original motion, camera, and audio are preserved by default. Its notation is Kling's own, different from the "image N" naming used everywhere else:
+
+- **`@Video1`** — the source clip (always; the transport anchors the instruction to it).
+- **`@Element1..`** — subjects to swap IN. Each element = one frontal image + up to 3 angle images (pass as `characterAssetIds`; @mention names in the prompt compile to @ElementN automatically).
+- **`@Image1..`** — style/appearance references (pass as `styleAssetIds`).
+- Max **4 combined** element + image refs per edit.
+
+**Prompt shape — the change, not the whole scene:**
+
+```
+Replace the man in @Video1 with @Element1, keeping his walk cycle, the camera move, and the rain unchanged.
+```
+
+```
+Edit @Video1: turn the daytime street into a neon-lit Tokyo alley at night, wet asphalt reflections. Apply the visual style of @Image1. Keep the subject and camera motion exactly as they are.
+```
+
+Rules:
+- Name what CHANGES; explicitly state what stays ("keep the motion / camera / everything else unchanged") — the model preserves better when told to.
+- One edit intent per pass. Chain passes for compound changes (each output is itself an editable clip, linked to its parent).
+- Billing is per second of OUTPUT ≈ the clip length, rounded UP to the next second. A 7.3s clip bills as 8s.
+- Clip constraints: 3-15s, 720-3840px, MP4/MOV. Agents can pre-trim on the timeline when a clip runs long.
+- Routing: Kling edit is the default edit tool (element lock + audio intact); Seedance edit/relocate wins style-transfer-heavy re-imaginings — see `slates-model-selection`.
+
 ## Sources
 
 - [fal.ai — Kling 3.0 Prompting Guide](https://blog.fal.ai/kling-3-0-prompting-guide/)

package/skills/slates-prompting-lip-sync.md

@@ -1,19 +1,33 @@
 ---
 name: slates-prompting-lip-sync
-description: How to set up Kling lip-sync. Read before calling slates_generate_lip_sync. Two flows — video→video re-dub and image→video avatar — with different inputs, pricing, and gotchas. Voice catalog, framing rules, audio file constraints, and which tier to pick.
+description: How to set up lip-sync — Kling (cheap utility lane) or Seedance 2.0 (premium single-pass lane). Read before calling slates_generate_lip_sync. Flows — video→video re-dub, image→video avatar, and Seedance native speech — with different inputs, pricing, and gotchas. Voice catalog, framing rules, audio file constraints, and which engine/tier to pick.
 ---
 
-# Kling lip-sync — setup guide
+# Lip-sync — setup guide
 
-Two distinct flows, both 5-second outputs:
+Two engines. Kling is the cheap utility lane (dedicated lip-sync endpoints, 5-second outputs); Seedance is the premium lane (speech generated IN the video itself, single pass):
 
-| Flow | Source | Model | Cost (5s) | Use case |
+| Flow | Source | Engine/Model | Cost | Use case |
 |------|--------|-------|-----------|----------|
-| Re-dub | video clip | kling-lip-sync-video | $0.11 | Replace dialogue on an existing talking head |
-| Avatar standard | still image | ai-avatar/v2/standard | $0.42 | Animate a portrait into a talking avatar |
-| Avatar pro | still image | ai-avatar/v2/pro | $0.86 | Higher facial fidelity for hero shots |
+| Re-dub | video clip | kling-lip-sync-video | $0.11 / 5s | Replace dialogue on an existing talking head |
+| Avatar standard | still image | ai-avatar/v2/standard | $0.42 / 5s | Animate a portrait into a talking avatar |
+| Avatar pro | still image | ai-avatar/v2/pro | $0.86 / 5s | Higher facial fidelity for hero shots |
+| **Seedance native** | image or video | `engine=seedance-2` | per second (`seedance-2-face-*`; video sources bill input+output seconds) | **Premium**: natural delivery, whole-body performance, voice cloned from a video source, audio included |
 
-Pick `sourceType` deliberately — it decides the pricing tier and the underlying endpoint.
+Pick engine + `sourceType` deliberately — they decide the pricing tier and the underlying endpoint.
+
+## Seedance engine (premium single-pass)
+
+Kling lip-sync moves the mouth on finished pixels; Seedance *generates* the performance — head movement, gesture, delivery energy — with the dialogue as a native conditioning signal. Key facts:
+
+- **`ttsText` becomes the spoken line, natively.** No TTS voice/speed params — a VIDEO source keeps its **own voice** (the model clones it from the clip's audio track); for an image source the voice follows the character's look, or describe it in the line's context.
+- **`audioMethod=upload`** drives the speech from a ≤15s audio file instead (a reference-audio input, no billing surcharge).
+- **Sources:** image (any style; same framing rules as the avatar flow below) or a 2–15s video clip. Output duration follows the source/audio/line length (4–15s), not a fixed 5s.
+- **Billing:** image sources bill the normal `seedance-2-face-{res}-{N}s` keys; video sources bill combined input+output seconds (`-vref-` keys) — pass `sourceSeconds` and quote via the confirm gate.
+- **Faces:** `seedanceFace` defaults true. A REAL person → `[REAL_FACE_DETECTED]` → confirm consent → retry with `seedanceRealFace=true, realFaceConsent=true` (premium realface pricing).
+- When to pick it: hero dialogue shots, natural delivery, "make this clip's person say X in their own voice". Stay on Kling for cheap utility re-dubs and long clips.
+
+Everything below applies to the **Kling** engine.
 
 ## Choosing video vs avatar
 

package/skills/slates-prompting-motion-transfer.md

@@ -1,18 +1,32 @@
 ---
 name: slates-prompting-motion-transfer
-description: How to set up Kling Motion Control (motion transfer). Read before calling slates_generate_motion_transfer. Reference image (character) + driving video (motion source) → new video of the character performing the motion. Asset selection rules, character_orientation choice, std vs pro tier, and prompt usage.
+description: How to set up motion transfer — Kling Motion Control (cheap utility lane) or Seedance 2.0 (premium single-pass lane). Read before calling slates_generate_motion_transfer. Reference image (character) + driving video (motion source) → new video of the character performing the motion. Asset selection rules, engine choice, character_orientation, tiers, and prompt usage.
 ---
 
-# Kling Motion Control — setup guide
+# Motion transfer — setup guide
 
-Take a still **target image** (your character) and a **source video** (the motion you want), produce a new 5s video of your character performing the source video's motion.
+Take a still **target image** (your character) and a **source video** (the motion you want), produce a new video of your character performing the source video's motion. Two engines:
 
-| Tier | Cost (5s) | Use case |
+| Engine | Cost | Use case |
 |------|-----------|----------|
-| std (`kling-mc-std-5s`) | $0.95 | General motion transfer |
-| pro (`kling-mc-pro-5s`) | $1.26 | Cleaner anatomy, better identity preservation |
+| Kling std (`kling-mc-std-5s`) | $0.95 / 5s | General motion transfer, budget lane |
+| Kling pro (`kling-mc-pro-5s`) | $1.26 / 5s | Cleaner anatomy, better identity preservation |
+| **Seedance 2.0** (`motionModel=seedance-2`) | per second of input+output (`seedance-2-face-vref-*`) | **Premium lane** — single-pass generation with the driving clip as a native conditioning signal: better motion fidelity, native audio, prompt-directed |
 
-Both tiers trip the >$0.50 confirm gate. User OK required every time.
+All tiers trip the >$0.50 confirm gate. User OK required every time.
+
+## Seedance engine (premium single-pass)
+
+Kling MC retargets a skeleton onto a finished image; Seedance *generates* the shot with the motion as a conditioning input — the difference shows on fast choreography, physical contact, cloth/hair, and camera motion. Key differences from Kling:
+
+- **Prompt-driven.** The prompt is the primary control, using ordinal references: `The character from image 1 performs the exact motion, choreography, and camera movement from video 1. Preserve the character's identity, appearance, and outfit.` Add style/setting/camera direction freely — Seedance re-generates the whole shot.
+- **Driving clip must be 2–15s** (all providers cap reference video at 15s). Longer clips: trim first, or use Kling MC (`character_orientation: 'video'` takes up to 30s).
+- **Billing = combined input+output seconds** (the vref keys). Pass `sourceVideoSeconds`; output `duration` defaults to the clip length (4–15s). The server probes the clip and corrects an understated key — quote via the confirm gate before spending.
+- **Faces route through the face cascade**: `seedanceFace` defaults true (driving clips contain people). A REAL person → `[REAL_FACE_DETECTED]` → confirm consent with the user → retry `seedanceRealFace=true, realFaceConsent=true` (premium realface vref pricing).
+- **Native audio included** — the output can carry sound from the prompt (or the driving clip's vibe); no audio surcharge.
+- `characterOrientation` is Kling-only; Seedance framing follows the prompt + `aspectRatio`.
+
+Everything below applies to the **Kling** engine.
 
 ## Inputs
 

package/skills/slates-prompting-nano-banana-2.md

@@ -144,3 +144,12 @@ Default to #1. Reach for #2 only when positive framing can't suppress the unwant
 ## The 3-strike rule
 
 If three iterations on the same prompt haven't produced what the user wants, stop. Hand back to the user with what you tried and what isn't working. The slot machine doesn't converge — the prompt structure is wrong, not the seed.
+
+## Family variants — Lite and Pro
+
+Everything in this skill applies to the whole Nano Banana family; two variants trade speed/ceiling around NB2 full:
+
+- **nano-banana-2-lite** — ~half the price, ~2.7× faster, **1K output only**, max 4 refs. The draft/iteration seat: explore compositions here, then re-run the winner on NB2 full at 2K/4K. Same Gemini filter.
+- **nano-banana-pro** — the hero-frame/typography ceiling (~2× NB2, 4K native). NB2 ≈ 95% of Pro; escalate only when spatial composition, cinematic lighting/skin, fine typography-in-scene, or deep multi-element frames must be perfect. Up to 14 refs — it takes a full subject library in one call.
+
+Routing between them (and vs GPT Image 2 / FLUX / Seedream): `slates-model-selection`.

package/skills/slates-prompting-seedance.md

@@ -83,6 +83,15 @@ Reference-to-video endpoint accepts up to **9 reference images, 3 reference vide
 
 **Mutually exclusive:** First-frame/last-frame mode CANNOT be combined with reference images. The error reads `"first/last frame content cannot be mixed with reference media content."` Pick one or the other.
 
+### Motion transfer & lip-sync recipes (reference video / audio)
+
+These aren't separate Seedance features — they're prompting strategies over reference media, and the Slates tools (`slates_generate_motion_transfer` / `slates_generate_lip_sync` with the seedance engine) compose them for you. When driving them by hand through `slates_generate_video`:
+
+- **Motion transfer:** subject image as a reference + the driving clip via `videoReferenceAssetId` (2–15s) + `The character from image 1 performs the exact motion, choreography, and camera movement from video 1. Preserve the character's identity, appearance, and outfit.`
+- **Lip-sync / dialogue:** write the line in the prompt — `The person in video 1 says: "…"` — with `generate_audio` on (always on in Slates). A **video** source's own voice is cloned natively; an **audio** reference (`audioReferenceAssetId`, ≤15s) drives speech from an existing recording: `…speaks the dialogue from audio 1 with accurate lip sync.`
+- **Voice + face from one clip (the talking-head recipe):** ONE unedited 2–15s clip of the person speaking (clear voice, no music, no cuts) as the video reference + prompt with the new script → their likeness AND voice deliver the new line.
+- **Billing:** a reference VIDEO switches the cost key to `seedance-2*-vref-{res}-{T}s` where T = clip seconds + output seconds — quote before confirming. Audio references are free (audio is included on every route).
+
 ## Faces — set `seedanceFace` for AI-character faces
 
 Seedance routes through **three tiers** depending on the face in the reference, exposed as the "Face in Reference" toggle plus the real-face params on `slates_generate_video`: