@slatesvideo/shared 0.6.1 → 0.6.2

package/skills/slates-previs-blocking.md

@@ -0,0 +1,153 @@
+---
+name: slates-previs-blocking
+description: Build a 3D blocking pass in Blender, render it grey-box, and use it as a reference video so the generated shot follows a camera path you designed instead of one the model invented. Use when the user wants precise camera control, a multi-cut sequence, a one-take move, spatial consistency across shots, or says the camera keeps drifting / they keep burning credits re-rolling.
+---
+
+# Previs blocking — design the shot, then generate it
+
+The spine of the whole workflow. Read this first; the other four previs skills are branches off it.
+
+## The mechanism (why this works at all)
+
+A text prompt asks the model to *invent* camera motion, so it invents differently every roll. You cannot iterate on a variable you do not control, so you re-roll and pay again.
+
+A **reference video** removes the invention. You build the shot in Blender as untextured proxies — a neutral grey set with colour-coded figures, free, instant, deterministic — render the camera's path to mp4, and hand the model that clip alongside the prompt. **Blender locks the motion; the model builds the world.** Iteration moves to the free half, and the paid half usually lands first try.
+
+Two halves, and keeping them separate is the whole discipline:
+
+| Half | Lives in | Changes when |
+|---|---|---|
+| **Structure** — cuts, camera, timing, who is where | the blocking clip | you re-block |
+| **Style** — what any of it looks like | references + prompt text | you restyle (see `slates-restyle-from-blocking`) |
+
+## Before you start
+
+1. `slates_blender_status` — confirms the bridge is up and returns fps, frame range, existing camera. If it reports `connected: false`, relay its hint and stop; nothing else here works.
+2. Settle **format first**, because the blocking render *is* the film's format: fps, aspect, duration. 24fps is the default and makes cut times land on clean frames. Duration ≤ 30s (seedance-2.5's reference-video ceiling; 15s on the others).
+3. Know the shot count. "One take" and "19 cuts" are different builds.
+
+## Build order
+
+Do these in order. Each stage is verifiable on its own, and a camera built before the geometry has nothing to frame.
+
+### 1. Set the format
+
+```python
+scene = bpy.context.scene
+scene.render.fps = 24
+scene.render.fps_base = 1.0
+scene.render.resolution_x, scene.render.resolution_y = 1920, 1080
+scene.frame_start, scene.frame_end = 1, 720   # 30s at 24fps
+result = {"seconds": 720 / 24}
+```
+
+Frame maths, stated once so you never redo it in your head: **frame = seconds × fps + 1**. A cut at 7.79s is frame 188.
+
+### 2. Geometry and light — grey set, coded figures, named
+
+Proxies only. A person is a box or a capsule with a sphere head. A car is a stretched cube. A can is a cylinder. **The SET is neutral grey — one light, a floor and enough wall that the space reads.** Colour is reserved for the figures, where it carries meaning (below); a grey set is what makes those few colours legible as notation rather than décor. Anything you spend on materials here you pay for twice, because the model repaints every surface anyway.
+
+**Name every object for what it *is* in the story**, not `Cube.003`. The name is how you refer to it later, and it is how you keep your own timeline honest.
+
+Two conventions that cost nothing now and save a re-roll later:
+
+- **Colour is identity.** Give each character a distinct viewport colour and *write the mapping down* — `red = the boss, green = the kid, blue = the driver`. The generation prompt will restate that mapping so the model knows which grey body is which person across cuts. Without it, characters swap.
+- **Encode facing on featureless proxies.** A box has no front. Mark one face red, the back black, the sides green, and say so in the prompt: `RED face = the direction he faces`. Otherwise the model guesses which way people are looking.
+- **Checker a surface when SCALE or SPEED has to read.** Flat grey gives a model no parallax cue, so a fast move over a featureless floor reads as slow, and a big room reads as a small one. A black-and-white checker on the ground (or the wall a camera races past) gives it something to measure against. ⚠️ **Build it as GEOMETRY, never as a Checker Texture node.** The blocking render is Workbench, which draws one flat colour per material and never evaluates a shader node tree — a `TEX_CHECKER` comes out flat grey and you lose the cue without being told. Subdivide the plane and alternate `material_index` per face. Like every other colour here it is notation, so it goes in the translation list and gets dressed over.
+
+```python
+# Two materials, alternated per face. `TILE` is the square size in metres.
+dark = bpy.data.materials.new("Checker_Dark")
+dark.diffuse_color = (0.05, 0.05, 0.05, 1.0)
+light = bpy.data.materials.new("Checker_Light")
+light.diffuse_color = (0.80, 0.80, 0.80, 1.0)
+floor.data.materials.append(dark)    # material_index 0
+floor.data.materials.append(light)   # material_index 1
+# Subdivide first (edit mode or a Subdivide modifier applied) so there ARE
+# faces to alternate — a 2-triangle plane can only ever be one colour.
+for face in floor.data.polygons:
+    cx, cy = face.center.x, face.center.y
+    face.material_index = (int(cx // TILE) + int(cy // TILE)) % 2
+```
+
+And the identity colour on each proxy:
+
+```python
+mat = bpy.data.materials.new("ID_Red")
+mat.diffuse_color = (0.8, 0.1, 0.1, 1.0)   # what the blocking render draws
+obj.data.materials.append(mat)
+obj.color = (0.8, 0.1, 0.1, 1.0)           # same value, for viewport parity
+```
+
+The blocking render pins Workbench to `MATERIAL` shading, so **`mat.diffuse_color` is the value that reaches the clip** — and an object with no material at all falls back to a neutral grey, which is why an unpainted set still reads correctly. Set `obj.color` to the same value anyway: it costs one line, it makes the user's viewport match what renders, and keeping the two equal means you never have to remember which one is authoritative.
+
+### 3. Camera
+
+The whole of `slates-camera-language`. Build the rig, then keyframe it. Then **read back what you built** with `slates_blender_scene` — its `cutSeconds` is your cut list, and it is the number you will write timings against. That field is the authoritative one on EITHER rig — marker frames when cameras are bound to markers, the active camera's own keyframes when they are not. `camera.keyframeSeconds` is empty on a marker-bound edit, which is the rig `slates-camera-language` recommends for anything past a handful of cuts.
+
+### 4. Handheld, last
+
+Add it after the moves are right, never before — noise on top of a wrong path just hides the wrong path.
+
+### 5. Verify the cuts
+
+The one check that catches the most damage: on a multi-cut blocking, camera position, target and focal length must all change **exactly on the cut frame, with no transition frame between**. One interpolated frame reads as a whip-pan the model will faithfully reproduce.
+
+```python
+# Every camera f-curve keyframe on a cut frame must be CONSTANT out of the
+# previous key, or the cut smears.
+for fc in cam.animation_data.action.fcurves:
+    for kp in fc.keyframe_points:
+        if int(kp.co[0]) in CUT_FRAMES:
+            kp.interpolation = 'CONSTANT'
+```
+
+Also check nothing interpenetrates — proxies through floors, clones through the hero object, letters through each other. The model renders intersections as faithfully as it renders everything else.
+
+### 6. Save a backup after every stage
+
+Cheap, and blocking is iterative by nature.
+
+```python
+bpy.ops.wm.save_as_mainfile(filepath=path, copy=True)
+```
+
+## Render and generate
+
+```
+slates_blender_render_blocking { projectId, fps: 24 }
+```
+
+Renders the **scene camera** through scene settings — never the user's viewport, so the result does not depend on where they left their mouse — imports the mp4 into the project, and returns `assetId` + `durationSeconds`.
+
+Then:
+
+```
+slates_generate_video {
+  model: "seedance-2.5",
+  videoReferenceAssetIds: [<the blocking asset>],
+  videoReferenceSecondsEach: [<durationSeconds>],
+  characterAssetIds: [...], environmentAssetIds: [...], styleAssetIds: [...],
+  prompt: <written per slates-blocking-to-prompt>
+}
+```
+
+**Four inputs, and that is the entire stack:** a character sheet each, one location/style reference, the blocking clip, and a prompt written against the blocking. Resist adding a fifth.
+
+Model note: seedance-2.5 is the seat for this — 10 reference videos at up to 30s each. seedance-2 and minimax-h3 take 3 at 15s. Route per `slates-model-selection`.
+
+## Leaving holes on purpose
+
+Where the model outperforms any blockout you could build — liquid, smoke, fire, cloth — **block a black gap instead** and say so in the prompt: `CUT 7 (14.5-17.0, black gap in the reference)`. You are reserving a slot, not forgetting one.
+
+## What not to do
+
+- **Don't texture, light or material the blocking.** Grey is the specification. The reference supplies motion; the references supply look.
+- **Don't animate what you don't need.** Heads especially — a proxy head turning wrong is worse than one that never turns.
+- **Don't build the camera before the geometry.** It has nothing to aim at, and every value you set gets redone.
+- **Don't skip reading the scene back.** Write timings from `slates_blender_scene`'s `cutSeconds`, never from what you intended to build.
+- **Don't exceed the model's reference-video ceiling.** A 40s blocking against a 30s cap silently truncates.
+
+## Related
+
+`slates-camera-language` (rigs and moves) · `slates-blocking-to-prompt` (writing the prompt against the clip) · `slates-dialogue-blocking` (multi-character continuity) · `slates-restyle-from-blocking` (one blocking, many worlds) · `slates-model-selection` (routing)

package/skills/slates-restyle-from-blocking.md

@@ -0,0 +1,121 @@
+---
+name: slates-restyle-from-blocking
+description: Render one blocking pass as several different visual worlds — live action, 2.5D painted, 2D ink, toybox — matching cut for cut. Use when a client needs style options, when someone wants to see the same edit in another look, or when an approved edit needs a new treatment without re-blocking.
+---
+
+# Restyle — one edit, many worlds
+
+The commercial payoff of the whole previs workflow, and the reason a blocking file is an asset rather than a step.
+
+## The idea
+
+Every prompt has two halves:
+
+- **Structure** — cuts, camera, timing, who is where. Lives in the blocking clip. **Never changes.**
+- **Style** — what any of it looks like. Lives in the references and the prompt text. **Changes freely.**
+
+Hold the structure, swap the style, and the same edit comes back as live action, painted 2.5D, ink on paper or a toybox — **matching frame for frame across all of them.** Cuts land on the same frames, the car drifts at the same moment, the same head turns at the same beat.
+
+For anyone pitching work: three visual worlds in a day, off one edit the client has already approved. The foundation is not up for renegotiation, so the conversation is only about look.
+
+## Before you restyle
+
+You need a blocking clip whose structure you are happy with, and a finished prompt for at least one style (per `slates-blocking-to-prompt`). The first style is the expensive one; every later style is an edit of its text.
+
+## What stays fixed
+
+Copy these across every style **verbatim**. Changing them is what desynchronises the outputs:
+
+- The blocking reference's own contract — that it is the master for all movement, the placement-only clause, the tie-break clause, the disambiguation clause
+- The shot count and every timestamp
+- Every shot's camera position, angle, framing and cut point
+- Screen direction and seating
+- The `HOLD FOR THE FULL TIMELINE` block
+- `videoReferenceAssetIds` and `videoReferenceSecondsEach`
+
+Lead each style's prompt with a lock so the style layer cannot leak into the structure:
+
+> VIDEO LOCK — the dominant rule of this prompt: the reference defines 100% of the motion, editing and object choreography. The text below defines only look, materials, locations and effects layered onto that motion. Wherever the text and the video could be read differently about motion, the video decides.
+
+## What changes
+
+| Layer | What you swap |
+|---|---|
+| Rendering style | photoreal · painted 2.5D · 2D ink · miniature/toybox |
+| Characters | different sheets entirely — a couple, grandparents, a robot and a cat |
+| Locations | the same four beats set in a different world |
+| Time of day / weather | night after rain · golden hour · hard noon |
+| Lighting and colour | per style |
+| Audio | SFX-only, or scored, or lip-synced dialogue |
+
+Characters can change species and still land, because the blocking only supplies where a body is and how it moves.
+
+## Dummy mapping — the mechanism that makes it work
+
+Each style needs its own explicit mapping from grey proxy to real object. The proxy is a slot; the style fills it:
+
+> DUMMY MAPPING: the front-LEFT sphere-head dummy (with its grey arm at the shifter and grey leg at the pedals) is THE GRANDPA; the front-RIGHT sphere-head dummy is THE GRANDMA; a front-seat dummy together with its loose blocks is that ONE whole person. Blocks on the rear bench are the luggage. The low-poly flying model in SHOT 18 is THE HELICOPTER. The two vehicles behind the hero car in SHOT 19 are THE POLICE CARS.
+
+Same clause per style, different right-hand side. And restate the placement-only rule in style terms:
+
+> The source defines only placement and motion, never appearance: every placeholder becomes the real object its position implies — spheres are always people, cabin blocks are always cases and bags, fully drawn.
+
+## Location continuity
+
+If the piece travels, name the places and pin each shot to one. Reusing labels across styles keeps the four prompts diffable:
+
+```
+LOCATION CONTINUITY — one journey through four fixed places; each looks
+identical in every shot where it appears:
+  LOC-A <opening>   LOC-B <middle>   LOC-C <turn>   LOC-D <finale>
+```
+
+Then tag every beat: `SHOT 9 — 7.79-9.33s — LOCKED, LOC-B: <description>`.
+
+**On a piece that visits many places, make the map absolute and countable** — otherwise the model reuses a room it liked and you get the same interior three times:
+
+> The location map is absolute — SEVEN locations, each appearing EXACTLY ONCE, in this exact order: 1) yard 00:00-00:03.3 … 7) rooftop 00:20-00:30. No location ever appears twice, and the three interiors are three COMPLETELY DIFFERENT rooms — different walls, furniture, people and light — never the same room repeated.
+
+## Style references
+
+A style reference is **not a keyframe**, and saying so prevents the model reproducing its composition as a shot:
+
+> STYLE MASTER — defines the painting and rendering style only: hand-painted look with visible brushstrokes, sculpted painterly volumes, textured matte surfaces, dramatic coloured rim light, deep moody shadows. NOT a keyframe, NOT a location to reproduce, NOT a frame that ever appears in the film. Its own subject, framing and composition are never seen in any shot.
+
+A style can also be **text-only** — no reference image at all. Ink and toybox looks usually specify better in words than they match from a still.
+
+## Keep performance inside the existing shots
+
+Style changes tempt the model to earn new coverage. Refuse it:
+
+> ACTING — inside the existing shots only: performance is visible only at the size and distance the reference already gives it, only where the source already shows a face; everywhere else it reads through posture and hands alone. The performance NEVER earns a new shot, a new angle or a closer framing.
+
+## Text-free worlds
+
+Stylised worlds are where invented signage and garbled lettering appear. One clause kills it:
+
+> TEXT-FREE WORLD: every sign is a blank painted shape, every gauge face carries tick marks only, every licence plate is a blank plate.
+
+## Running it
+
+Generate each style as its own `slates_generate_video` call against the **same** `videoReferenceAssetIds`. Keep them in one project so they sit side by side; name assets by style so the comparison reads at a glance.
+
+Quote the whole set before firing — `slates_estimate_generation_cost` per style — and confirm. Four styles is four generations, not one.
+
+🚨 Never fire a batch of style variants without showing the user the prompts and the total cost first.
+
+## Checklist per style
+
+- [ ] Same blocking asset, same `videoReferenceSecondsEach`
+- [ ] VIDEO LOCK leads the prompt
+- [ ] Every timestamp and shot count identical to style 1
+- [ ] Dummy mapping written for this style's cast
+- [ ] Style reference declared as style-only, or none used
+- [ ] Location labels reused; on a travelling piece the map is absolute and countable
+- [ ] Acting-inside-existing-shots clause present
+- [ ] HOLD block copied verbatim
+- [ ] Cost quoted and confirmed
+
+## Related
+
+`slates-previs-blocking` · `slates-blocking-to-prompt` · `slates-style-prompting` (style vocabulary per model) · `slates-cost-discipline` (batch quoting)