greenlight-motion 0.3.4 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -10,32 +10,36 @@
10
10
 
11
11
  ## Contents
12
12
 
13
- - [3D model layers (`mediaType: "model3d"`) — a live GLB/GLTF/OBJ rendered per frame](video-3d.md#3d-model-layers-mediatype-model3d--a-live-glbgltfobj-rendered-per-frame)
14
- - [Scene Rig — one set for every 3D layer](video-3d.md#scene-rig--one-set-for-every-3d-layer)
15
- - [Flat layers in the 3D scene — `item.shadowPlate`](video-3d.md#flat-layers-in-the-3d-scene--itemshadowplate)
13
+ - [3D model layers (`mediaType: "model3d"`) — a live GLB/GLTF/OBJ rendered per frame](#3d-model-layers-mediatype-model3d--a-live-glbgltfobj-rendered-per-frame)
14
+ - [Characters — the pose pipeline (`model3dConfig.pose`)](#characters--the-pose-pipeline-model3dconfigpose)
15
+ - [Skeletons for meshes that have none (`skeleton`)](#skeletons-for-meshes-that-have-none-skeleton)
16
+ - [Scene Rig — one set for every 3D layer](#scene-rig--one-set-for-every-3d-layer)
17
+ - [Turning flat layers in 3D — pivot and stacking](#turning-flat-layers-in-3d--pivot-and-stacking)
18
+ - [Flat layers in the 3D scene — `item.shadowPlate`](#flat-layers-in-the-3d-scene--itemshadowplate)
16
19
 
17
20
  ### 3D model layers (`mediaType: "model3d"`) — a live GLB/GLTF/OBJ rendered per frame
18
21
 
19
- Unlike a device mockup (pre-baked to a video file), a `model3d` layer is a **real 3D scene rendered live every frame** by the Player (three.js, its own depth-buffered pass), then composited like any other layer. The model can be one of the bundled device presets or a custom uploaded asset, and it participates in the project camera, keyframes, layer parenting and effects like any layer.
22
+ Unlike a device mockup (pre-baked to a video file), a `model3d` layer is a **real 3D scene rendered live every frame** by the Player (three.js, its own depth-buffered pass), then composited like any other layer. The model can be one of the bundled device presets, a primitive stand-in or a custom uploaded asset. It takes the project camera, keyframes and layer parenting like any layer. Effects work on it only in part: see [Real 3D on 3D model layers](video-effects.md#real-3d-on-3d-model-layers); layer effects marked `disabledWhen3d` in capabilities (Motion Blur, Rig (Puppet), the warps, Layer Mask…) do not run on it. It takes no transitions ([video-transitions.md](video-transitions.md)).
20
23
 
21
- **⚠ The clip `url` must NEVER be empty — an empty url silently deletes the layer.** `sanitizeClip` (VideoEditor.jsx) treats a `model3d` clip like a video clip: no `url` → the clip is dropped on load/render, and every timeline item referencing it is dropped with it. Nothing errors — the project saves fine and a headless render completes "successfully" with the model simply missing from every frame. This happens even when the item carries a complete, valid `model3dConfig`. The two valid url forms:
24
+ **Never leave the clip `url` empty: the layer disappears without a warning.** A `model3d` clip is loaded like a video clip, so a clip with no `url` is dropped on load and render, and every timeline item that uses it goes with it — even one carrying a complete, valid `model3dConfig`. The project still saves and a headless render still "succeeds", with the model missing from every frame. The valid url forms:
22
25
 
23
26
  - **Preset device:** `"url": "model-preset:<presetId>"` — a stable sentinel, NOT a fetchable path. (The bundled .glb's real URL is a content-hashed build artifact that changes every release; the sentinel survives app updates.)
24
27
  - **Primitive shape:** `"url": "model-primitive:<kind>"` — a stand-in built in code, no asset file. Pair it with `"source": "primitive"` and `"primitive": { "kind", "color", "text" }` in `model3dConfig` (see below).
25
28
  - **Custom model:** a real uploaded asset URL ending in `.glb`, `.gltf` or `.obj` (upload via the media API, then use the returned `/media/...` url).
29
+ - **Gaussian splat:** a `.ply`, `.splat` or `.spz` url. Only a 3D Scene draws splats ([video-world3d.md](video-world3d.md#gaussian-splats)).
26
30
 
27
31
  **Clip + item pair (preset device):**
28
32
 
29
33
  ```json
30
34
  // clip — url is the preset sentinel, source REMOTE, no proxy/thumbnail
31
35
  { "id": "m1", "source": "REMOTE", "mediaType": "model3d",
32
- "url": "model-preset:iphone_17", "proxyUrl": null, "name": "iPhone 17", "duration": 4 }
36
+ "url": "model-preset:iphone_pro_max", "proxyUrl": null, "name": "iPhone 17 Pro Max", "duration": 4 }
33
37
  // item — model3dConfig holds everything about fit, look and animation
34
38
  { "id": "mi1", "clipId": "m1", "trimStart": 0, "trimEnd": 4,
35
39
  "model3dConfig": {
36
- "source": "preset", "preset": "iphone_17",
40
+ "source": "preset", "preset": "iphone_pro_max",
37
41
  "fit": "height", "fitValue": 60, "anchor": "center", "placeOnRigFloor": true,
38
- "baseRotation": { "x": 0, "y": 0, "z": 0 },
42
+ "baseRotation": { "x": 0, "y": 180, "z": 0 },
39
43
  "env": { "hdr": "potsdamer_platz_1k", "intensity": 1, "exposure": 1, "rotation": 0 },
40
44
  "lights": { "key": 1, "fill": 0.45, "rim": 0.35 },
41
45
  "animation": { "clip": null, "speed": 1, "loop": "loop", "offset": 0 },
@@ -65,14 +69,13 @@ For a custom model use `"source": "custom", "preset": null` and point the clip `
65
69
  | `primitive.color` | any CSS colour, ≤ 64 chars | `#d9d9d9` | Tints the whole shape (the car's glass and tyres keep their own colour). |
66
70
  | `primitive.text` | string, ≤ 40 chars | `TEXT` | Only read by `kind: "text"` (extruded letters). |
67
71
 
68
- On a floor-bearing rig, `placeOnRigFloor: true` stands the shape on the cyclorama; `panX` (% of comp width) and `panZ` (px, + toward the camera) place it on the floor. Fit values the editor uses for a set at comp scale: cube/sphere 24, card 26, torus 22, cone/cylinder 28, human 58, car 20, text 14. Keep `source`/`preset` consistent with the clip url: the renderer resolves the asset from `model3dConfig` first (`source: "preset"` + a valid `preset` id wins), then falls back to the clip's sentinel/url — but the clip url is what keeps the clip **alive** through sanitization, so always set both sides to the same device.
72
+ On a floor-bearing rig, `placeOnRigFloor: true` stands the shape on the cyclorama; `panX` (% of comp width) and `panZ` (px, + toward the camera) place it on the floor. Fit values the editor uses for a set at comp scale: cube/sphere 24, card 26, torus 22, cone/cylinder 28, human 58, car 20, text 14. Keep `source`/`preset` consistent with the clip url: the renderer resolves the asset from `model3dConfig` first (`source: "preset"` + a valid `preset` id wins), then falls back to the clip's sentinel/url — but the clip url is what keeps the clip **alive** on load, so always set both sides to the same device.
69
73
 
70
- **Bundled device presets** (`MODEL_PRESETS` in `frontend/src/components/videoEditor/model3dPresets.js`):
74
+ **Bundled device presets:**
71
75
 
72
76
  | `preset` id | Device | Screen artwork | `baseRotation.y` to face the camera |
73
77
  | --- | --- | --- | --- |
74
78
  | `iphone_pro_max` | iPhone 17 Pro Max | yes | `180` |
75
- | `iphone_17` | iPhone 17 | yes | `0` |
76
79
  | `iphone_air` | iPhone Air | yes | `0` |
77
80
  | `double_iphone` | Two iPhones | yes | `180` |
78
81
  | `ipad` | iPad | yes | `0` |
@@ -81,18 +84,21 @@ On a floor-bearing rig, `placeOnRigFloor: true` stands the shape on the cycloram
81
84
  | `macbook_air` | MacBook Air | yes | `0` |
82
85
  | `imac` | iMac | yes | `180` |
83
86
 
84
- Some GLBs are authored facing away from the viewer — for those, set `baseRotation.y: 180` (the last column; the editor's Add-Layer flow applies this automatically via `presetBaseRotation`). A wrong value shows the device's back.
87
+ `iphone_17` is an alias of `iphone_pro_max`: it loads the same model, so it also needs `baseRotation.y: 180`. Write `iphone_pro_max`.
85
88
 
86
- **`model3dConfig` fields** (sanitized by `sanitizeModel3dConfig` in `editorState.js`; defaults in `MODEL_CONFIG_DEFAULTS` in `types.js` — out-of-range values are clamped, bad enums fall back):
89
+ Some GLBs are authored facing away from the viewer — for those, set `baseRotation.y: 180` (the last column; the editor sets it when it adds the layer). A wrong value shows the device's back.
90
+
91
+ **`model3dConfig` fields** (out-of-range values are clamped, unknown enum values fall back to the default):
87
92
 
88
93
  | Field | Values / range | Default | Meaning |
89
94
  | --- | --- | --- | --- |
90
- | `source` | `"preset"` \| `"custom"` | `custom` | Where the geometry comes from. Anything else coerces to `custom`. |
95
+ | `source` | `"preset"` \| `"primitive"` \| `"custom"` | `custom` | Where the geometry comes from. Anything else coerces to `custom`. |
91
96
  | `preset` | preset id string | `null` | Required when `source: "preset"`. An unknown id falls through to the clip url. |
92
- | `fit` | `"height"` \| `"width"` \| `"unit"` | `height` | How `fitValue` sizes the model: % of frame height, % of frame width, or raw pixels-per-model-unit. |
97
+ | `fit` | `"height"` \| `"width"` \| `"unit"` | `height` | How `fitValue` sizes the model: % of frame height, % of frame width, or raw pixels-per-model-unit. The box is the file's rest pose as it draws: a skinned character through its bones. |
93
98
  | `fitValue` | `0.01–10000` | `60` | With `fit: "height"`, `60` = the model's bounding box spans 60 % of the frame height. |
94
99
  | `anchor` | `"center"` \| `"base"` \| `"origin"` | `center` | Pivot: bbox centre, bbox floor (for objects standing on a surface), or the asset's own origin — no recentring, so a GLB exported in scene coordinates (Blender → GLEA) lands exactly where its scene put it, whatever its rest pose. |
95
100
  | `placeOnRigFloor` | bool | `true` | With a floor-bearing Scene Rig, place the model's real post-transform lower bound on the cyclorama floor. The ordinary Y transform becomes an offset from that floor. Set `false` for free/flying placement. |
101
+ | `physics` | `"off"` \| `"falls"` \| `"solid"` | `off` | Rigid-body physics, only in a 3D Scene ([Rigid bodies](video-world3d.md#rigid-bodies)): `falls` makes the model a rigid body that falls, hits and stacks from where the layer stands; `solid` a fixed one others hit. The classic renderer ignores it. |
96
102
  | `castShadow` | bool | `true` | With a Scene Rig casting real shadows: the model is in the shadow map. `false` = it casts nothing (a huge set piece that should only receive). Blended parts of a model (glTF BLEND — a baked contact-shadow card, a glow) never cast either way. Only stored when `false`. |
97
103
  | `receiveShadow` | bool | `true` | With a Scene Rig: rig shadows darken the model. `false` = they never land on it. Only stored when `false`. |
98
104
  | `baseRotation` | `{x,y,z}` deg, `±360` | `{0,0,0}` | Static orientation applied before keyframed rotation. |
@@ -101,10 +107,12 @@ Some GLBs are authored facing away from the viewer — for those, set `baseRotat
101
107
  | `env.exposure` | `0.05–8` | `1` | Tone-mapping exposure of the model pass. |
102
108
  | `env.rotation` | `±360` deg | `0` | Spins the environment (moves reflections). |
103
109
  | `lights.key/fill/rim` | `0–10` each | `1 / 0.45 / 0.35` | The three directional lights' intensities. **Accepted, normalised on read** — see "Scene Rig" below. Still the right shape to emit when a project has no rig layer; ignored on every model once one exists. |
104
- | `animation.clip` | glTF animation name \| `"*"` \| `null` | `null` | Plays the named animation baked into the asset; `"*"` plays EVERY clip together, each on its own length (Blender's exporter writes one clip per animated object); `null` = static. Names must match the glTF exactly. `loop: "once"` holds the last frame. A machine running or a device coming apart is made as such a GLB: video-mechanisms.md (GreenLight Dash only). |
110
+ | `animation.clip` | glTF animation name \| `"*"` \| `null` | `null` | Plays the named animation baked into the asset; `"*"` plays EVERY clip together, each on its own length (Blender's exporter writes one clip per animated object); `null` = static. Names must match the glTF exactly. `loop: "once"` holds the last frame. A machine running or a device coming apart is made as such a GLB: mechanisms.md (GreenLight Dash only). |
105
111
  | `animation.speed` | `-8–8` | `1` | Playback rate (negative = reversed). |
106
112
  | `animation.loop` | `"once"` \| `"loop"` \| `"pingpong"` | `loop` | Loop mode. |
107
113
  | `animation.offset` | `±600` s | `0` | Start offset into the animation. |
114
+ | `skeleton` | object, optional | absent | Bones and skin weights made for a file that has no skeleton — see [Skeletons](#skeletons-for-meshes-that-have-none-skeleton). Not on a procedural shape (`source: "primitive"`): build a figure to rig in the Scene's graph. |
115
+ | `pose` | object, optional | absent | A character's pose: clips crossfaded, morphs, bones, constraints, layers — see [Characters](#characters--the-pose-pipeline-model3dconfigpose). With it, `animation` only names the base clip when `pose.clips` is empty. |
108
116
  | `shadow.enabled` | bool | `false` | Soft contact-shadow blob under the model. |
109
117
  | `shadow.opacity` | `0–1` | `0.35` | Shadow strength. |
110
118
  | `shadow.blur` | `0–10` | `1` | Shadow softness. |
@@ -131,17 +139,227 @@ Some GLBs are authored facing away from the viewer — for those, set `baseRotat
131
139
  }]
132
140
  ```
133
141
 
134
- A `screen`-shaded mesh shows the media in **its own colours**: its pixels skip the 3D tone curve (a depth-tested coverage mask, so anything in front still hides it), so a UI on a slab matches the same file as a 2D layer. Being self-lit it takes no light — and so no shadow: a character walking on a page or a UI looks as if it floats. Set `receiveShadow: true` on such a screen and the Scene Rig's real shadows darken it where they land (by the casting lights' share of the set's light), while every other pixel keeps the media's exact colours. Use it for any screen something stands on or passes over. Videos are driven by the layer clock and are **frame-exact in exports and stills** for MP4/MOV (WebCodecs decoder lanes, settled before each frame is encoded; the device `screen` uses the same path). WebM falls back to a seeked `<video>` (best effort). The inspector's SCREEN section lists the entries and removes them; agents write them.
142
+ **A layer of the same project on a screen — `layer`.** Instead of `url`, a `screen` or a `screens` entry can take `"layer": "<item id>"` (or a channel id: its item at the playhead): the screen shows that layer's picture at the playhead, drawn live every frame. An HTML clip (including a HyperFrames page) runs its own page and needs no bake: an edit of the `.html` shows on the screen as soon as the editor sees the file change, and renders draw the same frames. A video layer plays in step with the timeline, an image layer shows its picture. Text, shape and sequence layers are not shown (make them an HTML clip). Hide the source layer's channel (`"visible": false`) to show it only on the screen. The screen's canvas is the source's own size (an HTML clip's `html.width` × `html.height`, a video's frame), so it is only as sharp as that; `"resolution": [w, h]` lays an HTML page out at another size. Nothing is uploaded again while the layer shows the same frame (paused, a still page), so Final keeps its samples.
143
+
144
+ ```jsonc
145
+ "screens": [{ "node": "Screen", "layer": "i-ui", "aspect": 2.3846, "fitMode": "cover", "look": "monitor" }]
146
+ ```
135
147
 
136
- **Transforms & keyframes.** The item's ordinary transform fields and `propertyKeyframes` (`scale`, `xPercent`/`yPercent`, `rotateX`/`rotateY`/`rotateZ`, `panZ`, opacity, …) are applied **inside the 3D pass** — a keyframed `rotateY` really orbits the object in 3D rather than spinning a flat card. The project camera films model layers like everything else (`cameraLocked` on the channel opts out).
148
+ **A real monitor's look — `look: "monitor"`** (on `screen` or a `screens` entry; a 3D Scene draws it): a glossy glass coat with soft reflections of the room over a near-black panel, and blacks lifted a few levels as an LCD's backlight shows through (`blackLevel`, 0–0.5, default 0.02 with the look, 0 without). `strength` scales the glow [1]; `glass` is the coat's roughness [0.1] (higher: a matte anti-glare screen). In Final a screen lights the room as a real light (sampled directly, so a keyboard under it is clean at the usual samples), and its weight as a light follows the picture it shows. In Draft the look adds a soft light from the panel in the colour its picture averages to, following it as it changes (no shadows; `"light": false` leaves it out). Without `look` a screen keeps the plain glowing panel.
137
149
 
138
- **Look.** The 3D pass is tone-mapped with **AgX** (the curve Blender renders with; three.js r183's AgXToneMapping) plus a GLEA look — ×1.35 gain so mid-grey sits where the earlier ACES default had it, and 1.2 saturation — in its own pass, so models, the Scene Rig, World View and real-3D effects share one filmic curve while 2D layers are untouched. It replaced the earlier ACES fit as the default.
150
+ A `screen`-shaded mesh shows the media in **its own colours**: its pixels skip the 3D tone curve (a depth-tested coverage mask, so anything in front still hides it), so a UI on a slab matches the same file as a 2D layer. Being self-lit it takes no light — and so no shadow: a character walking on a page or a UI looks as if it floats. Set `receiveShadow: true` on such a screen and the Scene Rig's real shadows darken it where they land (by the casting lights' share of the set's light), while every other pixel keeps the media's exact colours. Use it for any screen something stands on or passes over. Videos are driven by the layer clock and are **frame-exact in exports and stills** for MP4/MOV (the device `screen` uses the same path). WebM falls back to a seeked `<video>` (best effort).
151
+
152
+ **Transforms & keyframes.** The item's ordinary transform fields and `propertyKeyframes` (`scale`, `xPercent`/`yPercent`, `translateZ`, `rotateX`/`rotateY`/`rotate`, `opacity`, …) are applied **inside the 3D pass** — a keyframed `rotateY` really orbits the object in 3D rather than spinning a flat card. The project camera films model layers like everything else (`cameraLocked` on the channel opts out).
153
+
154
+ **Look.** The 3D pass is tone-mapped with **AgX** (the curve Blender renders with) plus a GLEA look (×1.35 gain and 1.2 saturation) in its own pass, so models, the Scene Rig and real-3D effects share one filmic curve while 2D layers are untouched. A 3D Scene takes its film look from `rig.look` and `rig.exposure` ([video-world3d.md](video-world3d.md)).
139
155
 
140
156
  **Rendering.** Model layers render in the editor preview, the desktop Download, **`/render-onserver`** / **`/render-onserver-job`**, and **`/render-still`**. The fast `/render` does **not** support them (video/image only). Very heavy assets (≳900 k triangles) render slowly enough that an export can look hung — prefer decimated models.
141
157
 
158
+ ### Characters — the pose pipeline (`model3dConfig.pose`)
159
+
160
+ A model layer's `pose` animates a character from its GLB: several clips crossfaded, morph targets, bones, constraints (look-at, two-bone
161
+ IK, attach, parent, follow path) and animation layers with masks. The file's clips are only read, never changed. Every frame runs one
162
+ fixed order, the same in the editor, Draft, Final, stills and exports:
163
+
164
+ 1. **clips** — the first is the base; each next one lies over what is below by its weight (a crossfade is the next clip's `fadeIn` while
165
+ the one before keeps playing);
166
+ 2. **morphs** — morph target weights by name;
167
+ 3. **bones** — local transform overrides by bone name;
168
+ 4. **constraints** — solved against the layer's transform of this frame;
169
+ 5. **world** — the layer's transform and floor (the floor is found on the pose before step 4, so an arm reaching down never lifts the feet).
170
+
171
+ `order` stores that list in the project (it is the only order; another is ignored with a warning). Layers (`layers`) run inside each
172
+ step after the pose's own entries, in their order. Names (clips, bones, morphs) are looked up in the file each frame, so a re-imported
173
+ file keeps the pose: a name it no longer has is skipped with a warning (`scene.warnings` of a still in a Scene). A bone's name as the file
174
+ writes it works (`"Palm2.L"`), and so does the loader's (`"Palm2L"`: it drops `.`, `:`, `/`, `[`, `]`). Nothing in a pose costs anything on
175
+ a layer without one: that layer plays `animation` exactly as before.
176
+
177
+ **Every number is keyable**: `keys: { "<field>": [{ "t", "v", "ease" }] }` on the control itself, in the **layer's** seconds (its clip's
178
+ clock, like `animation`), holding before the first key and after the last, `ease` `inOut` by default (`linear`, `in`, `out`, `back`,
179
+ `bounce`).
180
+
181
+ **Clips** (`clips: [...]`):
182
+
183
+ | Field | Values | Default | Meaning |
184
+ | --- | --- | --- | --- |
185
+ | `clip` | animation name | required | The GLB's animation (names exactly as in the file). |
186
+ | `start` / `end` | layer seconds | `0` / none | It lays nothing before `start` or after `end`. |
187
+ | `fadeIn` / `fadeOut` | seconds | `0` | Rises to its weight after `start` / falls to 0 before `end`, on an S-curve (the blend starts and ends gently: no pop). |
188
+ | `weight` | `0–1` | `1` | How much of it lies over what is below. Keyable. |
189
+ | `speed` / `offset` | `-8–8` / s | `1` / `0` | Playback rate; seconds into the clip at `start`. Keyable. |
190
+ | `time` | clip seconds | none | Set the clip's own time outright (key it to scrub by hand); `speed` and `offset` then do nothing. Keyable. |
191
+ | `loop` | `loop` \| `once` \| `pingpong` | `loop` | `once` holds the last frame. |
192
+ | `blend` | `over` \| `add` | `over` | `add` lays the clip's change from the file's rest pose on top (breathing over a walk). |
193
+
194
+ **Root motion** (`root`): `"normal"` (the clip moves the character as it was made), `"inPlace"` (the root's travel along the ground taken
195
+ out, its up and down kept), `"accumulate"` (a looping walk carries on from where the last loop ended instead of jumping back). `{ "mode":
196
+ "inPlace", "bone": "Hips" }` names the root bone; left out, of the bones the clips move by translation the one with the most of the
197
+ skeleton under it (the hips, never a foot).
198
+
199
+ **Morphs** (`morphs: { "<target>": 0.4 }` or `{ "value", "mode": "replace" | "add", "weight", "mesh", "keys" }`): replace sets the weight
200
+ over what the clips made, add adds to it. Draft and Final draw the mixed shape (Final traces it).
201
+
202
+ **Bones** (`bones: { "<bone>": { "rot": [x, y, z], "at", "scale", "mode": "add" | "replace", "weight", "keys" } }`): `rot` degrees in
203
+ the bone's own frame. `add` (default) turns, moves and scales on top of the animation; `replace` sets the local transform, by `weight`.
204
+ Any node of the file counts, not only bones.
205
+
206
+ **Constraints** (`constraints: [...]`), each with `type`, `bone`, `weight` (key it 0 → 1 to hand the bone from its animation to the
207
+ constraint: space switching) and `on`. A target is `"@id"` (a Scene object in `rig.scene.objects`, where it stands at the Scene's time,
208
+ keys and parents included), `[x, y, z]` (a point in the Scene's metres: the editor's 3D space, one composition pixel = 1 mm, the same in a
209
+ Scene and on a classic Rig) or `{ "bone": "<name>" }` (one of the character's own):
210
+
211
+ | `type` | Fields | What it does |
212
+ | --- | --- | --- |
213
+ | `lookAt` | `target`, `chain` [1, up to 4: the bone and its parents share the turn], `limit` [180 degrees from the animated pose], `axis` (the bone's front in its frame; default: the model's front, +Z of the file, as the bone holds it at rest) | Aims the bone at the target by the shortest turn. |
214
+ | `ik` | `target`, `pole`, `tip` | Two-bone IK: `bone` is the end (a wrist `Palm2.L`, an ankle); its parent and grandparent bend so it reaches the target, the middle joint toward `pole` (left out: the bend the pose has). `tip` (a bone under the end: `Middle2.L_end`, a toe) lands on the target instead of the wrist, so the hand touches what it reaches. Out of reach: the limb points at it, straight. |
215
+ | `parent` | `target`, `at`, `rot`, `orient` [true] | The bone is held by the target (a hand on a wheel), offset by `at` (metres) / `rot` (degrees). |
216
+ | `path` | `path` (2+ points), `progress` [0–1, keyable], `closed`, `orient` | The bone moves along a smooth curve through the points (a root along a route, a hand drawing a line). |
217
+ | `attach` | `object` (a Scene object id), `at`, `rot`, `keep`, `grab` | The Scene object is held by the bone (a cup in a hand): Draft and Final draw it there; `weight` between 0 and 1 puts it part way from its own place (key 0 → 1 the moment the hand reaches it). `keep: true` holds it where it was against the bone at the pick-up (`grab` seconds; left out, where `weight` starts to rise) instead of at `at`/`rot`, so it never jumps. |
218
+
219
+ `"@id"` targets are read from `rig.scene`'s JSON (keys and parents included), so they aim the same on a classic Rig; `attach` moves a
220
+ Scene object, which only a 3D Scene (`renderer: "world"`) draws.
221
+
222
+ **Layers** (`layers: [{ "id", "on", "weight", "mask", "clips", "morphs", "bones", "constraints", "keys" }]`): each lies over the pose below,
223
+ in each step, by its `weight` (keyable), only where its `mask` lets it: `bones` and `objects` (names, with their children unless
224
+ `"children": false`), `exclude`, `properties` (`rotation`, `position`, `scale`, `morphs`), `morphs` (names). `"on": false` leaves a layer
225
+ out without deleting it.
226
+
227
+ **What a file offers.** A still with `"inspect": true` answers, for each posed model layer in a Scene, `scene.poses["<item id>"]`: its
228
+ `clips` (name, seconds), `morphs`, `nodes` (every node with its parent, `bone` and `mesh` flags) and `solved` (this frame's constraints:
229
+ `lookAt` `errorDeg`, `ik` `distance` from the target in mm with `reach` and `targetDistance`) plus the pose's `warnings`. Write a small pose
230
+ (one clip), render one still with `inspect`, then name what the file really has. `POST /api/world3d/validate` with `{ "data": <project> }`
231
+ checks every pose with the project (unknown fields, a constraint without its `bone` or `target`, a Scene object that does not exist).
232
+
233
+ **Worked example** — a character idles, waves, turns its head to a flying target on a layer of its own and reaches a ball with its left
234
+ hand (three.js's RobotExpressive.glb, CC0; here placed by `fit: "unit"` from its own origin, so the pose's targets are in its file's units; `fit: "height"` / `"width"` measure a skinned character as it stands too, its bones included):
235
+
236
+ ```json
237
+ "model3dConfig": {
238
+ "source": "custom", "fit": "unit", "fitValue": 160, "anchor": "origin", "placeOnRigFloor": false,
239
+ "pose": {
240
+ "order": ["clips", "morphs", "bones", "constraints", "world"],
241
+ "clips": [
242
+ { "clip": "Idle" },
243
+ { "clip": "Wave", "start": 1.0, "fadeIn": 0.8, "end": 4.2, "fadeOut": 0.8 }
244
+ ],
245
+ "morphs": { "Surprised": { "value": 0, "keys": { "value": [{ "t": 0.9, "v": 0 }, { "t": 1.4, "v": 0.8 }, { "t": 2.4, "v": 0 }] } } },
246
+ "bones": { "Torso": { "rot": [0, 0, 0], "keys": { "rot": [{ "t": 2.8, "v": [0, 0, 0] }, { "t": 3.8, "v": [0, 0, -10] }] } } },
247
+ "layers": [
248
+ { "id": "look", "mask": { "bones": ["Neck"] }, "weight": 0, "keys": { "weight": [{ "t": 0.5, "v": 0 }, { "t": 1.2, "v": 1 }] },
249
+ "constraints": [{ "type": "lookAt", "bone": "Head", "chain": 2, "target": "@fly", "limit": 80 }] },
250
+ { "id": "reach", "mask": { "bones": ["UpperArm.L"] },
251
+ "constraints": [{ "type": "ik", "bone": "Palm2.L", "tip": "Middle2.L_end", "target": "@grip", "pole": [0.45, -0.3, -0.25],
252
+ "weight": 0, "keys": { "weight": [{ "t": 3.2, "v": 0 }, { "t": 4.2, "v": 1 }] } }] }
253
+ ]
254
+ }
255
+ }
256
+ ```
257
+
258
+ with `fly`, `ball` and `grip` as objects of the Scene (`rig.scene.objects`): `{ "id": "ball", "at": [0.24, -0.03, 0.17], "graph": [{ "op":
259
+ "sphere", "radius": 0.04 }] }`, the point the fingertips land on as an object with no graph on the ball's surface toward the shoulder
260
+ `{ "id": "grip", "parent": "ball", "at": [-0.027, -0.004, -0.032] }`, and `fly` keyed along `at`. The target must be within the arm's
261
+ reach of the shoulder (`solved` says `targetDistance` against `reach`). To pick the ball up afterwards add
262
+ `{ "type": "attach", "bone": "Palm2.L", "object": "ball", "keep": true, "weight": 0, "keys": { "weight": [{ "t": 4.3, "v": 0, "ease": "linear" }, { "t": 4.4, "v": 1 }] } }`
263
+ and key the IK's weight back to 0: the arm returns to its clip with the ball in its hand.
264
+
265
+ ### Skeletons for meshes that have none (`skeleton`)
266
+
267
+ `skeleton` gives a mesh with no skeleton one: joints, skin weights, and clips made of keys on the joints. It goes on a model layer
268
+ (`model3dConfig.skeleton`, a GLB/GLTF without a skeleton) or on a Scene object (`rig.scene.objects[i].skeleton`, a mesh built by its
269
+ `graph`, or a `model` file). The mesh is then a skinned character like a file that came with one: a `pose` moves its joints (on a model
270
+ layer `model3dConfig.pose`, on a Scene object the object's own `pose`, on the Scene's clock), Draft skins it, Final traces it skinned,
271
+ stills and exports alike. A mesh without `skeleton` is untouched.
272
+
273
+ Points are in the **mesh's own space**: a Scene object's local metres (before its `at`/`rot`/`scale`); a model file's own units (before the
274
+ layer's fit). The figure stands up +Y, faces +Z, its left is +X (`.L` names).
275
+
276
+ | Field | Values | Default | Meaning |
277
+ | --- | --- | --- | --- |
278
+ | `template` | `biped` \| `quadruped` | none | A ready joint tree placed by `landmarks`. biped: `hips` (root), `spine`, `chest`, `neck`, `head`; `upperArm`, `forearm`, `hand`; `thigh`, `shin`, `foot`, `toe` (each `.L` / `.R`). quadruped: `hips`, `spine`, `chest`, `neck`, `head`, `tail`; `upperArm`, `forearm`, `frontFoot`; `thigh`, `shin`, `hindFoot` (`.L` / `.R`). |
279
+ | `landmarks` | `{ name: [x, y, z] }` | `{}` | Points on the body. biped: `hips`, `spine`, `chest`, `neck` (foot of the neck), `head` (top of the neck), `headTop`, `shoulder`, `elbow`, `wrist`, `handTip`, `hip`, `knee`, `ankle`, `ball`, `toe` (the last ten `.L` / `.R`). quadruped: `hips`, `spine`, `chest`, `neck`, `head`, `nose`, `tail`, `tailTip`, `shoulder`, `elbow`, `wrist`, `frontFoot`, `hip`, `knee`, `hock`, `hindFoot`. **Every one left out is found from the mesh's shape** (cross-sections: the legs where a cut is two rings, a bent knee where the leg's line bends most; the neck at the narrowest cut; each arm, out in a T or A pose or hanging clear of the body at its side, followed from the fingertips by cuts square to it to where it meets the torso: the shoulder half the arm's radius past the armpit, an arm's radius under the shoulder's top; a quadruped's four legs near the ground, a hind leg's hock and knee where it zigzags, its head and neck followed back from the nose: `head` at the narrowest of the neck behind the skull, `neck` where it meets the body). |
280
+ | `symmetric` | bool | `true` | A `.L` point (given or found) also places its `.R`, mirrored across the middle. |
281
+ | `joints` | `[{ name, parent, at, tail, rot }]` | none | Joints written out: the whole tree without a template, or ones that replace a template's joint of the same name or are added to it (fingers, ears, a tail). `at` is where it turns, `tail` where a joint with no children ends, `rot` its rest turn (degrees; left out, its axes are the mesh's). |
282
+ | `bind` | `skin` \| `rigid` | `skin` | `skin`: smooth weights that bend the surface. `rigid`: each connected part follows the one joint that holds most of it (a robot of separate parts). |
283
+ | `skinning` | `dualQuaternion` \| `linear` | `dualQuaternion` | How the joints move the skin. Dual quaternions: every point follows a rigid blend of its joints, so a 90-degree elbow keeps its full thickness and a wrist twisted half a turn stays round; a joint's `scale` (a pose's or a clip's) scales its part of the skin about the joint, as linear blending does. `linear`: the joints' matrices averaged, as a file's own skeleton is drawn: a 90-degree bend thins to 0.71 at the crease, a half-turn twist collapses. |
284
+ | `parts` | `{ "<mesh name>": "<joint>" }` | none | A model file's meshes by name, each following one joint whole. |
285
+ | `weights` | `{ smooth, threshold, visibility }` | `0`, `0.01`, `true` | Blur passes after the solve; the smallest share kept (four joints a vertex at most); whether heat reaches only the points that see a bone from inside the body. |
286
+ | `fixes` | list | none | Manual weight edits, in order — below. |
287
+ | `clips` | `[{ name, duration, bones }]` | none | Animations made of keys — below. |
288
+
289
+ **Automatic weights** are heat diffusion over the surface (each joint's bone held hot, the surface's steady temperature is its weight):
290
+ smooth over every bend, never through the air, so two legs or two fingers that touch nothing but the body share nothing; a joint's heat
291
+ also stops where the nearest joint is neither it, its parent nor its child, so nothing creeps from one leg round the crotch into the other.
292
+ Four joints a vertex at most, adding up to exactly 1: the rest pose is the mesh itself (moves 0). Draft, Final and a model layer drawn
293
+ outside a Scene skin by the same dual quaternions.
294
+
295
+ **Fixes** (`fixes: [...]`), each `{ "joint", "value", "mode": "set" | "add" | "multiply" | "max" | "min", "strength" }` over a region, or
296
+ `{ "smooth": passes }` that blurs every joint's weights there. The other joints share what the fixed one leaves as they did. Regions (all
297
+ given at once multiply): `group` (a vertex group the graph wrote: `weightGradient`, `weightProximity`, `weightMask`, `weightNoise`),
298
+ `mesh` (a model file's mesh by name), `of` (the points a joint holds most now), `box: { min, max }` and `sphere: { at, radius }` with
299
+ `soft` metres of fade, `near: { at, radius, inner, falloff: "smooth" | "linear" | "sharp" }` (a brush).
300
+
301
+ **Clips** (`clips: [{ "name": "wave", "duration": 1, "bones": { "forearm.R": { "rot": [{ "t": 0, "v": [0, 0, -15] }, { "t": 0.5, "v": [0, 0, -55] }, { "t": 1, "v": [0, 0, -15] }] } } }]`):
302
+ `rot` turns from the joint's rest (degrees XYZ: the mesh's axes when the joint has no `rot`), `at` moves from it, `scale` multiplies;
303
+ keys in the clip's own seconds. A pose plays them like a file's own (`"clips": [{ "clip": "wave", "start": 2.8, "fadeIn": 0.6 }]`:
304
+ crossfades, loops, root motion), and its IK, look-at, bones and layers work on the joints by name (`"thigh.L"` or `"thighL"`).
305
+
306
+ **Check before posing.** A still with `"inspect": true` answers `scene.skeletons["<id>"]`: the `joints` (name, parent, where), the
307
+ `landmarks` used and which were `found` from the shape, and a `report` (`vertices`, `parts`, `unseen` points no bone could see,
308
+ `neutralMaxMove` — 0 —, `ms`, the `fixes` with the points each touched). A landmark found in the wrong place: give it in `landmarks`.
309
+ A joint's region that bends wrong in a pose: fix it (`fixes`) and render again. `scene.warnings` names a landmark a template could not
310
+ find, a fix that touched nothing, a joint name that is not a joint. `POST /api/world3d/validate` checks the fields and every joint name
311
+ a part names (a parent, a fix, a clip's track).
312
+
313
+ **Worked example** — a figure built from the graph, rigged, walking with its feet planted by IK, then waving (a Scene object; the walk's
314
+ clips are keys on the template's joints; each foot's IK target is a point keyed on the floor — held where the foot is planted for 60 % of
315
+ a step, then an arc (`ease: "linear"` keys, or one every 1/30 s) to the next plant one stride on; the pole moves with the body; the object's
316
+ own `at` keys carry it forward):
317
+
318
+ ```json
319
+ { "id": "walker", "at": [-1.8, -0.42, 0], "rot": [0, 90, 0], "material": "clay",
320
+ "keys": { "at": [{ "t": 0, "v": [-1.8, -0.42, 0], "ease": "linear" }, { "t": 2.2, "v": [-0.48, -0.42, 0] }] },
321
+ "graph": [{ "op": "skin", "blend": 0.03, "voxel": 0.012, "smooth": 2, "paths": [
322
+ [[0, 0.93, 0, 0.15], [0, 1.1, 0.005, 0.135], [0, 1.3, 0, 0.155], [0, 1.47, 0, 0.06], [0, 1.53, 0.005, 0.065], [0, 1.635, 0.012, 0.11]],
323
+ [[0.095, 0.9, 0, 0.085], [0.105, 0.5, 0.015, 0.06], [0.11, 0.085, 0, 0.045], [0.11, 0.035, 0.15, 0.035]],
324
+ [[-0.095, 0.9, 0, 0.085], [-0.105, 0.5, 0.015, 0.06], [-0.11, 0.085, 0, 0.045], [-0.11, 0.035, 0.15, 0.035]],
325
+ [[0.08, 1.36, 0, 0.09], [0.2, 1.4, 0, 0.055], [0.42, 1.24, 0, 0.045], [0.6, 1.1, 0.01, 0.035], [0.7, 1.02, 0.02, 0.028]],
326
+ [[-0.08, 1.36, 0, 0.09], [-0.2, 1.4, 0, 0.055], [-0.42, 1.24, 0, 0.045], [-0.6, 1.1, 0.01, 0.035], [-0.7, 1.02, 0.02, 0.028]] ] }],
327
+ "skeleton": { "template": "biped",
328
+ "landmarks": { "hips": [0, 0.95, 0], "chest": [0, 1.3, 0], "neck": [0, 1.46, 0], "head": [0, 1.55, 0.005], "headTop": [0, 1.75, 0.012],
329
+ "shoulder.L": [0.2, 1.4, 0], "elbow.L": [0.42, 1.24, 0], "wrist.L": [0.6, 1.1, 0.01], "handTip.L": [0.7, 1.02, 0.02],
330
+ "hip.L": [0.095, 0.88, 0], "knee.L": [0.105, 0.5, 0.015], "ankle.L": [0.11, 0.085, 0], "toe.L": [0.11, 0.035, 0.185] },
331
+ "clips": [
332
+ { "name": "walk", "duration": 1.1, "bones": {
333
+ "hips": { "at": [{ "t": 0, "v": [0, -0.038, 0] }, { "t": 0.275, "v": [0, -0.062, 0] }, { "t": 0.55, "v": [0, -0.038, 0] }, { "t": 0.825, "v": [0, -0.062, 0] }, { "t": 1.1, "v": [0, -0.038, 0] }] },
334
+ "upperArm.L": { "rot": [{ "t": 0, "v": [22, 0, -50] }, { "t": 0.55, "v": [-22, 0, -50] }, { "t": 1.1, "v": [22, 0, -50] }] },
335
+ "upperArm.R": { "rot": [{ "t": 0, "v": [-22, 0, 50] }, { "t": 0.55, "v": [22, 0, 50] }, { "t": 1.1, "v": [-22, 0, 50] }] } } },
336
+ { "name": "wave", "duration": 1, "bones": {
337
+ "upperArm.R": { "rot": [{ "t": 0, "v": [0, 0, -125] }] },
338
+ "forearm.R": { "rot": [{ "t": 0, "v": [0, 0, -15] }, { "t": 0.5, "v": [0, 0, -55] }, { "t": 1, "v": [0, 0, -15] }] } } } ] },
339
+ "pose": {
340
+ "clips": [{ "clip": "walk", "end": 2.55, "fadeOut": 0.5 }, { "clip": "wave", "start": 2.8, "fadeIn": 0.6 }],
341
+ "constraints": [
342
+ { "type": "ik", "bone": "foot.L", "target": [-1.6, -0.335, -0.11], "pole": [-0.8, 0.08, -0.11],
343
+ "keys": { "target": [{ "t": 0, "v": [-1.6, -0.335, -0.11] }, { "t": 0.66, "v": [-1.6, -0.335, -0.11], "ease": "linear" },
344
+ { "t": 0.88, "v": [-1.28, -0.265, -0.11], "ease": "linear" }, { "t": 1.1, "v": [-0.96, -0.335, -0.11] }, { "t": 1.76, "v": [-0.96, -0.335, -0.11] }] } },
345
+ { "type": "ik", "bone": "foot.R", "target": [-1.92, -0.335, 0.11], "pole": [-0.8, 0.08, 0.11], "keys": { "target": [ "… the same steps, half a cycle later" ] } }
346
+ ],
347
+ "layers": [{ "id": "look", "mask": { "bones": ["neck"] }, "weight": 0, "keys": { "weight": [{ "t": 2.6, "v": 0 }, { "t": 3.3, "v": 1 }] },
348
+ "constraints": [{ "type": "lookAt", "bone": "head", "chain": 2, "target": [0.1, 1.08, 4], "limit": 75 }] }]
349
+ } }
350
+ ```
351
+
352
+ A model file without a skeleton is rigged the same way on its layer: `"model3dConfig": { "fit": "unit", "fitValue": 1000, "skeleton":
353
+ { "template": "biped", "landmarks": { "shoulder.L": [0.16, 1.29, -0.06] }, "fixes": [{ "joint": "head", "value": 1, "box": { "min":
354
+ [-0.21, 1.4, -0.2], "max": [0.21, 1.76, 0.14] }, "soft": 0.03 }], "clips": [ … ] }, "pose": { … } }` — every other landmark found from its
355
+ shape, and one fix that gives the whole head to the head joint (its automatic weights left a flap at the back of the neck behind when
356
+ the head turned).
357
+
142
358
  ### Scene Rig — one set for every 3D layer
143
359
 
144
- A **rig** clip carries the project's lights, backdrop and environment. It is a singleton, like the camera: `mediaType: "rig"` on a channel with `kind: "rig"`, and the item carries `item.rig`. **When a rig exists it lights every model layer** and `model3dConfig.lights` is ignored; with no rig, each model keeps its own three lights exactly as before. Emitting a rig is therefore additive — it never changes an existing project until you add one.
360
+ > **Every Rig is a 3D Scene**, drawn by the world renderer unless `rig.renderer` is `"classic"` (or the machine has no WebGPU, below): the backdrop, the Rig's lights and environment and the model layers become one lit, shadowed world, path traced in Final 3D, and you can build in it yourself, with no Blender, by writing `rig.scene` ([video-world3d.md](video-world3d.md)). The fields below hold in both; where the world reads them differently, the section says so.
361
+
362
+ A **rig** clip carries the project's lights, backdrop and environment. It is a singleton, like the camera: `mediaType: "rig"` on a channel with `kind: "rig"`, and the item carries `item.rig`. **When a rig exists it lights every model layer** and `model3dConfig.lights` is ignored; with no rig, each model keeps its own three lights. Adding a rig is therefore additive — it never changes an existing project until you add one.
145
363
 
146
364
  ```jsonc
147
365
  "rig": {
@@ -150,7 +368,7 @@ A **rig** clip carries the project's lights, backdrop and environment. It is a s
150
368
  "width": 4, "height": 2.25, "depth": 3, // MULTIPLES OF COMP HEIGHT, not px
151
369
  "curve": 0.35, // floor→wall sweep, as a fraction of height
152
370
  "x": 0, "y": 0, "z": 0, "rotX": 0, "rotY": 0, "rotZ": 0, "scale": 1,
153
- "material": { "mode": "solid", // solid | shadowCatcher
371
+ "material": { "mode": "solid", // solid | shadowCatcher | image (a picture, `imageUrl`)
154
372
  "color": "#d8d8d8", "roughness": 0.8,
155
373
  "metalness": 0, "opacity": 1,
156
374
  "receiveShadow": true, "castShadow": false }
@@ -168,17 +386,22 @@ A **rig** clip carries the project's lights, backdrop and environment. It is a s
168
386
  }
169
387
  ```
170
388
 
171
- **Shadows.** `mode: "map"` gives real shadow maps: a model casts onto the cyclorama and
389
+ **Which renderer draws it.** On a machine with WebGPU, every Rig is drawn by the **world renderer** ([video-world3d.md](video-world3d.md)) unless `rig.renderer` is `"classic"`; without WebGPU it falls back to the classic Rig passes. Both read the same `rig`, with these differences:
390
+
391
+ - The world renderer ignores the `shadows` block (`mode`, `maxCasters`) and a light's `shadowBias`, `shadowNormalBias` and `shadowMapSize`: a light casts when its own `castShadow` is true. The shadow-map paragraphs below (**Shadows**, **The shadow map follows the camera**, the area light's companion caster) describe the classic renderer.
392
+ - `shadowSoftness` is the light's size in the world renderer (see the light fields below); in the classic renderer it widens the shadow-map penumbra.
393
+
394
+ **Shadows (classic renderer).** `mode: "map"` gives real shadow maps: a model casts onto the cyclorama and
172
395
  onto other models. A light only casts when it ALSO has `castShadow: true` — the mode
173
396
  alone does nothing. `maxCasters` (0-4, default 2) caps how many casting lights are
174
397
  honoured, because each one is a full extra render of the set every frame; lights past
175
398
  the cap are silently not casting, in array order. The cyclorama receives by default
176
399
  (`cyclorama.material.receiveShadow`) and does not cast. `mode: "contact"` keeps only
177
400
  the cheap soft blob under each model standing on the rig floor (`placeOnRigFloor`).
178
- A Scene Rig added in the editor starts with `mode: "map"` and only the Key light
179
- (`k1`) casting; a rig you write yourself gets exactly what you write.
401
+ A rig you write gets exactly what you write. In a 3D Scene none of this applies: every light with
402
+ `castShadow: true` casts (see **Which renderer draws it** above).
180
403
 
181
- **The shadow map follows the camera.** Each frame a casting light's map is fitted to
404
+ **The shadow map follows the camera (classic renderer).** Each frame a casting light's map is fitted to
182
405
  what the camera actually sees of the set (the frustum clipped to the receivers, in the
183
406
  casters' reach), in 1/8-octave size steps with a texel-snapped centre so a moving
184
407
  camera does not make shadow edges crawl. A small character on a long set (a ten-metre
@@ -189,38 +412,79 @@ override it with `shadowBias` / `shadowNormalBias` (0–20, in texels of its map
189
412
  them out for automatic — raise them if a surface still shows stripes, lower them if a
190
413
  shadow looks detached).
191
414
 
192
- **One shared world, but only when a rig exists.** With a rig, every 3D layer is placed
193
- in one scene so the shadow map can contain every caster; each layer still renders in
194
- its own pass, so opacity, blend modes, masks, effects and DOF are unchanged. With NO
195
- rig, each model keeps its own scene and its own `model3dConfig.env` — which is why
196
- adding a rig to an existing project is additive and never re-lights it by surprise.
197
- With a rig, the rig's `env` (and exposure) is the one in effect for every model;
198
- key `sc:env:intensity` to change reflections over time (e.g. only once a metal
199
- object takes the stage).
200
-
201
- **Camera depth of field is per pixel on 3D passes.** A model, a cyclorama or an
202
- occluding shadow plate hands the compositor its depth buffer, and every pixel blurs by
203
- the CoC of its OWN distance (`camAperture · |camFocusDistance − d| / d · camBlurLevel/100`,
204
- capped at 64 px, still held off inside the focus area) — a character in focus stays
205
- sharp in the same model as a wall metres behind it. The gather is depth-aware: a
206
- blurred background never smears over a sharp foreground edge, a sharp foreground
207
- leaves no halo on the blurred background, and a foreground in front of the focus plane
208
- blurs out over what is behind it and past its own silhouette. Taps read the picture
209
- prefiltered (mip level of the tap spacing), so thin bright detail under a wide blur —
210
- lettering far behind the focus — averages instead of sparkling; 2D layers get the same.
211
- A whole set (backdrop included) can therefore live in one model layer. Flat 2D layers
212
- keep one CoC per layer — the depth of their plane.
213
-
214
- **Light types and their fields.** `directional` / `point` / `spot` / `area` take `az` (±360°), `elev` (±90°), `dist` (world px); `ambient` and `hemisphere` take neither. `spot` adds `angle` (1–89°) and `penumbra` (0–1); `spot` and `area` aim at `targetX` / `targetY` / `targetZ` (world px, default the rig origin); `hemisphere` adds `groundColor`. Fields a type does not have are dropped by the sanitizer.
215
-
216
- **`area` — a softbox.** A lit rectangle of `width` × `height` world px (default 800 × 800) at its spherical position, facing its target: broad, soft highlights and soft wrap-around light, the way a large source lights in Blender or a studio — use it for keys and fills on products, glossy and metal surfaces. Bigger at the same `intensity` = the same light on the subject, only softer. It has no shadow map of its own: with `castShadow: true` a companion light from the same place carries half its light and casts the shadow (a lighter, softer shadow, as from a large source), and counts against `shadows.maxCasters`. Set the softness to match the box: `shadowSoftness` ≈ (box size / distance to the target) ÷ 0.22.
217
-
218
- **`intensity` means the same on every light: what the set receives.** A `point` or `spot` of `2` at `dist: 2400` (or an `area` of `2` at any size, at its target) lights the subject at the centre of the set like a `directional` of `2`, then falls off physically around it (closer is brighter, farther darker). A casting light's `shadowSoftness` (0–10, default 1) widens its penumbra, scaled by the light's distance — contact points stay sharp and the shadow softens away from them.
415
+ **One shared world, but only when a rig exists.** With `renderer: "classic"`, every 3D
416
+ layer is placed in one scene so the shadow map can contain every caster, and each layer
417
+ still renders in its own pass, so opacity, blend modes, masks, effects and DOF are
418
+ unchanged. In a 3D Scene the world draws the models, and a model it draws takes no
419
+ `opacity` or `blendMode`. With NO rig, each model keeps its own scene and its own
420
+ `model3dConfig.env`, which is why adding a rig to an existing project is additive and
421
+ never re-lights it by surprise. With a rig, the rig's `env` (and exposure) is the one in
422
+ effect for every model; key `sc:env:intensity` to change reflections over time (e.g.
423
+ only once a metal object takes the stage). A 3D Scene turns its environment by `rotY`
424
+ alone.
219
425
 
220
- ### Flat layers in the 3D scene — `item.shadowPlate`
426
+ **Camera depth of field is per pixel on 3D passes** (a model, the cyclorama, an occluding shadow plate), so a whole set, backdrop included, can live in one model layer; each pixel blurs by a radius in px: [video-camera.md](video-camera.md#depth-of-field).
427
+
428
+ **Light types and their fields.** Every light has `lid`, `type`, `name`, `on`, `color` and `intensity` (0–100, default 1). `directional` / `point` / `spot` / `area` also take `az` (±360°), `elev` (±90°), `dist` (0–100000 world px, default 3000), `castShadow` (default false; obeyed in Draft and Final on every kind of light: leave it off on a fill, or it throws a second shadow) and `shadowSoftness` (0–10, default 1); `ambient` and `hemisphere` take neither. `spot` adds `angle` (1–89°, default 32) and `penumbra` (0–1, default 0.5); `point` adds `range` (world px, default 0) and `decay` (0–4, default 2, the falloff exponent); `area` adds `width` × `height` (world px, default 800 × 800) and `spread` (below); `spot` and `area` aim at `targetX` / `targetY` / `targetZ` (world px, default the rig origin); `hemisphere` adds `groundColor`. Fields a type does not have are dropped on load.
429
+
430
+ - **`castShadow`** — a fill leaves it off, or it throws a second shadow.
431
+ - **`shadowSoftness`** — in the world renderer it is the light's size, and the subject stays as bright: a directional light becomes a sun `1 + 2 × shadowSoftness` degrees wide, an area light's panel is scaled by it, and a spot or point lamp grows with it and with its distance. In the classic renderer a casting light's `shadowSoftness` widens its penumbra, scaled by the light's distance — contact points stay sharp and the shadow softens away from them.
432
+ - **`spread`** (area only, 1–180, default 180) narrows a panel to a pool of light, like a grid on a soft box. The subject stays as bright. Stored only below 180.
433
+ - **`texture`** (on `directional`, `spot` and `area`) is what the light shines through: `{ "pattern": "window" }` (or `blinds`, `leaves`), or an image of the user's: `{ "url": "/media/…/gobo.png", "name": "gobo.png" }`. A spot projects it in colour, in Draft and Final. A directional and an area light cast the pattern from something out of frame. An image on an area light sits on the panel itself, so it shows in reflections in Final and tints the set in Draft.
434
+
435
+ **`area` — a softbox.** A lit rectangle of `width` × `height` world px at its spherical position, facing its target: broad, soft highlights and soft wrap-around light, the way a large source lights in Blender or a studio — use it for keys and fills on products, glossy and metal surfaces. Bigger at the same `intensity` = the same light on the subject, only softer. In the classic renderer it has no shadow map of its own: with `castShadow: true` a companion light from the same place carries half its light and casts the shadow (a lighter, softer shadow, as from a large source), and counts against `shadows.maxCasters`; match the softness to the box there with `shadowSoftness` ≈ (box size / distance to the target) ÷ 0.22.
436
+
437
+ **`intensity` means the same on every light: what the set receives.** A `point` or `spot` of `2` at `dist: 2400` (or an `area` of `2` at any size, at its target) lights the subject at the centre of the set like a `directional` of `2`, then falls off physically around it (closer is brighter, farther darker).
438
+
439
+ **`lid`** matches `/^[A-Za-z0-9_-]{1,12}$/` and must be unique within the rig — it is the key keyframe tracks address.
440
+
441
+ **Sizes are multiples of composition height**, so a rig authored at 1080p is correct at 4K with no migration. `width: 4` on a 1080-tall comp is 4320 world px.
442
+
443
+ **Switch a light off with `intensity: 0` or `on: false`, never by removing it.** The light count is a compiled shader constant, so changing it recompiles every material in the scene. `on: false` turns the light off the same way and keeps its `intensity` for later.
444
+
445
+ **Cyclorama shape `"none"`** draws nothing: use it for a rig that only carries lights. Any other shape with `material.mode: "solid"` writes depth, so layers behind it are cut by its surface.
446
+
447
+ **`material.mode: "image"`** with `material.imageUrl` (a media url) puts a picture on the backdrop in the world renderer; the picture is multiplied by `color`, so leave `color` white to show its own colours. The classic renderer does not read `imageUrl`.
448
+
449
+ **`material.mode: "shadowCatcher"`** keeps only the shadows the set receives and is transparent everywhere else, for models standing on the composition `background`. It writes no depth, so it never cuts a layer. Its shadows use `material.shadowColor` (default `#000000`), not `color` — `color` stays the Solid surface's colour, so a paper set switched to shadows does not paint them in the paper's own colour. `opacity` is the shadow strength (0-1), and the backdrop colour track `scc:cyc:col` drives `shadowColor` in this mode.
450
+
451
+ **`material.light`** (0–4, absent = 0) makes a `shadowCatcher` floor take more than shadows onto the footage under it, in the 3D Scene's Final: the light the objects add — their bounce in their own colour, their reflections on a glossy floor (`roughness`) — goes over the footage, and Final traces the floor in its own `color`, or wearing `imageUrl` (a picture of the real floor laid flat across the floor's `width` × `depth`, x along `width`, the far edge at the picture's top). Draft shows the shadow alone. The footage call (GreenLight Dash only) sets all of it from the clip; set it yourself for a model over any picture of a floor.
452
+
453
+ **Rig keyframes** live on the rig item's `propertyKeyframes` under two prefixes (`sc:` numeric, `scc:` colour — the prefix alone decides numeric vs RGBA interpolation):
221
454
 
222
- **Turning a flat layer in 3D — the pivot.** Every flat layer (image, video, shape, text, html clip) turns in 3D
223
- with `rotateX` / `rotateY` / `rotateZ`, static fields and `propertyKeyframes` tracks alike: a static angle and the
455
+ | Key | Range |
456
+ | --- | --- |
457
+ | `sc:<lid>:intensity` | 0–100 |
458
+ | `sc:<lid>:az` \| `elev` \| `dist` \| `shadowSoftness` | per the fields above; not on `ambient` / `hemisphere` |
459
+ | `sc:<lid>:angle` \| `penumbra` | spot only |
460
+ | `sc:<lid>:width` \| `height` \| `spread` | area only (world px; spread in degrees) |
461
+ | `scc:<lid>:col` | colour; on an `ambient` / `hemisphere` in a world Scene it tints the light it adds to the environment (a `hemisphere`'s `groundColor` tints the lower half) |
462
+ | `sc:cyc:x` \| `y` \| `z` \| `rotX` \| `rotY` \| `rotZ` \| `scale` \| `opacity` | `opacity` is the shadow strength on a `shadowCatcher`; on a Solid or picture backdrop it is the backdrop's own opacity (in a world Scene too) |
463
+ | `scc:cyc:col` | colour |
464
+ | `sc:env:intensity` \| `exposure` \| `rotX` \| `rotY` \| `rotZ` | — |
465
+ | `sc:g~<material>:<param>` | a number of a Scene material's animated texture graph (`edgeGlow`, `fractalNoise`, `motionBeam`, `sweep`, or a library name for one): the preset's range ([video-world3d.md](video-world3d.md#animated-materials-edge-glow-live-noise-beam-sweep-graph-presets)) |
466
+
467
+ **Refused, and dropped on load:** light `type`, adding/removing a light, `castShadow`, `shadowMapSize`, cyclorama `shape` and dimensions, `env.source`/`preset`/`url`. Each is a shader recompile or an environment re-bake, not a value — animating one would rebuild every frame. A track naming a `lid` that no longer exists is dropped on the next save.
468
+
469
+ **The playhead outside every rig clip does NOT mean darkness.** The nearest rig still lights the scene, and a project with no rig falls back to the default three directional lights.
470
+
471
+ **24 built-in presets**, each a **complete** rig — applying one replaces `item.rig` wholesale:
472
+
473
+ | Category | Preset ids |
474
+ | --- | --- |
475
+ | Studio | `threePoint` (the same lights as a project with no rig — use it to match one), `studioRoom`, `clamshellBeauty`, `highKeyWhite` |
476
+ | Product | `productSoftbox`, `topDownProduct`, `turntable`, `pastelSweep`, `pianoBlack`, `cosmeticGlow`, `windowFlatlay` |
477
+ | Dramatic | `rimLitDark`, `filmNoir`, `stageSpotlight`, `campfireGlow`, `rimHero` |
478
+ | Daylight | `sunsetWindow`, `overcastExterior`, `desertNoon`, `blueHour` |
479
+ | Location | `warehouse`, `sciFiRoom` |
480
+ | Stylized | `neonTwoTone`, `duotonePop` |
481
+
482
+ **Other rig-item fields the editor writes.** `item.rigLook = { id, name, hash }` records which look was applied (a preset id, or `saved:<id>` for a look the user saved) and a hash of the rig at that moment, so the editor can mark the look as edited once the rig drifts from it. It is informational — omit it when emitting a rig; a hand-made `hash` never matches, so the look would show as edited from the start. `item.rigAi` holds the AI video state — the prerender used as the source, the takes and which one is placed on the timeline. It is written by the editor from real render and generation results; do not write it yourself.
483
+
484
+ ### Turning flat layers in 3D — pivot and stacking
485
+
486
+ **The pivot.** Every flat layer (image, video, shape, text, html clip) turns in 3D
487
+ with the static `rotateX` / `rotateY` / `rotateZ` fields and the `rotateX` / `rotateY` / `rotate` keyframe tracks alike: a static angle and the
224
488
  same angle reached by a key give the same pose, and mixing static and keyed channels never moves the layer. The
225
489
  pivot is `keyframeTransformOrigin` (`"center"` default; `"top"`, `"bottom"`, `"left"`, `"right"`, the four
226
490
  corners such as `"bottom left"`, or CSS-style `"50% 100%"`): rotation and the keyed `scale` / `scaleX` / `scaleY`
@@ -235,6 +499,8 @@ in 3D is ordered against every flat layer it does not cross by which side of tha
235
499
  a card standing on a page or floor layer always draws over it, wherever the camera looks. Two layers that cut
236
500
  through each other still need `shadowPlate.occludes` (below) to intersect per pixel.
237
501
 
502
+ ### Flat layers in the 3D scene — `item.shadowPlate`
503
+
238
504
  `text`, `shape`, `image` and `video` items can join the 3D scene without becoming
239
505
  solids. The layer stays flat and keeps its normal renderer, so text animators, shape and
240
506
  video animation stay native and its alpha updates every frame.
@@ -255,53 +521,9 @@ ordinary 2D layer. The field is ignored on any other media type.
255
521
  - **`castShadow`** — the layer's live silhouette is added to the rig's shadow map, so it
256
522
  drops a shadow on models and on the cyclorama. Needs a rig with
257
523
  `shadows.mode: "map"` and at least one light with `castShadow: true`. The layer's own
258
- pixels are untouched.
524
+ pixels are untouched. A 3D Scene's world takes no flat silhouette, so there a scene
525
+ object or a model layer has to cast the shadow.
259
526
  - **`receiveShadow`** — the set's shadows fall across the layer. This is the only one
260
527
  that re-films the layer inside the lit scene, so **Motion Blur, Rig (Puppet) and 3D
261
528
  Depth are unavailable on that layer** while it is on; the other two leave them
262
529
  working.
263
-
264
- **`lid`** matches `/^[A-Za-z0-9_-]{1,12}$/` and must be unique within the rig — it is the key keyframe tracks address.
265
-
266
- **Sizes are multiples of composition height**, so a rig authored at 1080p is correct at 4K with no migration. `width: 4` on a 1080-tall comp is 4320 world px.
267
-
268
- **A light is switched off with `intensity: 0`, never by removing it** — the light count is a compiled shader constant, so changing it recompiles every material in the scene. `on: false` is presentation only and does the same thing.
269
-
270
- **Cyclorama shape `"none"`** draws nothing: use it for a rig that only carries lights. Any other shape with `material.mode: "solid"` writes depth, so layers behind it are cut by its surface.
271
-
272
- **`material.mode: "shadowCatcher"`** keeps only the shadows the set receives and is transparent everywhere else, for models standing on the composition `background`. It writes no depth, so it never cuts a layer. Its shadows use `material.shadowColor` (default `#000000`), not `color` — `color` stays the Solid surface's colour, so a paper set switched to shadows does not paint them in the paper's own colour. `opacity` is the shadow strength (0-1), and the backdrop colour track `scc:cyc:col` drives `shadowColor` in this mode.
273
-
274
- **Rig keyframes** live on the rig item's `propertyKeyframes` under two prefixes (`sc:` numeric, `scc:` colour — the prefix alone decides numeric vs RGBA interpolation):
275
-
276
- In the editor, the stopwatch beside an animatable rig field promotes that exact
277
- property into a flat 0%/100% track seeded from the field's current authored value.
278
- Once any rig track exists, moving a selected light in the 3D world view auto-keys
279
- its `az`, `elev` and `dist` tracks at the playhead instead of overwriting the base
280
- rig. API-authored projects use the same track shape directly.
281
-
282
- | Key | Range |
283
- | --- | --- |
284
- | `sc:<lid>:intensity` \| `az` \| `elev` \| `dist` | per the light's type |
285
- | `sc:<lid>:angle` \| `penumbra` | spot only |
286
- | `sc:<lid>:width` \| `height` | area only (world px) |
287
- | `scc:<lid>:col` | colour |
288
- | `sc:cyc:x` \| `y` \| `z` \| `rotX` \| `rotY` \| `rotZ` \| `scale` \| `opacity` | — |
289
- | `scc:cyc:col` | colour |
290
- | `sc:env:intensity` \| `exposure` \| `rotX` \| `rotY` \| `rotZ` | — |
291
-
292
- **Refused, and the sanitizer drops them:** light `type`, adding/removing a light, `castShadow`, `shadowMapSize`, cyclorama `shape` and dimensions, `env.source`/`preset`/`url`. Each is a shader recompile or an environment re-bake, not a value — animating one would rebuild every frame. A track naming a `lid` that no longer exists is dropped on the next save.
293
-
294
- **The playhead outside every rig clip does NOT mean darkness.** The nearest rig still lights the scene, and a project with no rig falls back to the legacy three directionals.
295
-
296
- **21 built-in presets** (`rigPresets.js`), each a **complete** rig — applying one replaces `item.rig` wholesale:
297
-
298
- | Category | Preset ids |
299
- | --- | --- |
300
- | Studio | `threePoint` (the exact legacy default — use it to match a pre-rig project), `studioRoom`, `clamshellBeauty`, `highKeyWhite` |
301
- | Product | `productSoftbox`, `topDownProduct`, `turntable`, `pastelSweep`, `pianoBlack` |
302
- | Dramatic | `rimLitDark`, `filmNoir`, `stageSpotlight`, `campfireGlow` |
303
- | Daylight | `sunsetWindow`, `overcastExterior`, `desertNoon`, `blueHour` |
304
- | Location | `warehouse`, `sciFiRoom` |
305
- | Stylized | `neonTwoTone`, `duotonePop` |
306
-
307
- **Other rig-item fields the editor writes.** `item.rigLook = { id, name, hash }` records which look was applied (a preset id, or `saved:<id>` for a look the user saved) and a hash of the rig at that moment, so the editor can mark the look as edited once the rig drifts from it. It is informational — omit it when emitting a rig; a hand-made `hash` never matches, so the look would show as edited from the start. `item.rigAi` holds the AI video state — the prerender used as the source, the takes and which one is placed on the timeline. It is written by the editor from real render and generation results; do not write it yourself.