@algorisys/tinyfly 0.91.0 → 0.93.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -6
- package/bin/tinyfly.mjs +34 -1
- package/docs/2d-animation-roadmap.md +191 -0
- package/docs/3d-rotations.md +164 -0
- package/docs/3d-support-plan.md +628 -0
- package/docs/DEPLOYMENT.md +414 -0
- package/docs/acting.md +466 -0
- package/docs/api-reference.md +2353 -0
- package/docs/camera.md +94 -0
- package/docs/cartoon-hands.md +96 -0
- package/docs/character-system-m1.md +218 -0
- package/docs/character-system-plan.md +384 -0
- package/docs/dance.md +315 -0
- package/docs/editor-guide.md +1249 -0
- package/docs/examples.md +765 -0
- package/docs/extending.md +168 -0
- package/docs/file-format.md +690 -0
- package/docs/getting-started.md +314 -0
- package/docs/grid-and-snapping.md +64 -0
- package/docs/gsap-compat.md +1241 -0
- package/docs/llm-guide.md +123 -0
- package/docs/maps.md +116 -0
- package/docs/onion-skinning.md +41 -0
- package/docs/pen-tool.md +65 -0
- package/docs/polygon-star.md +44 -0
- package/docs/scene-3d.md +299 -0
- package/docs/scroll-animation.md +314 -0
- package/docs/shape-morph.md +61 -0
- package/docs/sprite-sheet-export.md +54 -0
- package/docs/symbols-and-library.md +137 -0
- package/docs/teaching.md +348 -0
- package/docs/video-rendering.md +728 -0
- package/lib/addons/adapters.js +22 -18
- package/lib/addons/characters.js +2785 -1794
- package/lib/addons/target-properties-BtN9skb-.js +142 -0
- package/lib/browser/tinyfly.iife.js +9 -4
- package/lib/browser/tinyfly.js +6575 -5454
- package/lib/browser/tinyfly.umd.js +9 -4
- package/lib/engine/tinyfly-engine.js +286 -258
- package/lib/engine/tinyfly-engine.umd.cjs +2 -2
- package/lib/types/adapters/canvas/canvas-adapter.d.ts +6 -0
- package/lib/types/adapters/canvas/index.d.ts +1 -0
- package/lib/types/adapters/canvas/target-properties.d.ts +68 -0
- package/lib/types/characters/acting/beat-check.d.ts +95 -0
- package/lib/types/characters/acting/hand-path.d.ts +32 -0
- package/lib/types/characters/acting/index.d.ts +2 -0
- package/lib/types/characters/acting/script.d.ts +66 -7
- package/lib/types/characters/capabilities.d.ts +50 -0
- package/lib/types/characters/code-panel.d.ts +201 -0
- package/lib/types/characters/figure-actions.d.ts +11 -0
- package/lib/types/characters/figure-props.d.ts +87 -0
- package/lib/types/characters/gaits.d.ts +8 -0
- package/lib/types/characters/index.d.ts +4 -0
- package/lib/types/characters/species/human-motion.d.ts +45 -0
- package/lib/types/characters/species/human.d.ts +2 -0
- package/lib/types/characters/stick-figure.d.ts +2 -0
- package/lib/types/engine/authoring/did-you-mean.d.ts +14 -0
- package/lib/types/engine/authoring/index.d.ts +1 -0
- package/llms-full.txt +12548 -0
- package/llms.txt +57 -0
- package/package.json +6 -2
- package/skills/tinyfly/SKILL.md +64 -0
- package/lib/addons/camera-C7bZOWMB.js +0 -33
|
@@ -0,0 +1,628 @@
|
|
|
1
|
+
# 3D Support Plan
|
|
2
|
+
|
|
3
|
+
**Status:** Milestones 0 (rotation math, slerp tracks), 1 (CSS-style 3D in every adapter), 2 (scenes on Canvas 2D), 3 (WebGL2 renderer), 4 (the editor's 3D Scene element, first version) and 6 (3D characters) implemented; the rest planned
|
|
4
|
+
**Date:** 2026-10-04 (revised 2026-10-05: movies, two render modes, entries)
|
|
5
|
+
**Related:** [character-system-plan.md](character-system-plan.md),
|
|
6
|
+
[character-system-m1.md](character-system-m1.md), [camera.md](camera.md),
|
|
7
|
+
[video-rendering.md](video-rendering.md), [file-format.md](file-format.md)
|
|
8
|
+
|
|
9
|
+
This plan covers adding 3D to tinyfly without changing what tinyfly is: a
|
|
10
|
+
small, deterministic, JSON-first engine that computes values over time and
|
|
11
|
+
hands them to adapters. Each milestone below ships on its own, as one minor
|
|
12
|
+
release, with tests and a gallery demo.
|
|
13
|
+
|
|
14
|
+
3D is also meant for **movies**: short films made from data, rendered in a
|
|
15
|
+
**stylized** look by our own renderers or a **realistic** one by three.js, from
|
|
16
|
+
the same scene and timeline. The film work itself (shots, cuts, sets, sound,
|
|
17
|
+
long renders) is Phase 34 in `todo.md`; this plan makes sure the 3D foundation
|
|
18
|
+
leaves room for it (section 9).
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## 1. Goals and non-goals
|
|
23
|
+
|
|
24
|
+
### Goals
|
|
25
|
+
|
|
26
|
+
1. **3D is data.** A 3D scene, its camera, lights, objects and materials are
|
|
27
|
+
plain JSON. Their animation is ordinary tracks in an ordinary
|
|
28
|
+
`TimelineDefinition`. Nothing about 3D needs a new timeline or a new clock.
|
|
29
|
+
2. **The engine stays renderer-agnostic.** `src/engine/` gains only pure math
|
|
30
|
+
helpers (vectors, quaternions, matrices) and one new interpolation mode
|
|
31
|
+
(spherical interpolation of rotations). It does not learn about meshes,
|
|
32
|
+
cameras, lights or GPUs.
|
|
33
|
+
3. **Deterministic.** The same scene and timeline at the same time give the same
|
|
34
|
+
resolved scene (the same matrices, the same draw order), in the browser, in a
|
|
35
|
+
Worker and in Node. Frames can be rendered in any order.
|
|
36
|
+
4. **Headless works from day one.** `tinyfly video` and `renderVideo()` can
|
|
37
|
+
render a 3D scene to MP4/PNG in Node with the dependencies they already have
|
|
38
|
+
(`@napi-rs/canvas`, ffmpeg).
|
|
39
|
+
5. **Usable without the editor**, then usable in the editor, through the same
|
|
40
|
+
public API.
|
|
41
|
+
6. **Characters can go 3D.** Character v2 already solves its bones in 3D
|
|
42
|
+
(`SolvedChain.joints3`); a real camera should be able to look at them.
|
|
43
|
+
7. **Films.** A scene can hold several cameras and cut between them on the
|
|
44
|
+
timeline, and a whole film renders frame by frame, in any order, to MP4.
|
|
45
|
+
8. **Two render modes from one source.** `stylized` (our Canvas 2D and WebGL2
|
|
46
|
+
renderers: flat, toon, pencil; deterministic to the pixel on Canvas 2D) and
|
|
47
|
+
`realistic` (the three.js adapter: PBR, shadow maps, environment light). The
|
|
48
|
+
scene, timeline, shots and sound do not change between them; each renderer
|
|
49
|
+
reads the material and light fields it understands.
|
|
50
|
+
9. **Pay only for what you import.** 3D lives in its own entries, never in `.`
|
|
51
|
+
or `/player`; three.js is an optional peer dependency; nothing 3D is
|
|
52
|
+
downloaded into a bundle that does not import it (section 5, *Entries*).
|
|
53
|
+
|
|
54
|
+
### Non-goals
|
|
55
|
+
|
|
56
|
+
- A general-purpose 3D engine. No physics, no scene-graph editing at runtime, no
|
|
57
|
+
PBR pipeline in our own renderers (realistic mode gets PBR from three.js), no
|
|
58
|
+
general post-processing stack. Films do need a little, so the stylized
|
|
59
|
+
renderers get **blob / contact shadows** for characters by milestone 6, and
|
|
60
|
+
films get a **small deterministic post step** (supersampled anti-aliasing,
|
|
61
|
+
fog, vignette, motion blur by sub-frame sampling); shadow maps belong to
|
|
62
|
+
realistic mode.
|
|
63
|
+
- A hard dependency on three.js, Babylon or any 3D library in the engine, the
|
|
64
|
+
player, or the default adapters bundle. three.js is required only for
|
|
65
|
+
realistic mode, as an optional peer dependency.
|
|
66
|
+
- Full glTF coverage. Compressed geometry (Draco, meshopt), KHR material
|
|
67
|
+
extensions, cameras-from-file and multiple scenes per file are out of scope at
|
|
68
|
+
first.
|
|
69
|
+
- 3D in exported Lottie files. Lottie's 3D layer support is too thin to target.
|
|
70
|
+
- Bit-identical GPU output across machines. Determinism is guaranteed for the
|
|
71
|
+
*resolved scene* and for the CPU renderer; GPU renderers are deterministic in
|
|
72
|
+
input, not in pixels.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## 2. What "3D" means in tinyfly
|
|
77
|
+
|
|
78
|
+
There are four distinct things people mean by "3D". They have very different
|
|
79
|
+
costs, so the plan treats them separately.
|
|
80
|
+
|
|
81
|
+
### 2a. 3D transforms of existing 2D targets (CSS-style 3D)
|
|
82
|
+
|
|
83
|
+
A card flips, a carousel turns, a layer tilts. The targets stay flat; only their
|
|
84
|
+
transform is 3D.
|
|
85
|
+
|
|
86
|
+
**What exists today:**
|
|
87
|
+
|
|
88
|
+
| Adapter | 3D support today |
|
|
89
|
+
|---|---|
|
|
90
|
+
| DOM (`src/adapters/dom/dom-adapter.ts`) | `z`, `rotateX`, `rotateY`, `rotateZ`, `scaleZ` become `translateZ()`, `rotateX()` and so on; `perspective` is emitted first as a `perspective()` transform function on the element itself. |
|
|
91
|
+
| SVG (`src/adapters/svg/svg-adapter.ts`) | `rotateX`, `rotateY` and `perspective` composed into a CSS transform on the SVG element. |
|
|
92
|
+
| Canvas (`src/adapters/canvas/canvas-adapter.ts`) | An approximation: `rotateX`/`rotateY` scale the target by `cos(angle)` vertically/horizontally. No foreshortening, no `z`, no `perspective`. |
|
|
93
|
+
| WebGL (`src/adapters/webgl/webgl-adapter.ts`) | 2D only: a `mat3` per quad; `rotateX`/`rotateY`/`z` are ignored. |
|
|
94
|
+
|
|
95
|
+
Gallery precedent: `src/examples/live-demos/card-flip-3d.js` already gets a
|
|
96
|
+
convincing 3D flip by setting `perspective`, `transform-style: preserve-3d` and
|
|
97
|
+
`backface-visibility` in hand-written CSS and animating `rotateY`.
|
|
98
|
+
|
|
99
|
+
**What is missing:** a parent's `perspective` (as opposed to the
|
|
100
|
+
`perspective()` function on the element), `perspective-origin`,
|
|
101
|
+
`transform-style: preserve-3d`, `backface-visibility`, a documented transform
|
|
102
|
+
order, and quaternion rotations (`matrix3d`) for tumbling without gimbal lock.
|
|
103
|
+
These are small, adapter-only changes (milestone 1).
|
|
104
|
+
|
|
105
|
+
### 2b. A 3D scene adapter: cameras, lights, meshes
|
|
106
|
+
|
|
107
|
+
A real scene: primitives and models in world space, viewed through a perspective
|
|
108
|
+
or orthographic camera, lit by ambient, directional and point lights. This is
|
|
109
|
+
the bulk of the work (milestones 2–4).
|
|
110
|
+
|
|
111
|
+
### 2c. 3D characters
|
|
112
|
+
|
|
113
|
+
Character v2's skeleton is "2.5D": bones are solved in 3D in the character's own
|
|
114
|
+
space (`joints3`), then turned by `turn × 90°` and projected orthographically in
|
|
115
|
+
`project()` in `src/characters/rig/skeleton.ts`. Lifting it into a 3D scene
|
|
116
|
+
means:
|
|
117
|
+
|
|
118
|
+
- placing the character's space in the world with a transform (so `turn` becomes
|
|
119
|
+
a yaw on the object, not a special view rule);
|
|
120
|
+
- letting the scene camera do the projection (perspective, any angle, including
|
|
121
|
+
from above);
|
|
122
|
+
- drawing it either with its existing pens (the clean/pencil/silhouette look,
|
|
123
|
+
under a real camera) or as simple 3D shapes (capsule limbs, ellipsoid head).
|
|
124
|
+
|
|
125
|
+
### 2d. Importing glTF animations
|
|
126
|
+
|
|
127
|
+
glTF 2.0 is the standard interchange format. Its animations are already
|
|
128
|
+
keyframes: each channel targets a node's `translation`, `rotation`
|
|
129
|
+
(quaternion), `scale`, or morph `weights`, with `LINEAR`, `STEP` or
|
|
130
|
+
`CUBICSPLINE` sampling. That maps almost one-to-one onto tinyfly tracks. Import
|
|
131
|
+
is a pure function (JSON + binary buffers in, scene + tracks out), so it runs in
|
|
132
|
+
Node and in Workers.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## 3. Renderer options
|
|
137
|
+
|
|
138
|
+
### Candidates
|
|
139
|
+
|
|
140
|
+
| Option | Size (min+gz, ballpark) | Worker / OffscreenCanvas | Node / headless video | Fit with tinyfly |
|
|
141
|
+
|---|---|---|---|---|
|
|
142
|
+
| **Canvas 2D projected renderer** (ours) | ~5–8 KB | Yes (OffscreenCanvas 2D) | **Yes**, through `@napi-rs/canvas`, the dependency headless already uses | Flat or toon-shaded low-poly, painter's algorithm. Deterministic to the pixel on a given canvas backend. Can draw characters with the existing pens. Slow past a few thousand triangles; intersecting meshes sort wrongly. |
|
|
143
|
+
| **WebGL2** (ours, raw) | ~8–15 KB | Yes (`OffscreenCanvas.getContext('webgl2')`) | No, without a browser (headless-gl is WebGL1-only and a native build) | Depth buffer, per-pixel lighting, skinning in the vertex shader, tens of thousands of triangles. Universally available in browsers. |
|
|
144
|
+
| **WebGPU** (ours, raw) | ~10–20 KB | Yes | Only with Dawn bindings (native, heavy) | Best long-term performance; still not available on every browser/OS combination we must support, and adds a second shader language (WGSL). Later, behind the same interface. |
|
|
145
|
+
| **three.js** (optional adapter) | ~150–180 KB core | Yes | Only in a headless browser | PBR, shadows, glTF loader, huge ecosystem. Too large to justify inside tinyfly; fine as an *optional peer dependency* for users who already ship it. |
|
|
146
|
+
| **OGL** | ~10–30 KB | Yes | No | Small and clean, but a dependency for what our own ~15 KB WebGL2 renderer would do, and it would still need our resolved-scene mapping. |
|
|
147
|
+
| **Babylon.js** | several hundred KB+ | Yes | Only in a headless browser | Full engine with its own scene graph, animation system and clock. Conflicts with "the engine owns time". Not recommended even as an adapter. |
|
|
148
|
+
| **regl / twgl** | ~10–25 KB | Yes | No | Thin WebGL helpers. They save little over writing the 300–500 lines ourselves. |
|
|
149
|
+
|
|
150
|
+
Sizes are order-of-magnitude figures; measure the actual build before any
|
|
151
|
+
decision that depends on them.
|
|
152
|
+
|
|
153
|
+
### Dependency justification (per CLAUDE.md)
|
|
154
|
+
|
|
155
|
+
- **Our own renderers (Canvas 2D, WebGL2):** no dependency. A minimal renderer
|
|
156
|
+
for primitives, flat/Lambert/toon shading and a depth buffer is a few hundred
|
|
157
|
+
lines, readable, and testable through pure functions, like the existing
|
|
158
|
+
`quadMatrix` tests for the WebGL adapter.
|
|
159
|
+
- **three.js:** justified only as an *opt-in adapter* in its own entry point
|
|
160
|
+
(`@algorisys/tinyfly/three`), with `three` as an optional peer dependency,
|
|
161
|
+
externalised in the build exactly as the frameworks entries externalise React
|
|
162
|
+
and Vue. It never appears in the engine, player, or default adapters bundle.
|
|
163
|
+
Its value: users who already have a three.js scene can drive it from tinyfly
|
|
164
|
+
timelines, and get PBR/shadows for free.
|
|
165
|
+
- **Nothing else** clears the bar.
|
|
166
|
+
|
|
167
|
+
### Recommendation
|
|
168
|
+
|
|
169
|
+
1. **Canvas 2D projected renderer first.** It is the only option that works in
|
|
170
|
+
the browser, in a Worker and in Node video export with zero new dependencies,
|
|
171
|
+
gives pixel-stable golden tests, and can render characters in their pencil
|
|
172
|
+
look under a real camera, which is tinyfly's distinctive output.
|
|
173
|
+
2. **WebGL2 renderer second**, consuming the exact same resolved scene, for
|
|
174
|
+
performance, a real depth buffer and skinning.
|
|
175
|
+
3. **three.js adapter** once the data model is stable: optional for package
|
|
176
|
+
users, but it *is* realistic mode, so films depend on it (milestone 8).
|
|
177
|
+
4. **WebGPU** only when there is a measured need and it is available
|
|
178
|
+
everywhere we play; the renderer interface keeps that door open.
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## 4. Data model
|
|
183
|
+
|
|
184
|
+
### Coordinate conventions
|
|
185
|
+
|
|
186
|
+
- World space is **right-handed, +y up, +z toward the default camera**, in
|
|
187
|
+
**metres** (as glTF, and so that realistic mode's lights and cameras behave
|
|
188
|
+
physically). This matches glTF, WebGL and
|
|
189
|
+
three.js, *and* character v2's own space (+x the character's left, +y up, +z
|
|
190
|
+
the way it faces), so neither import nor characters need axis flips.
|
|
191
|
+
- This differs from the 2D stage (y down, pixels). The two never mix inside one
|
|
192
|
+
scene; a 3D scene element is a box on the 2D stage, like the map and character
|
|
193
|
+
elements.
|
|
194
|
+
- Angles in authored JSON are **degrees** (as everywhere else in tinyfly);
|
|
195
|
+
quaternions are `[x, y, z, w]` (glTF order).
|
|
196
|
+
- Euler rotation order is fixed at **`YXZ`** (yaw, then pitch, then roll), the
|
|
197
|
+
order that reads naturally for cameras and characters. It is documented, not
|
|
198
|
+
configurable, at first.
|
|
199
|
+
|
|
200
|
+
### Scene JSON
|
|
201
|
+
|
|
202
|
+
```ts
|
|
203
|
+
interface Scene3D {
|
|
204
|
+
formatVersion: number
|
|
205
|
+
id: string
|
|
206
|
+
/** Id of the camera at time 0; a film cuts between cameras with an `activeCamera` track */
|
|
207
|
+
camera: string
|
|
208
|
+
/** Ordered: array order is draw/tie-break order and child order */
|
|
209
|
+
objects: Object3D[]
|
|
210
|
+
materials?: Record<string, Material3D>
|
|
211
|
+
/** Background colour, or 'transparent' */
|
|
212
|
+
background?: string
|
|
213
|
+
/** Realistic mode: an equirectangular HDR asset for lighting and reflections */
|
|
214
|
+
environment?: string
|
|
215
|
+
/** Fog from `near` to `far` metres, both modes */
|
|
216
|
+
fog?: { color: string; near: number; far: number }
|
|
217
|
+
/** Binary assets (glTF buffers, textures) by id; data URIs or URLs, loaded before rendering */
|
|
218
|
+
assets?: Record<string, { uri: string; mimeType?: string }>
|
|
219
|
+
}
|
|
220
|
+
|
|
221
|
+
interface Transform3D {
|
|
222
|
+
position?: [number, number, number] // default [0, 0, 0]
|
|
223
|
+
/** Euler degrees, YXZ order; or a quaternion via `quaternion` */
|
|
224
|
+
rotation?: [number, number, number]
|
|
225
|
+
quaternion?: [number, number, number, number]
|
|
226
|
+
scale?: [number, number, number] | number // default 1
|
|
227
|
+
}
|
|
228
|
+
|
|
229
|
+
interface ObjectBase extends Transform3D {
|
|
230
|
+
id: string // track target id within the scene
|
|
231
|
+
name?: string
|
|
232
|
+
parent?: string // id of an earlier object
|
|
233
|
+
visible?: boolean
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
type Object3D =
|
|
237
|
+
| (ObjectBase & { kind: 'group' })
|
|
238
|
+
| (ObjectBase & { kind: 'mesh'; geometry: Geometry3D; material: string })
|
|
239
|
+
| (ObjectBase & { kind: 'camera'; projection: 'perspective'; fov: number; near: number; far: number })
|
|
240
|
+
| (ObjectBase & { kind: 'camera'; projection: 'orthographic'; height: number; near: number; far: number })
|
|
241
|
+
| (ObjectBase & { kind: 'light'; light: 'ambient' | 'directional' | 'point' | 'spot'; color: string
|
|
242
|
+
/** Physical units: lux for directional, candela for point and spot; stylized renderers normalise them */
|
|
243
|
+
intensity: number; range?: number; angle?: number; castShadow?: boolean })
|
|
244
|
+
| (ObjectBase & { kind: 'model'; asset: string; node?: string }) // a glTF (sub)tree
|
|
245
|
+
| (ObjectBase & { kind: 'character'; character: CharacterSpec; look?: 'pen' | 'solid'; model?: string }) // `model`: a skinned glTF for realistic mode
|
|
246
|
+
|
|
247
|
+
type Geometry3D =
|
|
248
|
+
| { type: 'box'; size: [number, number, number] }
|
|
249
|
+
| { type: 'sphere'; radius: number; segments?: number }
|
|
250
|
+
| { type: 'cylinder'; radius: number; height: number; segments?: number }
|
|
251
|
+
| { type: 'cone'; radius: number; height: number; segments?: number }
|
|
252
|
+
| { type: 'plane'; size: [number, number] }
|
|
253
|
+
| { type: 'torus'; radius: number; tube: number; segments?: number }
|
|
254
|
+
| { type: 'extrude'; path: string; depth: number } // an SVG path, extruded (logos, text outlines)
|
|
255
|
+
|
|
256
|
+
interface Material3D {
|
|
257
|
+
// Both modes
|
|
258
|
+
color: string
|
|
259
|
+
opacity?: number
|
|
260
|
+
texture?: string // asset id
|
|
261
|
+
doubleSided?: boolean
|
|
262
|
+
// Stylized mode (our renderers); realistic mode ignores these
|
|
263
|
+
shading: 'unlit' | 'flat' | 'lambert' | 'toon'
|
|
264
|
+
/** Toon only: number of light bands */
|
|
265
|
+
bands?: number
|
|
266
|
+
/** Ink outline, in px (Canvas 2D and WebGL renderers both support it) */
|
|
267
|
+
outline?: { width: number; color: string }
|
|
268
|
+
// Realistic mode (three.js, glTF's metallic-roughness); stylized mode ignores these
|
|
269
|
+
roughness?: number // 0..1, default 0.6
|
|
270
|
+
metalness?: number // 0..1, default 0
|
|
271
|
+
emissive?: string
|
|
272
|
+
normalMap?: string // asset id
|
|
273
|
+
}
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
A material is written once and looks right in both modes: stylized reads
|
|
277
|
+
`shading`, `bands` and `outline`; realistic reads `roughness`, `metalness`,
|
|
278
|
+
`emissive` and `normalMap`; both read `color`, `opacity` and `texture`. Neither
|
|
279
|
+
fails on the other's fields.
|
|
280
|
+
|
|
281
|
+
The `extrude` geometry reuses the engine's existing path parsing
|
|
282
|
+
(`src/engine/path/`), so any 2D shape in tinyfly can become a 3D object.
|
|
283
|
+
|
|
284
|
+
### Tracks: how 3D values animate
|
|
285
|
+
|
|
286
|
+
Tracks address scene objects by **target = `<sceneElementId>/<objectId>`** (for
|
|
287
|
+
example `stage3d/cube`), so a 3D scene's tracks can live in the same timeline as
|
|
288
|
+
2D elements without id collisions.
|
|
289
|
+
|
|
290
|
+
| Property | Value | Interpolation |
|
|
291
|
+
|---|---|---|
|
|
292
|
+
| `x`, `y`, `z` | number | linear (existing) |
|
|
293
|
+
| `rotateX`, `rotateY`, `rotateZ` | number, degrees | linear (existing); applied in YXZ order |
|
|
294
|
+
| `scale`, `scaleX`, `scaleY`, `scaleZ` | number | linear (existing) |
|
|
295
|
+
| `position`, `scale3` | `number[3]` | element-wise (existing `interpolateArray`) |
|
|
296
|
+
| `quaternion` | `number[4]` | **slerp** (new) |
|
|
297
|
+
| `color`, `material.color`, `light.color` | colour string | existing `interpolateColor` |
|
|
298
|
+
| `opacity`, `light.intensity`, `camera.fov`, `camera.height` | number | linear |
|
|
299
|
+
| `lookAt` | `number[3]` (a camera's target point) | element-wise |
|
|
300
|
+
| `weights` | `number[]` (morph targets) | element-wise |
|
|
301
|
+
| `visible` | number 0/1 | stepped |
|
|
302
|
+
| `activeCamera` (on the scene) | camera object id | stepped: a cut is one keyframe |
|
|
303
|
+
| `pose.*`, `turn`, `gait.*` (character objects) | number | as character v2 today |
|
|
304
|
+
|
|
305
|
+
Reusing the existing names (`x`, `rotateY`, …) means the editor's property
|
|
306
|
+
fields, curve editor, springs and staggers work on 3D objects unchanged.
|
|
307
|
+
|
|
308
|
+
**Validation rule (no hidden precedence):** an object may be animated by
|
|
309
|
+
component tracks (`x`/`y`/`z`, `rotateX…`) *or* by vector tracks
|
|
310
|
+
(`position`, `quaternion`) for the same channel, never both. The scene validator
|
|
311
|
+
reports the conflict instead of picking a winner.
|
|
312
|
+
|
|
313
|
+
### The one engine change: explicit slerp
|
|
314
|
+
|
|
315
|
+
Today `getInterpolator()` in `src/engine/interpolation/interpolators.ts` picks
|
|
316
|
+
an interpolator from the *shape* of the value. A 4-number array would therefore
|
|
317
|
+
be interpolated element-wise, which is wrong for a quaternion (it shrinks and
|
|
318
|
+
takes the wrong path). Guessing "4 numbers means quaternion" would be hidden
|
|
319
|
+
magic, so the choice becomes explicit data:
|
|
320
|
+
|
|
321
|
+
```ts
|
|
322
|
+
interface Track {
|
|
323
|
+
// ...existing fields
|
|
324
|
+
/** How keyframe values blend. Default: chosen from the value, as today. */
|
|
325
|
+
interpolation?: 'slerp'
|
|
326
|
+
}
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
`interpolateQuaternion(from, to, t)` normalises, takes the shortest arc (flips
|
|
330
|
+
`to` when the dot product is negative), falls back to normalised lerp when the
|
|
331
|
+
quaternions are nearly equal, and returns a unit quaternion. Easing still shapes
|
|
332
|
+
`t`, so every existing ease works on rotations.
|
|
333
|
+
|
|
334
|
+
Serialisation: a file that uses `interpolation` is written with
|
|
335
|
+
`formatVersion: 2`. `FORMAT_VERSION`'s existing rule ("a reader refuses files
|
|
336
|
+
from a newer format rather than playing them wrongly") then protects old players
|
|
337
|
+
from silently lerping quaternions. Files that don't use it stay at version 1.
|
|
338
|
+
|
|
339
|
+
Springs and inertia remain numeric: they work on `rotateX` and friends, not on
|
|
340
|
+
`quaternion` tracks (the validator says so).
|
|
341
|
+
|
|
342
|
+
### Deterministic ordering and evaluation
|
|
343
|
+
|
|
344
|
+
- `objects` is an **array**, never a map; parents must precede children. World
|
|
345
|
+
matrices are computed in array order in one pass.
|
|
346
|
+
- The resolver produces a **draw list** sorted by (opaque before transparent,
|
|
347
|
+
then depth, then object index, then triangle index). `Array.prototype.sort` is
|
|
348
|
+
stable, and the index tie-breakers make the order total, so equal depths never
|
|
349
|
+
flicker between runs.
|
|
350
|
+
- All math is in float64 (`number`) on the CPU; matrices are only converted to
|
|
351
|
+
`Float32Array` at the GPU boundary.
|
|
352
|
+
- No asset loads during `render()`. Assets are loaded (async) up front by
|
|
353
|
+
`loadScene3D()`; rendering a frame is synchronous and pure, so frame N never
|
|
354
|
+
depends on whether frame N−1 was drawn.
|
|
355
|
+
- `Math.sin`/`Math.cos` can differ in the last bit between JavaScript engines.
|
|
356
|
+
Golden tests compare on one engine (Node in CI) and use a 1e-9 tolerance for
|
|
357
|
+
matrices in unit tests.
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
## 5. Architecture and module layout
|
|
362
|
+
|
|
363
|
+
### The pipeline
|
|
364
|
+
|
|
365
|
+
```
|
|
366
|
+
TimelineDefinition ──► engine (unchanged + slerp) ──► AnimationState
|
|
367
|
+
│
|
|
368
|
+
Scene3D (JSON) ─────────────────────────────┐ │
|
|
369
|
+
▼ ▼
|
|
370
|
+
resolveScene3D(scene, state, size) pure, no GPU, no DOM
|
|
371
|
+
│
|
|
372
|
+
▼
|
|
373
|
+
ResolvedScene3D (plain data:
|
|
374
|
+
world matrices, camera matrices,
|
|
375
|
+
lights, sorted draw list)
|
|
376
|
+
│
|
|
377
|
+
┌──────────────────────┼───────────────────────┐
|
|
378
|
+
▼ ▼ ▼
|
|
379
|
+
canvas-2d renderer webgl2 renderer three adapter (optional)
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
The resolver is the 3D equivalent of what adapters already do with
|
|
383
|
+
`AnimationState`, split out so that every renderer shares one implementation of
|
|
384
|
+
transforms, cameras and ordering, and so that it can be unit-tested without any
|
|
385
|
+
rendering at all.
|
|
386
|
+
|
|
387
|
+
### Files
|
|
388
|
+
|
|
389
|
+
All new code; file and folder names lowercase-hyphenated.
|
|
390
|
+
|
|
391
|
+
```
|
|
392
|
+
src/engine/math/ pure helpers, exported from the engine
|
|
393
|
+
vec3.ts add, sub, scale, dot, cross, normalize, lerp
|
|
394
|
+
quat.ts fromEuler (YXZ), toEuler, multiply, slerp, rotateVec3
|
|
395
|
+
mat4.ts identity, multiply, compose(TRS), invert, perspective, orthographic, lookAt
|
|
396
|
+
index.ts
|
|
397
|
+
src/engine/interpolation/
|
|
398
|
+
interpolators.ts + interpolateQuaternion, chosen by Track.interpolation
|
|
399
|
+
|
|
400
|
+
src/scene-3d/ entry: @algorisys/tinyfly/scene-3d (no DOM, no GPU, no glTF)
|
|
401
|
+
scene-types.ts Scene3D, Object3D, Geometry3D, Material3D
|
|
402
|
+
validate-scene.ts ids, parent order, channel conflicts, slerp-only-on-quaternion
|
|
403
|
+
resolve-scene.ts Scene3D + AnimationState → ResolvedScene3D
|
|
404
|
+
geometry/
|
|
405
|
+
primitives.ts box, sphere, cylinder, cone, plane, torus → indexed triangles
|
|
406
|
+
extrude.ts SVG path → extruded mesh (uses engine/path)
|
|
407
|
+
camera.ts view/projection matrices, orbit and dolly helpers (pure)
|
|
408
|
+
shading.ts flat / lambert / toon colour for a normal + lights (pure)
|
|
409
|
+
draw-list.ts depth sorting with total-order tie-breaks
|
|
410
|
+
load-scene.ts async asset loading, returns a ready scene
|
|
411
|
+
characters/
|
|
412
|
+
character-object.ts character v2 skeleton placed in world space
|
|
413
|
+
scene-3d-adapter.ts registerScene / applyState / render / destroy
|
|
414
|
+
canvas-2d-renderer.ts painter's algorithm, outlines, pens, blob shadows (stylized)
|
|
415
|
+
index.ts
|
|
416
|
+
webgl/ entry: @algorisys/tinyfly/scene-3d/webgl
|
|
417
|
+
webgl2-renderer.ts depth buffer, lambert/toon shaders, skinning (stylized)
|
|
418
|
+
index.ts
|
|
419
|
+
gltf/ entry: @algorisys/tinyfly/scene-3d/gltf
|
|
420
|
+
glb-container.ts GLB header and chunks → JSON + binary
|
|
421
|
+
gltf-accessors.ts typed views over buffers
|
|
422
|
+
gltf-to-scene.ts nodes, meshes, materials → Scene3D objects
|
|
423
|
+
gltf-to-tracks.ts animation channels → tinyfly tracks
|
|
424
|
+
index.ts
|
|
425
|
+
|
|
426
|
+
src/adapters/three/ entry: @algorisys/tinyfly/three (three = optional peer)
|
|
427
|
+
three-adapter.ts Scene3D + AnimationState → three.js objects (realistic)
|
|
428
|
+
index.ts
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
### Entries: pay only for what you import
|
|
432
|
+
|
|
433
|
+
| Entry | Contains | Pulls in |
|
|
434
|
+
|---|---|---|
|
|
435
|
+
| `.`, `/player` | the engine, plus the math helpers and slerp | nothing 3D |
|
|
436
|
+
| `/scene-3d` | types, validator, resolver, primitives, Canvas 2D renderer | the engine |
|
|
437
|
+
| `/scene-3d/webgl` | the WebGL2 renderer | `/scene-3d` |
|
|
438
|
+
| `/scene-3d/gltf` | the glTF loader | `/scene-3d` |
|
|
439
|
+
| `/three` | realistic mode | `/scene-3d`, and `three` (optional peer, externalised) |
|
|
440
|
+
|
|
441
|
+
Each gets its own entry in `vite.config.addons.ts` (as `characters` and `maps`
|
|
442
|
+
have), and `package.json` keeps `sideEffects` limited to CSS, so bundlers drop
|
|
443
|
+
what is not imported. A test checks the built `lib/` files: each entry has a size
|
|
444
|
+
budget, and `.`, `/player` and `/scene-3d` contain no WebGL, glTF or three code.
|
|
445
|
+
`npm install` still downloads the whole package (812 kB at v0.82.0); if 3D makes
|
|
446
|
+
that noticeably larger, the renderers and loaders move to a separate
|
|
447
|
+
`@algorisys/tinyfly-3d` package rather than grow every install.
|
|
448
|
+
|
|
449
|
+
### The adapter interface
|
|
450
|
+
|
|
451
|
+
The existing adapters share a shape by convention (`registerTarget`,
|
|
452
|
+
`applyState`, `render`, `destroy`) rather than a declared interface. The 3D
|
|
453
|
+
adapter follows the same shape, and adds an explicit renderer interface so
|
|
454
|
+
renderers are swappable:
|
|
455
|
+
|
|
456
|
+
```ts
|
|
457
|
+
interface Renderer3D {
|
|
458
|
+
/** Draw one resolved frame. Synchronous; never loads anything. */
|
|
459
|
+
render(frame: ResolvedScene3D): void
|
|
460
|
+
resize(width: number, height: number, pixelRatio: number): void
|
|
461
|
+
destroy(): void
|
|
462
|
+
}
|
|
463
|
+
|
|
464
|
+
class Scene3DAdapter {
|
|
465
|
+
constructor(renderer: Renderer3D, options: { width: number; height: number })
|
|
466
|
+
registerScene(id: string, scene: LoadedScene3D): void
|
|
467
|
+
unregisterScene(id: string): void
|
|
468
|
+
applyState(state: AnimationState): void // resolves every registered scene
|
|
469
|
+
render(): void // hands each resolved frame to the renderer
|
|
470
|
+
destroy(): void
|
|
471
|
+
}
|
|
472
|
+
|
|
473
|
+
// Usage, no editor:
|
|
474
|
+
const scene = await loadScene3D(sceneJson)
|
|
475
|
+
const adapter = new Scene3DAdapter(new Canvas2DRenderer(ctx), { width: 800, height: 450 })
|
|
476
|
+
adapter.registerScene('stage3d', scene)
|
|
477
|
+
timeline.onUpdate = (state) => { adapter.applyState(state); adapter.render() }
|
|
478
|
+
```
|
|
479
|
+
|
|
480
|
+
The Canvas 2D renderer also exposes `drawScene3D(ctx, scene, state, size)`, so a
|
|
481
|
+
scene can be a `custom` target inside the existing `CanvasAdapter` and
|
|
482
|
+
therefore inside a `VideoScene`, with no change to `src/headless/`.
|
|
483
|
+
|
|
484
|
+
### Workers
|
|
485
|
+
|
|
486
|
+
Everything up to `ResolvedScene3D` is plain data, so a Worker can own the
|
|
487
|
+
timeline and an `OffscreenCanvas` and run either renderer entirely off the main
|
|
488
|
+
thread. Nothing in the engine or the timeline needs 3D-specific code for this.
|
|
489
|
+
|
|
490
|
+
### Headless
|
|
491
|
+
|
|
492
|
+
`FrameRenderer` draws on a 2D context. A 3D scene enters it as a custom canvas
|
|
493
|
+
target drawn by the Canvas 2D renderer, so `tinyfly video scene.mjs` renders 3D
|
|
494
|
+
to MP4 with today's dependencies: this is stylized mode, and the reference for
|
|
495
|
+
golden tests. WebGL2 is not available in Node, so heavy stylized scenes and
|
|
496
|
+
every realistic render go through headless Chromium (Playwright), driven by the
|
|
497
|
+
same scene module, frame by frame. Playwright is never a dependency: the CLI
|
|
498
|
+
asks for it when a film needs it. Realistic frames are deterministic in input,
|
|
499
|
+
not in pixels, and are checked against the stylized render of the same frame by
|
|
500
|
+
tolerance.
|
|
501
|
+
|
|
502
|
+
### The editor (SolidJS)
|
|
503
|
+
|
|
504
|
+
The editor stays a consumer of the public API; no 3D logic in components.
|
|
505
|
+
|
|
506
|
+
| Piece | Where (proposed) |
|
|
507
|
+
|---|---|
|
|
508
|
+
| `scene3d` element type: a box on the stage hosting one `Scene3D` | `src/editor/stores/scene-store.ts` (alongside `character` and `map`) |
|
|
509
|
+
| Element → `Scene3D`, loading, adapter wiring | `src/editor/utils/scene-3d-element.ts` |
|
|
510
|
+
| Preview: a canvas inside the element, repainted from `AnimationState` | `src/editor/components/preview-panel.tsx` |
|
|
511
|
+
| Outliner: objects, cameras, lights; add primitive / light / camera; import glTF | `src/editor/components/scene-3d-panel.tsx` |
|
|
512
|
+
| Property fields: position, rotation (Euler degrees), scale, material, light, camera, each with ◆ keyframe via `keyValuesAtPlayhead` | `src/editor/components/property-panel.tsx` |
|
|
513
|
+
| Viewport navigation (orbit, pan, dolly) for *authoring only*; it changes an editor view camera, not the scene, unless "look through camera" is on | calls `orbit()` / `dolly()` from `src/scene-3d/camera.ts` |
|
|
514
|
+
| Export (GIF, MP4, HTML embed) | `src/editor/utils/scene-to-canvas.ts` draws the scene with the Canvas 2D renderer |
|
|
515
|
+
|
|
516
|
+
The editor shows rotations as Euler degrees (people can type them); the library
|
|
517
|
+
stores what the user keyed (component tracks), and quaternion tracks appear only
|
|
518
|
+
for imported glTF animation, shown read-only with "convert to Euler tracks" as
|
|
519
|
+
an action.
|
|
520
|
+
|
|
521
|
+
---
|
|
522
|
+
|
|
523
|
+
## 6. Milestones
|
|
524
|
+
|
|
525
|
+
Each is one minor release with unit tests, a docs page, and a gallery demo (a
|
|
526
|
+
live demo in `src/examples/live-demos/` and, where it makes sense, a headless
|
|
527
|
+
example in `examples/headless-video/`).
|
|
528
|
+
|
|
529
|
+
| # | Release | Contents | Tests | Demo |
|
|
530
|
+
|---|---|---|---|---|
|
|
531
|
+
| 0 | **Rotation math** | `src/engine/math/` (vec3, quat, mat4); `Track.interpolation: 'slerp'` and `interpolateQuaternion`; `formatVersion` 2 only when used; DOM adapter accepts a `quaternion` value and emits `matrix3d()` | slerp shortest path, unit length, endpoints exact, Euler↔quat round trip, JSON round trip, old readers refuse v2 | A CSS cube tumbling on a quaternion track next to the same motion on Euler tracks (gimbal lock visible on the Euler one) |
|
|
532
|
+
| 1 | **CSS 3D, finished** | DOM/SVG: `transformStyle`, `perspective` on parents, `perspectiveOrigin`, `backfaceVisibility`; documented transform order; editor fields for `z`, `rotateX`, `rotateY`, perspective; Canvas adapter: true perspective foreshortening for flat targets (projected corners, image drawn as two affine triangles); WebGL adapter: `mat4` per quad so `rotateX/Y/z` work | transform string order, projected-corner maths, canvas vs DOM corner positions agree within 0.5 px | Cover-flow carousel and a 3D card flip, both built in the editor, both exported to GIF |
|
|
533
|
+
| 2 | **Scenes on Canvas 2D** | `@algorisys/tinyfly/scene-3d`: types, validator, resolver, primitives, extrude, camera, lights, flat/lambert/toon shading, outlines; Canvas 2D renderer; `drawScene3D` custom target for headless video | resolver matrices, camera matrices, draw-order total and stable, validator errors, golden PNGs rendered twice identically in Node | Orbiting camera around lit primitives and an extruded tinyfly logo; the same scene rendered to MP4 with `tinyfly video` |
|
|
534
|
+
| 3 | **WebGL2 renderer** | Same `ResolvedScene3D` on the GPU: depth buffer, Lambert and toon shaders, outlines, textures; OffscreenCanvas in a Worker | pure parts (buffer packing, uniforms) as `webgl-adapter.test.ts` does; a browser smoke test comparing WebGL2 and Canvas 2D output within a tolerance | 2,000 animated cubes with a staggered wave (the case Canvas 2D can't hold at 60 fps) |
|
|
535
|
+
| 4 | **Editor 3D element** | `scene3d` element, outliner, add primitives/lights/cameras, transform and material fields with keyframes, authoring viewport, look-through-camera, export | store tests for add/remove/key; exported JSON round-trips through `loadScene3D` | A product-spin scene made entirely in the editor, exported as an embed |
|
|
536
|
+
| 5 | **glTF import (rigid)** | GLB/glTF parsing, meshes, base-colour materials and textures, node hierarchy, animation channels → tracks (`LINEAR` → linear/slerp, `STEP` → stepped, `CUBICSPLINE` → baked and simplified with the existing `bake.ts`/`simplifyKeyframes`), morph `weights`; editor "Import glTF" | accessor decoding, every sampler type, channels to tracks, tracks reproduce glTF sample values at keyframe times | Khronos sample models (CC0/CC-BY, e.g. *AnimatedCube*, *BoxAnimated*) playing on the tinyfly timeline, scrubbable |
|
|
537
|
+
| 6 | **3D characters** | `character` scene objects: character v2 skeleton (`joints3`) placed in world space; `turn` becomes root yaw; perspective projection through the scene camera; `look: 'pen'` (existing clean/pencil/silhouette pens on Canvas 2D) or `look: 'solid'` (capsule limbs, ellipsoid head, toon-shaded) | joints in world space match the 2.5D solver at `turn` 0–4 under an orthographic front camera (so v2 goldens hold); depth order under any camera | A pencil-look walk cycle with the camera craning from side view to top-down; the same scene in the solid look |
|
|
538
|
+
| 7 | **Skinned glTF** | Skins, joints, inverse bind matrices; GPU skinning in WebGL2, CPU skinning in Canvas 2D; optional mapping of a humanoid glTF skeleton to character v2 pose fields | skinning matrices against hand-computed cases, bind pose renders as rest mesh | A rigged CC0 character playing its imported clip, then blended into a tinyfly pose |
|
|
539
|
+
| 8 | **three.js adapter** (realistic mode) | `@algorisys/tinyfly/three`: builds three objects from `Scene3D`, applies `AnimationState` each frame; `three` as an optional peer, externalised | mapping tests with three in devDependencies only | The milestone 2 scene rendered with three.js PBR and shadows, driven by the identical timeline JSON |
|
|
540
|
+
|
|
541
|
+
Milestones 0–2 are the foundation. After that, 3, 5 and 6 can go in any order
|
|
542
|
+
(for the story channel, 6 before 3). Milestone 2 already supports several
|
|
543
|
+
cameras and the `activeCamera` track, and milestone 6 adds blob shadows, so the
|
|
544
|
+
film work (Phase 34) needs no format change. Films in realistic mode need
|
|
545
|
+
milestone 8, and realistic characters need 7 (or the `solid` look).
|
|
546
|
+
|
|
547
|
+
---
|
|
548
|
+
|
|
549
|
+
## 7. Risks
|
|
550
|
+
|
|
551
|
+
| Risk | Mitigation |
|
|
552
|
+
|---|---|
|
|
553
|
+
| **Scope creep**: 3D invites "just one more feature" until tinyfly is a game engine | The non-goals list is part of this plan; each milestone's scope is fixed before it starts |
|
|
554
|
+
| **Painter's algorithm artefacts** (intersecting or long overlapping triangles sort wrongly) | Document it; sort per triangle, not per object; keep scenes low-poly in the Canvas 2D renderer; WebGL2 has a depth buffer |
|
|
555
|
+
| **Canvas 2D performance** past a few thousand triangles | Back-face culling and frustum culling in the resolver; the WebGL2 renderer for heavy scenes; the docs give a triangle budget |
|
|
556
|
+
| **GPU output differs between machines** | Goldens only for the Canvas 2D renderer; GPU tests check pure parts and use tolerances |
|
|
557
|
+
| **Floating-point differences between JS engines** | Unit tests use tolerances; goldens are produced and checked in one runtime (Node in CI) |
|
|
558
|
+
| **Async assets vs deterministic frames** | Loading is separate from rendering; `render()` is synchronous and refuses scenes whose assets are not loaded |
|
|
559
|
+
| **glTF breadth** (extensions, compression, sparse accessors) | Support core glTF 2.0 and sparse accessors; report unsupported extensions by name instead of rendering wrongly |
|
|
560
|
+
| **Two coordinate systems** (2D y-down px, 3D y-up units) confusing users | They never mix inside a scene; the 3D element is a box on the 2D stage; the docs open with one diagram of both |
|
|
561
|
+
| **Euler order surprises** | One fixed, documented order (YXZ); quaternion tracks when users need tumbling |
|
|
562
|
+
| **Editor UX for 3D** (gizmos, selection in depth) is a large design problem | Milestone 4 ships numeric fields and an orbit viewport first; on-canvas gizmos are a later, separate decision |
|
|
563
|
+
| **Bundle growth** | `scene-3d` and `three` are separate entries; the engine gains only math helpers and slerp, measured in the release notes |
|
|
564
|
+
|
|
565
|
+
---
|
|
566
|
+
|
|
567
|
+
## 8. Open questions
|
|
568
|
+
|
|
569
|
+
1. **Entry name:** `@algorisys/tinyfly/scene-3d` (proposed) or `/3d`?
|
|
570
|
+
2. **Target addressing:** `<sceneElementId>/<objectId>` (proposed), or one
|
|
571
|
+
target per scene with properties like `cube.x`? The first keeps every
|
|
572
|
+
existing property name and editor field unchanged.
|
|
573
|
+
3. **Euler order:** is YXZ right for all users, or should `rotationOrder` be a
|
|
574
|
+
per-object option from the start?
|
|
575
|
+
4. ~~**Units:**~~ Decided: metres, and physical light intensities (realistic
|
|
576
|
+
mode needs them; stylized renderers normalise).
|
|
577
|
+
5. **WebGL1 adapter:** upgrade the existing quad adapter to WebGL2 in milestone
|
|
578
|
+
1, or keep it WebGL1 for the widest reach and keep WebGL2 for scenes only?
|
|
579
|
+
6. ~~**Headless WebGL:**~~ Decided: yes. Films need it for realistic mode and
|
|
580
|
+
for heavy stylized scenes; Canvas 2D stays the reference renderer.
|
|
581
|
+
7. **Characters:** in a 3D scene, should `turn` stay animatable on a character
|
|
582
|
+
object (as yaw), or be replaced by `rotateY` to avoid two ways of doing the
|
|
583
|
+
same thing?
|
|
584
|
+
8. ~~**Shadows:**~~ Decided: blob shadows for characters in milestone 6
|
|
585
|
+
(stylized), projected ground shadows next; shadow maps in realistic mode.
|
|
586
|
+
9. **Sample assets:** which CC0 glTF models are acceptable to vendor into the
|
|
587
|
+
repo for tests and the gallery (size, licence)?
|
|
588
|
+
10. ~~**Render mode per shot:**~~ Decided: a film sets `renderMode`, and any
|
|
589
|
+
shot may override it. The resolver is shared, so mixing costs nothing; a
|
|
590
|
+
film can be blocked out in stylized and switched shot by shot.
|
|
591
|
+
11. ~~**Realistic characters:**~~ Decided: the `solid` look with PBR shading
|
|
592
|
+
first (no wait on milestone 7); skinned glTF characters (`model`) replace it
|
|
593
|
+
when milestone 7 lands. Both are driven by the same v2 pose tracks.
|
|
594
|
+
|
|
595
|
+
---
|
|
596
|
+
|
|
597
|
+
## 9. Movies and the two render modes
|
|
598
|
+
|
|
599
|
+
A film is a timeline like any other. Nothing below needs a new clock or a new
|
|
600
|
+
file format beyond what milestones 0–2 define.
|
|
601
|
+
|
|
602
|
+
- **Shots and cuts.** Shots are time ranges with a camera and a label; a cut is
|
|
603
|
+
one keyframe on the stepped `activeCamera` track. Transitions (crossfade, dip
|
|
604
|
+
to black, wipe) are pure functions of two frames and a progress, so any frame
|
|
605
|
+
of a film can be rendered on its own.
|
|
606
|
+
- **Cameras.** Moves are presets that write ordinary tracks: orbit, dolly,
|
|
607
|
+
crane, pan and tilt, follow, `lookAt`, focal-length zoom, and a seeded
|
|
608
|
+
handheld shake (deterministic).
|
|
609
|
+
- **Render modes.** `renderMode: 'stylized' | 'realistic'` is set for a film and
|
|
610
|
+
can be overridden per shot. The resolver is shared, so both modes see the same
|
|
611
|
+
world matrices, cameras and cuts; only the renderer differs.
|
|
612
|
+
|
|
613
|
+
| | Stylized | Realistic |
|
|
614
|
+
|---|---|---|
|
|
615
|
+
| Renderer | Canvas 2D (reference), WebGL2 | three.js |
|
|
616
|
+
| Look | flat, toon, pencil; outlines | PBR (roughness, metalness), environment map, tone mapping |
|
|
617
|
+
| Shadows | blob, then projected ground shadows | shadow maps |
|
|
618
|
+
| Characters | pen look, or `solid` toon | `solid` with PBR first; a skinned glTF (`model`) after milestone 7 |
|
|
619
|
+
| Headless | Node, pixel-identical between runs | headless Chromium, checked by tolerance |
|
|
620
|
+
|
|
621
|
+
- **Post step (both modes, small and deterministic).** Supersampled
|
|
622
|
+
anti-aliasing, fog, vignette, and motion blur by averaging sub-frame samples
|
|
623
|
+
at fixed offsets.
|
|
624
|
+
- **Long renders.** Frames are independent, so a film renders across worker
|
|
625
|
+
threads, in chunks, resumably, with a per-shot frame cache.
|
|
626
|
+
- **Sound, captions, narration.** Reused from video-from-code (Phase 31) and the
|
|
627
|
+
beat grid (Phase 32); see Phase 34 in `todo.md` for the full film plan.
|
|
628
|
+
|