@algorisys/tinyfly 0.92.0 → 0.93.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (61) hide show
  1. package/README.md +7 -6
  2. package/bin/tinyfly.mjs +34 -1
  3. package/docs/2d-animation-roadmap.md +191 -0
  4. package/docs/3d-rotations.md +164 -0
  5. package/docs/3d-support-plan.md +628 -0
  6. package/docs/DEPLOYMENT.md +414 -0
  7. package/docs/acting.md +466 -0
  8. package/docs/api-reference.md +2353 -0
  9. package/docs/camera.md +94 -0
  10. package/docs/cartoon-hands.md +96 -0
  11. package/docs/character-system-m1.md +218 -0
  12. package/docs/character-system-plan.md +384 -0
  13. package/docs/dance.md +315 -0
  14. package/docs/editor-guide.md +1249 -0
  15. package/docs/examples.md +765 -0
  16. package/docs/extending.md +168 -0
  17. package/docs/file-format.md +690 -0
  18. package/docs/getting-started.md +314 -0
  19. package/docs/grid-and-snapping.md +64 -0
  20. package/docs/gsap-compat.md +1241 -0
  21. package/docs/llm-guide.md +123 -0
  22. package/docs/maps.md +116 -0
  23. package/docs/onion-skinning.md +41 -0
  24. package/docs/pen-tool.md +65 -0
  25. package/docs/polygon-star.md +44 -0
  26. package/docs/scene-3d.md +299 -0
  27. package/docs/scroll-animation.md +314 -0
  28. package/docs/shape-morph.md +61 -0
  29. package/docs/sprite-sheet-export.md +54 -0
  30. package/docs/symbols-and-library.md +137 -0
  31. package/docs/teaching.md +348 -0
  32. package/docs/video-rendering.md +728 -0
  33. package/lib/addons/adapters.js +22 -18
  34. package/lib/addons/characters.js +2694 -1809
  35. package/lib/addons/target-properties-BtN9skb-.js +142 -0
  36. package/lib/browser/tinyfly.iife.js +9 -4
  37. package/lib/browser/tinyfly.js +6498 -5483
  38. package/lib/browser/tinyfly.umd.js +9 -4
  39. package/lib/engine/tinyfly-engine.js +286 -258
  40. package/lib/engine/tinyfly-engine.umd.cjs +2 -2
  41. package/lib/types/adapters/canvas/canvas-adapter.d.ts +6 -0
  42. package/lib/types/adapters/canvas/index.d.ts +1 -0
  43. package/lib/types/adapters/canvas/target-properties.d.ts +68 -0
  44. package/lib/types/characters/acting/beat-check.d.ts +95 -0
  45. package/lib/types/characters/acting/hand-path.d.ts +32 -0
  46. package/lib/types/characters/acting/index.d.ts +2 -0
  47. package/lib/types/characters/acting/script.d.ts +66 -7
  48. package/lib/types/characters/capabilities.d.ts +50 -0
  49. package/lib/types/characters/code-panel.d.ts +201 -0
  50. package/lib/types/characters/figure-actions.d.ts +11 -0
  51. package/lib/types/characters/figure-props.d.ts +87 -0
  52. package/lib/types/characters/gaits.d.ts +8 -0
  53. package/lib/types/characters/index.d.ts +3 -0
  54. package/lib/types/characters/stick-figure.d.ts +2 -0
  55. package/lib/types/engine/authoring/did-you-mean.d.ts +14 -0
  56. package/lib/types/engine/authoring/index.d.ts +1 -0
  57. package/llms-full.txt +12548 -0
  58. package/llms.txt +57 -0
  59. package/package.json +6 -2
  60. package/skills/tinyfly/SKILL.md +64 -0
  61. package/lib/addons/camera-C7bZOWMB.js +0 -33
@@ -0,0 +1,628 @@
1
+ # 3D Support Plan
2
+
3
+ **Status:** Milestones 0 (rotation math, slerp tracks), 1 (CSS-style 3D in every adapter), 2 (scenes on Canvas 2D), 3 (WebGL2 renderer), 4 (the editor's 3D Scene element, first version) and 6 (3D characters) implemented; the rest planned
4
+ **Date:** 2026-10-04 (revised 2026-10-05: movies, two render modes, entries)
5
+ **Related:** [character-system-plan.md](character-system-plan.md),
6
+ [character-system-m1.md](character-system-m1.md), [camera.md](camera.md),
7
+ [video-rendering.md](video-rendering.md), [file-format.md](file-format.md)
8
+
9
+ This plan covers adding 3D to tinyfly without changing what tinyfly is: a
10
+ small, deterministic, JSON-first engine that computes values over time and
11
+ hands them to adapters. Each milestone below ships on its own, as one minor
12
+ release, with tests and a gallery demo.
13
+
14
+ 3D is also meant for **movies**: short films made from data, rendered in a
15
+ **stylized** look by our own renderers or a **realistic** one by three.js, from
16
+ the same scene and timeline. The film work itself (shots, cuts, sets, sound,
17
+ long renders) is Phase 34 in `todo.md`; this plan makes sure the 3D foundation
18
+ leaves room for it (section 9).
19
+
20
+ ---
21
+
22
+ ## 1. Goals and non-goals
23
+
24
+ ### Goals
25
+
26
+ 1. **3D is data.** A 3D scene, its camera, lights, objects and materials are
27
+ plain JSON. Their animation is ordinary tracks in an ordinary
28
+ `TimelineDefinition`. Nothing about 3D needs a new timeline or a new clock.
29
+ 2. **The engine stays renderer-agnostic.** `src/engine/` gains only pure math
30
+ helpers (vectors, quaternions, matrices) and one new interpolation mode
31
+ (spherical interpolation of rotations). It does not learn about meshes,
32
+ cameras, lights or GPUs.
33
+ 3. **Deterministic.** The same scene and timeline at the same time give the same
34
+ resolved scene (the same matrices, the same draw order), in the browser, in a
35
+ Worker and in Node. Frames can be rendered in any order.
36
+ 4. **Headless works from day one.** `tinyfly video` and `renderVideo()` can
37
+ render a 3D scene to MP4/PNG in Node with the dependencies they already have
38
+ (`@napi-rs/canvas`, ffmpeg).
39
+ 5. **Usable without the editor**, then usable in the editor, through the same
40
+ public API.
41
+ 6. **Characters can go 3D.** Character v2 already solves its bones in 3D
42
+ (`SolvedChain.joints3`); a real camera should be able to look at them.
43
+ 7. **Films.** A scene can hold several cameras and cut between them on the
44
+ timeline, and a whole film renders frame by frame, in any order, to MP4.
45
+ 8. **Two render modes from one source.** `stylized` (our Canvas 2D and WebGL2
46
+ renderers: flat, toon, pencil; deterministic to the pixel on Canvas 2D) and
47
+ `realistic` (the three.js adapter: PBR, shadow maps, environment light). The
48
+ scene, timeline, shots and sound do not change between them; each renderer
49
+ reads the material and light fields it understands.
50
+ 9. **Pay only for what you import.** 3D lives in its own entries, never in `.`
51
+ or `/player`; three.js is an optional peer dependency; nothing 3D is
52
+ downloaded into a bundle that does not import it (section 5, *Entries*).
53
+
54
+ ### Non-goals
55
+
56
+ - A general-purpose 3D engine. No physics, no scene-graph editing at runtime, no
57
+ PBR pipeline in our own renderers (realistic mode gets PBR from three.js), no
58
+ general post-processing stack. Films do need a little, so the stylized
59
+ renderers get **blob / contact shadows** for characters by milestone 6, and
60
+ films get a **small deterministic post step** (supersampled anti-aliasing,
61
+ fog, vignette, motion blur by sub-frame sampling); shadow maps belong to
62
+ realistic mode.
63
+ - A hard dependency on three.js, Babylon or any 3D library in the engine, the
64
+ player, or the default adapters bundle. three.js is required only for
65
+ realistic mode, as an optional peer dependency.
66
+ - Full glTF coverage. Compressed geometry (Draco, meshopt), KHR material
67
+ extensions, cameras-from-file and multiple scenes per file are out of scope at
68
+ first.
69
+ - 3D in exported Lottie files. Lottie's 3D layer support is too thin to target.
70
+ - Bit-identical GPU output across machines. Determinism is guaranteed for the
71
+ *resolved scene* and for the CPU renderer; GPU renderers are deterministic in
72
+ input, not in pixels.
73
+
74
+ ---
75
+
76
+ ## 2. What "3D" means in tinyfly
77
+
78
+ There are four distinct things people mean by "3D". They have very different
79
+ costs, so the plan treats them separately.
80
+
81
+ ### 2a. 3D transforms of existing 2D targets (CSS-style 3D)
82
+
83
+ A card flips, a carousel turns, a layer tilts. The targets stay flat; only their
84
+ transform is 3D.
85
+
86
+ **What exists today:**
87
+
88
+ | Adapter | 3D support today |
89
+ |---|---|
90
+ | DOM (`src/adapters/dom/dom-adapter.ts`) | `z`, `rotateX`, `rotateY`, `rotateZ`, `scaleZ` become `translateZ()`, `rotateX()` and so on; `perspective` is emitted first as a `perspective()` transform function on the element itself. |
91
+ | SVG (`src/adapters/svg/svg-adapter.ts`) | `rotateX`, `rotateY` and `perspective` composed into a CSS transform on the SVG element. |
92
+ | Canvas (`src/adapters/canvas/canvas-adapter.ts`) | An approximation: `rotateX`/`rotateY` scale the target by `cos(angle)` vertically/horizontally. No foreshortening, no `z`, no `perspective`. |
93
+ | WebGL (`src/adapters/webgl/webgl-adapter.ts`) | 2D only: a `mat3` per quad; `rotateX`/`rotateY`/`z` are ignored. |
94
+
95
+ Gallery precedent: `src/examples/live-demos/card-flip-3d.js` already gets a
96
+ convincing 3D flip by setting `perspective`, `transform-style: preserve-3d` and
97
+ `backface-visibility` in hand-written CSS and animating `rotateY`.
98
+
99
+ **What is missing:** a parent's `perspective` (as opposed to the
100
+ `perspective()` function on the element), `perspective-origin`,
101
+ `transform-style: preserve-3d`, `backface-visibility`, a documented transform
102
+ order, and quaternion rotations (`matrix3d`) for tumbling without gimbal lock.
103
+ These are small, adapter-only changes (milestone 1).
104
+
105
+ ### 2b. A 3D scene adapter: cameras, lights, meshes
106
+
107
+ A real scene: primitives and models in world space, viewed through a perspective
108
+ or orthographic camera, lit by ambient, directional and point lights. This is
109
+ the bulk of the work (milestones 2–4).
110
+
111
+ ### 2c. 3D characters
112
+
113
+ Character v2's skeleton is "2.5D": bones are solved in 3D in the character's own
114
+ space (`joints3`), then turned by `turn × 90°` and projected orthographically in
115
+ `project()` in `src/characters/rig/skeleton.ts`. Lifting it into a 3D scene
116
+ means:
117
+
118
+ - placing the character's space in the world with a transform (so `turn` becomes
119
+ a yaw on the object, not a special view rule);
120
+ - letting the scene camera do the projection (perspective, any angle, including
121
+ from above);
122
+ - drawing it either with its existing pens (the clean/pencil/silhouette look,
123
+ under a real camera) or as simple 3D shapes (capsule limbs, ellipsoid head).
124
+
125
+ ### 2d. Importing glTF animations
126
+
127
+ glTF 2.0 is the standard interchange format. Its animations are already
128
+ keyframes: each channel targets a node's `translation`, `rotation`
129
+ (quaternion), `scale`, or morph `weights`, with `LINEAR`, `STEP` or
130
+ `CUBICSPLINE` sampling. That maps almost one-to-one onto tinyfly tracks. Import
131
+ is a pure function (JSON + binary buffers in, scene + tracks out), so it runs in
132
+ Node and in Workers.
133
+
134
+ ---
135
+
136
+ ## 3. Renderer options
137
+
138
+ ### Candidates
139
+
140
+ | Option | Size (min+gz, ballpark) | Worker / OffscreenCanvas | Node / headless video | Fit with tinyfly |
141
+ |---|---|---|---|---|
142
+ | **Canvas 2D projected renderer** (ours) | ~5–8 KB | Yes (OffscreenCanvas 2D) | **Yes**, through `@napi-rs/canvas`, the dependency headless already uses | Flat or toon-shaded low-poly, painter's algorithm. Deterministic to the pixel on a given canvas backend. Can draw characters with the existing pens. Slow past a few thousand triangles; intersecting meshes sort wrongly. |
143
+ | **WebGL2** (ours, raw) | ~8–15 KB | Yes (`OffscreenCanvas.getContext('webgl2')`) | No, without a browser (headless-gl is WebGL1-only and a native build) | Depth buffer, per-pixel lighting, skinning in the vertex shader, tens of thousands of triangles. Universally available in browsers. |
144
+ | **WebGPU** (ours, raw) | ~10–20 KB | Yes | Only with Dawn bindings (native, heavy) | Best long-term performance; still not available on every browser/OS combination we must support, and adds a second shader language (WGSL). Later, behind the same interface. |
145
+ | **three.js** (optional adapter) | ~150–180 KB core | Yes | Only in a headless browser | PBR, shadows, glTF loader, huge ecosystem. Too large to justify inside tinyfly; fine as an *optional peer dependency* for users who already ship it. |
146
+ | **OGL** | ~10–30 KB | Yes | No | Small and clean, but a dependency for what our own ~15 KB WebGL2 renderer would do, and it would still need our resolved-scene mapping. |
147
+ | **Babylon.js** | several hundred KB+ | Yes | Only in a headless browser | Full engine with its own scene graph, animation system and clock. Conflicts with "the engine owns time". Not recommended even as an adapter. |
148
+ | **regl / twgl** | ~10–25 KB | Yes | No | Thin WebGL helpers. They save little over writing the 300–500 lines ourselves. |
149
+
150
+ Sizes are order-of-magnitude figures; measure the actual build before any
151
+ decision that depends on them.
152
+
153
+ ### Dependency justification (per CLAUDE.md)
154
+
155
+ - **Our own renderers (Canvas 2D, WebGL2):** no dependency. A minimal renderer
156
+ for primitives, flat/Lambert/toon shading and a depth buffer is a few hundred
157
+ lines, readable, and testable through pure functions, like the existing
158
+ `quadMatrix` tests for the WebGL adapter.
159
+ - **three.js:** justified only as an *opt-in adapter* in its own entry point
160
+ (`@algorisys/tinyfly/three`), with `three` as an optional peer dependency,
161
+ externalised in the build exactly as the frameworks entries externalise React
162
+ and Vue. It never appears in the engine, player, or default adapters bundle.
163
+ Its value: users who already have a three.js scene can drive it from tinyfly
164
+ timelines, and get PBR/shadows for free.
165
+ - **Nothing else** clears the bar.
166
+
167
+ ### Recommendation
168
+
169
+ 1. **Canvas 2D projected renderer first.** It is the only option that works in
170
+ the browser, in a Worker and in Node video export with zero new dependencies,
171
+ gives pixel-stable golden tests, and can render characters in their pencil
172
+ look under a real camera, which is tinyfly's distinctive output.
173
+ 2. **WebGL2 renderer second**, consuming the exact same resolved scene, for
174
+ performance, a real depth buffer and skinning.
175
+ 3. **three.js adapter** once the data model is stable: optional for package
176
+ users, but it *is* realistic mode, so films depend on it (milestone 8).
177
+ 4. **WebGPU** only when there is a measured need and it is available
178
+ everywhere we play; the renderer interface keeps that door open.
179
+
180
+ ---
181
+
182
+ ## 4. Data model
183
+
184
+ ### Coordinate conventions
185
+
186
+ - World space is **right-handed, +y up, +z toward the default camera**, in
187
+ **metres** (as glTF, and so that realistic mode's lights and cameras behave
188
+ physically). This matches glTF, WebGL and
189
+ three.js, *and* character v2's own space (+x the character's left, +y up, +z
190
+ the way it faces), so neither import nor characters need axis flips.
191
+ - This differs from the 2D stage (y down, pixels). The two never mix inside one
192
+ scene; a 3D scene element is a box on the 2D stage, like the map and character
193
+ elements.
194
+ - Angles in authored JSON are **degrees** (as everywhere else in tinyfly);
195
+ quaternions are `[x, y, z, w]` (glTF order).
196
+ - Euler rotation order is fixed at **`YXZ`** (yaw, then pitch, then roll), the
197
+ order that reads naturally for cameras and characters. It is documented, not
198
+ configurable, at first.
199
+
200
+ ### Scene JSON
201
+
202
+ ```ts
203
+ interface Scene3D {
204
+ formatVersion: number
205
+ id: string
206
+ /** Id of the camera at time 0; a film cuts between cameras with an `activeCamera` track */
207
+ camera: string
208
+ /** Ordered: array order is draw/tie-break order and child order */
209
+ objects: Object3D[]
210
+ materials?: Record<string, Material3D>
211
+ /** Background colour, or 'transparent' */
212
+ background?: string
213
+ /** Realistic mode: an equirectangular HDR asset for lighting and reflections */
214
+ environment?: string
215
+ /** Fog from `near` to `far` metres, both modes */
216
+ fog?: { color: string; near: number; far: number }
217
+ /** Binary assets (glTF buffers, textures) by id; data URIs or URLs, loaded before rendering */
218
+ assets?: Record<string, { uri: string; mimeType?: string }>
219
+ }
220
+
221
+ interface Transform3D {
222
+ position?: [number, number, number] // default [0, 0, 0]
223
+ /** Euler degrees, YXZ order; or a quaternion via `quaternion` */
224
+ rotation?: [number, number, number]
225
+ quaternion?: [number, number, number, number]
226
+ scale?: [number, number, number] | number // default 1
227
+ }
228
+
229
+ interface ObjectBase extends Transform3D {
230
+ id: string // track target id within the scene
231
+ name?: string
232
+ parent?: string // id of an earlier object
233
+ visible?: boolean
234
+ }
235
+
236
+ type Object3D =
237
+ | (ObjectBase & { kind: 'group' })
238
+ | (ObjectBase & { kind: 'mesh'; geometry: Geometry3D; material: string })
239
+ | (ObjectBase & { kind: 'camera'; projection: 'perspective'; fov: number; near: number; far: number })
240
+ | (ObjectBase & { kind: 'camera'; projection: 'orthographic'; height: number; near: number; far: number })
241
+ | (ObjectBase & { kind: 'light'; light: 'ambient' | 'directional' | 'point' | 'spot'; color: string
242
+ /** Physical units: lux for directional, candela for point and spot; stylized renderers normalise them */
243
+ intensity: number; range?: number; angle?: number; castShadow?: boolean })
244
+ | (ObjectBase & { kind: 'model'; asset: string; node?: string }) // a glTF (sub)tree
245
+ | (ObjectBase & { kind: 'character'; character: CharacterSpec; look?: 'pen' | 'solid'; model?: string }) // `model`: a skinned glTF for realistic mode
246
+
247
+ type Geometry3D =
248
+ | { type: 'box'; size: [number, number, number] }
249
+ | { type: 'sphere'; radius: number; segments?: number }
250
+ | { type: 'cylinder'; radius: number; height: number; segments?: number }
251
+ | { type: 'cone'; radius: number; height: number; segments?: number }
252
+ | { type: 'plane'; size: [number, number] }
253
+ | { type: 'torus'; radius: number; tube: number; segments?: number }
254
+ | { type: 'extrude'; path: string; depth: number } // an SVG path, extruded (logos, text outlines)
255
+
256
+ interface Material3D {
257
+ // Both modes
258
+ color: string
259
+ opacity?: number
260
+ texture?: string // asset id
261
+ doubleSided?: boolean
262
+ // Stylized mode (our renderers); realistic mode ignores these
263
+ shading: 'unlit' | 'flat' | 'lambert' | 'toon'
264
+ /** Toon only: number of light bands */
265
+ bands?: number
266
+ /** Ink outline, in px (Canvas 2D and WebGL renderers both support it) */
267
+ outline?: { width: number; color: string }
268
+ // Realistic mode (three.js, glTF's metallic-roughness); stylized mode ignores these
269
+ roughness?: number // 0..1, default 0.6
270
+ metalness?: number // 0..1, default 0
271
+ emissive?: string
272
+ normalMap?: string // asset id
273
+ }
274
+ ```
275
+
276
+ A material is written once and looks right in both modes: stylized reads
277
+ `shading`, `bands` and `outline`; realistic reads `roughness`, `metalness`,
278
+ `emissive` and `normalMap`; both read `color`, `opacity` and `texture`. Neither
279
+ fails on the other's fields.
280
+
281
+ The `extrude` geometry reuses the engine's existing path parsing
282
+ (`src/engine/path/`), so any 2D shape in tinyfly can become a 3D object.
283
+
284
+ ### Tracks: how 3D values animate
285
+
286
+ Tracks address scene objects by **target = `<sceneElementId>/<objectId>`** (for
287
+ example `stage3d/cube`), so a 3D scene's tracks can live in the same timeline as
288
+ 2D elements without id collisions.
289
+
290
+ | Property | Value | Interpolation |
291
+ |---|---|---|
292
+ | `x`, `y`, `z` | number | linear (existing) |
293
+ | `rotateX`, `rotateY`, `rotateZ` | number, degrees | linear (existing); applied in YXZ order |
294
+ | `scale`, `scaleX`, `scaleY`, `scaleZ` | number | linear (existing) |
295
+ | `position`, `scale3` | `number[3]` | element-wise (existing `interpolateArray`) |
296
+ | `quaternion` | `number[4]` | **slerp** (new) |
297
+ | `color`, `material.color`, `light.color` | colour string | existing `interpolateColor` |
298
+ | `opacity`, `light.intensity`, `camera.fov`, `camera.height` | number | linear |
299
+ | `lookAt` | `number[3]` (a camera's target point) | element-wise |
300
+ | `weights` | `number[]` (morph targets) | element-wise |
301
+ | `visible` | number 0/1 | stepped |
302
+ | `activeCamera` (on the scene) | camera object id | stepped: a cut is one keyframe |
303
+ | `pose.*`, `turn`, `gait.*` (character objects) | number | as character v2 today |
304
+
305
+ Reusing the existing names (`x`, `rotateY`, …) means the editor's property
306
+ fields, curve editor, springs and staggers work on 3D objects unchanged.
307
+
308
+ **Validation rule (no hidden precedence):** an object may be animated by
309
+ component tracks (`x`/`y`/`z`, `rotateX…`) *or* by vector tracks
310
+ (`position`, `quaternion`) for the same channel, never both. The scene validator
311
+ reports the conflict instead of picking a winner.
312
+
313
+ ### The one engine change: explicit slerp
314
+
315
+ Today `getInterpolator()` in `src/engine/interpolation/interpolators.ts` picks
316
+ an interpolator from the *shape* of the value. A 4-number array would therefore
317
+ be interpolated element-wise, which is wrong for a quaternion (it shrinks and
318
+ takes the wrong path). Guessing "4 numbers means quaternion" would be hidden
319
+ magic, so the choice becomes explicit data:
320
+
321
+ ```ts
322
+ interface Track {
323
+ // ...existing fields
324
+ /** How keyframe values blend. Default: chosen from the value, as today. */
325
+ interpolation?: 'slerp'
326
+ }
327
+ ```
328
+
329
+ `interpolateQuaternion(from, to, t)` normalises, takes the shortest arc (flips
330
+ `to` when the dot product is negative), falls back to normalised lerp when the
331
+ quaternions are nearly equal, and returns a unit quaternion. Easing still shapes
332
+ `t`, so every existing ease works on rotations.
333
+
334
+ Serialisation: a file that uses `interpolation` is written with
335
+ `formatVersion: 2`. `FORMAT_VERSION`'s existing rule ("a reader refuses files
336
+ from a newer format rather than playing them wrongly") then protects old players
337
+ from silently lerping quaternions. Files that don't use it stay at version 1.
338
+
339
+ Springs and inertia remain numeric: they work on `rotateX` and friends, not on
340
+ `quaternion` tracks (the validator says so).
341
+
342
+ ### Deterministic ordering and evaluation
343
+
344
+ - `objects` is an **array**, never a map; parents must precede children. World
345
+ matrices are computed in array order in one pass.
346
+ - The resolver produces a **draw list** sorted by (opaque before transparent,
347
+ then depth, then object index, then triangle index). `Array.prototype.sort` is
348
+ stable, and the index tie-breakers make the order total, so equal depths never
349
+ flicker between runs.
350
+ - All math is in float64 (`number`) on the CPU; matrices are only converted to
351
+ `Float32Array` at the GPU boundary.
352
+ - No asset loads during `render()`. Assets are loaded (async) up front by
353
+ `loadScene3D()`; rendering a frame is synchronous and pure, so frame N never
354
+ depends on whether frame N−1 was drawn.
355
+ - `Math.sin`/`Math.cos` can differ in the last bit between JavaScript engines.
356
+ Golden tests compare on one engine (Node in CI) and use a 1e-9 tolerance for
357
+ matrices in unit tests.
358
+
359
+ ---
360
+
361
+ ## 5. Architecture and module layout
362
+
363
+ ### The pipeline
364
+
365
+ ```
366
+ TimelineDefinition ──► engine (unchanged + slerp) ──► AnimationState
367
+ │
368
+ Scene3D (JSON) ─────────────────────────────┐ │
369
+ ▼ ▼
370
+ resolveScene3D(scene, state, size) pure, no GPU, no DOM
371
+ │
372
+ ▼
373
+ ResolvedScene3D (plain data:
374
+ world matrices, camera matrices,
375
+ lights, sorted draw list)
376
+ │
377
+ ┌──────────────────────┼───────────────────────┐
378
+ ▼ ▼ ▼
379
+ canvas-2d renderer webgl2 renderer three adapter (optional)
380
+ ```
381
+
382
+ The resolver is the 3D equivalent of what adapters already do with
383
+ `AnimationState`, split out so that every renderer shares one implementation of
384
+ transforms, cameras and ordering, and so that it can be unit-tested without any
385
+ rendering at all.
386
+
387
+ ### Files
388
+
389
+ All new code; file and folder names lowercase-hyphenated.
390
+
391
+ ```
392
+ src/engine/math/ pure helpers, exported from the engine
393
+ vec3.ts add, sub, scale, dot, cross, normalize, lerp
394
+ quat.ts fromEuler (YXZ), toEuler, multiply, slerp, rotateVec3
395
+ mat4.ts identity, multiply, compose(TRS), invert, perspective, orthographic, lookAt
396
+ index.ts
397
+ src/engine/interpolation/
398
+ interpolators.ts + interpolateQuaternion, chosen by Track.interpolation
399
+
400
+ src/scene-3d/ entry: @algorisys/tinyfly/scene-3d (no DOM, no GPU, no glTF)
401
+ scene-types.ts Scene3D, Object3D, Geometry3D, Material3D
402
+ validate-scene.ts ids, parent order, channel conflicts, slerp-only-on-quaternion
403
+ resolve-scene.ts Scene3D + AnimationState → ResolvedScene3D
404
+ geometry/
405
+ primitives.ts box, sphere, cylinder, cone, plane, torus → indexed triangles
406
+ extrude.ts SVG path → extruded mesh (uses engine/path)
407
+ camera.ts view/projection matrices, orbit and dolly helpers (pure)
408
+ shading.ts flat / lambert / toon colour for a normal + lights (pure)
409
+ draw-list.ts depth sorting with total-order tie-breaks
410
+ load-scene.ts async asset loading, returns a ready scene
411
+ characters/
412
+ character-object.ts character v2 skeleton placed in world space
413
+ scene-3d-adapter.ts registerScene / applyState / render / destroy
414
+ canvas-2d-renderer.ts painter's algorithm, outlines, pens, blob shadows (stylized)
415
+ index.ts
416
+ webgl/ entry: @algorisys/tinyfly/scene-3d/webgl
417
+ webgl2-renderer.ts depth buffer, lambert/toon shaders, skinning (stylized)
418
+ index.ts
419
+ gltf/ entry: @algorisys/tinyfly/scene-3d/gltf
420
+ glb-container.ts GLB header and chunks → JSON + binary
421
+ gltf-accessors.ts typed views over buffers
422
+ gltf-to-scene.ts nodes, meshes, materials → Scene3D objects
423
+ gltf-to-tracks.ts animation channels → tinyfly tracks
424
+ index.ts
425
+
426
+ src/adapters/three/ entry: @algorisys/tinyfly/three (three = optional peer)
427
+ three-adapter.ts Scene3D + AnimationState → three.js objects (realistic)
428
+ index.ts
429
+ ```
430
+
431
+ ### Entries: pay only for what you import
432
+
433
+ | Entry | Contains | Pulls in |
434
+ |---|---|---|
435
+ | `.`, `/player` | the engine, plus the math helpers and slerp | nothing 3D |
436
+ | `/scene-3d` | types, validator, resolver, primitives, Canvas 2D renderer | the engine |
437
+ | `/scene-3d/webgl` | the WebGL2 renderer | `/scene-3d` |
438
+ | `/scene-3d/gltf` | the glTF loader | `/scene-3d` |
439
+ | `/three` | realistic mode | `/scene-3d`, and `three` (optional peer, externalised) |
440
+
441
+ Each gets its own entry in `vite.config.addons.ts` (as `characters` and `maps`
442
+ have), and `package.json` keeps `sideEffects` limited to CSS, so bundlers drop
443
+ what is not imported. A test checks the built `lib/` files: each entry has a size
444
+ budget, and `.`, `/player` and `/scene-3d` contain no WebGL, glTF or three code.
445
+ `npm install` still downloads the whole package (812 kB at v0.82.0); if 3D makes
446
+ that noticeably larger, the renderers and loaders move to a separate
447
+ `@algorisys/tinyfly-3d` package rather than grow every install.
448
+
449
+ ### The adapter interface
450
+
451
+ The existing adapters share a shape by convention (`registerTarget`,
452
+ `applyState`, `render`, `destroy`) rather than a declared interface. The 3D
453
+ adapter follows the same shape, and adds an explicit renderer interface so
454
+ renderers are swappable:
455
+
456
+ ```ts
457
+ interface Renderer3D {
458
+ /** Draw one resolved frame. Synchronous; never loads anything. */
459
+ render(frame: ResolvedScene3D): void
460
+ resize(width: number, height: number, pixelRatio: number): void
461
+ destroy(): void
462
+ }
463
+
464
+ class Scene3DAdapter {
465
+ constructor(renderer: Renderer3D, options: { width: number; height: number })
466
+ registerScene(id: string, scene: LoadedScene3D): void
467
+ unregisterScene(id: string): void
468
+ applyState(state: AnimationState): void // resolves every registered scene
469
+ render(): void // hands each resolved frame to the renderer
470
+ destroy(): void
471
+ }
472
+
473
+ // Usage, no editor:
474
+ const scene = await loadScene3D(sceneJson)
475
+ const adapter = new Scene3DAdapter(new Canvas2DRenderer(ctx), { width: 800, height: 450 })
476
+ adapter.registerScene('stage3d', scene)
477
+ timeline.onUpdate = (state) => { adapter.applyState(state); adapter.render() }
478
+ ```
479
+
480
+ The Canvas 2D renderer also exposes `drawScene3D(ctx, scene, state, size)`, so a
481
+ scene can be a `custom` target inside the existing `CanvasAdapter` and
482
+ therefore inside a `VideoScene`, with no change to `src/headless/`.
483
+
484
+ ### Workers
485
+
486
+ Everything up to `ResolvedScene3D` is plain data, so a Worker can own the
487
+ timeline and an `OffscreenCanvas` and run either renderer entirely off the main
488
+ thread. Nothing in the engine or the timeline needs 3D-specific code for this.
489
+
490
+ ### Headless
491
+
492
+ `FrameRenderer` draws on a 2D context. A 3D scene enters it as a custom canvas
493
+ target drawn by the Canvas 2D renderer, so `tinyfly video scene.mjs` renders 3D
494
+ to MP4 with today's dependencies: this is stylized mode, and the reference for
495
+ golden tests. WebGL2 is not available in Node, so heavy stylized scenes and
496
+ every realistic render go through headless Chromium (Playwright), driven by the
497
+ same scene module, frame by frame. Playwright is never a dependency: the CLI
498
+ asks for it when a film needs it. Realistic frames are deterministic in input,
499
+ not in pixels, and are checked against the stylized render of the same frame by
500
+ tolerance.
501
+
502
+ ### The editor (SolidJS)
503
+
504
+ The editor stays a consumer of the public API; no 3D logic in components.
505
+
506
+ | Piece | Where (proposed) |
507
+ |---|---|
508
+ | `scene3d` element type: a box on the stage hosting one `Scene3D` | `src/editor/stores/scene-store.ts` (alongside `character` and `map`) |
509
+ | Element → `Scene3D`, loading, adapter wiring | `src/editor/utils/scene-3d-element.ts` |
510
+ | Preview: a canvas inside the element, repainted from `AnimationState` | `src/editor/components/preview-panel.tsx` |
511
+ | Outliner: objects, cameras, lights; add primitive / light / camera; import glTF | `src/editor/components/scene-3d-panel.tsx` |
512
+ | Property fields: position, rotation (Euler degrees), scale, material, light, camera, each with ◆ keyframe via `keyValuesAtPlayhead` | `src/editor/components/property-panel.tsx` |
513
+ | Viewport navigation (orbit, pan, dolly) for *authoring only*; it changes an editor view camera, not the scene, unless "look through camera" is on | calls `orbit()` / `dolly()` from `src/scene-3d/camera.ts` |
514
+ | Export (GIF, MP4, HTML embed) | `src/editor/utils/scene-to-canvas.ts` draws the scene with the Canvas 2D renderer |
515
+
516
+ The editor shows rotations as Euler degrees (people can type them); the library
517
+ stores what the user keyed (component tracks), and quaternion tracks appear only
518
+ for imported glTF animation, shown read-only with "convert to Euler tracks" as
519
+ an action.
520
+
521
+ ---
522
+
523
+ ## 6. Milestones
524
+
525
+ Each is one minor release with unit tests, a docs page, and a gallery demo (a
526
+ live demo in `src/examples/live-demos/` and, where it makes sense, a headless
527
+ example in `examples/headless-video/`).
528
+
529
+ | # | Release | Contents | Tests | Demo |
530
+ |---|---|---|---|---|
531
+ | 0 | **Rotation math** | `src/engine/math/` (vec3, quat, mat4); `Track.interpolation: 'slerp'` and `interpolateQuaternion`; `formatVersion` 2 only when used; DOM adapter accepts a `quaternion` value and emits `matrix3d()` | slerp shortest path, unit length, endpoints exact, Euler↔quat round trip, JSON round trip, old readers refuse v2 | A CSS cube tumbling on a quaternion track next to the same motion on Euler tracks (gimbal lock visible on the Euler one) |
532
+ | 1 | **CSS 3D, finished** | DOM/SVG: `transformStyle`, `perspective` on parents, `perspectiveOrigin`, `backfaceVisibility`; documented transform order; editor fields for `z`, `rotateX`, `rotateY`, perspective; Canvas adapter: true perspective foreshortening for flat targets (projected corners, image drawn as two affine triangles); WebGL adapter: `mat4` per quad so `rotateX/Y/z` work | transform string order, projected-corner maths, canvas vs DOM corner positions agree within 0.5 px | Cover-flow carousel and a 3D card flip, both built in the editor, both exported to GIF |
533
+ | 2 | **Scenes on Canvas 2D** | `@algorisys/tinyfly/scene-3d`: types, validator, resolver, primitives, extrude, camera, lights, flat/lambert/toon shading, outlines; Canvas 2D renderer; `drawScene3D` custom target for headless video | resolver matrices, camera matrices, draw-order total and stable, validator errors, golden PNGs rendered twice identically in Node | Orbiting camera around lit primitives and an extruded tinyfly logo; the same scene rendered to MP4 with `tinyfly video` |
534
+ | 3 | **WebGL2 renderer** | Same `ResolvedScene3D` on the GPU: depth buffer, Lambert and toon shaders, outlines, textures; OffscreenCanvas in a Worker | pure parts (buffer packing, uniforms) as `webgl-adapter.test.ts` does; a browser smoke test comparing WebGL2 and Canvas 2D output within a tolerance | 2,000 animated cubes with a staggered wave (the case Canvas 2D can't hold at 60 fps) |
535
+ | 4 | **Editor 3D element** | `scene3d` element, outliner, add primitives/lights/cameras, transform and material fields with keyframes, authoring viewport, look-through-camera, export | store tests for add/remove/key; exported JSON round-trips through `loadScene3D` | A product-spin scene made entirely in the editor, exported as an embed |
536
+ | 5 | **glTF import (rigid)** | GLB/glTF parsing, meshes, base-colour materials and textures, node hierarchy, animation channels → tracks (`LINEAR` → linear/slerp, `STEP` → stepped, `CUBICSPLINE` → baked and simplified with the existing `bake.ts`/`simplifyKeyframes`), morph `weights`; editor "Import glTF" | accessor decoding, every sampler type, channels to tracks, tracks reproduce glTF sample values at keyframe times | Khronos sample models (CC0/CC-BY, e.g. *AnimatedCube*, *BoxAnimated*) playing on the tinyfly timeline, scrubbable |
537
+ | 6 | **3D characters** | `character` scene objects: character v2 skeleton (`joints3`) placed in world space; `turn` becomes root yaw; perspective projection through the scene camera; `look: 'pen'` (existing clean/pencil/silhouette pens on Canvas 2D) or `look: 'solid'` (capsule limbs, ellipsoid head, toon-shaded) | joints in world space match the 2.5D solver at `turn` 0–4 under an orthographic front camera (so v2 goldens hold); depth order under any camera | A pencil-look walk cycle with the camera craning from side view to top-down; the same scene in the solid look |
538
+ | 7 | **Skinned glTF** | Skins, joints, inverse bind matrices; GPU skinning in WebGL2, CPU skinning in Canvas 2D; optional mapping of a humanoid glTF skeleton to character v2 pose fields | skinning matrices against hand-computed cases, bind pose renders as rest mesh | A rigged CC0 character playing its imported clip, then blended into a tinyfly pose |
539
+ | 8 | **three.js adapter** (realistic mode) | `@algorisys/tinyfly/three`: builds three objects from `Scene3D`, applies `AnimationState` each frame; `three` as an optional peer, externalised | mapping tests with three in devDependencies only | The milestone 2 scene rendered with three.js PBR and shadows, driven by the identical timeline JSON |
540
+
541
+ Milestones 0–2 are the foundation. After that, 3, 5 and 6 can go in any order
542
+ (for the story channel, 6 before 3). Milestone 2 already supports several
543
+ cameras and the `activeCamera` track, and milestone 6 adds blob shadows, so the
544
+ film work (Phase 34) needs no format change. Films in realistic mode need
545
+ milestone 8, and realistic characters need 7 (or the `solid` look).
546
+
547
+ ---
548
+
549
+ ## 7. Risks
550
+
551
+ | Risk | Mitigation |
552
+ |---|---|
553
+ | **Scope creep**: 3D invites "just one more feature" until tinyfly is a game engine | The non-goals list is part of this plan; each milestone's scope is fixed before it starts |
554
+ | **Painter's algorithm artefacts** (intersecting or long overlapping triangles sort wrongly) | Document it; sort per triangle, not per object; keep scenes low-poly in the Canvas 2D renderer; WebGL2 has a depth buffer |
555
+ | **Canvas 2D performance** past a few thousand triangles | Back-face culling and frustum culling in the resolver; the WebGL2 renderer for heavy scenes; the docs give a triangle budget |
556
+ | **GPU output differs between machines** | Goldens only for the Canvas 2D renderer; GPU tests check pure parts and use tolerances |
557
+ | **Floating-point differences between JS engines** | Unit tests use tolerances; goldens are produced and checked in one runtime (Node in CI) |
558
+ | **Async assets vs deterministic frames** | Loading is separate from rendering; `render()` is synchronous and refuses scenes whose assets are not loaded |
559
+ | **glTF breadth** (extensions, compression, sparse accessors) | Support core glTF 2.0 and sparse accessors; report unsupported extensions by name instead of rendering wrongly |
560
+ | **Two coordinate systems** (2D y-down px, 3D y-up units) confusing users | They never mix inside a scene; the 3D element is a box on the 2D stage; the docs open with one diagram of both |
561
+ | **Euler order surprises** | One fixed, documented order (YXZ); quaternion tracks when users need tumbling |
562
+ | **Editor UX for 3D** (gizmos, selection in depth) is a large design problem | Milestone 4 ships numeric fields and an orbit viewport first; on-canvas gizmos are a later, separate decision |
563
+ | **Bundle growth** | `scene-3d` and `three` are separate entries; the engine gains only math helpers and slerp, measured in the release notes |
564
+
565
+ ---
566
+
567
+ ## 8. Open questions
568
+
569
+ 1. **Entry name:** `@algorisys/tinyfly/scene-3d` (proposed) or `/3d`?
570
+ 2. **Target addressing:** `<sceneElementId>/<objectId>` (proposed), or one
571
+ target per scene with properties like `cube.x`? The first keeps every
572
+ existing property name and editor field unchanged.
573
+ 3. **Euler order:** is YXZ right for all users, or should `rotationOrder` be a
574
+ per-object option from the start?
575
+ 4. ~~**Units:**~~ Decided: metres, and physical light intensities (realistic
576
+ mode needs them; stylized renderers normalise).
577
+ 5. **WebGL1 adapter:** upgrade the existing quad adapter to WebGL2 in milestone
578
+ 1, or keep it WebGL1 for the widest reach and keep WebGL2 for scenes only?
579
+ 6. ~~**Headless WebGL:**~~ Decided: yes. Films need it for realistic mode and
580
+ for heavy stylized scenes; Canvas 2D stays the reference renderer.
581
+ 7. **Characters:** in a 3D scene, should `turn` stay animatable on a character
582
+ object (as yaw), or be replaced by `rotateY` to avoid two ways of doing the
583
+ same thing?
584
+ 8. ~~**Shadows:**~~ Decided: blob shadows for characters in milestone 6
585
+ (stylized), projected ground shadows next; shadow maps in realistic mode.
586
+ 9. **Sample assets:** which CC0 glTF models are acceptable to vendor into the
587
+ repo for tests and the gallery (size, licence)?
588
+ 10. ~~**Render mode per shot:**~~ Decided: a film sets `renderMode`, and any
589
+ shot may override it. The resolver is shared, so mixing costs nothing; a
590
+ film can be blocked out in stylized and switched shot by shot.
591
+ 11. ~~**Realistic characters:**~~ Decided: the `solid` look with PBR shading
592
+ first (no wait on milestone 7); skinned glTF characters (`model`) replace it
593
+ when milestone 7 lands. Both are driven by the same v2 pose tracks.
594
+
595
+ ---
596
+
597
+ ## 9. Movies and the two render modes
598
+
599
+ A film is a timeline like any other. Nothing below needs a new clock or a new
600
+ file format beyond what milestones 0–2 define.
601
+
602
+ - **Shots and cuts.** Shots are time ranges with a camera and a label; a cut is
603
+ one keyframe on the stepped `activeCamera` track. Transitions (crossfade, dip
604
+ to black, wipe) are pure functions of two frames and a progress, so any frame
605
+ of a film can be rendered on its own.
606
+ - **Cameras.** Moves are presets that write ordinary tracks: orbit, dolly,
607
+ crane, pan and tilt, follow, `lookAt`, focal-length zoom, and a seeded
608
+ handheld shake (deterministic).
609
+ - **Render modes.** `renderMode: 'stylized' | 'realistic'` is set for a film and
610
+ can be overridden per shot. The resolver is shared, so both modes see the same
611
+ world matrices, cameras and cuts; only the renderer differs.
612
+
613
+ | | Stylized | Realistic |
614
+ |---|---|---|
615
+ | Renderer | Canvas 2D (reference), WebGL2 | three.js |
616
+ | Look | flat, toon, pencil; outlines | PBR (roughness, metalness), environment map, tone mapping |
617
+ | Shadows | blob, then projected ground shadows | shadow maps |
618
+ | Characters | pen look, or `solid` toon | `solid` with PBR first; a skinned glTF (`model`) after milestone 7 |
619
+ | Headless | Node, pixel-identical between runs | headless Chromium, checked by tolerance |
620
+
621
+ - **Post step (both modes, small and deterministic).** Supersampled
622
+ anti-aliasing, fog, vignette, and motion blur by averaging sub-frame samples
623
+ at fixed offsets.
624
+ - **Long renders.** Frames are independent, so a film renders across worker
625
+ threads, in chunks, resumably, with a per-shot frame cache.
626
+ - **Sound, captions, narration.** Reused from video-from-code (Phase 31) and the
627
+ beat grid (Phase 32); see Phase 34 in `todo.md` for the full film plan.
628
+