pixi-effects 0.48.2 → 0.49.0

package/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.49.0
+
+Nothing breaks: 3D scenes drawn by PixiJS 3D (`@pixi/3d`), the new main 3D engine, as a `scene3d` layer (entry `pixi-effects/3d`); three.js stays as the sub engine. PixiJS 8.23 in the template.
+
+- **3D scenes drawn by PixiJS 3D: the `scene3d` layer** (new optional entry `pixi-effects/3d`, `register3D()`, needs `@pixi/3d` and pixi.js 8.23+). PixiJS 3D (`@pixi/3d`, now public) becomes the main 3D engine; three.js stays as the sub engine under its own names. A scene3d layer is a free box like the three layer: `setup({ view, root, camera })` builds the scene with `@pixi/3d`, `update(t, ctx)` sets it from the time, keyframes reach `scene3d.<object>.<path>`. It is drawn inside PixiJS's own renderer, in the layer's place: no second renderer, no canvas, no copy, no context budget; masks, filters, nested compositions and `threeD` cards work on it, and the layer applies `alpha` (with its parents') and `add` / `screen` / `multiply`, which PixiJS 3D ignores on its own. A frame is a function of the time: models and animation players do not run on the ticker (a clip left playing is posed from the layer's time), the shader time `uTime` is the layer's time, `update` with a clock or `Math.random` warns, and `check` draws each scene3d layer in two orders and names one that differs. Typos (did you mean), keys only a three layer has, unknown keyframe paths, a failed `setup` and PixiJS 3D loaded after `movie.init` are each said in a sentence. Recipe `scene3d-basics`; template and README import maps carry `pixi-effects/3d`, `@pixi/3d`, `@pixi/3d/extras`. Measured: WebGL and WebGPU draw a lit scene within 0.03–0.04 of 255.
+- **scene3d, the basic features complete** (phase 2): `followCamera` / `followLights` / `followFog` put a scene3d layer in the composition's own camera, lights and fog (the scene is in composition pixels, y flipped; planes land on the composition's cards within 0.02 of 255; lights in PixiJS 3D's linear light, so a 3D surface is brighter in the shade than a card; shadows from parallel and spot lights, which PixiJS 3D's defaults lost in a scene of pixels: the shadow box and the spot bias are set for it); `movie.pick` / hover / `hit.scene3d.grab(plane).move(x, y)` into the scene (`kind: 'scene3d'`, the object named by its label, picked per triangle); `layerTexture(ctx, 'screen')` shows a 2D layer on a mesh (a PixiJS texture drawn at every frame: identical to the flat layer; a pick on the mesh names the 2D layer); `loadModel(url, { animation, speed, offset, loop })` / `fitCamera(ctx, object)` for glTF models posed from the time; `reuse`: a named layer keeps its scene across `setVars` (`setup` runs once). A light layer with `castsShadows` no longer warns about missing casters when a scene3d layer follows the lights; an unregistered `scene3d` / `three` layer names its entry. Recipes `scene3d-basics` (now with `loadModel`) and `scene3d-in-space`.
+- **PixiJS 8.23** in the template, the chat template and the library's own pages and tests (PixiJS 3D needs it; the main entry still accepts pixi.js 8.x).
+
 ## 0.48.2
 
 Nothing breaks: an app that bundles pixi-effects keeps about 100 KB (gzipped) less of pixi.js, and closing movies no longer uses up PixiJS's 500 warnings.

package/README.md

@@ -1,6 +1,6 @@
 # pixi-effects
 
-> **Status**: experimental — current release `0.48.2`. The API may still change between minor versions; see the [CHANGELOG](./CHANGELOG.md) for what each release changed.
+> **Status**: experimental — current release `0.49.0`. The API may still change between minor versions; see the [CHANGELOG](./CHANGELOG.md) for what each release changed.
 
 **[Gallery →](https://yjmtmtk.github.io/pixi-effects/examples/gallery/)** · 43 pieces written as plain data by AI models · **[For AI →](https://yjmtmtk.github.io/pixi-effects/for-ai.html)** · what an AI reads and runs · **[Playground →](https://yjmtmtk.github.io/pixi-effects/examples/playground.html)** in the browser.
 
@@ -25,16 +25,18 @@ Declarative composition and video rendering for the web. After Effects-style tim
 - **Music written as text** — `{ type: 'audio', music: { bpm: 96, tracks: [{ inst: 'keys', notes: 'Cmaj7:4 Am7:4' }], drums: { kick: 'x...x...' } } }`: a tune with no audio file, played by a small built-in synthesiser (keys, pluck, pad, bass, lead, bell, music box, drums; chords by name, swing, fades, a ritardando). Deterministic, in the exported file at the same moment; an AI writes it as easily as a title.
 - **Audio-reactive visuals** — `audioEnvelope('music.mp3')` analyses loudness, bass / mid / treble and the beats per frame; `react()` turns them into keyframes, so the picture follows the music and still renders identically.
 - **2.5D layers & camera** — add `threeD: true`, `z`, `rotationX/Y` and a `{ type: 'camera' }` layer for parallax, flips and dolly zooms; no three.js needed.
+- **3D scenes** *(optional)* — a real 3D scene (meshes, glTF models, PBR, lights, shadows) as a layer via `pixi-effects/3d`, drawn by [PixiJS 3D](https://pixijs.com/3d) inside PixiJS's own renderer, so masks, filters, blend modes and opacity work on it like on any layer; keyframes drive its objects (`scene3d.cube.rotation.y`). It can live in the composition's space (`followCamera`, `followLights`, `followFog`), load and pose a glTF from the time (`loadModel`), show a 2D layer on a mesh (`layerTexture`) and be clicked into (`movie.pick`).
 - **three.js layer** *(optional)* — drop a real three.js scene in as a layer via `pixi-effects/three`; keyframes drive its objects (`three.cube.rotation.y`). It can live in the composition's space: the composition's camera, lights, shadows and fog drive the scene, so one `cameraPath()` flies through the flat layers and a glTF model together (`threeGltf`).
 - **Built-in player UI — or your own** — a drop-in HTML5-`<video>`-style overlay controller (play, scrub, mute, volume, fullscreen, export-to-file) whose colours are one option (`theme: { accent: '#ff4d6d' }`); or build a player yourself on the movie's `<video>`-named events (`play pause ended seeking seeked volumechange error`).
 - **MP4 / WebM / MOV export** — pick container and quality from the controller, call `movie.render()` from code, or render headless from the command line (`pixi-effects render`).
 - **A timeline you can read and scrub** — `movie.timelineChart()` draws every layer as a bar on a time axis; `pixi-effects view` opens the page with that timeline under it, zoomable, with a playhead you can drag.
-- **Tiny dependency surface** — only `mediabunny` (runtime) plus PixiJS (a peer dep). three.js is an *optional* peer, needed only if you import `pixi-effects/three`. mediabunny is fetched only when you export or use a video layer: a page that only plays never loads it.
+- **Tiny dependency surface** — only `mediabunny` (runtime) plus PixiJS (a peer dep). `@pixi/3d` and three.js are *optional* peers, needed only if you import `pixi-effects/3d` or `pixi-effects/three`. mediabunny is fetched only when you export or use a video layer: a page that only plays never loads it.
 
 ## Install
 
 ```bash
 npm install pixi-effects pixi.js
+npm install @pixi/3d     # optional — only if you import `pixi-effects/3d` (needs pixi.js 8.23+)
 npm install three        # optional — only if you import `pixi-effects/three`
 npm install pixi-filters # optional — only if a layer uses a named filter other than blur / noise / alpha / colorMatrix
 ```
@@ -49,8 +51,8 @@ Drop the imports into an [importmap](https://developer.mozilla.org/docs/Web/HTML
   "imports": {
     "pixi.js":                 "https://esm.sh/pixi.js@8.22.0?bundle-deps",
     "mediabunny":              "https://esm.sh/mediabunny",
-    "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.48.2/dist/index.js",
-    "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.48.2/dist/Controller.js"
+    "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/index.js",
+    "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/Controller.js"
   }
 }
 </script>
@@ -63,12 +65,54 @@ Drop the imports into an [importmap](https://developer.mozilla.org/docs/Web/HTML
 
 The `mediabunny` line is needed only to export (the download button, `movie.render()`) or for a `video` layer; a page that only plays can leave it out and never fetches it (or call `useMediabunny(url)` instead, see [the API](docs/api.md#where-the-encoder-comes-from-usemediabunnysource)).
 
-Load the `dist/` files **as they are** (jsDelivr's `/npm/…/dist/…`, or unpkg's `https://unpkg.com/pixi-effects@0.48.2/dist/index.js`) rather than a CDN-rebundled build such as `esm.sh/pixi-effects` or jsDelivr's `+esm`: the entries (`pixi-effects`, `…/controller`, `…/three`) share internal chunks, which only works when each file is served untouched.
+Load the `dist/` files **as they are** (jsDelivr's `/npm/…/dist/…`, or unpkg's `https://unpkg.com/pixi-effects@0.49.0/dist/index.js`) rather than a CDN-rebundled build such as `esm.sh/pixi-effects` or jsDelivr's `+esm`: the entries (`pixi-effects`, `…/controller`, `…/three`) share internal chunks, which only works when each file is served untouched.
 
-> **Using three.js?** Add two more entries (`three` and `pixi-effects/three`) to this importmap — see [Adding three.js](#adding-threejs-optional) below.
+> **A 3D scene?** Add three more entries (`pixi-effects/3d`, `@pixi/3d`, `@pixi/3d/extras`) — see [Adding a 3D scene](#adding-a-3d-scene-optional) below. **Using three.js?** Add two more entries (`three` and `pixi-effects/three`) to this importmap — see [Adding three.js](#adding-threejs-optional) below.
 >
 > **Want the unreleased `main`?** Point the `pixi-effects*` entries at the GitHub Pages build instead: `https://yjmtmtk.github.io/pixi-effects/dist/index.js` (and `Controller.js`, `three.js`).
 
+#### Adding a 3D scene (optional)
+
+Add three entries to the importmap above (PixiJS 3D needs `pixi.js` 8.23.0 or later):
+
+```html
+<script type="importmap">
+{
+  "imports": {
+    "...":                     "(everything from the importmap above)",
+    "pixi-effects/3d":         "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/3d.js",
+    "@pixi/3d":                "https://cdn.jsdelivr.net/npm/@pixi/3d@0.1.0/dist/pixijs-3d.min.mjs",
+    "@pixi/3d/extras":         "https://cdn.jsdelivr.net/npm/@pixi/3d@0.1.0/dist/pixijs-3d-extras.min.mjs"
+  }
+}
+</script>
+```
+
+```html
+<script type="module">
+  import { Movie } from 'pixi-effects';
+  import { register3D, scene3d } from 'pixi-effects/3d';         // BEFORE movie.init()
+  import { Mesh3D, TorusKnotGeometry, Material3D, DirectionalLight } from '@pixi/3d';
+
+  register3D();
+
+  // ...inside composition.sequences:
+  scene3d({
+    type: 'scene3d', name: 'knot',
+    setup: ({ root, camera }) => {
+      const knot = new Mesh3D({ geometry: new TorusKnotGeometry(), material: new Material3D({ baseColor: 0x7fb4ff, roughness: 0.4 }) });
+      const sun = new DirectionalLight({ intensity: 3 }); sun.rotation.set(-0.8, 0.5, 0);
+      root.addChild(knot, sun);
+      camera.position.set(0, 0, 6); camera.lookAt({ x: 0, y: 0, z: 0 });
+      return { objects: { knot } };        // addressable from keyframes
+    },
+    keyframes: [{ at: 0, to: { 'scene3d.knot.rotation.y': Math.PI * 2 }, duration: 6 }],
+  })
+</script>
+```
+
+The scene is drawn inside PixiJS's renderer in the layer's place, so masks, filters, blend modes and opacity apply. `update(t)` must set everything from `t`. Full spec: [DSL reference § scene3d](./docs/dsl.md#scene3d) and [API reference](./docs/api.md#pixi-effects3d).
+
 #### Adding three.js (optional)
 
 Add two more entries to the importmap above: three.js itself, and the `pixi-effects/three` entry (and `three/addons/` if you load a glTF model with `threeGltf`).
@@ -80,7 +124,7 @@ Add two more entries to the importmap above: three.js itself, and the `pixi-effe
     "...":                     "(everything from the importmap above)",
     "three":                   "https://esm.sh/three@0.178.0",
     "three/addons/":           "https://cdn.jsdelivr.net/npm/three@0.178.0/examples/jsm/",
-    "pixi-effects/three":      "https://cdn.jsdelivr.net/npm/pixi-effects@0.48.2/dist/three.js"
+    "pixi-effects/three":      "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/three.js"
   }
 }
 </script>

package/ai/chat-template.html

@@ -57,11 +57,14 @@
 <script type="importmap">
   {
     "imports": {
-      "pixi.js":                 "https://cdn.jsdelivr.net/npm/pixi.js@8.22.0/dist/pixi.min.mjs",
+      "pixi.js":                 "https://cdn.jsdelivr.net/npm/pixi.js@8.23.0/dist/pixi.min.mjs",
       "mediabunny":              "https://cdn.jsdelivr.net/npm/mediabunny@1.61.3/+esm",
-      "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.48.2/dist/index.js",
-      "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.48.2/dist/Controller.js",
-      "pixi-effects/present":  "https://cdn.jsdelivr.net/npm/pixi-effects@0.48.2/dist/present.js"
+      "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/index.js",
+      "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/Controller.js",
+      "pixi-effects/present":  "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/present.js",
+      "pixi-effects/3d":         "https://cdn.jsdelivr.net/npm/pixi-effects@0.49.0/dist/3d.js",
+      "@pixi/3d":                "https://cdn.jsdelivr.net/npm/@pixi/3d@0.1.0/dist/pixijs-3d.min.mjs",
+      "@pixi/3d/extras":         "https://cdn.jsdelivr.net/npm/@pixi/3d@0.1.0/dist/pixijs-3d-extras.min.mjs"
     }
   }
   </script>
@@ -70,6 +73,7 @@
     import { Movie, kenBurns, withFade, wiggle, stagger, animateText, followPath, particles } from 'pixi-effects';
     import { Controller } from 'pixi-effects/controller';
     // A talk instead of a video: import { deck, Presenter } from 'pixi-effects/present'; use deck({ pages: [...] }) and new Presenter(movie, { canvas }) instead of the Controller.
+    // A 3D scene as a layer (inside the edit block, before init): const { register3D } = await import('pixi-effects/3d'); const { Mesh3D, CubeGeometry, Material3D, DirectionalLight } = await import('@pixi/3d'); register3D(); then { type: 'scene3d', setup, update }.
 
     // ===================== EDIT FROM HERE =====================
     const W = 1280, H = 720, FPS = 30, DURATION = 6;

package/ai/tools/check.mjs

@@ -520,7 +520,7 @@ export async function runCheck(opts, log = console.log) {
     const layoutProblems = rv.problems.filter(g => g.kind !== 'canvas'), canvasProblems = rv.problems.filter(g => g.kind === 'canvas');
     report.inspect = { checkedFrames: rv.frames, issues: layoutProblems, review: rv.review, canvas: canvasProblems.map(g => g.message), canvasLayers: rv.canvas ?? [] };
     if (layoutProblems.length) report.problems.push(`${layoutProblems.length} kind(s) of layout issue from movie.inspect`);
-    if (canvasProblems.length) report.problems.push(`${canvasProblems.length} layer(s) (canvas / three) that do not draw the same picture for the same time (movie.canvasProblems)`);
+    if (canvasProblems.length) report.problems.push(`${canvasProblems.length} layer(s) (canvas / three / scene3d) that do not draw the same picture for the same time (movie.canvasProblems)`);
     // fonts: a web font whose file did not load is a problem (the library lists a layer with no available font for review)
     report.fonts = rv.fonts;
     if (rv.fonts.failed.length) report.problems.push(`${rv.fonts.failed.length} web font(s) failed to load: ${rv.fonts.failed.join(', ')} (the layers that use them are drawn in a fallback)`);
@@ -553,7 +553,7 @@ export async function runCheck(opts, log = console.log) {
         if (rows.some(([, px]) => px > 0)) w.depthBlur = Object.fromEntries(rows);
         // pick: what is under each --pick point at this moment (the frame is on screen after the snapshot above)
         if (opts.pick.length) {
-          w.pick = await cdp.eval(`(${JSON.stringify(opts.pick)}).map(p => ({ x: p.x, y: p.y, hits: movie.pickAll(p.x, p.y).map(h => ({ name: h.name ?? h.path, path: h.path, kind: h.kind, object: h.three ? h.three.object : undefined })) }))`).catch(() => []);
+          w.pick = await cdp.eval(`(${JSON.stringify(opts.pick)}).map(p => ({ x: p.x, y: p.y, hits: movie.pickAll(p.x, p.y).map(h => ({ name: h.name ?? h.path, path: h.path, kind: h.kind, object: h.three ? h.three.object : h.scene3d ? h.scene3d.object : undefined })) }))`).catch(() => []);
         }
         // light: how lit each threeD layer is at this moment (left out when the composition has no light)
         const lit = await cdp.eval(`movie.inspect(${w.frame}).then(r => r.layers.filter(l => l.light !== undefined).map(l => [l.path, l.light]))`).catch(() => []);
@@ -895,7 +895,7 @@ export async function runSteps(cdp, steps, outDir, size, consoleLines, seenLogs)
     await sleep(300);                                                       // the page's handlers and rebuilds settle
     const after = await cdp.eval('JSON.parse(JSON.stringify(movie.vars || {}))').catch(() => ({}));
     res.vars = Object.fromEntries(Object.keys({ ...before, ...after }).filter(k => JSON.stringify(before[k]) !== JSON.stringify(after[k])).map(k => [k, [before[k], after[k]]]));
-    if (last) res.hits = await cdp.eval(`movie.pickAll(${last.x}, ${last.y}).map(h => h.kind === 'three' ? h.path + ' (three: ' + (h.three && h.three.object || '?') + ')' : h.path)`).catch(() => []);
+    if (last) res.hits = await cdp.eval(`movie.pickAll(${last.x}, ${last.y}).map(h => h.kind === 'three' ? h.path + ' (three: ' + (h.three && h.three.object || '?') + ')' : h.kind === 'scene3d' ? h.path + ' (scene3d: ' + (h.scene3d && h.scene3d.object || '?') + ')' : h.path)`).catch(() => []);
     res.frame = await cdp.eval('movie.currentFrame').catch(() => null);
     res.playing = await cdp.eval('!!movie.isPlaying').catch(() => null);
     const now = await pageLogs();
@@ -930,8 +930,8 @@ function finish(report, outDir, log) {
     if (g.length > 8) L.push(`            … ${g.length - 8} more (report.json)`);
     const cv = report.inspect.canvas ?? [];
     const rows = report.inspect.canvasLayers ?? [];
-    if (cv.length) { L.push(`  canvas    ${cv.length} layer(s) (canvas / three) that do not draw the same picture for the same time:`); for (const m of cv.slice(0, 4)) L.push(`            - ${m}`); }
-    for (const kind of ['canvas', 'three']) {                       // silence must not be the only sign that the check ran
+    if (cv.length) { L.push(`  canvas    ${cv.length} layer(s) (canvas / three / scene3d) that do not draw the same picture for the same time:`); for (const m of cv.slice(0, 4)) L.push(`            - ${m}`); }
+    for (const kind of ['canvas', 'three', 'scene3d']) {                       // silence must not be the only sign that the check ran
       const mine = rows.filter(r => (r.kind ?? 'canvas') === kind);
       if (!mine.length) continue;
       const good = mine.filter(r => r.ok), slow = [...mine].sort((a, b) => b.drawMs - a.drawMs)[0];
@@ -952,7 +952,7 @@ function finish(report, outDir, log) {
   }
   for (const w of (report.at ?? []).filter(x => x.pick)) {
     for (const p of w.pick) {
-      const says = p.hits.length ? p.hits.map(h => h.kind === 'three' ? `${h.name} (three: ${h.object || '?'})` : h.name).join(', ') : 'nothing';
+      const says = p.hits.length ? p.hits.map(h => h.kind === 'three' || h.kind === 'scene3d' ? `${h.name} (${h.kind}: ${h.object || '?'})` : h.name).join(', ') : 'nothing';
       L.push(`  pick      ${w.label} (${p.x}, ${p.y}) — ${says}`);
     }
   }