@relu-interactives/spatial-ecs 0.5.3 → 0.9.0

package/dist/docs/guide/webgpu-and-tsl.md

@@ -0,0 +1,415 @@
+# WebGPU and TSL
+
+Since 0.6.0 a `World` can render with three's `WebGPURenderer` as well as the classic `WebGLRenderer`. With WebGPU you write materials and post effects in **TSL** (three's node-based shading language, `three/tsl`) instead of GLSL strings.
+
+## Choosing a renderer
+
+| | `WebGLRenderer` (default) | `WebGPURenderer` + `webgpuBackend` |
+| --- | --- | --- |
+| Import | `three` | `three/webgpu` + `@relu-interactives/spatial-ecs/webgpu` |
+| Custom shaders | GLSL (`ShaderMaterial`, `onBeforeCompile`) | TSL node materials |
+| Post-processing | `EffectComposer` passes | `RenderPipeline` of TSL nodes |
+| Browser support | every WebGL 2 browser | WebGPU where available, **WebGL 2 fallback otherwise** (automatic) |
+| XR (VR / mixed reality) | ✅ | ❌ use WebGL |
+| Image tracking (AR) | ✅ | ❌ use WebGL (the tracking engine owns a WebGL renderer) |
+
+- `WebGPURenderer` falls back to WebGL 2 by itself when the browser has no WebGPU. TSL and the WebGPU backend work on both, so you do not need a second code path for older browsers. `renderer.backend.isWebGPUBackend` says which one you got.
+- XR and image-tracking projects stay on `WebGLRenderer`.
+- Check the renderer from a script with `this.world.getRendererKind()`, which returns `"webgl"` or `"webgpu"`. The WebGL 2 fallback still reports `"webgpu"`: it is still a `WebGPURenderer`, and TSL still works.
+
+```ts
+import { Behaviour } from "@relu-interactives/spatial-ecs";
+
+export default class RendererInfo extends Behaviour {
+  init() {
+    if (this.world.getRendererKind() === "webgpu") {
+      console.log("TSL materials and the TSL post stack are available");
+    } else {
+      console.log("WebGL: use GLSL ShaderMaterial and EffectComposer");
+    }
+  }
+}
+```
+
+## Constructing a World on WebGPU
+
+Pass the renderer and `webgpuBackend`, and `await renderer.init()` before the first `world.update()`. Until the renderer is initialised, `RenderSystem` skips frames and warns once instead of throwing.
+
+```ts
+import * as THREE from "three";
+import { WebGPURenderer } from "three/webgpu";
+import { World } from "@relu-interactives/spatial-ecs";
+import { webgpuBackend } from "@relu-interactives/spatial-ecs/webgpu";
+
+const renderer = new WebGPURenderer({ antialias: true });
+renderer.setSize(window.innerWidth, window.innerHeight);
+document.body.appendChild(renderer.domElement);
+await renderer.init();
+
+const scene = new THREE.Scene();
+const camera = new THREE.PerspectiveCamera(60, window.innerWidth / window.innerHeight, 0.1, 200);
+
+const world = new World({ scene, camera, renderer, backend: webgpuBackend });
+world.setup({ systems: [] });
+
+let last = performance.now();
+renderer.setAnimationLoop((now: number) => {
+  world.update((now - last) / 1000);
+  last = now;
+});
+```
+
+`webgpuBackend` lives in its own entry point (`@relu-interactives/spatial-ecs/webgpu`) so WebGL apps never load `three/webgpu`. It is ESM only.
+
+A `WebGPURenderer` without `webgpuBackend` still renders, but the world warns once and turns off post-processing, green-screen keying and area lights, because their WebGL versions cannot run there.
+
+## What changes on WebGPU
+
+- **No GLSL.** `ShaderMaterial`, `RawShaderMaterial` and `material.onBeforeCompile` are WebGL only. `WebGPURenderer` logs that the material is not compatible and draws a blank material instead. Write a node material (`MeshStandardNodeMaterial`, `MeshBasicNodeMaterial`, ...) with TSL.
+- **No `EffectComposer`.** `three/examples/jsm/postprocessing/*` passes are WebGL only. Post effects are TSL nodes in a `RenderPipeline` (see [Custom post effect](#custom-post-effect)).
+- **Built-in materials still work.** `MeshStandardMaterial` and friends are converted to their node versions automatically.
+- **No `renderer.capabilities`.** Ask `renderer.getMaxAnisotropy()` on WebGPU.
+- **Environment maps** are prefiltered by the renderer: `EnvironmentSyncSystem` assigns the equirect texture to `scene.environment` directly instead of running the WebGL `PMREMGenerator`.
+- **Video green screen** uses a TSL material with the same keying maths as the WebGL shader.
+
+### Post-processing effects per backend
+
+The `Postprocessing` component is the same on both renderers. The WebGPU backend maps each effect to the closest TSL node:
+
+| Effect | WebGL (`EffectComposer`) | WebGPU (`RenderPipeline`) |
+| --- | --- | --- |
+| Depth of field | `BokehPass` | `dof` (`DepthOfFieldNode`); `focusDistance` / `focalLength` in world units |
+| Bloom | `UnrealBloomPass` | `bloom` (`BloomNode`), added to the image |
+| Vignette | `VignetteShader` | same formula as a TSL function |
+| Hue / saturation | `HueSaturationShader` | `hue()` + `saturation()` with matching curves |
+| Sepia | `SepiaShader` | `sepia` |
+| Film noise | `FilmPass` | `film` (`FilmNode`) |
+| Dot screen | `DotScreenPass` | `dotScreen` (`DotScreenNode`) |
+| SSAO | `SSAOPass` | ≈ GTAO (`ao`, `GTAONode`): `radius` → radius, `intensity` → occlusion exponent; normals come from depth |
+| Chromatic aberration | `RGBShiftShader` | `rgbShift` (`RGBShiftNode`) |
+| Pixelation | custom pass | TSL block sampling |
+| SMAA | `SMAAPass` | `smaa` (`SMAANode`) on tone-mapped sRGB colour |
+| Tone mapping | renderer + `OutputPass` | renderer + `RenderPipeline` output transform |
+| Glitch | `GlitchPass` | ❌ **not available**; skipped with one warning |
+
+On WebGPU, changing an effect's parameters updates its uniforms in place. Switching an effect on or off rebuilds the chain.
+
+With SSAO or depth of field on, the scene pass renders without MSAA: those effects read the depth buffer, and WGSL can't size a multisampled depth texture. Turn on SMAA to smooth edges in those scenes.
+
+## TSL in Script Objects
+
+A [Script Object](./authoring-a-script-object) builds its visual in code, so it is where TSL materials go. Imports come from `three/tsl` (functions such as `uniform`, `time`, `sin`) and `three/webgpu` (node material classes).
+
+> [!IMPORTANT]
+> Node materials need a `WebGPURenderer`. A `WebGLRenderer` cannot draw them. A Script Object that may run on either renderer should check `this.world.getRendererKind()` in `build()` and return a plain material on WebGL, as the first example does.
+
+TSL values that change at runtime are `uniform(...)` nodes: write `uniform.value` in `update()` and the GPU picks it up. Do not rebuild the material per frame.
+
+### Animated emissive pulse
+
+```ts
+import { ScriptObject, type ColorInput, type FloatInput } from "@relu-interactives/spatial-ecs";
+import * as THREE from "three";
+import { MeshStandardNodeMaterial } from "three/webgpu";
+import { sin, time, uniform } from "three/tsl";
+
+export default class PulseOrb extends ScriptObject {
+  data = { color: "#ff6b35" as ColorInput, speed: 2 as FloatInput };
+
+  private readonly glowColor = uniform(new THREE.Color());
+  private readonly speed = uniform(2);
+  private material: THREE.Material | null = null;
+
+  build(): THREE.Object3D {
+    const geometry = new THREE.SphereGeometry(0.5, 48, 32);
+
+    if (this.world.getRendererKind() !== "webgpu") {
+      // WebGL cannot draw node materials: show a static stand-in.
+      this.material = new THREE.MeshStandardMaterial({
+        color: 0x222222,
+        emissive: new THREE.Color(this.data.color),
+      });
+      return new THREE.Mesh(geometry, this.material);
+    }
+
+    const material = new MeshStandardNodeMaterial({ color: 0x222222 });
+    // 0..1, once per 2π / speed seconds.
+    const pulse = sin(time.mul(this.speed)).mul(0.5).add(0.5);
+    material.emissiveNode = this.glowColor.rgb.mul(pulse);
+    this.material = material;
+    this.update();
+    return new THREE.Mesh(geometry, material);
+  }
+
+  update(): void {
+    // Uniform writes are cheap: no shader rebuild.
+    this.glowColor.value.set(this.data.color);
+    this.speed.value = this.data.speed;
+  }
+
+  dispose(): void {
+    this.material?.dispose();
+  }
+}
+```
+
+### Vertex displacement with `positionLocal` and `time`
+
+`material.positionNode` replaces the vertex position. Offsetting along `normalLocal` keeps the wave on the surface.
+
+```ts
+import { ScriptObject, type FloatInput } from "@relu-interactives/spatial-ecs";
+import * as THREE from "three";
+import { MeshStandardNodeMaterial } from "three/webgpu";
+import { normalLocal, positionLocal, sin, time, uniform } from "three/tsl";
+
+export default class WobbleBlob extends ScriptObject {
+  data = { amplitude: 0.08 as FloatInput, frequency: 6 as FloatInput };
+
+  private readonly amplitude = uniform(0.08);
+  private readonly frequency = uniform(6);
+  private material: MeshStandardNodeMaterial | null = null;
+
+  build(): THREE.Object3D {
+    const material = new MeshStandardNodeMaterial({ color: 0x66ccff, roughness: 0.35 });
+    const wave = sin(positionLocal.y.mul(this.frequency).add(time.mul(3)));
+    material.positionNode = positionLocal.add(normalLocal.mul(wave.mul(this.amplitude)));
+    this.material = material;
+    this.update();
+    return new THREE.Mesh(new THREE.IcosahedronGeometry(0.5, 32), material);
+  }
+
+  update(): void {
+    this.amplitude.value = this.data.amplitude;
+    this.frequency.value = this.data.frequency;
+  }
+
+  dispose(): void {
+    this.material?.dispose();
+  }
+}
+```
+
+### Fresnel rim light
+
+Brightens the silhouette where the surface turns away from the camera.
+
+```ts
+import { ScriptObject, type ColorInput } from "@relu-interactives/spatial-ecs";
+import * as THREE from "three";
+import { MeshStandardNodeMaterial } from "three/webgpu";
+import {
+  cameraPosition,
+  dot,
+  normalWorld,
+  oneMinus,
+  positionWorld,
+  pow,
+  saturate,
+  uniform,
+} from "three/tsl";
+
+export default class RimShell extends ScriptObject {
+  data = { rimColor: "#7fd8ff" as ColorInput };
+
+  private readonly rimColor = uniform(new THREE.Color());
+  private material: MeshStandardNodeMaterial | null = null;
+
+  build(): THREE.Object3D {
+    const material = new MeshStandardNodeMaterial({ color: 0x101820, metalness: 0.2 });
+    const viewDirection = cameraPosition.sub(positionWorld).normalize();
+    const facing = saturate(dot(normalWorld, viewDirection));
+    const rim = pow(oneMinus(facing), 3);
+    material.emissiveNode = this.rimColor.rgb.mul(rim);
+    this.material = material;
+    this.update();
+    return new THREE.Mesh(new THREE.TorusKnotGeometry(0.35, 0.12, 160, 24), material);
+  }
+
+  update(): void {
+    this.rimColor.value.set(this.data.rimColor);
+  }
+
+  dispose(): void {
+    this.material?.dispose();
+  }
+}
+```
+
+### Dissolve with noise
+
+`Discard()` throws away fragments below the threshold; a thin band just above it glows.
+
+```ts
+import { ScriptObject, type ColorInput, type FloatInput } from "@relu-interactives/spatial-ecs";
+import * as THREE from "three";
+import { MeshStandardNodeMaterial } from "three/webgpu";
+import {
+  Discard,
+  Fn,
+  If,
+  mx_noise_float,
+  oneMinus,
+  positionLocal,
+  smoothstep,
+  uniform,
+} from "three/tsl";
+
+export default class DissolveCube extends ScriptObject {
+  data = {
+    baseColor: "#8899aa" as ColorInput,
+    edgeColor: "#ffb347" as ColorInput,
+    speed: 0.25 as FloatInput,
+  };
+
+  private readonly threshold = uniform(0);
+  private readonly baseColor = uniform(new THREE.Color());
+  private readonly edgeColor = uniform(new THREE.Color());
+  private material: MeshStandardNodeMaterial | null = null;
+  private elapsed = 0;
+
+  build(): THREE.Object3D {
+    const material = new MeshStandardNodeMaterial();
+    // mx_noise_float returns roughly -1..1; remap to 0..1.
+    const noise = mx_noise_float(positionLocal.mul(4)).mul(0.5).add(0.5);
+
+    material.colorNode = Fn(() => {
+      If(noise.lessThan(this.threshold), () => {
+        Discard();
+      });
+      return this.baseColor.rgb;
+    })();
+    const band = oneMinus(smoothstep(this.threshold, this.threshold.add(0.06), noise));
+    material.emissiveNode = this.edgeColor.rgb.mul(band.mul(4));
+
+    this.material = material;
+    this.update(0);
+    return new THREE.Mesh(new THREE.BoxGeometry(1, 1, 1), material);
+  }
+
+  update(deltaTime: number): void {
+    // Sweep the threshold 0 → 1 and start over.
+    this.elapsed = (this.elapsed + deltaTime * this.data.speed) % 1;
+    this.threshold.value = this.elapsed;
+    this.baseColor.value.set(this.data.baseColor);
+    this.edgeColor.value.set(this.data.edgeColor);
+  }
+
+  dispose(): void {
+    this.material?.dispose();
+  }
+}
+```
+
+### Scrolling UVs
+
+Offset the UV by `time` to scroll a texture. The texture needs `RepeatWrapping` so it tiles.
+
+```ts
+import { ScriptObject, type FloatInput } from "@relu-interactives/spatial-ecs";
+import * as THREE from "three";
+import { MeshBasicNodeMaterial } from "three/webgpu";
+import { texture, time, uniform, uv, vec2 } from "three/tsl";
+
+export default class ScrollingStripes extends ScriptObject {
+  data = { speed: 0.2 as FloatInput };
+
+  private readonly speed = uniform(0.2);
+  private map: THREE.DataTexture | null = null;
+  private material: MeshBasicNodeMaterial | null = null;
+
+  build(): THREE.Object3D {
+    // 2×1 stripe texture made in code; any loaded texture works the same way.
+    const pixels = new Uint8Array([255, 107, 53, 255, 20, 20, 30, 255]);
+    const map = new THREE.DataTexture(pixels, 2, 1);
+    map.wrapS = THREE.RepeatWrapping;
+    map.wrapT = THREE.RepeatWrapping;
+    map.colorSpace = THREE.SRGBColorSpace;
+    map.needsUpdate = true;
+
+    const material = new MeshBasicNodeMaterial();
+    const offset = vec2(time.mul(this.speed), 0);
+    material.colorNode = texture(map, uv().mul(4).add(offset));
+
+    this.map = map;
+    this.material = material;
+    this.update();
+    return new THREE.Mesh(new THREE.PlaneGeometry(2, 1), material);
+  }
+
+  update(): void {
+    this.speed.value = this.data.speed;
+  }
+
+  dispose(): void {
+    this.material?.dispose();
+    this.map?.dispose();
+  }
+}
+```
+
+## Custom post effect
+
+To run your own post chain, build a `RenderPipeline` and hand it to `RenderSystem` through `scene.userData`, the same slot `PostprocessingSyncSystem` uses:
+
+- `scene.userData.ecsPostprocessingComposer`: any object with `render()` (a `RenderPipeline` qualifies; `setSize()` and `setPixelRatio()` are called only if present).
+- `scene.userData.ecsPostprocessingComposerActive`: `true` while it should render.
+
+Set both **every frame**. Scripts run before `RenderSystem`, and `PostprocessingSyncSystem` rewrites both fields whenever the scene's `Postprocessing` component changes.
+
+```ts
+import { Behaviour, type FloatInput } from "@relu-interactives/spatial-ecs";
+import { RenderPipeline, type WebGPURenderer } from "three/webgpu";
+import { float, mix, pass, screenSize, screenUV, sin, uniform, vec4 } from "three/tsl";
+
+export default class ScanlinePost extends Behaviour {
+  data = { strength: 0.3 as FloatInput };
+
+  private pipeline: RenderPipeline | null = null;
+  private readonly strength = uniform(0.3);
+
+  init() {
+    // RenderPipeline needs a WebGPURenderer. On WebGL use EffectComposer instead.
+    if (this.world.getRendererKind() !== "webgpu") return;
+
+    const renderer = this.world.getRenderer() as WebGPURenderer;
+    const scenePass = pass(this.world.getScene(), this.world.getCamera());
+    const color = scenePass.getTextureNode("output");
+
+    // One dark line every two device pixels.
+    const lines = sin(screenUV.y.mul(screenSize.y).mul(Math.PI)).mul(0.5).add(0.5);
+    const shaded = color.rgb.mul(mix(float(1), lines, this.strength));
+
+    // The pipeline applies tone mapping and the output colour space itself.
+    this.pipeline = new RenderPipeline(renderer, vec4(shaded, color.a));
+  }
+
+  onUpdate() {
+    if (!this.pipeline) return;
+    this.strength.value = this.data.strength;
+
+    const userData = this.world.getScene().userData;
+    userData.ecsPostprocessingComposer = this.pipeline;
+    userData.ecsPostprocessingComposerActive = true;
+  }
+
+  dispose() {
+    if (!this.pipeline) return;
+    const userData = this.world.getScene().userData;
+    if (userData.ecsPostprocessingComposer === this.pipeline) {
+      delete userData.ecsPostprocessingComposer;
+      userData.ecsPostprocessingComposerActive = false;
+    }
+    this.pipeline.dispose();
+    this.pipeline = null;
+  }
+}
+```
+
+A custom pipeline replaces the scene's `Postprocessing` effects while it is installed. To combine them, chain the built-in effect nodes yourself (`three/examples/jsm/tsl/display/*`).
+
+## See also
+
+- [Postprocessing](./scene/postprocessing): the component and its per-backend support
+- [Script Objects](./authoring-a-script-object)
+- [`RenderSystem`](./systems/render-system) and [`PostprocessingSyncSystem`](./systems/postprocessing-sync-system)

package/dist/docs/guide/world-api.md

@@ -86,7 +86,9 @@ export default class Spinner extends Behaviour {
 | --- | --- |
 | `getScene()` | The live `THREE.Scene`. |
 | `getCamera()` | The active rendering camera. |
-| `getRenderer()` | The `THREE.WebGLRenderer`. |
+| `getRenderer()` | The renderer: a `THREE.WebGLRenderer`, or a `WebGPURenderer` from `three/webgpu` ([WebGPU and TSL](./webgpu-and-tsl)). Typed `EcsRenderer`; check `getRendererKind()` before using an API only one has (e.g. `capabilities` is WebGL only). |
+| `getRendererKind()` | `"webgl"` or `"webgpu"` (also `"webgpu"` on `WebGPURenderer`'s WebGL 2 fallback). Use it to pick GLSL vs TSL. |
+| `getRenderBackend()` | The `RenderBackend` in use (post-processing, green-screen video, area lights). `webglBackend` by default. |
 | `getPhysicsWorld()` | The Rapier `World` (or `null` if physics is disabled). |
 
 ### Raycasting
@@ -105,9 +107,17 @@ Both methods accept `Vec3` (`THREE.Vector3` or `{ x, y, z }`) and return `Inters
 | `getSystem(instance)` | Resolve the REGISTERED system matching the argument's class — pass a throwaway instance (`world.getSystem(new XRHitTestSystem())`), never the class itself. |
 | `hasSystem(SystemClass)` | `true` if the system is registered. |
 | `getObjectManagementSystem()` | Shorthand for the built-in `ObjectManagementSystem`. Use this to spawn or destroy entities at runtime (`requestCreate`, `createNow`, `deleteEntity`). |
-| `getEnvironmentComponent()` | Returns the singleton [`Environment`](./scene/environment) component, or `undefined` if not yet initialized. Mutate `values` to change skybox, fog, or gravity at runtime. |
+| `getEnvironmentComponent()` | Returns the singleton [`Environment`](./scene/environment) component, or `undefined` if not yet initialized. Mutate `values` to change skybox, fog, gravity, or HDRI lighting and tone mapping at runtime. |
 | `getPostprocessingComponent()` | Returns the singleton [`Postprocessing`](./scene/postprocessing) component, or `undefined` if not yet initialized. Mutate `values` to toggle effects or adjust parameters at runtime. |
 
+### Script errors
+
+| Method | Purpose |
+| --- | --- |
+| `onScriptError(listener)` | Called once per distinct Behaviour or ScriptObject error with `{ entityId, entityName, scriptUrl, className, phase, message, stack?, source }`. Returns an unsubscribe function. |
+| `reportScriptError(report, error)` | Used by systems that call Behaviour hooks. Logs and emits the error the first time it occurs; returns `true` when it was new. |
+| `clearScriptErrors(scriptUrl?)` | Forget reported errors (all, or one script's) so they report again, for example after a hot reload. |
+
 ### World lifecycle
 
 | Method | Purpose |
@@ -440,6 +450,46 @@ export default class PauseOnHit extends Behaviour {
 > stop a subset of entities (for example, freeze enemies while a menu is open)
 > use a flag in a custom script (`Behaviour`) instead and guard your `onUpdate` logic.
 
+### Catch script errors
+
+When a script throws, the engine logs the error once with `console.error`, naming the entity, class and phase, and sets the script's `runtimeState` to `"error"`. A Behaviour that throws waits about one second before it is re-initialised, so a script that throws every frame does not rebuild itself every frame.
+
+Subscribe with `onScriptError` to collect errors yourself, for example to show them in an in-game debug overlay:
+
+```ts
+import { Behaviour, type ScriptErrorEvent } from "@relu-interactives/spatial-ecs";
+
+export default class ErrorOverlay extends Behaviour {
+  private errors: ScriptErrorEvent[] = [];
+  private stop: (() => void) | null = null;
+
+  init() {
+    this.stop = this.world.onScriptError((event) => {
+      this.errors.push(event);
+      console.warn(
+        `${event.className} on "${event.entityName}" failed in ${event.phase}: ${event.message}`,
+      );
+    });
+  }
+
+  dispose() {
+    this.stop?.();
+  }
+}
+```
+
+`phase` tells you where the error came from:
+
+| `phase` | Meaning |
+| --- | --- |
+| `"load"` | The module failed to import, or it has no export with the entry's class name. |
+| `"init"` | The constructor or `init()` threw. |
+| `"build"` | A ScriptObject's `build()` threw or returned something that is not a `THREE.Object3D`. |
+| `"update"` | `onUpdate()` (Behaviour) or `update()` (ScriptObject) threw. |
+| hook name | A Behaviour hook threw: `"onSelect"`, `"onUiPress"`, `"onGrab"`, `"onXRSessionStart"`, and so on. |
+
+The listener receives each distinct error once. The engine treats two errors as the same when the script URL, class name, phase and message all match, so ten enemies running one broken script report a single event. `source` is `"behaviour"` or `"scriptObject"`, and `stack` carries the stack trace when the error had one.
+
 ## See also
 
 - [Custom Behaviours](./authoring-a-behaviour) — `init`, `onUpdate`, `dispose`, `data`, lifecycle.