three-realtime-rt 0.3.0 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,394 +1,391 @@
1
- # three-realtime-rt
2
-
3
- **Turn-on ray traced lighting for three.js.** Build your scene with ordinary
4
- three.js — meshes, `MeshStandardMaterial`, `PointLight` / `DirectionalLight` —
5
- then swap one render call and get BVH-traced **soft shadows**, **one-bounce
6
- global illumination**, **emissive-mesh area lights**, **mirror/glossy
7
- reflections**, **glass refraction**, **volumetric god rays** (BVH-shadowed
8
- single scatter, not a screen-space trick), a **procedural sky** that lights
9
- the scene, **ReSTIR many-light sampling** (flat cost in light count),
10
- **blue-noise sampling**, and real-time **temporal denoising + anti-aliasing**.
11
- Runs on plain WebGL2.
12
-
13
- The library ships as **untranspiled ES modules** (the `src/` folder) — it has no
14
- build step of its own, so you consume it through your bundler (Vite, webpack,
15
- esbuild, …) or a browser import map that resolves the bare `three` /
16
- `three-mesh-bvh` specifiers. MIT licensed. **Not on npm yet** — until the first
17
- release lands, install straight from the Git repo (see [Getting started](#getting-started)).
18
-
19
- ### ▶ [Live demo](https://goldwinxs.github.io/three-realtime-rt/) — drag to orbit, drop the pile, toggle every feature.
20
-
21
- > **Support this project:** the [supporter pack on itch.io](https://goldwinxs.itch.io/three-realtime-rt-supporter-pack) gets you a ready-to-run starter template, all example scenes, and a 12-section deep-dive guide to how the whole pipeline works. The library itself is and stays MIT.
22
-
23
- ![Ray traced room: emissive area light, reflections, glass, volumetric haze](docs/hero.png)
24
-
25
- Same scene, same camera, same lights — plain three.js (shadow maps + ACES) on
26
- the left, `rt.render` on the right:
27
-
28
- | Rasterized three.js | three-realtime-rt |
29
- |---|---|
30
- | ![raster](docs/compare-raster.jpg) | ![ray traced](docs/compare-rt.jpg) |
31
-
32
- ## Getting started
33
-
34
- Install the library plus its two **peer dependencies** — you bring your own copy
35
- of `three` and `three-mesh-bvh`:
36
-
37
- ```bash
38
- npm i three-realtime-rt three three-mesh-bvh
39
- ```
40
-
41
- > **Not published to npm yet.** Until the first release, install from the repo:
42
- > `npm i github:GoldwinXS/three-realtime-rt three three-mesh-bvh`
43
-
44
- No bundler? [`standalone.html`](standalone.html) is a single copy-paste file
45
- that runs the raytracer via CDN import maps — open it from any static server.
46
-
47
- A complete, copy-pasteable minimal app — a lit sphere on a floor, one point light:
48
-
49
- ```js
50
- import * as THREE from "three";
51
- import { RealtimeRaytracer } from "three-realtime-rt";
52
-
53
- // 1. An ordinary three.js renderer. Size it BEFORE constructing the raytracer —
54
- // it reads the drawing-buffer size at construction.
55
- const renderer = new THREE.WebGLRenderer({ antialias: false });
56
- renderer.setPixelRatio(window.devicePixelRatio);
57
- renderer.setSize(window.innerWidth, window.innerHeight);
58
- document.body.appendChild(renderer.domElement);
59
-
60
- // 2. An ordinary scene: meshes with MeshStandardMaterial + a real light.
61
- const scene = new THREE.Scene();
62
- const camera = new THREE.PerspectiveCamera(
63
- 60, window.innerWidth / window.innerHeight, 0.1, 100
64
- );
65
- camera.position.set(0, 2, 6);
66
-
67
- scene.add(new THREE.Mesh(
68
- new THREE.SphereGeometry(1, 48, 48),
69
- new THREE.MeshStandardMaterial({ color: 0xdddddd, roughness: 0.4, metalness: 0.0 })
70
- ));
71
- const floor = new THREE.Mesh(
72
- new THREE.PlaneGeometry(20, 20),
73
- new THREE.MeshStandardMaterial({ color: 0x808080, roughness: 1.0 })
74
- );
75
- floor.rotation.x = -Math.PI / 2;
76
- floor.position.y = -1;
77
- scene.add(floor);
78
-
79
- const light = new THREE.PointLight(0xffffff, 40); // PointLight / DirectionalLight (up to 16)
80
- light.position.set(3, 5, 2);
81
- scene.add(light);
82
-
83
- // 3. Turn on ray tracing.
84
- const rt = new RealtimeRaytracer(renderer);
85
- rt.compileScene(scene); // builds the BVH + material/light tables
86
-
87
- // 4. Resize: pass DRAWING-BUFFER (device) pixels, not CSS pixels.
88
- addEventListener("resize", () => {
89
- camera.aspect = window.innerWidth / window.innerHeight;
90
- camera.updateProjectionMatrix();
91
- renderer.setSize(window.innerWidth, window.innerHeight);
92
- const db = renderer.getDrawingBufferSize(new THREE.Vector2());
93
- rt.setSize(db.x, db.y);
94
- });
95
-
96
- // 5. Render loop — replace renderer.render(scene, camera) with rt.render.
97
- function loop() {
98
- requestAnimationFrame(loop);
99
- rt.render(scene, camera);
100
- }
101
- loop();
102
- ```
103
-
104
- On hardware that can't trace, `rt.render` transparently falls back to
105
- `renderer.render` — no capability branch needed (see [Running everywhere](#running-everywhere-capability-tiers)).
106
-
107
- ### Integrating into an existing app
108
-
109
- A checklist for dropping the tracer into a scene you already have:
110
-
111
- 1. **Swap the render call** — `renderer.render(scene, camera)`
112
- `rt.render(scene, camera)`. Construct the `RealtimeRaytracer` once, *after*
113
- the renderer has its final size.
114
- 2. **Compile once; recompile after structural changes** `rt.compileScene(scene)`
115
- bakes geometry into a static BVH and snapshots materials + emissive area
116
- lights. Call it again after you add/remove meshes, swap geometry, or change a
117
- material's `emissive` / `color` / `roughness` / `metalness`.
118
- 3. **Declare movers** pass moving meshes to
119
- `rt.compileScene(scene, { dynamicMeshes: [...] })`, then call
120
- `rt.updateDynamic()` each frame after you move them (e.g. after a physics
121
- step). Skip it on frames where nothing moved.
122
- 4. **Update lights when they change** — after moving, toggling (`.visible`),
123
- recolouring or dimming a light, call `rt.updateLights(scene)`. No recompile.
124
- 5. **Resize** — in your resize handler, after `renderer.setSize(...)`, call
125
- `rt.setSize(width, height)` with **drawing-buffer (device) pixels**
126
- (`renderer.getDrawingBufferSize(...)`).
127
- 6. **Reset after jumps** — call `rt.resetAccumulation()` after a camera teleport
128
- or a scene cut, so stale temporal history doesn't ghost.
129
-
130
- That's the whole integration. Everything below is optional.
131
-
132
- ---
133
-
134
- ## Why "hybrid deferred" (the "RTX on" model)
135
-
136
- Primary visibility is **rasterized** by three.js into a G-buffer — free, fast,
137
- and pixel-perfect on materials and textures. Only the *lighting* is ray traced,
138
- in a fragment shader, against a GPU BVH ([three-mesh-bvh]):
139
-
140
- 1. **G-buffer pass** MRT: albedo+roughness, world normal+metalness, world
141
- position, emissive.
142
- 2. **RT lighting pass** per pixel: soft shadow rays to each light (area
143
- sampled) + a 1-bounce cosine-weighted GI ray with next-event estimation. GI
144
- rays that escape sample the **procedural sky**, so the sky is a soft area
145
- light. **Emissive meshes are real area lights**: their triangles are sampled
146
- directly (NEE with the area→solid-angle pdf), so a glowing panel casts soft
147
- light and shadows instead of waiting for a lucky GI ray to hit it. Metallic
148
- pixels trace a **mirror/glossy reflection ray**; transmissive pixels trace a
149
- Fresnel-weighted **reflection + two-interface refraction**. Output is
150
- *demodulated irradiance* (albedo divided out) so it denoises cleanly while
151
- textures stay sharp.
152
- 3. **Temporal reprojection** — motion-validated history keeps samples alive as
153
- the camera and objects move.
154
- 4. **À-trous denoise** — an edge-avoiding (SVGF-lite) wavelet filter guided by
155
- the G-buffer, so 1 sample/pixel looks converged.
156
- 5. **Composite** `albedo × irradiance + emissive`, distance fog, ACES tonemap.
157
- 6. **TAA** — sub-pixel jitter + a neighbourhood-clamped history resolve:
158
- supersampled anti-aliasing that also clears disocclusion speckles. This is
159
- the analytic (FSR2 / TAAU) approach, not a learned upscaler.
160
-
161
- Lighting is traced at half resolution by default and reconstructed by a joint
162
- bilateral upsample + the denoiser + TAA — the same "render few pixels, rebuild
163
- temporally" idea DLSS uses, done with hand-written math.
164
-
165
- ## Moving objects (dynamic BVH)
166
-
167
- Mark meshes as dynamic and their motion casts **correct ray traced shadows** —
168
- the demo drops 40 rigid bodies (Rapier physics) that shadow each other and the
169
- ground in real time:
170
-
171
- ```js
172
- rt.compileScene(scene, { dynamicMeshes: crates }); // meshes that will move
173
-
174
- // each frame, after you move them (e.g. after a physics step):
175
- rt.updateDynamic(); // re-bakes them into the BVH (refit) — cheap
176
- rt.render(scene, camera);
177
- ```
178
-
179
- Under the hood this is a **two-level BVH**: static geometry lives in one BVH
180
- uploaded to the GPU once at compile time, dynamic meshes in a second small BVH
181
- that is re-baked and refit per frame. `updateDynamic()` therefore costs
182
- ~1 ms for dozens of moving objects *regardless of how big the static world is* —
183
- skip it entirely on frames where nothing moved.
184
-
185
- ## Live lighting & sky
186
-
187
- Lights can be toggled, moved, and recoloured every frame without recompiling:
188
-
189
- ```js
190
- warmLight.visible = false; // or change .color / .intensity / .position
191
- rt.updateLights(scene); // re-reads the scene's lights
192
- ```
193
-
194
- The procedural sky doubles as the ambient light source:
195
-
196
- ```js
197
- const rt = new RealtimeRaytracer(renderer, {
198
- sky: {
199
- enabled: true,
200
- sunDir: new THREE.Vector3(0.55, 0.62, 0.55).normalize(), // toward the sun
201
- sunColor: new THREE.Color(1.0, 0.92, 0.78),
202
- zenith: new THREE.Color(0.20, 0.40, 0.72),
203
- horizon: new THREE.Color(0.78, 0.85, 0.92),
204
- intensity: 1.0,
205
- },
206
- fog: { enabled: true, color: new THREE.Color(0.72, 0.8, 0.88), density: 0.03 },
207
- });
208
- ```
209
-
210
- ![Physics: 40 rigid bodies with dynamic ray traced shadows](docs/physics.png)
211
-
212
- ## What is and isn't supported
213
-
214
- Primary visibility is ordinary rasterized three.js, so **whatever three.js draws,
215
- you still see** transparency, blending, sorting, custom shaders and post effects
216
- behave exactly as normal. The ray tracer computes only the *lighting*, and it
217
- reads a deliberately small, fixed slice of the material and light model. This is
218
- the honest map of what actually feeds the traced lighting.
219
-
220
- ### Materials
221
-
222
- Lighting reads the **first material only** on a multi-material mesh, pulling the
223
- scalar fields of `MeshStandardMaterial` / `MeshPhysicalMaterial` (Basic / Lambert
224
- / Phong contribute whatever of those fields they have).
225
-
226
- | Property | Feeds lighting? | Notes |
227
- |----------|-----------------|-------|
228
- | `color` + `map` | ✅ | Albedo = `color × map.rgb`. Textures stay sharp (irradiance is demodulated, then re-multiplied). |
229
- | `roughness` | | **Scalar only.** Drives shadow / GI softness and reflection sharpness. |
230
- | `metalness` | ✅ | **Scalar only.** Metallic pixels trace a reflection ray. |
231
- | `emissive` | | A *static* emissive mesh becomes a real **area light** (NEE) casts soft light + shadows. |
232
- | `emissiveMap` | ⚠️ visible only | A map-masked emissive **glows on screen** but the map **zeroes its area-light table** — it lights nothing. Use a flat `emissive` colour (no map) for an emitter that should illuminate. |
233
- | `transmission` (Physical) | | Glass: Fresnel reflection + two-interface refraction. |
234
- | `roughnessMap` / `metalnessMap` | ❌ | Only the scalar `roughness` / `metalness` are used; these maps are ignored by lighting. |
235
- | `normalMap` | ❌ | Lighting uses geometric normals; normal maps don't perturb shading. |
236
- | `clearcoat`, `sheen`, `iridescence` | ❌ | Not modelled. |
237
- | vertex colors | ❌ | Not read into albedo. |
238
- | per-material `ior` | ❌ | Refraction uses the single **global** `rt.ior` (default 1.5), never `material.ior`. |
239
- | 2nd+ material of a group | ❌ | Only `material[0]` of a multi-material mesh reaches the G-buffer and BVH. |
240
-
241
- ### Lights
242
-
243
- | Light | Supported | Notes |
244
- |-------|-----------|-------|
245
- | `PointLight` | ✅ | `light.userData.rtRadius` (default `0.15`) sets soft-shadow size. |
246
- | `DirectionalLight` | | `light.userData.rtRadius` (default `0.02`) sets sun softness; keep its direction in sync with `sky.sunDir`. |
247
- | Emissive meshes | static | Sampled directly as area lights. **Dynamic** emitters are *not* in the NEE list they light only via GI-ray hits. |
248
- | `SpotLight` | ❌ | Ignored. |
249
- | `RectAreaLight` | | Use an emissive mesh instead. |
250
- | `HemisphereLight` / `AmbientLight` | | Ignored the procedural `sky` (or `envColor`) provides ambient. |
251
-
252
- - Up to **16** point/directional lights (`MAX_LIGHTS`); further lights are dropped.
253
- - Moving, toggling, recolouring or dimming a light → `rt.updateLights(scene)` (cheap, no recompile).
254
- - Changing a mesh's **emissive** (it's an area light baked at compile time) → `rt.compileScene(...)` again.
255
- - Emissive area lights are capped at **256 triangles** (largest by area kept, with a console warning) — prefer low-poly emitter meshes.
256
-
257
- ### Geometry & occlusion
258
-
259
- - Every non-excluded visible mesh is **merged into one static BVH at compile time**. Add / remove geometry → recompile.
260
- - Meshes that move must be declared via `dynamicMeshes` and driven with `updateDynamic()`. Anything not declared is treated as static moving it on screen won't move its traced shadow.
261
- - **Transparent materials never occlude** (by design — a glass case shouldn't cast an opaque shadow). They still rasterize normally.
262
- - **`alphaTest` cut-outs** (`transparent: false`) *do* occlude — but as **full triangles**, not per-texel, so their shadows are blocky.
263
- - `mesh.userData.rtExclude = true` removes a mesh from the BVH entirely (it still rasterizes and gets lit) — handy for water / translucent surfaces.
264
-
265
- ### Rendering model
266
-
267
- - **1-bounce GI + direct light.** No multi-bounce diffuse, no caustics, no specular-chain paths.
268
- - **Reflections** are a single traced bounce into a **diffuse-shaded** view of the world no recursive mirror-in-mirror; a metal shows its surroundings, not a second full render.
269
- - **Refraction** is two-interface (front + back) with a single global IOR — convincing glass, not a spectral / dispersive renderer.
270
- - **Volumetric** is **single-scatter** god rays, not multiple-scattering fog.
271
- - Because primary visibility is rasterized, **blending, sorting and transparency composite exactly like normal three.js** — the tracer never changes how your transparent surfaces layer.
272
-
273
- ### Platform
274
-
275
- - Requires **WebGL2 + `EXT_color_buffer_float`**. Software rasterizers (SwiftShader / llvmpipe) are treated as unsupported.
276
- - On anything unsupported, `rt.supported === false` and `rt.render()` **falls back to `renderer.render()`** after one console warning — your app still runs everywhere. Branch yourself with `rt.supported` or `RealtimeRaytracer.isSupported(renderer)` if you want.
277
- - WebGL2 only; no WebGPU backend (on the roadmap).
278
-
279
- ## Options
280
-
281
- | Option | Default | What |
282
- |--------|---------|------|
283
- | `renderScale` | `0.5` | Lighting resolution vs. the G-buffer. `1.0` = max quality. |
284
- | `taa` | `true` | Temporal anti-aliasing (jitter + neighbourhood clamp). |
285
- | `denoise` | `true` | Edge-aware à-trous denoiser. |
286
- | `gi` | `true` | 1-bounce global illumination (vs. direct-only). |
287
- | `emissiveNEE` | `true` | Sample static emissive meshes as area lights (next-event estimation). Off = emitters only light via lucky GI rays. |
288
- | `reflections` | `true` | Traced mirror/glossy reflections on metallic surfaces (sharpest at `renderScale: 1`). |
289
- | `refraction` | `true` | Traced two-interface refraction for `MeshPhysicalMaterial.transmission` surfaces. |
290
- | `restir` | `true` | ReSTIR direct lighting: per-pixel reservoirs with temporal + spatial reuse, one visibility ray regardless of light count. Flat cost in light count; cuts emissive area-light noise. |
291
- | `ior` | `1.5` | Index of refraction used by `refraction`. |
292
- | `volumetric` | *off* | Physically-based god rays: single-scatter fog, one BVH-shadowed light sample per lighting pixel per frame, temporally accumulated. `{ enabled, density, maxDist }`. |
293
- | `stochasticLights` | `false` | One direct shadow ray per pixel per frame (random source) instead of one per light. |
294
- | `temporalReprojection` | `true` | Keep samples across camera/object motion. |
295
- | `maxHistory` | `128` | History cap — higher is smoother, slower to react. |
296
- | `fireflyClamp` | `4.0` | Clamp on indirect luminance to suppress fireflies. |
297
- | `sky` | *off* | Procedural sky as background + GI ambient (see above). |
298
- | `fog` | *off* | Distance fog, composited before tonemap. |
299
-
300
- Per-light: set `light.userData.rtRadius` for soft-shadow size. Set
301
- `mesh.userData.rtExclude = true` to keep a mesh out of the BVH (it still
302
- rasterizes and gets lit — useful for water / translucent surfaces).
303
- Transparent materials never act as occluders (a glass case shouldn't cast an
304
- opaque shadow); `alphaTest` cut-outs still do.
305
-
306
- ## Running everywhere (capability tiers)
307
-
308
- Nothing adaptive is imposed everything below is **opt-in**:
309
-
310
- - **No usable GPU** (missing WebGL2 float targets, or a software rasterizer
311
- like SwiftShader): the library logs one console warning and `rt.render()`
312
- silently falls back to plain `renderer.render()`. Your app runs everywhere
313
- with zero capability checks; query `rt.supported` or the static
314
- `RealtimeRaytracer.isSupported(renderer)` if you want to branch yourself.
315
- - **Tier presets**: `RealtimeRaytracer.detectTier(renderer)` returns
316
- `"none" | "mid" | "high"`, and `recommendedOptions(tier)` gives matching
317
- constructor options spread them, then override what you like:
318
-
319
- ```js
320
- const tier = RealtimeRaytracer.detectTier(renderer);
321
- const rt = new RealtimeRaytracer(renderer, {
322
- ...RealtimeRaytracer.recommendedOptions(tier),
323
- targetFps: 55,
324
- });
325
- ```
326
-
327
- - **Adaptive quality** (`adaptiveQuality: true`, default **false**): continuous
328
- dynamic resolution scaling. Watches real frame time and steers the lighting
329
- resolution smoothly toward `targetFps` (in 5% steps with a cooldown), pairing
330
- low resolutions with MORE denoise passes (they run at lighting res, so
331
- they're nearly free exactly where they're needed) and stochastic direct
332
- light. While enabled it drives `renderScale` / `stochasticLights` /
333
- `denoiseIterations`; turn it off for manual control.
334
- - **`stochasticLights: true`** (default false): one direct shadow ray per
335
- pixel per frame instead of one per light the biggest ray-count lever for
336
- many-light scenes and mobile GPUs.
337
-
338
- WebGPU: not used as a backend (this is a WebGL2 library); a WGSL compute
339
- backend is on the roadmap.
340
-
341
- ## Running the demo
342
-
343
- ```bash
344
- npm install
345
- npm run dev # http://localhost:8115
346
- ```
347
-
348
- The demo ([`examples/`](examples/)) is an ordinary three.js app — see
349
- [`examples/main.js`](examples/main.js) for the full, commented integration
350
- (scene physics → compile → render loop). `npm run deploy` builds and publishes
351
- it to GitHub Pages.
352
-
353
- ## Gallery & benchmarks
354
-
355
- [`gallery.html`](gallery.html) drops the raytracer into **stock glTF scenes it
356
- was never authored for** LittlestTokyo, Lantern, Antique Camera, BoomBox,
357
- Corset, Water Bottle, and the Damaged Helmet streamed straight from their
358
- public hosts (no assets committed). A one-button toggle A/Bs ray tracing against
359
- plain rasterized three.js with an fps + triangle readout, and a compact options
360
- strip exposes GI / emissive NEE / reflections / refraction / ReSTIR / denoise /
361
- TAA / volumetric plus lighting-resolution and auto-quality controls.
362
-
363
- [`bench.html?autorun=1`](bench.html) runs a matrix of feature configs with
364
- GPU-**fence-timed** frame costs and a temporal **ghosting metric**, writing each
365
- run's results to [`bench-results/`](bench-results/) for tracking regressions.
366
-
367
- ## Roadmap
368
-
369
- | Stage | Status | What |
370
- |-------|--------|------|
371
- | 1. Core | ✅ | Scene→GPU sync, BVH, G-buffer, traced shadows + 1-bounce GI, accumulation |
372
- | 2. Reprojection | ✅ | Motion-validated history samples survive camera motion |
373
- | 3. Denoiser | ✅ | Edge-avoiding à-trous (SVGF-lite)clean 1spp |
374
- | 4. TAA | ✅ | Sub-pixel jitter + neighbourhood-clamped resolve AA, no speckles |
375
- | 4b. Sky | ✅ | Procedural sky as background + GI ambient light source |
376
- | 5. Two-level BVH | ✅ | Static BVH uploaded once; movers in a small per-frame BVH → dynamic shadows at ~1 ms |
377
- | 5b. Area lights | ✅ | Emissive meshes sampled directly (NEE)glowing panels cast soft light + shadows |
378
- | 6. Specular | | Mirror/glossy reflections on metals + two-interface glass refraction |
379
- | 6b. Sampling | ✅ | Blue-noise sampling + ReSTIR direct lighting (temporal + spatial reuse) |
380
- | 6c. Any-hit shadows | ✅ | Unordered early-out BVH traversal for occlusion rays — same image, up to ~2× cheaper shadows |
381
- | 7. Publish | — | npm publish; refractive water; per-material IOR |
382
-
383
- ## Credits
384
-
385
- - [three-mesh-bvh] by Garrett Johnson — the GPU BVH this is built on.
386
- - Inspired by [Erich Loftis'][erichlof] `THREE.js-PathTracing-Renderer`.
387
- - Demo models: Khronos glTF sample assets (Damaged Helmet, Duck).
388
-
389
- ## License
390
-
391
- MIT © Goldwin Stewart
392
-
393
- [three-mesh-bvh]: https://github.com/gkjohnson/three-mesh-bvh
394
- [erichlof]: https://github.com/erichlof/THREE.js-PathTracing-Renderer
1
+ # three-realtime-rt
2
+
3
+ **Turn-on ray traced lighting for three.js.** Build your scene with ordinary
4
+ three.js — meshes, `MeshStandardMaterial`, `PointLight` / `DirectionalLight` —
5
+ then swap one render call and get BVH-traced **soft shadows**, **one-bounce
6
+ global illumination**, **emissive-mesh area lights**, **mirror/glossy
7
+ reflections**, **glass refraction**, **volumetric god rays** (BVH-shadowed
8
+ single scatter, not a screen-space trick), a **procedural sky** that lights
9
+ the scene, **ReSTIR many-light sampling** (flat cost in light count),
10
+ **blue-noise sampling**, and real-time **temporal denoising + anti-aliasing**.
11
+ Runs on plain WebGL2.
12
+
13
+ The library ships as **untranspiled ES modules** (the `src/` folder) — it has no
14
+ build step of its own, so you consume it through your bundler (Vite, webpack,
15
+ esbuild, …) or a browser import map that resolves the bare `three` /
16
+ `three-mesh-bvh` specifiers. MIT licensed.
17
+ [On npm](https://www.npmjs.com/package/three-realtime-rt): `npm i three-realtime-rt three three-mesh-bvh`.
18
+
19
+ ### ▶ [Live demo](https://goldwinxs.github.io/three-realtime-rt/) — drag to orbit, drop the pile, toggle every feature.
20
+
21
+ > **Support this project:** the [supporter pack on itch.io](https://goldwinxs.itch.io/three-realtime-rt-supporter-pack) gets you a ready-to-run starter template, all example scenes, and a 12-section deep-dive guide to how the whole pipeline works. The library itself is and stays MIT.
22
+
23
+ ![Ray traced room: emissive area light, reflections, glass, volumetric haze](docs/hero.png)
24
+
25
+ Same scene, same camera, same lights — plain three.js (shadow maps + ACES) on
26
+ the left, `rt.render` on the right:
27
+
28
+ | Rasterized three.js | three-realtime-rt |
29
+ |---|---|
30
+ | ![raster](docs/compare-raster.jpg) | ![ray traced](docs/compare-rt.jpg) |
31
+
32
+ ## Getting started
33
+
34
+ Install the library plus its two **peer dependencies** — you bring your own copy
35
+ of `three` and `three-mesh-bvh`:
36
+
37
+ ```bash
38
+ npm i three-realtime-rt three three-mesh-bvh
39
+ ```
40
+
41
+ No bundler? [`standalone.html`](standalone.html) is a single copy-paste file
42
+ that runs the raytracer via CDN import maps — open it from any static server.
43
+
44
+ A complete, copy-pasteable minimal app — a lit sphere on a floor, one point light:
45
+
46
+ ```js
47
+ import * as THREE from "three";
48
+ import { RealtimeRaytracer } from "three-realtime-rt";
49
+
50
+ // 1. An ordinary three.js renderer. Size it BEFORE constructing the raytracer —
51
+ // it reads the drawing-buffer size at construction.
52
+ const renderer = new THREE.WebGLRenderer({ antialias: false });
53
+ renderer.setPixelRatio(window.devicePixelRatio);
54
+ renderer.setSize(window.innerWidth, window.innerHeight);
55
+ document.body.appendChild(renderer.domElement);
56
+
57
+ // 2. An ordinary scene: meshes with MeshStandardMaterial + a real light.
58
+ const scene = new THREE.Scene();
59
+ const camera = new THREE.PerspectiveCamera(
60
+ 60, window.innerWidth / window.innerHeight, 0.1, 100
61
+ );
62
+ camera.position.set(0, 2, 6);
63
+
64
+ scene.add(new THREE.Mesh(
65
+ new THREE.SphereGeometry(1, 48, 48),
66
+ new THREE.MeshStandardMaterial({ color: 0xdddddd, roughness: 0.4, metalness: 0.0 })
67
+ ));
68
+ const floor = new THREE.Mesh(
69
+ new THREE.PlaneGeometry(20, 20),
70
+ new THREE.MeshStandardMaterial({ color: 0x808080, roughness: 1.0 })
71
+ );
72
+ floor.rotation.x = -Math.PI / 2;
73
+ floor.position.y = -1;
74
+ scene.add(floor);
75
+
76
+ const light = new THREE.PointLight(0xffffff, 40); // Point / Spot / Directional (up to 32)
77
+ light.position.set(3, 5, 2);
78
+ scene.add(light);
79
+
80
+ // 3. Turn on ray tracing.
81
+ const rt = new RealtimeRaytracer(renderer);
82
+ rt.compileScene(scene); // builds the BVH + material/light tables
83
+
84
+ // 4. Resize: pass DRAWING-BUFFER (device) pixels, not CSS pixels.
85
+ addEventListener("resize", () => {
86
+ camera.aspect = window.innerWidth / window.innerHeight;
87
+ camera.updateProjectionMatrix();
88
+ renderer.setSize(window.innerWidth, window.innerHeight);
89
+ const db = renderer.getDrawingBufferSize(new THREE.Vector2());
90
+ rt.setSize(db.x, db.y);
91
+ });
92
+
93
+ // 5. Render loop — replace renderer.render(scene, camera) with rt.render.
94
+ function loop() {
95
+ requestAnimationFrame(loop);
96
+ rt.render(scene, camera);
97
+ }
98
+ loop();
99
+ ```
100
+
101
+ On hardware that can't trace, `rt.render` transparently falls back to
102
+ `renderer.render` — no capability branch needed (see [Running everywhere](#running-everywhere-capability-tiers)).
103
+
104
+ ### Integrating into an existing app
105
+
106
+ A checklist for dropping the tracer into a scene you already have:
107
+
108
+ 1. **Swap the render call** — `renderer.render(scene, camera)` →
109
+ `rt.render(scene, camera)`. Construct the `RealtimeRaytracer` once, *after*
110
+ the renderer has its final size.
111
+ 2. **Compile once; recompile after structural changes** — `rt.compileScene(scene)`
112
+ bakes geometry into a static BVH and snapshots materials + emissive area
113
+ lights. Call it again after you add/remove meshes, swap geometry, or change a
114
+ material's `emissive` / `color` / `roughness` / `metalness`.
115
+ 3. **Declare movers** pass moving meshes to
116
+ `rt.compileScene(scene, { dynamicMeshes: [...] })`, then call
117
+ `rt.updateDynamic()` each frame after you move them (e.g. after a physics
118
+ step). Skip it on frames where nothing moved.
119
+ 4. **Update lights when they change** — after moving, toggling (`.visible`),
120
+ recolouring or dimming a light, call `rt.updateLights(scene)`. No recompile.
121
+ 5. **Resize** in your resize handler, after `renderer.setSize(...)`, call
122
+ `rt.setSize(width, height)` with **drawing-buffer (device) pixels**
123
+ (`renderer.getDrawingBufferSize(...)`).
124
+ 6. **Reset after jumps** — call `rt.resetAccumulation()` after a camera teleport
125
+ or a scene cut, so stale temporal history doesn't ghost.
126
+
127
+ That's the whole integration. Everything below is optional.
128
+
129
+ ---
130
+
131
+ ## Why "hybrid deferred" (the "RTX on" model)
132
+
133
+ Primary visibility is **rasterized** by three.js into a G-buffer — free, fast,
134
+ and pixel-perfect on materials and textures. Only the *lighting* is ray traced,
135
+ in a fragment shader, against a GPU BVH ([three-mesh-bvh]):
136
+
137
+ 1. **G-buffer pass** MRT: albedo+roughness, world normal+metalness, world
138
+ position, emissive.
139
+ 2. **RT lighting pass** — per pixel: soft shadow rays to each light (area
140
+ sampled) + a 1-bounce cosine-weighted GI ray with next-event estimation. GI
141
+ rays that escape sample the **procedural sky**, so the sky is a soft area
142
+ light. **Emissive meshes are real area lights**: their triangles are sampled
143
+ directly (NEE with the area→solid-angle pdf), so a glowing panel casts soft
144
+ light and shadows instead of waiting for a lucky GI ray to hit it. Metallic
145
+ pixels trace a **mirror/glossy reflection ray**; transmissive pixels trace a
146
+ Fresnel-weighted **reflection + two-interface refraction**. Output is
147
+ *demodulated irradiance* (albedo divided out) so it denoises cleanly while
148
+ textures stay sharp.
149
+ 3. **Temporal reprojection** — motion-validated history keeps samples alive as
150
+ the camera and objects move.
151
+ 4. **À-trous denoise** — an edge-avoiding (SVGF-lite) wavelet filter guided by
152
+ the G-buffer, so 1 sample/pixel looks converged.
153
+ 5. **Composite** `albedo × irradiance + emissive`, distance fog, ACES tonemap.
154
+ 6. **TAA**sub-pixel jitter + a neighbourhood-clamped history resolve:
155
+ supersampled anti-aliasing that also clears disocclusion speckles. This is
156
+ the analytic (FSR2 / TAAU) approach, not a learned upscaler.
157
+
158
+ Lighting is traced at half resolution by default and reconstructed by a joint
159
+ bilateral upsample + the denoiser + TAA the same "render few pixels, rebuild
160
+ temporally" idea DLSS uses, done with hand-written math.
161
+
162
+ ## Moving objects (dynamic BVH)
163
+
164
+ Mark meshes as dynamic and their motion casts **correct ray traced shadows** —
165
+ the demo drops 40 rigid bodies (Rapier physics) that shadow each other and the
166
+ ground in real time:
167
+
168
+ ```js
169
+ rt.compileScene(scene, { dynamicMeshes: crates }); // meshes that will move
170
+
171
+ // each frame, after you move them (e.g. after a physics step):
172
+ rt.updateDynamic(); // re-bakes them into the BVH (refit) cheap
173
+ rt.render(scene, camera);
174
+ ```
175
+
176
+ Under the hood this is a **two-level BVH**: static geometry lives in one BVH
177
+ uploaded to the GPU once at compile time, dynamic meshes in a second small BVH
178
+ that is re-baked and refit per frame. `updateDynamic()` therefore costs
179
+ ~1 ms for dozens of moving objects *regardless of how big the static world is* —
180
+ skip it entirely on frames where nothing moved.
181
+
182
+ ## Live lighting & sky
183
+
184
+ Lights can be toggled, moved, and recoloured every frame without recompiling:
185
+
186
+ ```js
187
+ warmLight.visible = false; // or change .color / .intensity / .position
188
+ rt.updateLights(scene); // re-reads the scene's lights
189
+ ```
190
+
191
+ The procedural sky doubles as the ambient light source:
192
+
193
+ ```js
194
+ const rt = new RealtimeRaytracer(renderer, {
195
+ sky: {
196
+ enabled: true,
197
+ sunDir: new THREE.Vector3(0.55, 0.62, 0.55).normalize(), // toward the sun
198
+ sunColor: new THREE.Color(1.0, 0.92, 0.78),
199
+ zenith: new THREE.Color(0.20, 0.40, 0.72),
200
+ horizon: new THREE.Color(0.78, 0.85, 0.92),
201
+ intensity: 1.0,
202
+ },
203
+ fog: { enabled: true, color: new THREE.Color(0.72, 0.8, 0.88), density: 0.03 },
204
+ });
205
+ ```
206
+
207
+ ![Physics: 40 rigid bodies with dynamic ray traced shadows](docs/physics.png)
208
+
209
+ ## What is and isn't supported
210
+
211
+ Primary visibility is ordinary rasterized three.js, so **whatever three.js draws,
212
+ you still see** — transparency, blending, sorting, custom shaders and post effects
213
+ behave exactly as normal. The ray tracer computes only the *lighting*, and it
214
+ reads a deliberately small, fixed slice of the material and light model. This is
215
+ the honest map of what actually feeds the traced lighting.
216
+
217
+ ### Materials
218
+
219
+ Lighting reads the **first material only** on a multi-material mesh, pulling the
220
+ scalar fields of `MeshStandardMaterial` / `MeshPhysicalMaterial` (Basic / Lambert
221
+ / Phong contribute whatever of those fields they have).
222
+
223
+ | Property | Feeds lighting? | Notes |
224
+ |----------|-----------------|-------|
225
+ | `color` + `map` | ✅ | Albedo = `color × map.rgb`. Textures stay sharp (irradiance is demodulated, then re-multiplied). |
226
+ | `roughness` | | **Scalar only.** Drives shadow / GI softness and reflection sharpness. |
227
+ | `metalness` | ✅ | **Scalar only.** Metallic pixels trace a reflection ray. |
228
+ | `emissive` | ✅ | A *static* emissive mesh becomes a real **area light** (NEE) casts soft light + shadows. |
229
+ | `emissiveMap` | ⚠️ visible only | A map-masked emissive **glows on screen** but the map **zeroes its area-light table** — it lights nothing. Use a flat `emissive` colour (no map) for an emitter that should illuminate. |
230
+ | `transmission` (Physical) | ✅ | Glass: Fresnel reflection + two-interface refraction. |
231
+ | `roughnessMap` / `metalnessMap` | | Only the scalar `roughness` / `metalness` are used; these maps are ignored by lighting. |
232
+ | `normalMap` | | Lighting uses geometric normals; normal maps don't perturb shading. |
233
+ | `clearcoat`, `sheen`, `iridescence` | | Not modelled. |
234
+ | vertex colors | ❌ | Not read into albedo. |
235
+ | per-material `ior` | ❌ | Refraction uses the single **global** `rt.ior` (default 1.5), never `material.ior`. |
236
+ | 2nd+ material of a group | ❌ | Only `material[0]` of a multi-material mesh reaches the G-buffer and BVH. |
237
+
238
+ ### Lights
239
+
240
+ | Light | Supported | Notes |
241
+ |-------|-----------|-------|
242
+ | `PointLight` | ✅ | `light.userData.rtRadius` (default `0.15`) sets soft-shadow size. |
243
+ | `DirectionalLight` | | `light.userData.rtRadius` (default `0.02`) sets sun softness; keep its direction in sync with `sky.sunDir`. |
244
+ | Emissive meshes | ✅ static | Sampled directly as area lights. **Dynamic** emitters are *not* in the NEE list — they light only via GI-ray hits. |
245
+ | `SpotLight` | ✅ | Cone + penumbra respected; soft shadows via `rtRadius`; visible light cones in volumetric fog. |
246
+ | `RectAreaLight` | | Use an emissive mesh instead. |
247
+ | `HemisphereLight` / `AmbientLight` | | Ignored the procedural `sky` (or `envColor`) provides ambient. |
248
+
249
+ - Up to **32** point/directional lights (`MAX_LIGHTS`); further lights are dropped.
250
+ - Moving, toggling, recolouring or dimming a light `rt.updateLights(scene)` (cheap, no recompile).
251
+ - Changing a mesh's **emissive** (it's an area light baked at compile time) → `rt.compileScene(...)` again.
252
+ - Emissive area lights are capped at **256 triangles** (largest by area kept, with a console warning) prefer low-poly emitter meshes.
253
+
254
+ ### Geometry & occlusion
255
+
256
+ - Every non-excluded visible mesh is **merged into one static BVH at compile time**. Add / remove geometry → recompile.
257
+ - Meshes that move must be declared via `dynamicMeshes` and driven with `updateDynamic()`. Anything not declared is treated as static — moving it on screen won't move its traced shadow.
258
+ - **Transparent materials never occlude** (by design — a glass case shouldn't cast an opaque shadow). They still rasterize normally.
259
+ - **`alphaTest` cut-outs** (`transparent: false`) *do* occlude but as **full triangles**, not per-texel, so their shadows are blocky.
260
+ - `mesh.userData.rtExclude = true` removes a mesh from the BVH entirely (it still rasterizes and gets lit)handy for water / translucent surfaces.
261
+
262
+ ### Rendering model
263
+
264
+ - **1-bounce GI + direct light.** No multi-bounce diffuse, no caustics, no specular-chain paths.
265
+ - **Reflections** are a single traced bounce into a **diffuse-shaded** view of the world — no recursive mirror-in-mirror; a metal shows its surroundings, not a second full render.
266
+ - **Refraction** is two-interface (front + back) with a single global IOR — convincing glass, not a spectral / dispersive renderer.
267
+ - **Volumetric** is **single-scatter** god rays, not multiple-scattering fog.
268
+ - Because primary visibility is rasterized, **blending, sorting and transparency composite exactly like normal three.js** the tracer never changes how your transparent surfaces layer.
269
+
270
+ ### Platform
271
+
272
+ - Requires **WebGL2 + `EXT_color_buffer_float`**. Software rasterizers (SwiftShader / llvmpipe) are treated as unsupported.
273
+ - On anything unsupported, `rt.supported === false` and `rt.render()` **falls back to `renderer.render()`** after one console warning — your app still runs everywhere. Branch yourself with `rt.supported` or `RealtimeRaytracer.isSupported(renderer)` if you want.
274
+ - WebGL2 only; no WebGPU backend (on the roadmap).
275
+
276
+ ## Options
277
+
278
+ | Option | Default | What |
279
+ |--------|---------|------|
280
+ | `renderScale` | `0.5` | Lighting resolution vs. the G-buffer. `1.0` = max quality. |
281
+ | `taa` | `true` | Temporal anti-aliasing (jitter + neighbourhood clamp). |
282
+ | `denoise` | `true` | Edge-aware à-trous denoiser. |
283
+ | `gi` | `true` | 1-bounce global illumination (vs. direct-only). |
284
+ | `emissiveNEE` | `true` | Sample static emissive meshes as area lights (next-event estimation). Off = emitters only light via lucky GI rays. |
285
+ | `reflections` | `true` | Traced mirror/glossy reflections on metallic surfaces (sharpest at `renderScale: 1`). |
286
+ | `refraction` | `true` | Traced two-interface refraction for `MeshPhysicalMaterial.transmission` surfaces. |
287
+ | `restir` | `true` | ReSTIR direct lighting: per-pixel reservoirs with temporal + spatial reuse, one visibility ray regardless of light count. Flat cost in light count; cuts emissive area-light noise. |
288
+ | `ior` | `1.5` | Index of refraction used by `refraction`. |
289
+ | `volumetric` | *off* | Physically-based god rays: single-scatter fog, one BVH-shadowed light sample per lighting pixel per frame, temporally accumulated. `{ enabled, density, maxDist, zones }`, where `zones` is an optional array of up to 8 AABBs `{ min:[x,y,z], max:[x,y,z], density }` that add localized fog on top of (or instead of) the global `density`. |
290
+ | `stochasticLights` | `false` | One direct shadow ray per pixel per frame (random source) instead of one per light. |
291
+ | `temporalReprojection` | `true` | Keep samples across camera/object motion. |
292
+ | `maxHistory` | `128` | History cap higher is smoother, slower to react. |
293
+ | `fireflyClamp` | `4.0` | Clamp on indirect luminance to suppress fireflies. |
294
+ | `sky` | *off* | Procedural sky as background + GI ambient (see above). |
295
+ | `fog` | *off* | Distance fog, composited before tonemap. |
296
+
297
+ Per-light: set `light.userData.rtRadius` for soft-shadow size. Set
298
+ `mesh.userData.rtExclude = true` to keep a mesh out of the BVH (it still
299
+ rasterizes and gets lit — useful for water / translucent surfaces).
300
+ Transparent materials never act as occluders (a glass case shouldn't cast an
301
+ opaque shadow); `alphaTest` cut-outs still do.
302
+
303
+ ## Running everywhere (capability tiers)
304
+
305
+ Nothing adaptive is imposed — everything below is **opt-in**:
306
+
307
+ - **No usable GPU** (missing WebGL2 float targets, or a software rasterizer
308
+ like SwiftShader): the library logs one console warning and `rt.render()`
309
+ silently falls back to plain `renderer.render()`. Your app runs everywhere
310
+ with zero capability checks; query `rt.supported` or the static
311
+ `RealtimeRaytracer.isSupported(renderer)` if you want to branch yourself.
312
+ - **Tier presets**: `RealtimeRaytracer.detectTier(renderer)` returns
313
+ `"none" | "mid" | "high"`, and `recommendedOptions(tier)` gives matching
314
+ constructor options spread them, then override what you like:
315
+
316
+ ```js
317
+ const tier = RealtimeRaytracer.detectTier(renderer);
318
+ const rt = new RealtimeRaytracer(renderer, {
319
+ ...RealtimeRaytracer.recommendedOptions(tier),
320
+ targetFps: 55,
321
+ });
322
+ ```
323
+
324
+ - **Adaptive quality** (`adaptiveQuality: true`, default **false**): continuous
325
+ dynamic resolution scaling. Watches real frame time and steers the lighting
326
+ resolution smoothly toward `targetFps` (in 5% steps with a cooldown), pairing
327
+ low resolutions with MORE denoise passes (they run at lighting res, so
328
+ they're nearly free exactly where they're needed) and stochastic direct
329
+ light. While enabled it drives `renderScale` / `stochasticLights` /
330
+ `denoiseIterations`; turn it off for manual control.
331
+ - **`stochasticLights: true`** (default false): one direct shadow ray per
332
+ pixel per frame instead of one per light — the biggest ray-count lever for
333
+ many-light scenes and mobile GPUs.
334
+
335
+ WebGPU: not used as a backend (this is a WebGL2 library); a WGSL compute
336
+ backend is on the roadmap.
337
+
338
+ ## Running the demo
339
+
340
+ ```bash
341
+ npm install
342
+ npm run dev # http://localhost:8115
343
+ ```
344
+
345
+ The demo ([`examples/`](examples/)) is an ordinary three.js app — see
346
+ [`examples/main.js`](examples/main.js) for the full, commented integration
347
+ (scene → physics → compile → render loop). `npm run deploy` builds and publishes
348
+ it to GitHub Pages.
349
+
350
+ ## Gallery & benchmarks
351
+
352
+ [`gallery.html`](gallery.html) drops the raytracer into **stock glTF scenes it
353
+ was never authored for** — LittlestTokyo, Lantern, Antique Camera, BoomBox,
354
+ Corset, Water Bottle, and the Damaged Helmet — streamed straight from their
355
+ public hosts (no assets committed). A one-button toggle A/Bs ray tracing against
356
+ plain rasterized three.js with an fps + triangle readout, and a compact options
357
+ strip exposes GI / emissive NEE / reflections / refraction / ReSTIR / denoise /
358
+ TAA / volumetric plus lighting-resolution and auto-quality controls.
359
+
360
+ [`bench.html?autorun=1`](bench.html) runs a matrix of feature configs with
361
+ GPU-**fence-timed** frame costs and a temporal **ghosting metric**, writing each
362
+ run's results to [`bench-results/`](bench-results/) for tracking regressions.
363
+
364
+ ## Roadmap
365
+
366
+ | Stage | Status | What |
367
+ |-------|--------|------|
368
+ | 1. Core | ✅ | Scene→GPU sync, BVH, G-buffer, traced shadows + 1-bounce GI, accumulation |
369
+ | 2. Reprojection | | Motion-validated history — samples survive camera motion |
370
+ | 3. Denoiser | ✅ | Edge-avoiding à-trous (SVGF-lite) → clean 1spp |
371
+ | 4. TAA | ✅ | Sub-pixel jitter + neighbourhood-clamped resolve AA, no speckles |
372
+ | 4b. Sky | ✅ | Procedural sky as background + GI ambient light source |
373
+ | 5. Two-level BVH | ✅ | Static BVH uploaded once; movers in a small per-frame BVH dynamic shadows at ~1 ms |
374
+ | 5b. Area lights | ✅ | Emissive meshes sampled directly (NEE) glowing panels cast soft light + shadows |
375
+ | 6. Specular | ✅ | Mirror/glossy reflections on metals + two-interface glass refraction |
376
+ | 6b. Sampling | ✅ | Blue-noise sampling + ReSTIR direct lighting (temporal + spatial reuse) |
377
+ | 6c. Any-hit shadows | ✅ | Unordered early-out BVH traversal for occlusion rays same image, up to ~2× cheaper shadows |
378
+ | 7. Publish | | npm publish; refractive water; per-material IOR |
379
+
380
+ ## Credits
381
+
382
+ - [three-mesh-bvh] by Garrett Johnson — the GPU BVH this is built on.
383
+ - Inspired by [Erich Loftis'][erichlof] `THREE.js-PathTracing-Renderer`.
384
+ - Demo models: Khronos glTF sample assets (Damaged Helmet, Duck).
385
+
386
+ ## License
387
+
388
+ MIT © Goldwin Stewart
389
+
390
+ [three-mesh-bvh]: https://github.com/gkjohnson/three-mesh-bvh
391
+ [erichlof]: https://github.com/erichlof/THREE.js-PathTracing-Renderer