three-realtime-rt 0.3.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +187 -19
- package/package.json +1 -1
- package/src/CompositePass.js +59 -21
- package/src/DenoisePass.js +16 -2
- package/src/GBufferPass.js +84 -22
- package/src/RTLightingPass.js +411 -17
- package/src/RealtimeRaytracer.js +288 -16
- package/src/RestirPass.js +20 -3
- package/src/SceneCompiler.js +127 -32
- package/src/TAAPass.js +20 -4
- package/src/index.d.ts +87 -4
package/README.md
CHANGED
|
@@ -10,6 +10,18 @@ the scene, **ReSTIR many-light sampling** (flat cost in light count),
|
|
|
10
10
|
**blue-noise sampling**, and real-time **temporal denoising + anti-aliasing**.
|
|
11
11
|
Runs on plain WebGL2.
|
|
12
12
|
|
|
13
|
+
This round adds **GGX PBR specular** — Cook-Torrance dielectric highlights in a
|
|
14
|
+
separate specular buffer, so `roughness` finally matters on non-metals — plus
|
|
15
|
+
**normal / roughness / metalness maps**, **alpha-blended transparency**
|
|
16
|
+
(single-layer deferred blend), **deforming dynamic meshes** (a mirror-water pool
|
|
17
|
+
whose traced reflections follow the live wave surface), **overscan** to hide
|
|
18
|
+
leading-edge convergence noise while the camera turns, and **emissive importance
|
|
19
|
+
sampling** (area × luminance) for calmer area lights. The zero-config renderer
|
|
20
|
+
now ships **conservative, self-scaling defaults** — it starts low-but-ray-traced
|
|
21
|
+
and an on-by-default governor scales quality up toward `targetFps`; an optional
|
|
22
|
+
async **GPU tier probe** reads real WebGPU adapter limits for a smarter starting
|
|
23
|
+
point.
|
|
24
|
+
|
|
13
25
|
The library ships as **untranspiled ES modules** (the `src/` folder) — it has no
|
|
14
26
|
build step of its own, so you consume it through your bundler (Vite, webpack,
|
|
15
27
|
esbuild, …) or a browser import map that resolves the bare `three` /
|
|
@@ -101,6 +113,17 @@ loop();
|
|
|
101
113
|
On hardware that can't trace, `rt.render` transparently falls back to
|
|
102
114
|
`renderer.render` — no capability branch needed (see [Running everywhere](#running-everywhere-capability-tiers)).
|
|
103
115
|
|
|
116
|
+
> **Defaults are conservative.** Zero-config construction
|
|
117
|
+
> (`new RealtimeRaytracer(renderer)`) starts *low but still ray traced* — half
|
|
118
|
+
> lighting resolution, stochastic direct light, a lean denoise — so it runs
|
|
119
|
+
> acceptably on weak discrete and integrated GPUs out of the box. The adaptive
|
|
120
|
+
> governor is **on by default** and scales quality **up** toward `targetFps`
|
|
121
|
+
> when it measures headroom (a strong desktop climbs to full-resolution
|
|
122
|
+
> lighting within a couple of seconds). Want to start higher, or pin a level?
|
|
123
|
+
> Pass `RealtimeRaytracer.recommendedOptions(RealtimeRaytracer.detectTier(renderer))`
|
|
124
|
+
> (or `probeGPUTier()` — see [Running everywhere](#running-everywhere-capability-tiers)),
|
|
125
|
+
> or explicit options.
|
|
126
|
+
|
|
104
127
|
### Integrating into an existing app
|
|
105
128
|
|
|
106
129
|
A checklist for dropping the tracer into a scene you already have:
|
|
@@ -179,6 +202,46 @@ that is re-baked and refit per frame. `updateDynamic()` therefore costs
|
|
|
179
202
|
~1 ms for dozens of moving objects *regardless of how big the static world is* —
|
|
180
203
|
skip it entirely on frames where nothing moved.
|
|
181
204
|
|
|
205
|
+
### Deforming meshes (water, cloth)
|
|
206
|
+
|
|
207
|
+
By default a dynamic mesh is **rigid**: its vertices are snapshotted at compile
|
|
208
|
+
time and only re-transformed by `mesh.matrixWorld` each frame — so CPU edits to
|
|
209
|
+
its `position` attribute show in the rasterized image but the *traced* rays
|
|
210
|
+
(shadows, GI, reflections) still hit the original shape. For a mesh whose
|
|
211
|
+
vertices actually move on the CPU (a water surface, cloth, a morph target), set
|
|
212
|
+
`userData.rtDeforming` so the raytracer re-reads its live geometry every frame:
|
|
213
|
+
|
|
214
|
+
```js
|
|
215
|
+
water.userData.rtDeforming = true; // opt in
|
|
216
|
+
rt.compileScene(scene, { dynamicMeshes: [water, ...crates] });
|
|
217
|
+
|
|
218
|
+
// each frame:
|
|
219
|
+
deformWaterVertices(water.geometry, t); // your CPU wave/cloth solver
|
|
220
|
+
water.geometry.attributes.position.needsUpdate = true;
|
|
221
|
+
water.geometry.computeVertexNormals(); // REQUIRED — see below
|
|
222
|
+
rt.updateDynamic(); // re-reads the live vertices
|
|
223
|
+
rt.render(scene, camera);
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Two requirements:
|
|
227
|
+
|
|
228
|
+
- **You own the normals.** The tracer reads the mesh's live `normal` attribute
|
|
229
|
+
for deforming segments — it does not recompute them. Call
|
|
230
|
+
`geometry.computeVertexNormals()` (or update the attribute yourself) after
|
|
231
|
+
moving the vertices, or the shading/reflections will track the old silhouette.
|
|
232
|
+
- **The vertex count is fixed at compile time.** Deforming an existing surface is
|
|
233
|
+
free; changing its topology (adding/removing vertices) is not — `updateDynamic()`
|
|
234
|
+
throws with a clear message telling you to `compileScene()` again.
|
|
235
|
+
|
|
236
|
+
**Cost model.** A deforming segment is re-baked from its live vertices every
|
|
237
|
+
frame and its normals are re-uploaded every frame (rigid movers amortize the
|
|
238
|
+
normal upload over 8 frames). Both are O(dynamic triangles), and the cheap
|
|
239
|
+
per-frame BVH `refit()` (kept for surfaces that stay roughly in place, like a
|
|
240
|
+
water plane) is O(dynamic tris) too — so **keep deforming meshes low-poly**. A
|
|
241
|
+
`48×48` plane is ≈ 4.6k triangles, which refits in well under a millisecond; a
|
|
242
|
+
`256×256` plane (~131k tris) will dominate the frame. The demo's mirror-water
|
|
243
|
+
pool is a `48×48` plane.
|
|
244
|
+
|
|
182
245
|
## Live lighting & sky
|
|
183
246
|
|
|
184
247
|
Lights can be toggled, moved, and recoloured every frame without recompiling:
|
|
@@ -208,10 +271,12 @@ const rt = new RealtimeRaytracer(renderer, {
|
|
|
208
271
|
|
|
209
272
|
## What is and isn't supported
|
|
210
273
|
|
|
211
|
-
Primary visibility is
|
|
212
|
-
you still see** —
|
|
213
|
-
|
|
214
|
-
|
|
274
|
+
Primary visibility is rasterized into a G-buffer, so **whatever three.js draws,
|
|
275
|
+
you still see** — the ray tracer computes only the *lighting*, reading a
|
|
276
|
+
deliberately small, fixed slice of the material and light model. The one place
|
|
277
|
+
the G-buffer diverges from a plain three.js draw is transparency: it is a
|
|
278
|
+
**single-layer deferred blend** (see the `transparent` row below and the
|
|
279
|
+
Rendering-model notes), not three.js's per-fragment sorted over-blend. This is
|
|
215
280
|
the honest map of what actually feeds the traced lighting.
|
|
216
281
|
|
|
217
282
|
### Materials
|
|
@@ -223,13 +288,17 @@ scalar fields of `MeshStandardMaterial` / `MeshPhysicalMaterial` (Basic / Lamber
|
|
|
223
288
|
| Property | Feeds lighting? | Notes |
|
|
224
289
|
|----------|-----------------|-------|
|
|
225
290
|
| `color` + `map` | ✅ | Albedo = `color × map.rgb`. Textures stay sharp (irradiance is demodulated, then re-multiplied). |
|
|
226
|
-
| `roughness` | ✅ |
|
|
227
|
-
| `metalness` | ✅ |
|
|
228
|
-
| `emissive` | ✅ | A *static* emissive mesh becomes a real **area light** (NEE) — casts soft light + shadows. |
|
|
291
|
+
| `roughness` | ✅ | Drives shadow / GI softness, reflection sharpness **and the GGX specular lobe width** (see *dielectric specular* below). |
|
|
292
|
+
| `metalness` | ✅ | Metallic pixels trace a reflection ray whose analytic-light glints are shadowed; `F0 = mix(0.04, albedo, metalness)`. |
|
|
293
|
+
| `emissive` | ✅ | A *static* emissive mesh becomes a real **area light** (NEE) — casts soft light + shadows, including a GGX highlight. |
|
|
229
294
|
| `emissiveMap` | ⚠️ visible only | A map-masked emissive **glows on screen** but the map **zeroes its area-light table** — it lights nothing. Use a flat `emissive` colour (no map) for an emitter that should illuminate. |
|
|
230
|
-
| `transmission` (Physical) | ✅ | Glass: Fresnel reflection + two-interface refraction. |
|
|
231
|
-
| `
|
|
232
|
-
| `
|
|
295
|
+
| `transmission` (Physical) | ✅ | Glass: Fresnel reflection (with analytic-light glints) + two-interface refraction. |
|
|
296
|
+
| `transparent` + `opacity` | ✅ | Alpha blend: the surface is composited over the geometry behind it (a straight-through traced ray), weighted by scalar `opacity` and **tinted by `color`/`map`**. The behind-radiance rides the **specular buffer** and the opacity blend happens at **composite** (where the pane's albedo lives), so **needs `specular: true`** — with the specular buffer off, blend surfaces degrade to opaque. Single layer — nearest transparent surface wins, overlapping panes don't inter-sort. Kept out of the BVH, so it casts no shadow. Toggle with `transparency`. |
|
|
297
|
+
| `opacity` on an opaque material | ❌ | Only read when `transparent: true`; an opaque material always writes at full coverage. |
|
|
298
|
+
| **dielectric specular** | ✅ | Cook-Torrance **GGX** direct highlights for *every* surface, in a separate white (`F0 ≈ 0.04`) specular buffer the composite adds without the albedo multiply. Toggle with `specular` (default on). |
|
|
299
|
+
| `roughnessMap` | ✅ | `roughness × roughnessMap.g` (three.js convention) — sampled in the G-buffer. |
|
|
300
|
+
| `metalnessMap` | ✅ | `metalness × metalnessMap.b` (a packed ORM texture works — G = roughness, B = metalness). |
|
|
301
|
+
| `normalMap` | ✅ | Perturbs the shading normal via a screen-space cotangent frame (no tangent attribute needed); respects `material.normalScale`. |
|
|
233
302
|
| `clearcoat`, `sheen`, `iridescence` | ❌ | Not modelled. |
|
|
234
303
|
| vertex colors | ❌ | Not read into albedo. |
|
|
235
304
|
| per-material `ior` | ❌ | Refraction uses the single **global** `rt.ior` (default 1.5), never `material.ior`. |
|
|
@@ -246,6 +315,13 @@ scalar fields of `MeshStandardMaterial` / `MeshPhysicalMaterial` (Basic / Lamber
|
|
|
246
315
|
| `RectAreaLight` | ❌ | Use an emissive mesh instead. |
|
|
247
316
|
| `HemisphereLight` / `AmbientLight` | ❌ | Ignored — the procedural `sky` (or `envColor`) provides ambient. |
|
|
248
317
|
|
|
318
|
+
- **Emissive noise caveat:** emissive NEE is the noisiest direct-light path — one
|
|
319
|
+
uniformly-picked triangle sample per pixel per frame, with a `1/dist²` term that
|
|
320
|
+
sparkles into fireflies near small, close emitters. **Keep `restir: true` in any
|
|
321
|
+
scene that leans on emissive lighting** (the reservoirs converge each pixel onto
|
|
322
|
+
the emitter that matters; the library logs a hint if you compile emissive
|
|
323
|
+
geometry with ReSTIR off). Prefer larger/dimmer emitter surfaces over tiny
|
|
324
|
+
bright ones, and let `fireflyClamp` do its job.
|
|
249
325
|
- Up to **32** point/directional lights (`MAX_LIGHTS`); further lights are dropped.
|
|
250
326
|
- Moving, toggling, recolouring or dimming a light → `rt.updateLights(scene)` (cheap, no recompile).
|
|
251
327
|
- Changing a mesh's **emissive** (it's an area light baked at compile time) → `rt.compileScene(...)` again.
|
|
@@ -265,7 +341,7 @@ scalar fields of `MeshStandardMaterial` / `MeshPhysicalMaterial` (Basic / Lamber
|
|
|
265
341
|
- **Reflections** are a single traced bounce into a **diffuse-shaded** view of the world — no recursive mirror-in-mirror; a metal shows its surroundings, not a second full render.
|
|
266
342
|
- **Refraction** is two-interface (front + back) with a single global IOR — convincing glass, not a spectral / dispersive renderer.
|
|
267
343
|
- **Volumetric** is **single-scatter** god rays, not multiple-scattering fog.
|
|
268
|
-
-
|
|
344
|
+
- **Transparency** is a **single-layer deferred blend**: a `transparent` surface writes as the nearest layer of the G-buffer and the lighting pass composites it over the geometry behind by tracing one straight-through ray. The behind-radiance is fully lit (direct + 1-bounce GI) and tinted by the pane's albedo. Overlapping transparent surfaces do **not** inter-sort (only the nearest is kept), and there is no per-pixel back-to-front over-blend of many layers as in raster three.js. Turn it off with `transparency: false` (blend surfaces then render fully opaque).
|
|
269
345
|
|
|
270
346
|
### Platform
|
|
271
347
|
|
|
@@ -278,16 +354,22 @@ scalar fields of `MeshStandardMaterial` / `MeshPhysicalMaterial` (Basic / Lamber
|
|
|
278
354
|
| Option | Default | What |
|
|
279
355
|
|--------|---------|------|
|
|
280
356
|
| `renderScale` | `0.5` | Lighting resolution vs. the G-buffer. `1.0` = max quality. |
|
|
357
|
+
| `overscan` | `0` | Render past the canvas edges and crop the centre back, so leading-edge disocclusion noise during camera motion is born off-screen. Padding fraction per edge (0–0.25); `0.1` costs 1.44× the pixels. See *Edge convergence and overscan*. |
|
|
358
|
+
| `adaptiveQuality` | `true` | Governor that steers `renderScale` / `denoiseIterations` / `stochasticLights` toward `targetFps` — scales **up** on strong hardware, **down** on weak. Turn off for manual control. |
|
|
359
|
+
| `denoiseIterations` | `2` | À-trous denoise passes; the governor raises this as it lowers resolution. |
|
|
281
360
|
| `taa` | `true` | Temporal anti-aliasing (jitter + neighbourhood clamp). |
|
|
282
361
|
| `denoise` | `true` | Edge-aware à-trous denoiser. |
|
|
283
362
|
| `gi` | `true` | 1-bounce global illumination (vs. direct-only). |
|
|
284
363
|
| `emissiveNEE` | `true` | Sample static emissive meshes as area lights (next-event estimation). Off = emitters only light via lucky GI rays. |
|
|
364
|
+
| `emissiveImportance` | `true` | Pick WHICH emissive triangle NEE samples proportional to **area × emitted luminance** (compile-time power CDF) instead of a uniform 1-of-N. Same mean, far less sparkle when emitters differ in size/brightness. Off = legacy uniform pick. |
|
|
365
|
+
| `specular` | `true` | Cook-Torrance **GGX** dielectric highlights for every surface, in a separate white (`F0 ≈ 0.04`) buffer the composite adds without the albedo multiply. Off = the old Lambert-only look. Also required by `transparency`. |
|
|
285
366
|
| `reflections` | `true` | Traced mirror/glossy reflections on metallic surfaces (sharpest at `renderScale: 1`). |
|
|
286
367
|
| `refraction` | `true` | Traced two-interface refraction for `MeshPhysicalMaterial.transmission` surfaces. |
|
|
368
|
+
| `transparency` | `true` | Alpha-blended transparency: composite `transparent` meshes over the geometry behind them (single-layer, weighted by `opacity`, tinted by albedo). Needs the specular buffer (`specular: true`). Off = they render fully opaque. |
|
|
287
369
|
| `restir` | `true` | ReSTIR direct lighting: per-pixel reservoirs with temporal + spatial reuse, one visibility ray regardless of light count. Flat cost in light count; cuts emissive area-light noise. |
|
|
288
370
|
| `ior` | `1.5` | Index of refraction used by `refraction`. |
|
|
289
371
|
| `volumetric` | *off* | Physically-based god rays: single-scatter fog, one BVH-shadowed light sample per lighting pixel per frame, temporally accumulated. `{ enabled, density, maxDist, zones }`, where `zones` is an optional array of up to 8 AABBs `{ min:[x,y,z], max:[x,y,z], density }` that add localized fog on top of (or instead of) the global `density`. |
|
|
290
|
-
| `stochasticLights` | `
|
|
372
|
+
| `stochasticLights` | `true` | One direct shadow ray per pixel per frame (random source) instead of one per light. The governor turns it off once it has scaled resolution up on strong hardware. |
|
|
291
373
|
| `temporalReprojection` | `true` | Keep samples across camera/object motion. |
|
|
292
374
|
| `maxHistory` | `128` | History cap — higher is smoother, slower to react. |
|
|
293
375
|
| `fireflyClamp` | `4.0` | Clamp on indirect luminance to suppress fireflies. |
|
|
@@ -300,9 +382,42 @@ rasterizes and gets lit — useful for water / translucent surfaces).
|
|
|
300
382
|
Transparent materials never act as occluders (a glass case shouldn't cast an
|
|
301
383
|
opaque shadow); `alphaTest` cut-outs still do.
|
|
302
384
|
|
|
385
|
+
## Edge convergence and overscan
|
|
386
|
+
|
|
387
|
+
Lighting is accumulated over time (temporal reprojection). When the camera
|
|
388
|
+
moves, pixels newly revealed at the **leading screen edge** have no history to
|
|
389
|
+
reproject from, so they start from a single noisy sample and take several frames
|
|
390
|
+
to converge — a shimmering band that rides the edge you are turning toward.
|
|
391
|
+
|
|
392
|
+
`overscan` hides it by rendering *bigger than the screen*. Every internal pass
|
|
393
|
+
(G-buffer, lighting, denoise, volumetric, composite, TAA history) runs at a
|
|
394
|
+
padded resolution with a proportionally **widened field of view**, and only the
|
|
395
|
+
final on-screen draw crops the central canvas-sized region out. Disoccluded
|
|
396
|
+
pixels are then born in the padding — off-screen — and have already converged by
|
|
397
|
+
the time the camera turns far enough to bring them into view.
|
|
398
|
+
|
|
399
|
+
```js
|
|
400
|
+
const rt = new RealtimeRaytracer(renderer, { overscan: 0.1 });
|
|
401
|
+
// or live: rt.overscan = 0.05; (reallocates targets; resets accumulation)
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
`overscan` is the padding fraction **per edge**. `0.1` on a 1000×600 canvas
|
|
405
|
+
renders 1200×720 internally and crops the central 1000×600 — both axes pad by
|
|
406
|
+
the same fraction, so aspect ratio is preserved and the widened frustum stays
|
|
407
|
+
centred on your camera's. The cost is purely the extra pixels: `1 + 2·overscan`
|
|
408
|
+
per axis, so **0.1 → 1.44×** the work of every pass. **0.05–0.1** is the useful
|
|
409
|
+
range; more just spends pixels on padding you will rarely turn fast enough to
|
|
410
|
+
need. Changing it live reallocates the targets and resets accumulation, so treat
|
|
411
|
+
it as a settings-time knob rather than a per-frame one. The camera you pass to
|
|
412
|
+
`render()` is never mutated — the widened projection is applied and restored
|
|
413
|
+
internally each frame, exactly like the TAA jitter.
|
|
414
|
+
|
|
303
415
|
## Running everywhere (capability tiers)
|
|
304
416
|
|
|
305
|
-
|
|
417
|
+
The zero-config defaults are **conservative and self-scaling**: construction
|
|
418
|
+
starts low-but-ray-traced and the adaptive governor (on by default) walks
|
|
419
|
+
quality up or down toward `targetFps`. The knobs below let you set a smarter
|
|
420
|
+
starting point or take manual control:
|
|
306
421
|
|
|
307
422
|
- **No usable GPU** (missing WebGL2 float targets, or a software rasterizer
|
|
308
423
|
like SwiftShader): the library logs one console warning and `rt.render()`
|
|
@@ -321,7 +436,30 @@ Nothing adaptive is imposed — everything below is **opt-in**:
|
|
|
321
436
|
});
|
|
322
437
|
```
|
|
323
438
|
|
|
324
|
-
- **
|
|
439
|
+
- **GPU probe** (`await RealtimeRaytracer.probeGPUTier(renderer?)`): an optional,
|
|
440
|
+
async, more-informed alternative to `detectTier`. When the browser exposes
|
|
441
|
+
**WebGPU** it inspects the real adapter limits; otherwise it falls back to the
|
|
442
|
+
WebGL heuristic. Returns `{ tier, source: "webgpu"|"webgl"|"fallback", details }`.
|
|
443
|
+
|
|
444
|
+
```js
|
|
445
|
+
const probe = await RealtimeRaytracer.probeGPUTier(renderer);
|
|
446
|
+
const rt = new RealtimeRaytracer(renderer, RealtimeRaytracer.recommendedOptions(probe.tier));
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
**Honest heuristic — WebGPU does NOT expose VRAM.** There is no API for actual
|
|
450
|
+
video memory, so the probe uses `adapter.limits` (`maxBufferSize`,
|
|
451
|
+
`maxTextureDimension2D`, …) as a *proxy* for GPU class — two cards with wildly
|
|
452
|
+
different VRAM can report the same limits. `adapter.info`
|
|
453
|
+
(vendor/architecture/description) is masked on many browsers, so it is a hint
|
|
454
|
+
only. Screen resolution is factored in: `screenPixels = screen.width *
|
|
455
|
+
screen.height * min(devicePixelRatio, 2)`; a 4K-class panel (`>= 6e6`) has to
|
|
456
|
+
fill ~4× the pixels of 1080p, so a GPU is only rated `"high"` on such a screen
|
|
457
|
+
when `maxBufferSize >= 4GiB` (otherwise it is demoted to `"mid"`). Software
|
|
458
|
+
renderers (SwiftShader / llvmpipe) rate `"none"`. Every threshold and the raw
|
|
459
|
+
values are echoed back in `details`. The constructor stays synchronous — this
|
|
460
|
+
is a pre-construction, opt-in call.
|
|
461
|
+
|
|
462
|
+
- **Adaptive quality** (`adaptiveQuality: true`, default **true**): continuous
|
|
325
463
|
dynamic resolution scaling. Watches real frame time and steers the lighting
|
|
326
464
|
resolution smoothly toward `targetFps` (in 5% steps with a cooldown), pairing
|
|
327
465
|
low resolutions with MORE denoise passes (they run at lighting res, so
|
|
@@ -342,16 +480,29 @@ npm install
|
|
|
342
480
|
npm run dev # http://localhost:8115
|
|
343
481
|
```
|
|
344
482
|
|
|
345
|
-
The demo ([`examples/`](examples/)) is
|
|
483
|
+
The demo ([`examples/`](examples/)) is a **panoramic museum gallery** — a
|
|
484
|
+
Cornell-style room (saturated red/teal side walls for obvious colour bleed, open
|
|
485
|
+
top) staged as an exhibit where every renderer feature gets its own vignette: a
|
|
486
|
+
**deforming mirror-water pool** under the emissive gallery light, the
|
|
487
|
+
DamagedHelmet hero on a pedestal under a toggleable **spotlight** (normal /
|
|
488
|
+
roughness maps + analytic-light glints), a glossy teapot showing **GGX dielectric
|
|
489
|
+
specular** against the teal wall, a gold torus knot and mirror sphere (traced
|
|
490
|
+
reflections), a glass sphere (refraction), a roughness ramp on plinths, and the
|
|
491
|
+
**duck in a glass vitrine** (alpha-blended transparency that casts no shadow onto
|
|
492
|
+
the exhibit). An always-on **fps readout** sits top-left and a **collapsible
|
|
493
|
+
control panel** (starts collapsed on phones) toggles every feature, drives a
|
|
494
|
+
clerestory-window light slider, and spawns the 40-body physics pile. See
|
|
346
495
|
[`examples/main.js`](examples/main.js) for the full, commented integration
|
|
347
496
|
(scene → physics → compile → render loop). `npm run deploy` builds and publishes
|
|
348
497
|
it to GitHub Pages.
|
|
349
498
|
|
|
350
499
|
## Gallery & benchmarks
|
|
351
500
|
|
|
352
|
-
[`gallery.html`](gallery.html)
|
|
353
|
-
|
|
354
|
-
|
|
501
|
+
[`gallery.html`](gallery.html) opens on a built-in **Cornell box** (the classic
|
|
502
|
+
GI reference — coloured walls, an emissive ceiling panel) and drops the raytracer
|
|
503
|
+
into **stock glTF scenes it was never authored for** — Littlest Tokyo, Lantern,
|
|
504
|
+
Damaged Helmet, Antique Camera, BoomBox, Corset, Water Bottle, Toy Car,
|
|
505
|
+
Iridescence Lamp, Mosquito in Amber, and Fox — streamed straight from their
|
|
355
506
|
public hosts (no assets committed). A one-button toggle A/Bs ray tracing against
|
|
356
507
|
plain rasterized three.js with an fps + triangle readout, and a compact options
|
|
357
508
|
strip exposes GI / emissive NEE / reflections / refraction / ReSTIR / denoise /
|
|
@@ -361,6 +512,22 @@ TAA / volumetric plus lighting-resolution and auto-quality controls.
|
|
|
361
512
|
GPU-**fence-timed** frame costs and a temporal **ghosting metric**, writing each
|
|
362
513
|
run's results to [`bench-results/`](bench-results/) for tracking regressions.
|
|
363
514
|
|
|
515
|
+
### Movement-artifact harness
|
|
516
|
+
|
|
517
|
+
[`harness.html`](harness.html) makes the **edge-of-screen convergence noise seen
|
|
518
|
+
while the camera moves** measurable and eyeball-able. It drives the demo scene
|
|
519
|
+
along a deterministic path (`strafe` — sinusoidal side-to-side — or `orbit`;
|
|
520
|
+
pose is a pure function of sim-time) and, every couple of frames, reads three
|
|
521
|
+
vertical bands off the drawing buffer — the outer 10% at each edge and the
|
|
522
|
+
central 10% — tracking **per-pixel temporal luminance variance** over a sliding
|
|
523
|
+
window. The HUD reports the three mean-variance numbers; the headline is the
|
|
524
|
+
**edge-vs-center ratio** (>1 = edges noisier than the middle — the artifact the
|
|
525
|
+
overscan feature targets). A magnified side-by-side inset shows a left-edge strip
|
|
526
|
+
next to a center strip for human comparison, and the metric triple is logged as a
|
|
527
|
+
JSON line to the console every 2s for automated scraping. The overscan control
|
|
528
|
+
is **feature-detected** — it appears only when the loaded build exposes an
|
|
529
|
+
`overscan` property.
|
|
530
|
+
|
|
364
531
|
## Roadmap
|
|
365
532
|
|
|
366
533
|
| Stage | Status | What |
|
|
@@ -375,7 +542,8 @@ run's results to [`bench-results/`](bench-results/) for tracking regressions.
|
|
|
375
542
|
| 6. Specular | ✅ | Mirror/glossy reflections on metals + two-interface glass refraction |
|
|
376
543
|
| 6b. Sampling | ✅ | Blue-noise sampling + ReSTIR direct lighting (temporal + spatial reuse) |
|
|
377
544
|
| 6c. Any-hit shadows | ✅ | Unordered early-out BVH traversal for occlusion rays — same image, up to ~2× cheaper shadows |
|
|
378
|
-
|
|
|
545
|
+
| 6d. PBR materials | ✅ | Cook-Torrance GGX dielectric specular + normal/roughness/metalness maps, alpha-blended transparency, deforming (water) meshes, overscan |
|
|
546
|
+
| 7. Next | — | Skinned-mesh (animated) shadows; DDGI irradiance probes; ReSTIR GI (indirect reservoir reuse); WGSL / WebGPU backend; per-material IOR |
|
|
379
547
|
|
|
380
548
|
## Credits
|
|
381
549
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "three-realtime-rt",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Turn-on ray traced lighting for three.js: a hybrid deferred renderer with BVH-traced soft shadows, one-bounce global illumination, temporal reprojection, an a-trous denoiser and TAA.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "src/index.js",
|
package/src/CompositePass.js
CHANGED
|
@@ -19,6 +19,8 @@ in vec2 vUv;
|
|
|
19
19
|
${SKY_GLSL}
|
|
20
20
|
|
|
21
21
|
uniform sampler2D uIrradiance;
|
|
22
|
+
uniform sampler2D uSpecular; // dielectric direct specular (added WITHOUT albedo)
|
|
23
|
+
uniform bool uSpecEnabled;
|
|
22
24
|
uniform sampler2D uGAlbedoRough;
|
|
23
25
|
uniform sampler2D uGNormalMetal;
|
|
24
26
|
uniform sampler2D uGWorldPos;
|
|
@@ -27,7 +29,8 @@ uniform sampler2D uVolumetric; // in-scattered light (quarter canvas res, smooth
|
|
|
27
29
|
uniform vec2 uVolTexelSize;
|
|
28
30
|
uniform bool uVolEnabled;
|
|
29
31
|
uniform vec3 uBackgroundColor;
|
|
30
|
-
// 0 composite, 1 albedo, 2 normal, 3 irradiance (direct+GI), 4 worldPos,
|
|
32
|
+
// 0 composite, 1 albedo, 2 normal, 3 irradiance (direct+GI), 4 worldPos,
|
|
33
|
+
// 5 emissive, 6 specular
|
|
31
34
|
uniform int uOutputMode;
|
|
32
35
|
|
|
33
36
|
// joint bilateral upsample (lighting may be rendered below full resolution)
|
|
@@ -35,6 +38,11 @@ uniform bool uUpsample;
|
|
|
35
38
|
uniform vec2 uIrrTexelSize;
|
|
36
39
|
uniform vec3 uCameraPos;
|
|
37
40
|
|
|
41
|
+
// Overscan crop: maps this on-screen pixel's UV into the central region of the
|
|
42
|
+
// padded internal image (scale.xy, offset.zw). Identity (1,1,0,0) when overscan
|
|
43
|
+
// is 0 or when compositing into the offscreen target that TAA later crops.
|
|
44
|
+
uniform vec4 uCrop;
|
|
45
|
+
|
|
38
46
|
// distance fog (applied in linear space, before tonemap)
|
|
39
47
|
uniform bool uFogEnabled;
|
|
40
48
|
uniform vec3 uFogColor;
|
|
@@ -63,8 +71,8 @@ vec3 acesFilm(vec3 x) {
|
|
|
63
71
|
// Upsample low-res irradiance to this full-res pixel: 4 nearest low-res taps,
|
|
64
72
|
// bilinear weights modulated by geometric similarity (plane distance + normal)
|
|
65
73
|
// so lighting never bleeds across depth or orientation discontinuities.
|
|
66
|
-
vec3
|
|
67
|
-
if (!uUpsample) return texture(
|
|
74
|
+
vec3 upsampleGuided(sampler2D tex, vec2 uv, vec3 P, vec3 N) {
|
|
75
|
+
if (!uUpsample) return texture(tex, uv).rgb;
|
|
68
76
|
|
|
69
77
|
float planeTol = 0.01 * distance(P, uCameraPos) + 1e-3;
|
|
70
78
|
vec2 base = uv / uIrrTexelSize - 0.5;
|
|
@@ -80,7 +88,7 @@ vec3 sampleIrradiance(vec2 uv, vec3 P, vec3 N) {
|
|
|
80
88
|
for (int dy = 0; dy <= 1; dy++) {
|
|
81
89
|
for (int dx = 0; dx <= 1; dx++) {
|
|
82
90
|
vec2 tuv = uv00 + vec2(float(dx), float(dy)) * uIrrTexelSize;
|
|
83
|
-
vec3 irr = texture(
|
|
91
|
+
vec3 irr = texture(tex, tuv).rgb;
|
|
84
92
|
float bw = (dx == 0 ? 1.0 - f.x : f.x) * (dy == 0 ? 1.0 - f.y : f.y);
|
|
85
93
|
if (bw > bestBilW) { bestBilW = bw; bestBil = irr; }
|
|
86
94
|
|
|
@@ -106,22 +114,41 @@ vec3 sampleIrradiance(vec2 uv, vec3 P, vec3 N) {
|
|
|
106
114
|
}
|
|
107
115
|
|
|
108
116
|
void main() {
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
117
|
+
// Sample the padded internal image at the cropped UV (identity when no crop).
|
|
118
|
+
// Everything below lives in padded space, so one remap here covers all taps.
|
|
119
|
+
vec2 uv = vUv * uCrop.xy + uCrop.zw;
|
|
120
|
+
vec4 wp = texture(uGWorldPos, uv);
|
|
121
|
+
vec4 albedoRough = texture(uGAlbedoRough, uv);
|
|
122
|
+
vec4 nmFull = texture(uGNormalMetal, uv);
|
|
123
|
+
vec3 N = normalize(nmFull.xyz);
|
|
124
|
+
vec3 irradiance = upsampleGuided(uIrradiance, uv, wp.xyz, N);
|
|
125
|
+
vec3 specular = uSpecEnabled ? upsampleGuided(uSpecular, uv, wp.xyz, N) : vec3(0.0);
|
|
126
|
+
vec3 emissive = texture(uGEmissive, uv).rgb;
|
|
114
127
|
|
|
115
128
|
vec3 color;
|
|
116
129
|
if (wp.w < 0.5) {
|
|
117
130
|
// Background: the procedural sky (with sun), else fog colour, else flat.
|
|
118
131
|
if (uSkyEnabled) {
|
|
119
|
-
color = skyColor(viewRay(
|
|
132
|
+
color = skyColor(viewRay(uv), uSunDir, uSunColor, uSkyZenith, uSkyHorizon, uSkyIntensity);
|
|
120
133
|
} else {
|
|
121
134
|
color = uFogEnabled ? uFogColor : uBackgroundColor;
|
|
122
135
|
}
|
|
123
136
|
} else {
|
|
124
|
-
|
|
137
|
+
// Diffuse is demodulated (albedo re-applied here); the dielectric specular
|
|
138
|
+
// highlight is white (F0 ~= 0.04) and is added WITHOUT the albedo multiply.
|
|
139
|
+
if (nmFull.w >= 4.0) {
|
|
140
|
+
// Alpha blend (packed word >= 4, opacity = w - 4): the irradiance slot
|
|
141
|
+
// holds the pane's own demodulated surface light and the SPECULAR slot
|
|
142
|
+
// carries the traced radiance from BEHIND the pane (see RTLightingPass) —
|
|
143
|
+
// the only place both quantities exist at final-pixel scale alongside the
|
|
144
|
+
// pane's albedo, so the opacity blend happens here. With the specular
|
|
145
|
+
// buffer disabled there is no behind-image; degrade to an opaque pane.
|
|
146
|
+
float opacity = clamp(nmFull.w - 4.0, 0.0, 1.0);
|
|
147
|
+
vec3 paneCol = albedoRough.rgb * irradiance + emissive;
|
|
148
|
+
color = uSpecEnabled ? mix(specular, paneCol, opacity) : paneCol;
|
|
149
|
+
} else {
|
|
150
|
+
color = albedoRough.rgb * irradiance + specular + emissive;
|
|
151
|
+
}
|
|
125
152
|
// Volumetric in-scatter (already radiance, not modulated by albedo). Fog
|
|
126
153
|
// is low-frequency, so a wide 9-tap blur costs nothing visually and eats
|
|
127
154
|
// the single-sample grain — crucial with MOVING lights, where the
|
|
@@ -130,15 +157,15 @@ void main() {
|
|
|
130
157
|
if (uVolEnabled) {
|
|
131
158
|
vec2 o1 = uVolTexelSize * 1.5;
|
|
132
159
|
vec2 o2 = uVolTexelSize * 3.5;
|
|
133
|
-
vec3 vol = texture(uVolumetric,
|
|
134
|
-
+ texture(uVolumetric,
|
|
135
|
-
+ texture(uVolumetric,
|
|
136
|
-
+ texture(uVolumetric,
|
|
137
|
-
+ texture(uVolumetric,
|
|
138
|
-
+ texture(uVolumetric,
|
|
139
|
-
+ texture(uVolumetric,
|
|
140
|
-
+ texture(uVolumetric,
|
|
141
|
-
+ texture(uVolumetric,
|
|
160
|
+
vec3 vol = texture(uVolumetric, uv).rgb * 0.24
|
|
161
|
+
+ texture(uVolumetric, uv + vec2( o1.x, o1.y)).rgb * 0.12
|
|
162
|
+
+ texture(uVolumetric, uv + vec2(-o1.x, o1.y)).rgb * 0.12
|
|
163
|
+
+ texture(uVolumetric, uv + vec2( o1.x, -o1.y)).rgb * 0.12
|
|
164
|
+
+ texture(uVolumetric, uv + vec2(-o1.x, -o1.y)).rgb * 0.12
|
|
165
|
+
+ texture(uVolumetric, uv + vec2( o2.x, 0.0 )).rgb * 0.07
|
|
166
|
+
+ texture(uVolumetric, uv + vec2(-o2.x, 0.0 )).rgb * 0.07
|
|
167
|
+
+ texture(uVolumetric, uv + vec2( 0.0 , o2.y)).rgb * 0.07
|
|
168
|
+
+ texture(uVolumetric, uv + vec2( 0.0 , -o2.y)).rgb * 0.07;
|
|
142
169
|
color += vol;
|
|
143
170
|
}
|
|
144
171
|
if (uFogEnabled) {
|
|
@@ -153,6 +180,7 @@ void main() {
|
|
|
153
180
|
else if (uOutputMode == 3) color = irradiance;
|
|
154
181
|
else if (uOutputMode == 4) color = fract(wp.xyz * 0.1);
|
|
155
182
|
else if (uOutputMode == 5) color = emissive;
|
|
183
|
+
else if (uOutputMode == 6) color = specular;
|
|
156
184
|
else color = acesFilm(color);
|
|
157
185
|
|
|
158
186
|
outColor = vec4(pow(color, vec3(1.0 / 2.2)), 1.0);
|
|
@@ -168,6 +196,8 @@ export class CompositePass {
|
|
|
168
196
|
fragmentShader: compositeFrag,
|
|
169
197
|
uniforms: {
|
|
170
198
|
uIrradiance: { value: null },
|
|
199
|
+
uSpecular: { value: null },
|
|
200
|
+
uSpecEnabled: { value: false },
|
|
171
201
|
uGAlbedoRough: { value: null },
|
|
172
202
|
uGNormalMetal: { value: null },
|
|
173
203
|
uGWorldPos: { value: null },
|
|
@@ -180,6 +210,7 @@ export class CompositePass {
|
|
|
180
210
|
uUpsample: { value: false },
|
|
181
211
|
uIrrTexelSize: { value: new THREE.Vector2() },
|
|
182
212
|
uCameraPos: { value: new THREE.Vector3() },
|
|
213
|
+
uCrop: { value: new THREE.Vector4(1, 1, 0, 0) },
|
|
183
214
|
uFogEnabled: { value: false },
|
|
184
215
|
uFogColor: { value: new THREE.Color(0.5, 0.6, 0.7) },
|
|
185
216
|
uFogDensity: { value: 0.05 },
|
|
@@ -202,13 +233,20 @@ export class CompositePass {
|
|
|
202
233
|
this.scene.add(this.quad);
|
|
203
234
|
}
|
|
204
235
|
|
|
205
|
-
|
|
236
|
+
// `crop` is the overscan central-crop transform (THREE.Vector4 scale.xy /
|
|
237
|
+
// offset.zw); pass null for no crop (identity) — used when compositing into the
|
|
238
|
+
// padded offscreen target that TAA crops later.
|
|
239
|
+
render(renderer, irradianceTexture, gbuffer, background, target = null, specularTexture = null, crop = null) {
|
|
206
240
|
const u = this.material.uniforms;
|
|
207
241
|
u.uIrradiance.value = irradianceTexture;
|
|
242
|
+
u.uSpecular.value = specularTexture;
|
|
243
|
+
u.uSpecEnabled.value = specularTexture !== null;
|
|
208
244
|
u.uGAlbedoRough.value = gbuffer.albedoRough;
|
|
209
245
|
u.uGNormalMetal.value = gbuffer.normalMetal;
|
|
210
246
|
u.uGWorldPos.value = gbuffer.worldPos;
|
|
211
247
|
u.uGEmissive.value = gbuffer.emissive;
|
|
248
|
+
if (crop) u.uCrop.value.copy(crop);
|
|
249
|
+
else u.uCrop.value.set(1, 1, 0, 0);
|
|
212
250
|
if (background && background.isColor) {
|
|
213
251
|
u.uBackgroundColor.value.copy(background);
|
|
214
252
|
}
|
package/src/DenoisePass.js
CHANGED
|
@@ -23,6 +23,7 @@ uniform float uStep; // à-trous step: 1, 2, 4, ...
|
|
|
23
23
|
uniform vec3 uCameraPos;
|
|
24
24
|
uniform float uEps;
|
|
25
25
|
uniform float uLumSigma;
|
|
26
|
+
uniform bool uBlendIsSpec; // this instance filters the specular buffer
|
|
26
27
|
|
|
27
28
|
float luminance(vec3 c) {
|
|
28
29
|
return dot(c, vec3(0.299, 0.587, 0.114));
|
|
@@ -42,8 +43,15 @@ void main() {
|
|
|
42
43
|
// detail is NOT in the G-buffer guides — filtering would smear them, and
|
|
43
44
|
// their signal is nearly deterministic anyway. Scale the filter down as the
|
|
44
45
|
// surface gets more mirror-like.
|
|
46
|
+
// Packed word ranges (see GBufferPass): [4,5] alpha blend, [2,4) glass,
|
|
47
|
+
// [0,1] metal. In the IRRADIANCE buffer a blend surface carries diffuse-lit
|
|
48
|
+
// direct + GI that DOES need filtering (specAmount 0); in the SPECULAR buffer
|
|
49
|
+
// (uBlendIsSpec) it carries the traced behind-the-pane image, which must be
|
|
50
|
+
// spared like a mirror — the pane is flat, so the G-buffer guides would let
|
|
51
|
+
// the filter smear the see-through content into mush.
|
|
45
52
|
float matW = nm.w;
|
|
46
|
-
float specAmount = matW >=
|
|
53
|
+
float specAmount = matW >= 4.0 ? (uBlendIsSpec ? 1.0 : 0.0)
|
|
54
|
+
: (matW >= 2.0 ? clamp(matW - 2.0, 0.0, 1.0) : matW);
|
|
47
55
|
float specKeep = specAmount * (1.0 - clamp(wp.w - 1.0, 0.0, 1.0));
|
|
48
56
|
|
|
49
57
|
// Fewer accumulated samples -> noisier pixel -> wider luminance tolerance.
|
|
@@ -123,7 +131,12 @@ void main() {
|
|
|
123
131
|
* pixels get blurred hard, converged pixels stay crisp.
|
|
124
132
|
*/
|
|
125
133
|
export class DenoisePass {
|
|
126
|
-
|
|
134
|
+
// `blendIsSpec`: the SPECULAR-buffer instance passes true — for blend pixels
|
|
135
|
+
// that buffer holds the traced behind-the-pane image, whose detail is not in
|
|
136
|
+
// the G-buffer guides (the pane itself is flat), so filtering would smear it
|
|
137
|
+
// into mush. The irradiance instance keeps false: there a blend pixel holds
|
|
138
|
+
// the pane's own diffuse-lit surface, which does want filtering.
|
|
139
|
+
constructor(width, height, { blendIsSpec = false } = {}) {
|
|
127
140
|
this.targetA = this._makeTarget(width, height);
|
|
128
141
|
this.targetB = this._makeTarget(width, height);
|
|
129
142
|
|
|
@@ -140,6 +153,7 @@ export class DenoisePass {
|
|
|
140
153
|
uCameraPos: { value: new THREE.Vector3() },
|
|
141
154
|
uEps: { value: 1e-3 },
|
|
142
155
|
uLumSigma: { value: 0.25 },
|
|
156
|
+
uBlendIsSpec: { value: blendIsSpec },
|
|
143
157
|
},
|
|
144
158
|
depthTest: false,
|
|
145
159
|
depthWrite: false,
|