gaussian-splat-lite 1.2.0 → 1.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +44 -2
- package/README.md +22 -13
- package/dist/gaussian-splat-lite.cjs +819 -88
- package/dist/gaussian-splat-lite.cjs.map +1 -1
- package/dist/gaussian-splat-lite.min.cjs +81 -20
- package/dist/gaussian-splat-lite.min.cjs.map +1 -1
- package/dist/gaussian-splat-lite.module.js +821 -90
- package/dist/gaussian-splat-lite.module.js.map +1 -1
- package/dist/gaussian-splat-lite.module.min.js +2984 -2427
- package/dist/gaussian-splat-lite.module.min.js.map +1 -1
- package/dist/types/addons/TAANode.d.cts +4 -2
- package/dist/types/addons/TAANode.d.ts +4 -2
- package/dist/types/addons/TAAPass.d.cts +6 -15
- package/dist/types/addons/TAAPass.d.ts +6 -15
- package/dist/types/addons/taaShared.d.cts +4 -2
- package/dist/types/addons/taaShared.d.ts +4 -2
- package/dist/types/capture/SplatCapture.d.cts +66 -0
- package/dist/types/capture/SplatCapture.d.ts +66 -0
- package/dist/types/capture/WebGLCapture.d.cts +13 -0
- package/dist/types/capture/WebGLCapture.d.ts +13 -0
- package/dist/types/capture/WebGPUCapture.d.cts +17 -0
- package/dist/types/capture/WebGPUCapture.d.ts +17 -0
- package/dist/types/capture/pixels.d.cts +5 -0
- package/dist/types/capture/pixels.d.ts +5 -0
- package/dist/types/capture/rendererState.d.cts +3 -0
- package/dist/types/capture/rendererState.d.ts +3 -0
- package/dist/types/capture/sceneView.d.cts +3 -0
- package/dist/types/capture/sceneView.d.ts +3 -0
- package/dist/types/index.d.cts +1 -1
- package/dist/types/index.d.ts +1 -1
- package/dist/types/rendering/GaussianSplatRenderer.d.cts +17 -11
- package/dist/types/rendering/GaussianSplatRenderer.d.ts +17 -11
- package/dist/types/rendering/SplatAccumulator.d.cts +2 -1
- package/dist/types/rendering/SplatAccumulator.d.ts +2 -1
- package/dist/types/rendering/backend.d.cts +1 -1
- package/dist/types/rendering/backend.d.ts +1 -1
- package/dist/types/rendering/rendererUtils.d.cts +1 -5
- package/dist/types/rendering/rendererUtils.d.ts +1 -5
- package/dist/types/rendering/tsl/ProjectionProgram.d.cts +2 -0
- package/dist/types/rendering/tsl/ProjectionProgram.d.ts +2 -0
- package/dist/types/rendering/tsl/SplatMaterial.d.cts +1 -0
- package/dist/types/rendering/tsl/SplatMaterial.d.ts +1 -0
- package/dist/types/rendering/tsl/tslCompat.d.cts +1 -0
- package/dist/types/rendering/tsl/tslCompat.d.ts +1 -0
- package/dist/types/rendering/uniforms.d.cts +3 -3
- package/dist/types/rendering/uniforms.d.ts +3 -3
- package/dist/types/scene/SplatMesh.d.cts +1 -1
- package/dist/types/scene/SplatMesh.d.ts +1 -1
- package/docs/Architecture.md +54 -3
- package/docs/GaussianSplatRenderer.md +71 -55
- package/docs/PostDecode.md +52 -36
- package/docs/RadStreamScheduler.md +53 -41
- package/docs/SogStreamScheduler.md +49 -39
- package/docs/SplatCapture.md +147 -0
- package/docs/SplatEdit.md +33 -26
- package/docs/SplatFileType.md +11 -13
- package/docs/SplatLoader.md +19 -14
- package/docs/SplatMesh.md +69 -49
- package/docs/Splats.md +33 -26
- package/docs/StochasticRendering.md +27 -30
- package/docs/TAAPass.md +98 -73
- package/package.json +1 -1
- package/docs/SplatAccumulator.md +0 -37
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[Back to documentation](../README.md#documentation)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Shows large RAD scenes by loading only the detail the camera needs, and updating it as the camera moves. It supports RAD version 1 files with levels of detail (LOD), either as a single `.rad` file or split into a header and `.radc` pages. It works on WebGPU and WebGL2, and needs no Spark runtime.
|
|
6
6
|
|
|
7
7
|
```js
|
|
8
8
|
import { RadStreamScheduler } from "gaussian-splat-lite";
|
|
@@ -21,7 +21,7 @@ renderer.setAnimationLoop(() => {
|
|
|
21
21
|
renderer.render(scene, camera);
|
|
22
22
|
});
|
|
23
23
|
|
|
24
|
-
//
|
|
24
|
+
// Resolves when the first data is on screen. update() must be running.
|
|
25
25
|
await streaming.firstRenderable;
|
|
26
26
|
|
|
27
27
|
// When removing the model:
|
|
@@ -29,11 +29,23 @@ await streaming.firstRenderable;
|
|
|
29
29
|
// streaming.group.removeFromParent();
|
|
30
30
|
```
|
|
31
31
|
|
|
32
|
-
|
|
32
|
+
The three steps that matter:
|
|
33
33
|
|
|
34
|
-
|
|
34
|
+
1. Add `streaming.group` to the scene. Move, rotate, scale, or hide the scene through this group.
|
|
35
|
+
2. Register the camera and its resolution.
|
|
36
|
+
3. Call `streaming.update()` before every render.
|
|
35
37
|
|
|
36
|
-
|
|
38
|
+
Streamed models support [SDF edits](SplatEdit.md) and [picking](SplatMesh.md#raycasting). Their data is read-only.
|
|
39
|
+
|
|
40
|
+
## Cameras and resolution
|
|
41
|
+
|
|
42
|
+
Detail depends on how large the scene appears, so each camera needs a resolution. It is in CSS pixels, without the device pixel ratio, so a scene loads the same detail on standard and high-DPI displays. Set it again whenever the canvas is resized.
|
|
43
|
+
|
|
44
|
+
Register several cameras to load detail for all of them. They share one selection, at the highest detail any of them needs.
|
|
45
|
+
|
|
46
|
+
### WebXR
|
|
47
|
+
|
|
48
|
+
Register `renderer.xr.getCamera()` in place of your camera. Detail is chosen for each eye at the headset's resolution, so no resolution is needed:
|
|
37
49
|
|
|
38
50
|
```js
|
|
39
51
|
const xrCamera = renderer.xr.getCamera();
|
|
@@ -53,50 +65,48 @@ renderer.setAnimationLoop(() => {
|
|
|
53
65
|
});
|
|
54
66
|
```
|
|
55
67
|
|
|
56
|
-
Ordinary camera matrices are updated by the scheduler. The XR camera's matrices are read directly from Three.js, so detail selection before rendering may use the previous frame's pose.
|
|
57
|
-
|
|
58
|
-
For original Splat IDs, use the picking hit's [`sourceIndex`](SplatMesh.md#raycasting).
|
|
59
|
-
|
|
60
68
|
## Options
|
|
61
69
|
|
|
62
70
|
| Option | Default | Description |
|
|
63
71
|
| --- | --- | --- |
|
|
64
72
|
| `url` / `file` / `fileBytes` | Exactly one required | URL, `Blob`/`File`, or complete `Uint8Array`/`ArrayBuffer` |
|
|
65
|
-
| `resolveFile` | Relative
|
|
66
|
-
| `group` | New `THREE.Group` |
|
|
67
|
-
| `splatBudget` | `3_000_000` | Maximum
|
|
68
|
-
| `
|
|
69
|
-
| `
|
|
70
|
-
| `maxConcurrentLoads` | `4` | Maximum
|
|
71
|
-
| `manager` | `THREE.DefaultLoadingManager` |
|
|
72
|
-
| `requestHeader` / `withCredentials` | `{}` / `false` | Request settings
|
|
73
|
-
| `onChange` | — |
|
|
74
|
-
| `onError` |
|
|
73
|
+
| `resolveFile` | Relative URLs | Supply the `.radc` pages of a split scene yourself: `(filename, signal) => URL \| Blob \| Uint8Array \| ArrayBuffer`, directly or as a promise |
|
|
74
|
+
| `group` | New `THREE.Group` | Group that holds the scene |
|
|
75
|
+
| `splatBudget` | `3_000_000` | Maximum number of Splats shown at once. Higher values show more detail and use more memory |
|
|
76
|
+
| `fadeDurationMs` | `200` | Duration of the fade when the scene first appears and when detail changes; `0` switches instantly |
|
|
77
|
+
| `cooldownMs` | `2000` | How long detail that is no longer shown stays in memory, ready for reuse; `0` releases it at once |
|
|
78
|
+
| `maxConcurrentLoads` | `4` | Maximum number of loads in progress at the same time |
|
|
79
|
+
| `manager` | `THREE.DefaultLoadingManager` | Three.js loading manager, including its URL modifiers |
|
|
80
|
+
| `requestHeader` / `withCredentials` | `{}` / `false` | Request settings. Sensitive settings are not sent to other origins |
|
|
81
|
+
| `onChange` | — | Called when the scene needs a redraw |
|
|
82
|
+
| `onError` | Logs to the console | Called when loading fails: `(error, url)` |
|
|
75
83
|
|
|
76
|
-
|
|
84
|
+
Detail that is already on screen stays there while its replacement loads. During a fade, old and new detail overlap, so the scene can briefly exceed `splatBudget`; use `fadeDurationMs: 0` to prevent this. The budget limits what is shown, not total memory: detail kept for reuse takes memory too.
|
|
77
85
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
## Common properties and methods
|
|
86
|
+
## Properties and methods
|
|
81
87
|
|
|
82
88
|
| API | Description |
|
|
83
89
|
| --- | --- |
|
|
84
|
-
| `group` |
|
|
85
|
-
| `splatBudget` |
|
|
86
|
-
| `initialized` | Resolves when scene
|
|
87
|
-
| `firstRenderable` | Resolves when the first data
|
|
88
|
-
| `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that
|
|
89
|
-
| `setResolution(camera, width, height)` | Set a registered camera's
|
|
90
|
-
| `setResolutionFromRenderer(camera, renderer)` | Set a registered camera's
|
|
91
|
-
| `update()` |
|
|
92
|
-
| `getBoundingBox()` | Approximate
|
|
93
|
-
| `getGlobalIndex(mesh, renderedIndex)` | Convert a picking
|
|
94
|
-
| `stats` |
|
|
95
|
-
| `dispose()` |
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
90
|
+
| `group` | Group that holds the scene |
|
|
91
|
+
| `splatBudget` | Change it at any time to adjust detail, even while the camera is still. Must be a positive integer |
|
|
92
|
+
| `initialized` | Resolves when the scene is set up; rejects for invalid or unsupported files |
|
|
93
|
+
| `firstRenderable` | Resolves when the first data is on screen, or the scene is empty; rejects if the first data cannot be loaded. Keep calling `update()` while you wait |
|
|
94
|
+
| `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that chooses detail. `hasCamera(camera)` and `cameras` report the registered ones |
|
|
95
|
+
| `setResolution(camera, width, height)` | Set a registered camera's size in CSS pixels; also accepts a `THREE.Vector2` |
|
|
96
|
+
| `setResolutionFromRenderer(camera, renderer)` | Set a registered camera's size from `renderer.getSize()` |
|
|
97
|
+
| `update()` | Advance detail selection, loading, and fades. Call it before every render; it throws if no camera is registered |
|
|
98
|
+
| `getBoundingBox()` | Approximate box around the scene, in the group's coordinates; empty until the first data loads |
|
|
99
|
+
| `getGlobalIndex(mesh, renderedIndex)` | Convert a picking hit into a node index that stays the same for the whole RAD file |
|
|
100
|
+
| `stats` | Numbers for monitoring, such as `visibleSplats`, `loadingChunks`, `downloadedBytes`, and `residentBytes`. Memory figures are estimates, not total browser memory |
|
|
101
|
+
| `dispose()` | Stop loading and release the scene. Pending `initialized` and `firstRenderable` promises reject with `AbortError` |
|
|
102
|
+
|
|
103
|
+
### On-demand rendering
|
|
104
|
+
|
|
105
|
+
Request a redraw from `onChange`, and keep calling `update()` on every animation tick, also when you skip the render. Loading, retries, and fades only make progress inside `update()`.
|
|
106
|
+
|
|
107
|
+
### Picking
|
|
108
|
+
|
|
109
|
+
Hits from a streamed scene refer to the detail that is currently shown. Use `getGlobalIndex()`, or the hit's [`sourceIndex`](SplatMesh.md#raycasting), for an ID that does not change:
|
|
100
110
|
|
|
101
111
|
```js
|
|
102
112
|
const hits = raycaster.intersectObject(streaming.group, true);
|
|
@@ -108,6 +118,8 @@ if (hits.length && hits[0].index !== undefined) {
|
|
|
108
118
|
|
|
109
119
|
## Error recovery
|
|
110
120
|
|
|
111
|
-
Temporary download failures
|
|
121
|
+
Temporary download failures are retried automatically, waiting longer after each attempt, up to 30 seconds. This covers network errors and HTTP 408, 425, 429, and 5xx responses. Each retry calls `onChange`, so an on-demand render loop resumes loading when the connection returns.
|
|
122
|
+
|
|
123
|
+
Invalid data, out-of-memory errors, and other HTTP errors are reported to `onError` and are not retried.
|
|
112
124
|
|
|
113
|
-
If
|
|
125
|
+
If the scheduler can no longer choose detail after an internal failure, it calls `onError` and keeps showing its current detail without refining further. Create a new scheduler to try again.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[Back to documentation](../README.md#documentation)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Shows large SOG `lod-meta.json` scenes by loading only the detail the camera needs, and updating it as the camera moves. It supports version 1 and older unversioned indexes, on WebGPU and WebGL2. For an ordinary `.sog` file or `meta.json`, use [SplatMesh](SplatMesh.md).
|
|
6
6
|
|
|
7
7
|
```js
|
|
8
8
|
import { SogStreamScheduler } from "gaussian-splat-lite";
|
|
@@ -15,65 +15,75 @@ scene.add(streaming.group);
|
|
|
15
15
|
streaming.setCamera(camera);
|
|
16
16
|
await streaming.initialized;
|
|
17
17
|
|
|
18
|
-
|
|
19
|
-
streaming.update();
|
|
20
|
-
renderer.render(scene, camera);
|
|
18
|
+
renderer.setAnimationLoop(() => {
|
|
19
|
+
streaming.update();
|
|
20
|
+
renderer.render(scene, camera);
|
|
21
|
+
});
|
|
22
|
+
|
|
23
|
+
// Resolves when the first data is on screen. update() must be running.
|
|
24
|
+
await streaming.firstRenderable;
|
|
21
25
|
|
|
22
26
|
// When removing the model:
|
|
23
|
-
streaming.dispose();
|
|
24
|
-
streaming.group.removeFromParent();
|
|
27
|
+
// streaming.dispose();
|
|
28
|
+
// streaming.group.removeFromParent();
|
|
25
29
|
```
|
|
26
30
|
|
|
27
|
-
|
|
31
|
+
The three steps that matter:
|
|
32
|
+
|
|
33
|
+
1. Add `streaming.group` to the scene. Move, rotate, scale, or hide the scene through this group.
|
|
34
|
+
2. Register the camera.
|
|
35
|
+
3. Call `streaming.update()` before every render.
|
|
28
36
|
|
|
29
|
-
|
|
37
|
+
Streamed models support [SDF edits](SplatEdit.md) and [picking](SplatMesh.md#raycasting). Their data is read-only. For a Splat's original ID, use the picking hit's [`sourceIndex`](SplatMesh.md#raycasting).
|
|
30
38
|
|
|
31
|
-
|
|
39
|
+
## Cameras
|
|
32
40
|
|
|
33
|
-
|
|
41
|
+
Detail depends on distance from the camera, so cameras need no resolution. Register several cameras to load detail for all of them; each part of the scene follows the nearest camera that sees it.
|
|
42
|
+
|
|
43
|
+
In WebXR, register `renderer.xr.getCamera()` in place of your camera, as shown for [RadStreamScheduler](RadStreamScheduler.md#webxr). Detail is chosen for each eye.
|
|
34
44
|
|
|
35
45
|
## Options
|
|
36
46
|
|
|
37
47
|
| Option | Default | Description |
|
|
38
48
|
| --- | --- | --- |
|
|
39
|
-
| `url` | Required |
|
|
40
|
-
| `group` | New `THREE.Group` |
|
|
41
|
-
| `splatBudget` | `3_000_000` | Target
|
|
42
|
-
| `
|
|
43
|
-
| `
|
|
44
|
-
| `maxConcurrentLoads` | `4` | Maximum
|
|
45
|
-
| `manager` | `THREE.DefaultLoadingManager` |
|
|
46
|
-
| `requestHeader` / `withCredentials` | `{}` / `false` |
|
|
47
|
-
| `onChange` | — |
|
|
48
|
-
| `onError` |
|
|
49
|
-
| `loadChunk` | Built-in
|
|
50
|
-
|
|
51
|
-
The Splat budget is a target: available detail levels, fades, and the environment can exceed it. Cached data also uses memory, so the budget is not a total memory limit.
|
|
49
|
+
| `url` | Required | URL of the scene's `lod-meta.json` |
|
|
50
|
+
| `group` | New `THREE.Group` | Group that holds the scene |
|
|
51
|
+
| `splatBudget` | `3_000_000` | Target number of Splats shown at once, including the environment. Higher values show more detail and use more memory |
|
|
52
|
+
| `fadeDurationMs` | `200` | Duration of the fade when the scene first appears and when detail changes; `0` switches instantly |
|
|
53
|
+
| `cooldownMs` | `2000` | How long detail that is no longer shown stays in memory, ready for reuse; `0` releases it at once |
|
|
54
|
+
| `maxConcurrentLoads` | `4` | Maximum number of loads in progress at the same time |
|
|
55
|
+
| `manager` | `THREE.DefaultLoadingManager` | Three.js loading manager, including its URL modifiers |
|
|
56
|
+
| `requestHeader` / `withCredentials` | `{}` / `false` | Request settings. Headers and credentials are not sent to files on other origins |
|
|
57
|
+
| `onChange` | — | Called when the scene needs a redraw |
|
|
58
|
+
| `onError` | Logs to the console | Called when loading fails: `(error, url)` |
|
|
59
|
+
| `loadChunk` | Built-in loader | Load the scene's files yourself: `(url, signal) => Promise<Splats>` |
|
|
52
60
|
|
|
53
|
-
|
|
61
|
+
The budget is a target, not a hard limit: the available detail levels, fades, and the environment can take the scene above it. It limits what is shown, not total memory: detail kept for reuse takes memory too.
|
|
54
62
|
|
|
55
|
-
|
|
63
|
+
Some parts of a scene exist only at finer detail levels. They can be empty from far away and fade in as the camera approaches.
|
|
56
64
|
|
|
57
|
-
##
|
|
65
|
+
## Properties and methods
|
|
58
66
|
|
|
59
67
|
| API | Description |
|
|
60
68
|
| --- | --- |
|
|
61
|
-
| `group` |
|
|
62
|
-
| `splatBudget` |
|
|
63
|
-
| `initialized` | Resolves when the index is
|
|
64
|
-
| `firstRenderable` | Resolves when the first data
|
|
65
|
-
| `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that
|
|
66
|
-
| `update()` |
|
|
67
|
-
| `getBoundingBox()` |
|
|
68
|
-
| `stats` |
|
|
69
|
-
| `dispose()` |
|
|
69
|
+
| `group` | Group that holds the scene |
|
|
70
|
+
| `splatBudget` | Change it at any time to adjust detail. Must be a positive integer; includes the environment |
|
|
71
|
+
| `initialized` | Resolves when the index is loaded; rejects for invalid input |
|
|
72
|
+
| `firstRenderable` | Resolves when the first data is on screen, or the scene is empty; rejects if the first data cannot be loaded. Keep calling `update()` while you wait |
|
|
73
|
+
| `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that chooses detail. `hasCamera(camera)` and `cameras` report the registered ones |
|
|
74
|
+
| `update()` | Advance detail selection, loading, and fades. Call it before every render; it throws if no camera is registered |
|
|
75
|
+
| `getBoundingBox()` | Box around the scene, in the group's coordinates; available once `initialized` resolves |
|
|
76
|
+
| `stats` | Numbers for monitoring, such as `visibleSplats`, `visibleRegions`, `loadingChunks`, `downloadedBytes`, and `residentBytes`. Memory figures are estimates, not total browser memory |
|
|
77
|
+
| `dispose()` | Stop loading and release the scene. Pending `initialized` and `firstRenderable` promises reject with `AbortError` |
|
|
70
78
|
|
|
71
|
-
|
|
79
|
+
### On-demand rendering
|
|
72
80
|
|
|
73
|
-
|
|
81
|
+
Request a redraw from `onChange`, and keep calling `update()` on every animation tick, also when you skip the render. Loading, retries, and fades only make progress inside `update()`.
|
|
74
82
|
|
|
75
83
|
## Error recovery
|
|
76
84
|
|
|
77
|
-
Temporary download failures
|
|
85
|
+
Temporary download failures are retried automatically, waiting longer after each attempt, up to 30 seconds. This covers network errors and HTTP 408, 425, 429, and 5xx responses. Each retry calls `onChange`, so an on-demand render loop resumes loading when the connection returns.
|
|
86
|
+
|
|
87
|
+
Invalid data, out-of-memory errors, and other HTTP errors are reported to `onError` and are not retried. Parts of the scene that are already on screen stay visible when loading more detail fails.
|
|
78
88
|
|
|
79
|
-
|
|
89
|
+
If the scheduler can no longer choose detail after an internal failure, it calls `onError` and disposes itself. Create a new scheduler to try again.
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# SplatCapture
|
|
2
|
+
|
|
3
|
+
[Back to documentation](../README.md#documentation)
|
|
4
|
+
|
|
5
|
+
Renders a scene and its Splats offscreen: into a render target, as pixels you can read back, as a cube map, or as an environment map for reflections. It works with `WebGLRenderer` and with `WebGPURenderer` on both WebGPU and its WebGL2 fallback.
|
|
6
|
+
|
|
7
|
+
```js
|
|
8
|
+
import { SplatCapture } from "gaussian-splat-lite";
|
|
9
|
+
|
|
10
|
+
const capture = new SplatCapture({
|
|
11
|
+
splatRenderer, // The GaussianSplatRenderer already in your scene.
|
|
12
|
+
target: { width: 1920, height: 1080 },
|
|
13
|
+
});
|
|
14
|
+
|
|
15
|
+
const rgba = await capture.renderReadTarget({ scene, camera });
|
|
16
|
+
// Uint8Array of 1920 × 1080 × 4 bytes, bottom row first.
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Captures do not disturb what is on screen. Your render loop can keep running while one is in progress, and the display keeps its own camera and settings.
|
|
20
|
+
|
|
21
|
+
[SDF edits](SplatEdit.md) apply to captures as they do on screen, whether they are attached to the scene or to a model.
|
|
22
|
+
|
|
23
|
+
Coming from 1.1.8? These were methods and options of `GaussianSplatRenderer`. They keep the same names on `SplatCapture`, except the cube captures' `update` option, which is gone: every capture shows the current scene and reuses earlier work while nothing has changed.
|
|
24
|
+
|
|
25
|
+
## Render to a target
|
|
26
|
+
|
|
27
|
+
Pass `target` to capture 2D images. It accepts the options below plus `THREE.RenderTargetOptions`, except `stencilBuffer`, which follows the Three.js renderer.
|
|
28
|
+
|
|
29
|
+
| Option | Default | Description |
|
|
30
|
+
| --- | --- | --- |
|
|
31
|
+
| `width` / `height` | Required | Image size in pixels |
|
|
32
|
+
| `superXY` | `1` | Supersampling, an integer from 1 to 4. The scene renders at `superXY` times the size on each axis and is averaged down when read back |
|
|
33
|
+
| `doubleBuffer` | `false` | Alternate between two targets, so `capture.target` keeps a finished image while the next one renders |
|
|
34
|
+
|
|
35
|
+
`width` and `height` multiplied by `superXY` can be at most 8192.
|
|
36
|
+
|
|
37
|
+
| Method | Description |
|
|
38
|
+
| --- | --- |
|
|
39
|
+
| `renderTarget({ scene, camera })` | Render the view and return the `THREE.RenderTarget` holding it |
|
|
40
|
+
| `readTarget()` | Read the latest render as RGBA bytes |
|
|
41
|
+
| `renderReadTarget({ scene, camera })` | Do both in one call |
|
|
42
|
+
|
|
43
|
+
`capture.target` always holds the latest finished render. Use `capture.target.texture` as a texture; its size includes supersampling.
|
|
44
|
+
|
|
45
|
+
Pixels come back as a `Uint8Array` of `width × height × 4` bytes, bottom row first, in the same layout on every renderer. The array is reused by later reads, so copy it with `rgba.slice()` if you need to keep it.
|
|
46
|
+
|
|
47
|
+
Targets are 8-bit RGBA in sRGB unless you say otherwise:
|
|
48
|
+
|
|
49
|
+
- `samples` antialiases the meshes in the scene.
|
|
50
|
+
- Stencil masking follows the Three.js renderer's stencil setting. Stencil settings on your Splat renderer's `material` apply to the Splats in the capture too.
|
|
51
|
+
- `type: THREE.HalfFloatType` or `THREE.FloatType` keeps values above 1. Such targets can be rendered but not read back; reading requires 8-bit RGBA.
|
|
52
|
+
|
|
53
|
+
2D captures also keep the scene's depth in the target.
|
|
54
|
+
|
|
55
|
+
## Cube maps and environment maps
|
|
56
|
+
|
|
57
|
+
```js
|
|
58
|
+
const cube = await capture.renderCubeMap({
|
|
59
|
+
scene,
|
|
60
|
+
worldCenter: new THREE.Vector3(0, 1, 0),
|
|
61
|
+
size: 256,
|
|
62
|
+
hideObjects: [reflectiveMesh],
|
|
63
|
+
});
|
|
64
|
+
const faces = await capture.readCubeTargets(); // Six RGBA arrays.
|
|
65
|
+
|
|
66
|
+
const envMap = await capture.renderEnvMap({
|
|
67
|
+
scene,
|
|
68
|
+
worldCenter: reflectiveMesh.getWorldPosition(new THREE.Vector3()),
|
|
69
|
+
hideObjects: [reflectiveMesh],
|
|
70
|
+
});
|
|
71
|
+
capture.recurseSetEnvMap(reflectiveMesh, envMap);
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
`target` is not needed when you only capture cube or environment maps.
|
|
75
|
+
|
|
76
|
+
| Method | Description |
|
|
77
|
+
| --- | --- |
|
|
78
|
+
| `renderCubeMap(options)` | Render the six directions around a point and return a `THREE.CubeTexture` |
|
|
79
|
+
| `readCubeTargets()` | Read the latest cube map as six RGBA arrays |
|
|
80
|
+
| `renderEnvMap(options)` | Render a cube map and filter it into an environment map ready for `material.envMap` |
|
|
81
|
+
| `recurseSetEnvMap(root, envMap)` | Assign an environment map to every `MeshStandardMaterial` under `root` |
|
|
82
|
+
|
|
83
|
+
| Option | Default | Description |
|
|
84
|
+
| --- | --- | --- |
|
|
85
|
+
| `scene` | Required | Scene to capture |
|
|
86
|
+
| `worldCenter` | Required | Capture position in world space |
|
|
87
|
+
| `size` | `256` | Size of each face in pixels, up to 8192 |
|
|
88
|
+
| `near` / `far` | `0.1` / `1000` | Clipping distances |
|
|
89
|
+
| `hideObjects` | `[]` | Objects to leave out of the capture, such as the mesh that will show the reflection. Their children, Splat models included, are left out too, except SDF edits, which still apply. They stay visible on screen |
|
|
90
|
+
| `filter` | `false` | `renderCubeMap()` only. Capture linear color with mipmaps, for filtering the cube map into an environment map yourself |
|
|
91
|
+
|
|
92
|
+
Cube faces are read in +X, −X, +Y, −Y, +Z, −Z order, with the same layout on every renderer. Each side face starts with its top (+Y) row.
|
|
93
|
+
|
|
94
|
+
Cube and environment captures show the objects on Three.js layer 0.
|
|
95
|
+
|
|
96
|
+
## Moving reflections
|
|
97
|
+
|
|
98
|
+
To keep a reflection up to date while an object moves, start a capture from your render loop without awaiting it, and keep showing the previous environment map until the next one is ready. Run one capture at a time. For a `MeshStandardMaterial` with `WebGLRenderer`:
|
|
99
|
+
|
|
100
|
+
```js
|
|
101
|
+
let reflectionPending = false;
|
|
102
|
+
let nextCaptureTime = 0;
|
|
103
|
+
|
|
104
|
+
renderer.setAnimationLoop((time) => {
|
|
105
|
+
// Update the sphere's position and other animation here.
|
|
106
|
+
if (!reflectionPending && time >= nextCaptureTime) {
|
|
107
|
+
reflectionPending = true;
|
|
108
|
+
nextCaptureTime = time + 100; // Adjust the update interval for your scene.
|
|
109
|
+
capture.renderEnvMap({
|
|
110
|
+
scene,
|
|
111
|
+
worldCenter: sphere.getWorldPosition(new THREE.Vector3()),
|
|
112
|
+
hideObjects: [sphere],
|
|
113
|
+
size: 256,
|
|
114
|
+
}).then((envMap) => {
|
|
115
|
+
const previous = sphere.material.envMap;
|
|
116
|
+
capture.recurseSetEnvMap(sphere, envMap);
|
|
117
|
+
previous?.dispose();
|
|
118
|
+
}).catch(console.error).finally(() => {
|
|
119
|
+
reflectionPending = false;
|
|
120
|
+
});
|
|
121
|
+
}
|
|
122
|
+
renderer.render(scene, camera);
|
|
123
|
+
});
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
With `WebGPURenderer`, use a `MeshStandardNodeMaterial` and assign one `pmremTexture(envMap)` node to `material.envNode`. For each later capture, set that node's `.value` to the new environment map, then dispose the old one.
|
|
127
|
+
|
|
128
|
+
Each capture draws the scene six times, so update reflections less often than you render.
|
|
129
|
+
|
|
130
|
+
## Good to know
|
|
131
|
+
|
|
132
|
+
- **Await the result.** Every render and read method returns a promise.
|
|
133
|
+
- **The view is fixed when you call.** A capture uses the camera or `worldCenter` as it was at the call, even if it moves before the capture finishes.
|
|
134
|
+
- **Captures run in order.** A call made while another capture is in progress waits for it.
|
|
135
|
+
- **Splat settings follow your renderer.** Captures use the quality and blending options of the `splatRenderer` you passed in.
|
|
136
|
+
- **Stencil follows your renderer.** 2D, cube, and environment captures enable their stencil buffers when the Three.js renderer does. No capture option is needed. A 2D target created with `depthBuffer: false` has no stencil buffer either.
|
|
137
|
+
- **Captures always use sorted rendering,** also when the display uses [stochastic rendering](StochasticRendering.md).
|
|
138
|
+
- **Colors match the screen.** Splats blend as they do when the same renderer draws to the canvas, and the image is stored in the target's color space. Environment maps are linear.
|
|
139
|
+
- **Animation does not advance.** Captures do not call [`onFrame`](SplatMesh.md#scene-integration); only your display renders do.
|
|
140
|
+
- **Visibility and layers apply.** Models appear according to their visibility and the capture camera's layers.
|
|
141
|
+
|
|
142
|
+
## Cleanup
|
|
143
|
+
|
|
144
|
+
`capture.dispose()` releases the targets, cube maps, and other memory the capture holds. It does not dispose your Splat renderer or models.
|
|
145
|
+
|
|
146
|
+
- The cube texture from `renderCubeMap()` belongs to the capture. Later calls redraw the same texture, or replace it when `size`, `near`, `far`, or the renderer's stencil setting change. `renderEnvMap()` leaves it untouched unless you captured it with `filter: true`.
|
|
147
|
+
- Environment maps from `renderEnvMap()` belong to you. Call `envMap.dispose()` when you are done with one.
|
package/docs/SplatEdit.md
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
|
-
#
|
|
1
|
+
# SplatEdit and SplatEditSdf
|
|
2
2
|
|
|
3
3
|
[Back to documentation](../README.md#documentation)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Recolor, fade, or hide the part of a model that lies inside a shape, such as a sphere or a box. Splats are not moved or deleted, so an edit can be changed or removed at any time. The shapes are signed distance fields (SDFs), which is where the class names come from.
|
|
6
|
+
|
|
7
|
+
A `SplatEdit` holds the settings, and one or more `SplatEditSdf` shapes say where it applies. Attach the edit to a `SplatMesh` to affect only that mesh:
|
|
6
8
|
|
|
7
9
|
```js
|
|
8
10
|
import * as THREE from "three";
|
|
@@ -29,52 +31,57 @@ edit.add(sphere);
|
|
|
29
31
|
splat.add(edit);
|
|
30
32
|
```
|
|
31
33
|
|
|
32
|
-
To affect
|
|
34
|
+
To affect every editable mesh, attach it to the scene instead:
|
|
33
35
|
|
|
34
36
|
```js
|
|
35
37
|
scene.add(edit);
|
|
36
38
|
```
|
|
37
39
|
|
|
40
|
+
A mesh created with `editable: false` ignores edits. Edits also work on [streamed scenes](RadStreamScheduler.md).
|
|
41
|
+
|
|
38
42
|
## SplatEdit options
|
|
39
43
|
|
|
40
44
|
| Option | Type | Default | Description |
|
|
41
45
|
| --- | --- | --- | --- |
|
|
42
46
|
| `name` | `string` | Generated | Object name |
|
|
43
|
-
| `rgbaBlendMode` | `SplatEditRgbaBlendMode` | `MULTIPLY_RGBA` |
|
|
44
|
-
| `
|
|
45
|
-
| `
|
|
46
|
-
| `invert` | `boolean` | `false` | Affect the outside instead of the inside |
|
|
47
|
-
| `sdfs` | `SplatEditSdf[]` | `null` | Explicit
|
|
48
|
-
|
|
49
|
-
## SplatEdit methods
|
|
47
|
+
| `rgbaBlendMode` | `SplatEditRgbaBlendMode` | `MULTIPLY_RGBA` | How the shapes' color and opacity combine with the model's; see [blend modes](#blend-modes) |
|
|
48
|
+
| `softEdge` | `number` | `0` | Soften the edge of the edited region; `0` gives a hard edge |
|
|
49
|
+
| `sdfSmooth` | `number` | `0` | Blend neighboring shapes into each other smoothly |
|
|
50
|
+
| `invert` | `boolean` | `false` | Affect the outside of the shapes instead of the inside |
|
|
51
|
+
| `sdfs` | `SplatEditSdf[]` | `null` | Explicit list of shapes, instead of using child objects |
|
|
50
52
|
|
|
51
|
-
|
|
52
|
-
| --- | --- |
|
|
53
|
-
| `addSdf(sdf)` | Add a shape to the explicit `sdfs` list without duplicates |
|
|
54
|
-
| `removeSdf(sdf)` | Remove a shape from the explicit `sdfs` list |
|
|
53
|
+
Shapes are normally children of the edit: use `edit.add(sdf)` and `edit.remove(sdf)`. If you set an explicit `sdfs` list, it is used instead of the children; manage it with `edit.addSdf(sdf)`, which skips duplicates, and `edit.removeSdf(sdf)`.
|
|
55
54
|
|
|
56
|
-
|
|
55
|
+
Edits apply in the order they were created. Change `edit.ordering` to reorder them.
|
|
57
56
|
|
|
58
|
-
##
|
|
57
|
+
## Blend modes
|
|
59
58
|
|
|
60
59
|
| Mode | Description |
|
|
61
60
|
| --- | --- |
|
|
62
|
-
| `MULTIPLY_RGBA` |
|
|
63
|
-
| `SET_RGBA` |
|
|
64
|
-
| `ADD_RGBA` |
|
|
61
|
+
| `MULTIPLY_RGBA` | Multiply the model's color and opacity by the shape's values |
|
|
62
|
+
| `SET_RGBA` | Replace the model's color and opacity with the shape's values |
|
|
63
|
+
| `ADD_RGBA` | Add the shape's values to the model's color and opacity |
|
|
65
64
|
|
|
66
|
-
|
|
65
|
+
Only the channels a shape assigns are affected. A `THREE.Color` assigns red, green, and blue; `opacity` assigns alpha. For example, `color: { r: 1, g: 0.5 }` with `SET_RGBA` replaces only red and green. Set a channel to `undefined` to stop assigning it.
|
|
67
66
|
|
|
68
67
|
## SplatEditSdf options
|
|
69
68
|
|
|
70
69
|
| Option | Type | Default | Description |
|
|
71
70
|
| --- | --- | --- | --- |
|
|
72
|
-
| `type` | `SplatEditSdfType` | `SPHERE` |
|
|
73
|
-
| `
|
|
74
|
-
| `opacity` | `number` | Unassigned | Opacity applied inside the shape
|
|
75
|
-
| `
|
|
76
|
-
| `
|
|
71
|
+
| `type` | `SplatEditSdfType` | `SPHERE` | Shape |
|
|
72
|
+
| `color` | `THREE.Color \| { r?: number; g?: number; b?: number }` | Unassigned | Color applied inside the shape |
|
|
73
|
+
| `opacity` | `number` | Unassigned | Opacity applied inside the shape |
|
|
74
|
+
| `radius` | `number` | `0` | Radius of a sphere, cylinder, capsule, and similar shapes |
|
|
75
|
+
| `invert` | `boolean` | `false` | Affect the outside of this shape instead of the inside |
|
|
77
76
|
|
|
78
77
|
Shapes (`SplatEditSdfType`): `ALL`, `PLANE`, `SPHERE`, `BOX`, `ELLIPSOID`, `CYLINDER`, `CAPSULE`, `INFINITE_CONE`.
|
|
79
78
|
|
|
80
|
-
|
|
79
|
+
## Placing and sizing shapes
|
|
80
|
+
|
|
81
|
+
Move, rotate, and scale shapes like any Three.js object, including through their parents.
|
|
82
|
+
|
|
83
|
+
- `BOX` and `ELLIPSOID` take their half-size from `scale`.
|
|
84
|
+
- `CAPSULE` takes its length from `scale.y`.
|
|
85
|
+
- For those three shapes, scaling does not change `radius`. Other shapes, `SPHERE` included, scale as a whole.
|
|
86
|
+
- A shape scaled to zero on any axis has no effect, even when inverted.
|
|
87
|
+
- With uneven scaling, soft edges are approximate.
|
package/docs/SplatFileType.md
CHANGED
|
@@ -2,18 +2,16 @@
|
|
|
2
2
|
|
|
3
3
|
[Back to documentation](../README.md#documentation)
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
}
|
|
14
|
-
```
|
|
5
|
+
The model formats you can load:
|
|
6
|
+
|
|
7
|
+
| Name | Value | Files |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| `SplatFileType.PLY` | `"ply"` | `.ply` |
|
|
10
|
+
| `SplatFileType.SPZ` | `"spz"` | `.spz` |
|
|
11
|
+
| `SplatFileType.SOG` | `"sog"` | `.sog`, or a SOG `meta.json` with its images |
|
|
12
|
+
| `SplatFileType.RAD` | `"rad"` | `.rad` |
|
|
15
13
|
|
|
16
|
-
|
|
14
|
+
The format is normally detected from the URL or `fileName`, so you rarely need this. Set it when the name does not show the format:
|
|
17
15
|
|
|
18
16
|
```js
|
|
19
17
|
import { SplatFileType, SplatMesh } from "gaussian-splat-lite";
|
|
@@ -24,6 +22,6 @@ const splat = new SplatMesh({
|
|
|
24
22
|
});
|
|
25
23
|
```
|
|
26
24
|
|
|
27
|
-
`SplatMesh` and `Splats` accept
|
|
25
|
+
`SplatMesh` and `Splats` accept `fileType` for URLs, files, and bytes. `SplatLoader.load()` and `loadAsync()` always detect the format themselves.
|
|
28
26
|
|
|
29
|
-
|
|
27
|
+
Large scenes with levels of detail load through their own classes: [SogStreamScheduler](SogStreamScheduler.md) for a SOG `lod-meta.json`, and [RadStreamScheduler](RadStreamScheduler.md) for RAD.
|