gaussian-splat-lite 1.2.0 → 1.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. package/CHANGELOG.md +44 -2
  2. package/README.md +22 -13
  3. package/dist/gaussian-splat-lite.cjs +819 -88
  4. package/dist/gaussian-splat-lite.cjs.map +1 -1
  5. package/dist/gaussian-splat-lite.min.cjs +81 -20
  6. package/dist/gaussian-splat-lite.min.cjs.map +1 -1
  7. package/dist/gaussian-splat-lite.module.js +821 -90
  8. package/dist/gaussian-splat-lite.module.js.map +1 -1
  9. package/dist/gaussian-splat-lite.module.min.js +2984 -2427
  10. package/dist/gaussian-splat-lite.module.min.js.map +1 -1
  11. package/dist/types/addons/TAANode.d.cts +4 -2
  12. package/dist/types/addons/TAANode.d.ts +4 -2
  13. package/dist/types/addons/TAAPass.d.cts +6 -15
  14. package/dist/types/addons/TAAPass.d.ts +6 -15
  15. package/dist/types/addons/taaShared.d.cts +4 -2
  16. package/dist/types/addons/taaShared.d.ts +4 -2
  17. package/dist/types/capture/SplatCapture.d.cts +66 -0
  18. package/dist/types/capture/SplatCapture.d.ts +66 -0
  19. package/dist/types/capture/WebGLCapture.d.cts +13 -0
  20. package/dist/types/capture/WebGLCapture.d.ts +13 -0
  21. package/dist/types/capture/WebGPUCapture.d.cts +17 -0
  22. package/dist/types/capture/WebGPUCapture.d.ts +17 -0
  23. package/dist/types/capture/pixels.d.cts +5 -0
  24. package/dist/types/capture/pixels.d.ts +5 -0
  25. package/dist/types/capture/rendererState.d.cts +3 -0
  26. package/dist/types/capture/rendererState.d.ts +3 -0
  27. package/dist/types/capture/sceneView.d.cts +3 -0
  28. package/dist/types/capture/sceneView.d.ts +3 -0
  29. package/dist/types/index.d.cts +1 -1
  30. package/dist/types/index.d.ts +1 -1
  31. package/dist/types/rendering/GaussianSplatRenderer.d.cts +17 -11
  32. package/dist/types/rendering/GaussianSplatRenderer.d.ts +17 -11
  33. package/dist/types/rendering/SplatAccumulator.d.cts +2 -1
  34. package/dist/types/rendering/SplatAccumulator.d.ts +2 -1
  35. package/dist/types/rendering/backend.d.cts +1 -1
  36. package/dist/types/rendering/backend.d.ts +1 -1
  37. package/dist/types/rendering/rendererUtils.d.cts +1 -5
  38. package/dist/types/rendering/rendererUtils.d.ts +1 -5
  39. package/dist/types/rendering/tsl/ProjectionProgram.d.cts +2 -0
  40. package/dist/types/rendering/tsl/ProjectionProgram.d.ts +2 -0
  41. package/dist/types/rendering/tsl/SplatMaterial.d.cts +1 -0
  42. package/dist/types/rendering/tsl/SplatMaterial.d.ts +1 -0
  43. package/dist/types/rendering/tsl/tslCompat.d.cts +1 -0
  44. package/dist/types/rendering/tsl/tslCompat.d.ts +1 -0
  45. package/dist/types/rendering/uniforms.d.cts +3 -3
  46. package/dist/types/rendering/uniforms.d.ts +3 -3
  47. package/dist/types/scene/SplatMesh.d.cts +1 -1
  48. package/dist/types/scene/SplatMesh.d.ts +1 -1
  49. package/docs/Architecture.md +54 -3
  50. package/docs/GaussianSplatRenderer.md +71 -55
  51. package/docs/PostDecode.md +52 -36
  52. package/docs/RadStreamScheduler.md +53 -41
  53. package/docs/SogStreamScheduler.md +49 -39
  54. package/docs/SplatCapture.md +147 -0
  55. package/docs/SplatEdit.md +33 -26
  56. package/docs/SplatFileType.md +11 -13
  57. package/docs/SplatLoader.md +19 -14
  58. package/docs/SplatMesh.md +69 -49
  59. package/docs/Splats.md +33 -26
  60. package/docs/StochasticRendering.md +27 -30
  61. package/docs/TAAPass.md +98 -73
  62. package/package.json +1 -1
  63. package/docs/SplatAccumulator.md +0 -37
@@ -2,7 +2,7 @@
2
2
 
3
3
  [Back to documentation](../README.md#documentation)
4
4
 
5
- Loads large RAD scenes with detail that adapts as the camera moves. Supports RAD version 1 with levels of detail (LOD), as a single `.rad` file or split files with `.radc` companions. Works on WebGPU and WebGL2 without a Spark runtime.
5
+ Shows large RAD scenes by loading only the detail the camera needs, and updating it as the camera moves. It supports RAD version 1 files with levels of detail (LOD), either as a single `.rad` file or split into a header and `.radc` pages. It works on WebGPU and WebGL2, and needs no Spark runtime.
6
6
 
7
7
  ```js
8
8
  import { RadStreamScheduler } from "gaussian-splat-lite";
@@ -21,7 +21,7 @@ renderer.setAnimationLoop(() => {
21
21
  renderer.render(scene, camera);
22
22
  });
23
23
 
24
- // Requires update() to keep running.
24
+ // Resolves when the first data is on screen. update() must be running.
25
25
  await streaming.firstRenderable;
26
26
 
27
27
  // When removing the model:
@@ -29,11 +29,23 @@ await streaming.firstRenderable;
29
29
  // streaming.group.removeFromParent();
30
30
  ```
31
31
 
32
- Use `streaming.group` to position, rotate, scale, or hide the scene. Streamed models support SDF edits and picking; their data is read-only.
32
+ The three steps that matter:
33
33
 
34
- Resolution is in CSS pixels, without the device pixel ratio, so a scene selects the same detail on standard and high-DPI displays. Register several cameras to load detail for all of them; they share one selection at the greatest detail any of them needs.
34
+ 1. Add `streaming.group` to the scene. Move, rotate, scale, or hide the scene through this group.
35
+ 2. Register the camera and its resolution.
36
+ 3. Call `streaming.update()` before every render.
35
37
 
36
- In WebXR, register `renderer.xr.getCamera()` in place of your camera. Detail is selected for each eye at the headset's resolution, so no resolution is needed:
38
+ Streamed models support [SDF edits](SplatEdit.md) and [picking](SplatMesh.md#raycasting). Their data is read-only.
39
+
40
+ ## Cameras and resolution
41
+
42
+ Detail depends on how large the scene appears, so each camera needs a resolution. It is in CSS pixels, without the device pixel ratio, so a scene loads the same detail on standard and high-DPI displays. Set it again whenever the canvas is resized.
43
+
44
+ Register several cameras to load detail for all of them. They share one selection, at the highest detail any of them needs.
45
+
46
+ ### WebXR
47
+
48
+ Register `renderer.xr.getCamera()` in place of your camera. Detail is chosen for each eye at the headset's resolution, so no resolution is needed:
37
49
 
38
50
  ```js
39
51
  const xrCamera = renderer.xr.getCamera();
@@ -53,50 +65,48 @@ renderer.setAnimationLoop(() => {
53
65
  });
54
66
  ```
55
67
 
56
- Ordinary camera matrices are updated by the scheduler. The XR camera's matrices are read directly from Three.js, so detail selection before rendering may use the previous frame's pose.
57
-
58
- For original Splat IDs, use the picking hit's [`sourceIndex`](SplatMesh.md#raycasting).
59
-
60
68
  ## Options
61
69
 
62
70
  | Option | Default | Description |
63
71
  | --- | --- | --- |
64
72
  | `url` / `file` / `fileBytes` | Exactly one required | URL, `Blob`/`File`, or complete `Uint8Array`/`ArrayBuffer` |
65
- | `resolveFile` | Relative URL resolution | Companion-page resolver: `(filename, signal) => URL \| Blob \| Uint8Array \| ArrayBuffer`, optionally asynchronous |
66
- | `group` | New `THREE.Group` | Parent group for moving, rotating, scaling, or hiding the scene |
67
- | `splatBudget` | `3_000_000` | Maximum selected Splat count before transition overlap |
68
- | `cooldownMs` | `2000` | Milliseconds to retain unused pages after fade-out; `0` releases eligible pages immediately |
69
- | `fadeDurationMs` | `200` | Initial visibility and LOD fade duration in milliseconds; `0` switches immediately |
70
- | `maxConcurrentLoads` | `4` | Maximum simultaneous loads |
71
- | `manager` | `THREE.DefaultLoadingManager` | Loading manager and URL modifiers |
72
- | `requestHeader` / `withCredentials` | `{}` / `false` | Request settings; sensitive settings are not forwarded to other origins |
73
- | `onChange` | — | Request a redraw when data or visibility changes |
74
- | `onError` | Console error | Failure callback: `(error, url)` |
73
+ | `resolveFile` | Relative URLs | Supply the `.radc` pages of a split scene yourself: `(filename, signal) => URL \| Blob \| Uint8Array \| ArrayBuffer`, directly or as a promise |
74
+ | `group` | New `THREE.Group` | Group that holds the scene |
75
+ | `splatBudget` | `3_000_000` | Maximum number of Splats shown at once. Higher values show more detail and use more memory |
76
+ | `fadeDurationMs` | `200` | Duration of the fade when the scene first appears and when detail changes; `0` switches instantly |
77
+ | `cooldownMs` | `2000` | How long detail that is no longer shown stays in memory, ready for reuse; `0` releases it at once |
78
+ | `maxConcurrentLoads` | `4` | Maximum number of loads in progress at the same time |
79
+ | `manager` | `THREE.DefaultLoadingManager` | Three.js loading manager, including its URL modifiers |
80
+ | `requestHeader` / `withCredentials` | `{}` / `false` | Request settings. Sensitive settings are not sent to other origins |
81
+ | `onChange` | — | Called when the scene needs a redraw |
82
+ | `onError` | Logs to the console | Called when loading fails: `(error, url)` |
75
83
 
76
- Existing detail stays visible while replacement detail loads. Fades can temporarily exceed `splatBudget`; use `fadeDurationMs: 0` to switch immediately. Cached data also uses memory, so the budget is not a total memory limit.
84
+ Detail that is already on screen stays there while its replacement loads. During a fade, old and new detail overlap, so the scene can briefly exceed `splatBudget`; use `fadeDurationMs: 0` to prevent this. The budget limits what is shown, not total memory: detail kept for reuse takes memory too.
77
85
 
78
- Ready pages are written during `update()` without a per-update byte limit. Loading concurrency and pending decode memory remain bounded.
79
-
80
- ## Common properties and methods
86
+ ## Properties and methods
81
87
 
82
88
  | API | Description |
83
89
  | --- | --- |
84
- | `group` | Parent group for moving, rotating, scaling, or hiding the scene |
85
- | `splatBudget` | Positive safe integer; change it at runtime to adjust detail, even while the camera is stationary |
86
- | `initialized` | Resolves when scene setup is ready; rejects invalid or unsupported input |
87
- | `firstRenderable` | Resolves when the first data becomes visible, or the scene is empty; rejects if the first data cannot be loaded. Keep calling `update()` while waiting |
88
- | `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that selects detail; `hasCamera(camera)` and `cameras` list them |
89
- | `setResolution(camera, width, height)` | Set a registered camera's render size in CSS pixels; also accepts a `THREE.Vector2` |
90
- | `setResolutionFromRenderer(camera, renderer)` | Set a registered camera's resolution from `renderer.getSize()` |
91
- | `update()` | Update detail, loading, fades, and cleanup for the registered cameras; throws if none are registered |
92
- | `getBoundingBox()` | Approximate bounds in group coordinates; empty until the first data loads |
93
- | `getGlobalIndex(mesh, renderedIndex)` | Convert a picking result to a stable node index in the RAD file |
94
- | `stats` | Visible and retained Splat data, loading progress, and memory estimates; not total browser memory |
95
- | `dispose()` | Cancel loading and release scene resources; pending readiness promises reject with `AbortError` |
96
-
97
- For on-demand rendering, use `onChange` to request redraws and keep calling `update()` each animation tick so loading, retries, fades, and cleanup progress.
98
-
99
- Cooldown uses elapsed time, independent of update frequency. Pending LOD decisions defer expired-page cleanup so newly needed pages can be reused. Each accepted decision updates demand before cleanup, even during camera movement; existing page pins still apply.
90
+ | `group` | Group that holds the scene |
91
+ | `splatBudget` | Change it at any time to adjust detail, even while the camera is still. Must be a positive integer |
92
+ | `initialized` | Resolves when the scene is set up; rejects for invalid or unsupported files |
93
+ | `firstRenderable` | Resolves when the first data is on screen, or the scene is empty; rejects if the first data cannot be loaded. Keep calling `update()` while you wait |
94
+ | `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that chooses detail. `hasCamera(camera)` and `cameras` report the registered ones |
95
+ | `setResolution(camera, width, height)` | Set a registered camera's size in CSS pixels; also accepts a `THREE.Vector2` |
96
+ | `setResolutionFromRenderer(camera, renderer)` | Set a registered camera's size from `renderer.getSize()` |
97
+ | `update()` | Advance detail selection, loading, and fades. Call it before every render; it throws if no camera is registered |
98
+ | `getBoundingBox()` | Approximate box around the scene, in the group's coordinates; empty until the first data loads |
99
+ | `getGlobalIndex(mesh, renderedIndex)` | Convert a picking hit into a node index that stays the same for the whole RAD file |
100
+ | `stats` | Numbers for monitoring, such as `visibleSplats`, `loadingChunks`, `downloadedBytes`, and `residentBytes`. Memory figures are estimates, not total browser memory |
101
+ | `dispose()` | Stop loading and release the scene. Pending `initialized` and `firstRenderable` promises reject with `AbortError` |
102
+
103
+ ### On-demand rendering
104
+
105
+ Request a redraw from `onChange`, and keep calling `update()` on every animation tick, also when you skip the render. Loading, retries, and fades only make progress inside `update()`.
106
+
107
+ ### Picking
108
+
109
+ Hits from a streamed scene refer to the detail that is currently shown. Use `getGlobalIndex()`, or the hit's [`sourceIndex`](SplatMesh.md#raycasting), for an ID that does not change:
100
110
 
101
111
  ```js
102
112
  const hits = raycaster.intersectObject(streaming.group, true);
@@ -108,6 +118,8 @@ if (hits.length && hits[0].index !== undefined) {
108
118
 
109
119
  ## Error recovery
110
120
 
111
- Temporary download failures and lost decoder workers are retried automatically, waiting longer after each attempt, up to 30 seconds. Retryable HTTP errors are 408, 425, 429, and 5xx. Invalid data, out-of-memory errors, and other HTTP errors are reported to `onError` and not retried. Each retry calls `onChange`, so on-demand render loops resume loading once the connection returns.
121
+ Temporary download failures are retried automatically, waiting longer after each attempt, up to 30 seconds. This covers network errors and HTTP 408, 425, 429, and 5xx responses. Each retry calls `onChange`, so an on-demand render loop resumes loading when the connection returns.
122
+
123
+ Invalid data, out-of-memory errors, and other HTTP errors are reported to `onError` and are not retried.
112
124
 
113
- If LOD selection or preparation fails, or the LOD worker is lost during page loading, `onError` is called and the scene keeps its current detail but stops refining; create a new scheduler to try again.
125
+ If the scheduler can no longer choose detail after an internal failure, it calls `onError` and keeps showing its current detail without refining further. Create a new scheduler to try again.
@@ -2,7 +2,7 @@
2
2
 
3
3
  [Back to documentation](../README.md#documentation)
4
4
 
5
- Loads large SOG `lod-meta.json` scenes with detail that adapts as the camera moves. Supports version 1 and older unversioned indexes on WebGPU and WebGL2. For an ordinary `.sog` file or `meta.json`, use [SplatMesh](SplatMesh.md).
5
+ Shows large SOG `lod-meta.json` scenes by loading only the detail the camera needs, and updating it as the camera moves. It supports version 1 and older unversioned indexes, on WebGPU and WebGL2. For an ordinary `.sog` file or `meta.json`, use [SplatMesh](SplatMesh.md).
6
6
 
7
7
  ```js
8
8
  import { SogStreamScheduler } from "gaussian-splat-lite";
@@ -15,65 +15,75 @@ scene.add(streaming.group);
15
15
  streaming.setCamera(camera);
16
16
  await streaming.initialized;
17
17
 
18
- // Call each animation tick, before rendering.
19
- streaming.update();
20
- renderer.render(scene, camera);
18
+ renderer.setAnimationLoop(() => {
19
+ streaming.update();
20
+ renderer.render(scene, camera);
21
+ });
22
+
23
+ // Resolves when the first data is on screen. update() must be running.
24
+ await streaming.firstRenderable;
21
25
 
22
26
  // When removing the model:
23
- streaming.dispose();
24
- streaming.group.removeFromParent();
27
+ // streaming.dispose();
28
+ // streaming.group.removeFromParent();
25
29
  ```
26
30
 
27
- Use `streaming.group` to position, rotate, scale, or hide the scene. Streamed models support SDF edits and picking; their data is read-only.
31
+ The three steps that matter:
32
+
33
+ 1. Add `streaming.group` to the scene. Move, rotate, scale, or hide the scene through this group.
34
+ 2. Register the camera.
35
+ 3. Call `streaming.update()` before every render.
28
36
 
29
- Detail follows distance from the camera, so cameras need no resolution. Register several cameras to load detail for all of them; each region follows the nearest camera that sees it.
37
+ Streamed models support [SDF edits](SplatEdit.md) and [picking](SplatMesh.md#raycasting). Their data is read-only. For a Splat's original ID, use the picking hit's [`sourceIndex`](SplatMesh.md#raycasting).
30
38
 
31
- In WebXR, register `renderer.xr.getCamera()` in place of your camera, as shown for [RadStreamScheduler](RadStreamScheduler.md). Detail is selected for each eye. Calling `update()` before rendering uses the previous frame's eye poses, and until the headset reports its first poses, the current detail and downloads are kept.
39
+ ## Cameras
32
40
 
33
- For original Splat IDs, use the picking hit's [`sourceIndex`](SplatMesh.md#raycasting).
41
+ Detail depends on distance from the camera, so cameras need no resolution. Register several cameras to load detail for all of them; each part of the scene follows the nearest camera that sees it.
42
+
43
+ In WebXR, register `renderer.xr.getCamera()` in place of your camera, as shown for [RadStreamScheduler](RadStreamScheduler.md#webxr). Detail is chosen for each eye.
34
44
 
35
45
  ## Options
36
46
 
37
47
  | Option | Default | Description |
38
48
  | --- | --- | --- |
39
- | `url` | Required | Streamed SOG index URL |
40
- | `group` | New `THREE.Group` | Parent group for moving, rotating, scaling, or hiding the scene |
41
- | `splatBudget` | `3_000_000` | Target visible Splat count, including the environment |
42
- | `cooldownMs` | `2000` | Milliseconds to retain unused data after fade-out; `0` releases eligible data immediately |
43
- | `fadeDurationMs` | `200` | Visibility and LOD fade duration in milliseconds; `0` disables fades |
44
- | `maxConcurrentLoads` | `4` | Maximum simultaneous loads |
45
- | `manager` | `THREE.DefaultLoadingManager` | Loading manager and URL modifiers |
46
- | `requestHeader` / `withCredentials` | `{}` / `false` | Fetch settings; headers and credentials are not forwarded to cross-origin chunks |
47
- | `onChange` | — | Request a redraw when data or visibility changes |
48
- | `onError` | Console error | Chunk failure callback: `(error, url)` |
49
- | `loadChunk` | Built-in worker loader | Custom `(url, signal) => Promise<Splats>` loader |
50
-
51
- The Splat budget is a target: available detail levels, fades, and the environment can exceed it. Cached data also uses memory, so the budget is not a total memory limit.
49
+ | `url` | Required | URL of the scene's `lod-meta.json` |
50
+ | `group` | New `THREE.Group` | Group that holds the scene |
51
+ | `splatBudget` | `3_000_000` | Target number of Splats shown at once, including the environment. Higher values show more detail and use more memory |
52
+ | `fadeDurationMs` | `200` | Duration of the fade when the scene first appears and when detail changes; `0` switches instantly |
53
+ | `cooldownMs` | `2000` | How long detail that is no longer shown stays in memory, ready for reuse; `0` releases it at once |
54
+ | `maxConcurrentLoads` | `4` | Maximum number of loads in progress at the same time |
55
+ | `manager` | `THREE.DefaultLoadingManager` | Three.js loading manager, including its URL modifiers |
56
+ | `requestHeader` / `withCredentials` | `{}` / `false` | Request settings. Headers and credentials are not sent to files on other origins |
57
+ | `onChange` | — | Called when the scene needs a redraw |
58
+ | `onError` | Logs to the console | Called when loading fails: `(error, url)` |
59
+ | `loadChunk` | Built-in loader | Load the scene's files yourself: `(url, signal) => Promise<Splats>` |
52
60
 
53
- Regions that disappear at coarser LODs can remain empty until selected for more detail. Empty levels do not load files; visible data fades out when a region returns to an empty level.
61
+ The budget is a target, not a hard limit: the available detail levels, fades, and the environment can take the scene above it. It limits what is shown, not total memory: detail kept for reuse takes memory too.
54
62
 
55
- Ready regions are attached during `update()` once their previous fade completes, without a per-update byte limit. Loading concurrency and pending extraction memory remain bounded.
63
+ Some parts of a scene exist only at finer detail levels. They can be empty from far away and fade in as the camera approaches.
56
64
 
57
- ## Common properties and methods
65
+ ## Properties and methods
58
66
 
59
67
  | API | Description |
60
68
  | --- | --- |
61
- | `group` | Parent group for moving, rotating, scaling, or hiding the scene |
62
- | `splatBudget` | Positive safe integer including the environment; change it at runtime to adjust detail |
63
- | `initialized` | Resolves when the index is ready; rejects invalid input |
64
- | `firstRenderable` | Resolves when the first data becomes visible, or the scene is empty; rejects if the first data cannot be loaded or prepared. Keep calling `update()` while waiting |
65
- | `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that selects detail; `hasCamera(camera)` and `cameras` list them |
66
- | `update()` | Update detail, loading, fades, and cleanup for the registered cameras; throws if none are registered |
67
- | `getBoundingBox()` | Group-local index bounds, available after initialization |
68
- | `stats` | Visible and retained Splat data, loading progress, and memory estimates; not total browser memory |
69
- | `dispose()` | Cancel loading and release scene resources; pending readiness promises reject with `AbortError` |
69
+ | `group` | Group that holds the scene |
70
+ | `splatBudget` | Change it at any time to adjust detail. Must be a positive integer; includes the environment |
71
+ | `initialized` | Resolves when the index is loaded; rejects for invalid input |
72
+ | `firstRenderable` | Resolves when the first data is on screen, or the scene is empty; rejects if the first data cannot be loaded. Keep calling `update()` while you wait |
73
+ | `setCamera(camera)` / `deleteCamera(camera)` | Add or remove a camera that chooses detail. `hasCamera(camera)` and `cameras` report the registered ones |
74
+ | `update()` | Advance detail selection, loading, and fades. Call it before every render; it throws if no camera is registered |
75
+ | `getBoundingBox()` | Box around the scene, in the group's coordinates; available once `initialized` resolves |
76
+ | `stats` | Numbers for monitoring, such as `visibleSplats`, `visibleRegions`, `loadingChunks`, `downloadedBytes`, and `residentBytes`. Memory figures are estimates, not total browser memory |
77
+ | `dispose()` | Stop loading and release the scene. Pending `initialized` and `firstRenderable` promises reject with `AbortError` |
70
78
 
71
- For on-demand rendering, use `onChange` to request redraws and keep calling `update()` each animation tick so loading, retries, fades, and cleanup progress.
79
+ ### On-demand rendering
72
80
 
73
- Cooldown uses elapsed time, independent of update frequency. Expired data releases during updates or after accepted LOD decisions once no longer needed. Pending decisions defer cleanup so newly needed data can be reused; each accepted decision permits cleanup even during camera movement.
81
+ Request a redraw from `onChange`, and keep calling `update()` on every animation tick, also when you skip the render. Loading, retries, and fades only make progress inside `update()`.
74
82
 
75
83
  ## Error recovery
76
84
 
77
- Temporary download failures and lost decoder workers are retried automatically, waiting longer after each attempt, up to 30 seconds. Retryable HTTP errors are 408, 425, 429, and 5xx. Invalid data, out-of-memory errors, and other HTTP errors are reported to `onError` and not retried. Each retry calls `onChange`, so on-demand render loops resume loading once the connection returns.
85
+ Temporary download failures are retried automatically, waiting longer after each attempt, up to 30 seconds. This covers network errors and HTTP 408, 425, 429, and 5xx responses. Each retry calls `onChange`, so an on-demand render loop resumes loading when the connection returns.
86
+
87
+ Invalid data, out-of-memory errors, and other HTTP errors are reported to `onError` and are not retried. Parts of the scene that are already on screen stay visible when loading more detail fails.
78
88
 
79
- Regions already on screen stay visible when loading more detail fails. If detail selection itself fails, `onError` is called and the scheduler disposes itself; create a new one to try again.
89
+ If the scheduler can no longer choose detail after an internal failure, it calls `onError` and disposes itself. Create a new scheduler to try again.
@@ -0,0 +1,147 @@
1
+ # SplatCapture
2
+
3
+ [Back to documentation](../README.md#documentation)
4
+
5
+ Renders a scene and its Splats offscreen: into a render target, as pixels you can read back, as a cube map, or as an environment map for reflections. It works with `WebGLRenderer` and with `WebGPURenderer` on both WebGPU and its WebGL2 fallback.
6
+
7
+ ```js
8
+ import { SplatCapture } from "gaussian-splat-lite";
9
+
10
+ const capture = new SplatCapture({
11
+ splatRenderer, // The GaussianSplatRenderer already in your scene.
12
+ target: { width: 1920, height: 1080 },
13
+ });
14
+
15
+ const rgba = await capture.renderReadTarget({ scene, camera });
16
+ // Uint8Array of 1920 × 1080 × 4 bytes, bottom row first.
17
+ ```
18
+
19
+ Captures do not disturb what is on screen. Your render loop can keep running while one is in progress, and the display keeps its own camera and settings.
20
+
21
+ [SDF edits](SplatEdit.md) apply to captures as they do on screen, whether they are attached to the scene or to a model.
22
+
23
+ Coming from 1.1.8? These were methods and options of `GaussianSplatRenderer`. They keep the same names on `SplatCapture`, except the cube captures' `update` option, which is gone: every capture shows the current scene and reuses earlier work while nothing has changed.
24
+
25
+ ## Render to a target
26
+
27
+ Pass `target` to capture 2D images. It accepts the options below plus `THREE.RenderTargetOptions`, except `stencilBuffer`, which follows the Three.js renderer.
28
+
29
+ | Option | Default | Description |
30
+ | --- | --- | --- |
31
+ | `width` / `height` | Required | Image size in pixels |
32
+ | `superXY` | `1` | Supersampling, an integer from 1 to 4. The scene renders at `superXY` times the size on each axis and is averaged down when read back |
33
+ | `doubleBuffer` | `false` | Alternate between two targets, so `capture.target` keeps a finished image while the next one renders |
34
+
35
+ `width` and `height` multiplied by `superXY` can be at most 8192.
36
+
37
+ | Method | Description |
38
+ | --- | --- |
39
+ | `renderTarget({ scene, camera })` | Render the view and return the `THREE.RenderTarget` holding it |
40
+ | `readTarget()` | Read the latest render as RGBA bytes |
41
+ | `renderReadTarget({ scene, camera })` | Do both in one call |
42
+
43
+ `capture.target` always holds the latest finished render. Use `capture.target.texture` as a texture; its size includes supersampling.
44
+
45
+ Pixels come back as a `Uint8Array` of `width × height × 4` bytes, bottom row first, in the same layout on every renderer. The array is reused by later reads, so copy it with `rgba.slice()` if you need to keep it.
46
+
47
+ Targets are 8-bit RGBA in sRGB unless you say otherwise:
48
+
49
+ - `samples` antialiases the meshes in the scene.
50
+ - Stencil masking follows the Three.js renderer's stencil setting. Stencil settings on your Splat renderer's `material` apply to the Splats in the capture too.
51
+ - `type: THREE.HalfFloatType` or `THREE.FloatType` keeps values above 1. Such targets can be rendered but not read back; reading requires 8-bit RGBA.
52
+
53
+ 2D captures also keep the scene's depth in the target.
54
+
55
+ ## Cube maps and environment maps
56
+
57
+ ```js
58
+ const cube = await capture.renderCubeMap({
59
+ scene,
60
+ worldCenter: new THREE.Vector3(0, 1, 0),
61
+ size: 256,
62
+ hideObjects: [reflectiveMesh],
63
+ });
64
+ const faces = await capture.readCubeTargets(); // Six RGBA arrays.
65
+
66
+ const envMap = await capture.renderEnvMap({
67
+ scene,
68
+ worldCenter: reflectiveMesh.getWorldPosition(new THREE.Vector3()),
69
+ hideObjects: [reflectiveMesh],
70
+ });
71
+ capture.recurseSetEnvMap(reflectiveMesh, envMap);
72
+ ```
73
+
74
+ `target` is not needed when you only capture cube or environment maps.
75
+
76
+ | Method | Description |
77
+ | --- | --- |
78
+ | `renderCubeMap(options)` | Render the six directions around a point and return a `THREE.CubeTexture` |
79
+ | `readCubeTargets()` | Read the latest cube map as six RGBA arrays |
80
+ | `renderEnvMap(options)` | Render a cube map and filter it into an environment map ready for `material.envMap` |
81
+ | `recurseSetEnvMap(root, envMap)` | Assign an environment map to every `MeshStandardMaterial` under `root` |
82
+
83
+ | Option | Default | Description |
84
+ | --- | --- | --- |
85
+ | `scene` | Required | Scene to capture |
86
+ | `worldCenter` | Required | Capture position in world space |
87
+ | `size` | `256` | Size of each face in pixels, up to 8192 |
88
+ | `near` / `far` | `0.1` / `1000` | Clipping distances |
89
+ | `hideObjects` | `[]` | Objects to leave out of the capture, such as the mesh that will show the reflection. Their children, Splat models included, are left out too, except SDF edits, which still apply. They stay visible on screen |
90
+ | `filter` | `false` | `renderCubeMap()` only. Capture linear color with mipmaps, for filtering the cube map into an environment map yourself |
91
+
92
+ Cube faces are read in +X, −X, +Y, −Y, +Z, −Z order, with the same layout on every renderer. Each side face starts with its top (+Y) row.
93
+
94
+ Cube and environment captures show the objects on Three.js layer 0.
95
+
96
+ ## Moving reflections
97
+
98
+ To keep a reflection up to date while an object moves, start a capture from your render loop without awaiting it, and keep showing the previous environment map until the next one is ready. Run one capture at a time. For a `MeshStandardMaterial` with `WebGLRenderer`:
99
+
100
+ ```js
101
+ let reflectionPending = false;
102
+ let nextCaptureTime = 0;
103
+
104
+ renderer.setAnimationLoop((time) => {
105
+ // Update the sphere's position and other animation here.
106
+ if (!reflectionPending && time >= nextCaptureTime) {
107
+ reflectionPending = true;
108
+ nextCaptureTime = time + 100; // Adjust the update interval for your scene.
109
+ capture.renderEnvMap({
110
+ scene,
111
+ worldCenter: sphere.getWorldPosition(new THREE.Vector3()),
112
+ hideObjects: [sphere],
113
+ size: 256,
114
+ }).then((envMap) => {
115
+ const previous = sphere.material.envMap;
116
+ capture.recurseSetEnvMap(sphere, envMap);
117
+ previous?.dispose();
118
+ }).catch(console.error).finally(() => {
119
+ reflectionPending = false;
120
+ });
121
+ }
122
+ renderer.render(scene, camera);
123
+ });
124
+ ```
125
+
126
+ With `WebGPURenderer`, use a `MeshStandardNodeMaterial` and assign one `pmremTexture(envMap)` node to `material.envNode`. For each later capture, set that node's `.value` to the new environment map, then dispose the old one.
127
+
128
+ Each capture draws the scene six times, so update reflections less often than you render.
129
+
130
+ ## Good to know
131
+
132
+ - **Await the result.** Every render and read method returns a promise.
133
+ - **The view is fixed when you call.** A capture uses the camera or `worldCenter` as it was at the call, even if it moves before the capture finishes.
134
+ - **Captures run in order.** A call made while another capture is in progress waits for it.
135
+ - **Splat settings follow your renderer.** Captures use the quality and blending options of the `splatRenderer` you passed in.
136
+ - **Stencil follows your renderer.** 2D, cube, and environment captures enable their stencil buffers when the Three.js renderer does. No capture option is needed. A 2D target created with `depthBuffer: false` has no stencil buffer either.
137
+ - **Captures always use sorted rendering,** also when the display uses [stochastic rendering](StochasticRendering.md).
138
+ - **Colors match the screen.** Splats blend as they do when the same renderer draws to the canvas, and the image is stored in the target's color space. Environment maps are linear.
139
+ - **Animation does not advance.** Captures do not call [`onFrame`](SplatMesh.md#scene-integration); only your display renders do.
140
+ - **Visibility and layers apply.** Models appear according to their visibility and the capture camera's layers.
141
+
142
+ ## Cleanup
143
+
144
+ `capture.dispose()` releases the targets, cube maps, and other memory the capture holds. It does not dispose your Splat renderer or models.
145
+
146
+ - The cube texture from `renderCubeMap()` belongs to the capture. Later calls redraw the same texture, or replace it when `size`, `near`, `far`, or the renderer's stencil setting change. `renderEnvMap()` leaves it untouched unless you captured it with `filter: true`.
147
+ - Environment maps from `renderEnvMap()` belong to you. Call `envMap.dispose()` when you are done with one.
package/docs/SplatEdit.md CHANGED
@@ -1,8 +1,10 @@
1
- # SDF color and opacity editing
1
+ # SplatEdit and SplatEditSdf
2
2
 
3
3
  [Back to documentation](../README.md#documentation)
4
4
 
5
- SDF edits change color and opacity within a shape without moving or deleting Splats. Attach an edit to a `SplatMesh` to affect only that mesh:
5
+ Recolor, fade, or hide the part of a model that lies inside a shape, such as a sphere or a box. Splats are not moved or deleted, so an edit can be changed or removed at any time. The shapes are signed distance fields (SDFs), which is where the class names come from.
6
+
7
+ A `SplatEdit` holds the settings, and one or more `SplatEditSdf` shapes say where it applies. Attach the edit to a `SplatMesh` to affect only that mesh:
6
8
 
7
9
  ```js
8
10
  import * as THREE from "three";
@@ -29,52 +31,57 @@ edit.add(sphere);
29
31
  splat.add(edit);
30
32
  ```
31
33
 
32
- To affect all editable meshes, attach it to the scene:
34
+ To affect every editable mesh, attach it to the scene instead:
33
35
 
34
36
  ```js
35
37
  scene.add(edit);
36
38
  ```
37
39
 
40
+ A mesh created with `editable: false` ignores edits. Edits also work on [streamed scenes](RadStreamScheduler.md).
41
+
38
42
  ## SplatEdit options
39
43
 
40
44
  | Option | Type | Default | Description |
41
45
  | --- | --- | --- | --- |
42
46
  | `name` | `string` | Generated | Object name |
43
- | `rgbaBlendMode` | `SplatEditRgbaBlendMode` | `MULTIPLY_RGBA` | Multiply, replace, or add to the assigned color and opacity channels |
44
- | `sdfSmooth` | `number` | `0` | Soften the join between shapes |
45
- | `softEdge` | `number` | `0` | Soften the region boundary |
46
- | `invert` | `boolean` | `false` | Affect the outside instead of the inside |
47
- | `sdfs` | `SplatEditSdf[]` | `null` | Explicit shape list; SDFs can instead be child objects |
48
-
49
- ## SplatEdit methods
47
+ | `rgbaBlendMode` | `SplatEditRgbaBlendMode` | `MULTIPLY_RGBA` | How the shapes' color and opacity combine with the model's; see [blend modes](#blend-modes) |
48
+ | `softEdge` | `number` | `0` | Soften the edge of the edited region; `0` gives a hard edge |
49
+ | `sdfSmooth` | `number` | `0` | Blend neighboring shapes into each other smoothly |
50
+ | `invert` | `boolean` | `false` | Affect the outside of the shapes instead of the inside |
51
+ | `sdfs` | `SplatEditSdf[]` | `null` | Explicit list of shapes, instead of using child objects |
50
52
 
51
- | API | Description |
52
- | --- | --- |
53
- | `addSdf(sdf)` | Add a shape to the explicit `sdfs` list without duplicates |
54
- | `removeSdf(sdf)` | Remove a shape from the explicit `sdfs` list |
53
+ Shapes are normally children of the edit: use `edit.add(sdf)` and `edit.remove(sdf)`. If you set an explicit `sdfs` list, it is used instead of the children; manage it with `edit.addSdf(sdf)`, which skips duplicates, and `edit.removeSdf(sdf)`.
55
54
 
56
- An explicit `sdfs` list takes precedence over child objects. For child shapes, use `edit.add(sdf)` and `edit.remove(sdf)`.
55
+ Edits apply in the order they were created. Change `edit.ordering` to reorder them.
57
56
 
58
- ## RGBA blend modes
57
+ ## Blend modes
59
58
 
60
59
  | Mode | Description |
61
60
  | --- | --- |
62
- | `MULTIPLY_RGBA` | Multiplies each assigned channel by the corresponding SDF value |
63
- | `SET_RGBA` | Replaces each assigned channel with the corresponding SDF value |
64
- | `ADD_RGBA` | Adds the corresponding SDF value to each assigned channel |
61
+ | `MULTIPLY_RGBA` | Multiply the model's color and opacity by the shape's values |
62
+ | `SET_RGBA` | Replace the model's color and opacity with the shape's values |
63
+ | `ADD_RGBA` | Add the shape's values to the model's color and opacity |
65
64
 
66
- Unassigned channels stay unchanged. For example, `color: { r: 1, g: 0.5 }` with `SET_RGBA` replaces only R and G. A `THREE.Color` assigns all RGB channels; `opacity` assigns alpha. Set a channel to `undefined` to unassign it.
65
+ Only the channels a shape assigns are affected. A `THREE.Color` assigns red, green, and blue; `opacity` assigns alpha. For example, `color: { r: 1, g: 0.5 }` with `SET_RGBA` replaces only red and green. Set a channel to `undefined` to stop assigning it.
67
66
 
68
67
  ## SplatEditSdf options
69
68
 
70
69
  | Option | Type | Default | Description |
71
70
  | --- | --- | --- | --- |
72
- | `type` | `SplatEditSdfType` | `SPHERE` | SDF shape |
73
- | `invert` | `boolean` | `false` | Affect the outside instead of the inside |
74
- | `opacity` | `number` | Unassigned | Opacity applied inside the shape; leave unassigned to preserve it |
75
- | `color` | `THREE.Color \| { r?: number; g?: number; b?: number }` | Unassigned | Color applied inside the shape; unassigned channels stay unchanged |
76
- | `radius` | `number` | `0` | Radius used by sphere, cylinder, capsule, and related shapes |
71
+ | `type` | `SplatEditSdfType` | `SPHERE` | Shape |
72
+ | `color` | `THREE.Color \| { r?: number; g?: number; b?: number }` | Unassigned | Color applied inside the shape |
73
+ | `opacity` | `number` | Unassigned | Opacity applied inside the shape |
74
+ | `radius` | `number` | `0` | Radius of a sphere, cylinder, capsule, and similar shapes |
75
+ | `invert` | `boolean` | `false` | Affect the outside of this shape instead of the inside |
77
76
 
78
77
  Shapes (`SplatEditSdfType`): `ALL`, `PLANE`, `SPHERE`, `BOX`, `ELLIPSOID`, `CYLINDER`, `CAPSULE`, `INFINITE_CONE`.
79
78
 
80
- Move, rotate, and scale shapes with normal Three.js transforms, including transforms inherited from parents. `BOX` and `ELLIPSOID` take their half-size from `scale`, and `CAPSULE` takes its length from `scale.y`; scaling these shapes does not change their `radius`. Other shapes, including `SPHERE`, scale as a whole. A shape scaled to zero on any axis has no effect, even when inverted. With uneven scaling, soft edges are approximate. Edits apply in creation order; use `edit.ordering` to change it.
79
+ ## Placing and sizing shapes
80
+
81
+ Move, rotate, and scale shapes like any Three.js object, including through their parents.
82
+
83
+ - `BOX` and `ELLIPSOID` take their half-size from `scale`.
84
+ - `CAPSULE` takes its length from `scale.y`.
85
+ - For those three shapes, scaling does not change `radius`. Other shapes, `SPHERE` included, scale as a whole.
86
+ - A shape scaled to zero on any axis has no effect, even when inverted.
87
+ - With uneven scaling, soft edges are approximate.
@@ -2,18 +2,16 @@
2
2
 
3
3
  [Back to documentation](../README.md#documentation)
4
4
 
5
- Supported formats for ordinary model loading:
6
-
7
- ```ts
8
- enum SplatFileType {
9
- PLY = "ply",
10
- SPZ = "spz",
11
- SOG = "sog",
12
- RAD = "rad",
13
- }
14
- ```
5
+ The model formats you can load:
6
+
7
+ | Name | Value | Files |
8
+ | --- | --- | --- |
9
+ | `SplatFileType.PLY` | `"ply"` | `.ply` |
10
+ | `SplatFileType.SPZ` | `"spz"` | `.spz` |
11
+ | `SplatFileType.SOG` | `"sog"` | `.sog`, or a SOG `meta.json` with its images |
12
+ | `SplatFileType.RAD` | `"rad"` | `.rad` |
15
13
 
16
- Usually inferred from a `.ply`, `.spz`, `.sog`, `.rad`, or SOG `meta.json` URL or `fileName`. Otherwise, set it explicitly:
14
+ The format is normally detected from the URL or `fileName`, so you rarely need this. Set it when the name does not show the format:
17
15
 
18
16
  ```js
19
17
  import { SplatFileType, SplatMesh } from "gaussian-splat-lite";
@@ -24,6 +22,6 @@ const splat = new SplatMesh({
24
22
  });
25
23
  ```
26
24
 
27
- `SplatMesh` and `Splats` accept this option for URL, file, and byte input. `SplatLoader.load()` and `loadAsync()` detect the format automatically.
25
+ `SplatMesh` and `Splats` accept `fileType` for URLs, files, and bytes. `SplatLoader.load()` and `loadAsync()` always detect the format themselves.
28
26
 
29
- For a SOG `lod-meta.json` scene, use [SogStreamScheduler](SogStreamScheduler.md). For RAD scenes with levels of detail, use [RadStreamScheduler](RadStreamScheduler.md).
27
+ Large scenes with levels of detail load through their own classes: [SogStreamScheduler](SogStreamScheduler.md) for a SOG `lod-meta.json`, and [RadStreamScheduler](RadStreamScheduler.md) for RAD.