@viggle/splat-engine-three 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +450 -2
- package/dist/chunks/chunk-WN5WBKM4.js +8606 -0
- package/dist/chunks/chunk-WN5WBKM4.js.map +7 -0
- package/dist/index.js +56 -0
- package/dist/index.js.map +7 -0
- package/dist/types/anim/animation-conversion.d.ts +25 -0
- package/dist/types/anim/bind-pose.d.ts +15 -0
- package/dist/types/anim/keyframe-splines.d.ts +59 -0
- package/dist/types/anim/rest-relative.d.ts +32 -0
- package/dist/types/anim/spline.d.ts +14 -0
- package/dist/types/animation.d.ts +216 -0
- package/dist/types/armature.d.ts +524 -0
- package/dist/types/character.d.ts +582 -0
- package/dist/types/engine.d.ts +166 -0
- package/dist/types/environment.d.ts +53 -0
- package/dist/types/index.d.ts +40 -0
- package/dist/types/lighting/color.d.ts +13 -0
- package/dist/types/lighting/point-light.d.ts +33 -0
- package/dist/types/lighting/sky-light.d.ts +22 -0
- package/dist/types/lighting/sun-light.d.ts +34 -0
- package/dist/types/loader/animation/animation-glb.d.ts +117 -0
- package/dist/types/loader/character/index.d.ts +33 -0
- package/dist/types/loader/character/legacy/index.d.ts +34 -0
- package/dist/types/loader/character/legacy/interleaved-splat.d.ts +101 -0
- package/dist/types/loader/character/legacy/planar-splat.d.ts +139 -0
- package/dist/types/loader/character/load.d.ts +33 -0
- package/dist/types/loader/character/types.d.ts +129 -0
- package/dist/types/loader/character/v1/decode.d.ts +32 -0
- package/dist/types/loader/environment/ply.d.ts +16 -0
- package/dist/types/loader/shared/bin-cache.d.ts +19 -0
- package/dist/types/loader/skeleton/index.d.ts +19 -0
- package/dist/types/loader/skeleton/skeleton-data-441.d.ts +5 -0
- package/dist/types/loader/skeleton/skeleton-data.d.ts +6 -0
- package/dist/types/math.d.ts +82 -0
- package/dist/types/skeletal/animation-player.d.ts +185 -0
- package/dist/types/skeletal/bone-mask.d.ts +27 -0
- package/dist/types/skeletal/bone-stretch.d.ts +65 -0
- package/dist/types/skeletal/pose.d.ts +27 -0
- package/dist/types/skeletal/skeleton.d.ts +57 -0
- package/dist/types/skeletal/splat-skinning.d.ts +69 -0
- package/dist/types/splat/backend.d.ts +57 -0
- package/dist/types/splat/bone-binding.d.ts +38 -0
- package/dist/types/splat/depth-sort.d.ts +76 -0
- package/dist/types/splat/importance.d.ts +60 -0
- package/dist/types/splat/packed-splats.d.ts +36 -0
- package/dist/types/splat/prune.d.ts +60 -0
- package/dist/types/splat/quality.d.ts +175 -0
- package/dist/types/splat/splat-mesh.d.ts +71 -0
- package/dist/types/splat/splat.d.ts +252 -0
- package/dist/types/splat/transform-palette.d.ts +42 -0
- package/dist/types/splat/webgl/backend.d.ts +17 -0
- package/dist/types/splat/webgl/shaders.d.ts +2 -0
- package/dist/types/splat/webgl/splat-mesh.d.ts +54 -0
- package/dist/types/splat/webgpu/backend.d.ts +25 -0
- package/dist/types/splat/webgpu/lighting.d.ts +34 -0
- package/dist/types/splat/webgpu/radix-sort.d.ts +46 -0
- package/dist/types/splat/webgpu/splat-mesh.d.ts +73 -0
- package/dist/types/three/loaders.d.ts +77 -0
- package/dist/types/three/splat-character.d.ts +220 -0
- package/dist/types/types.d.ts +30 -0
- package/dist/types/webgpu.d.ts +5 -0
- package/dist/webgpu.js +843 -0
- package/dist/webgpu.js.map +7 -0
- package/package.json +46 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Viggle / WarpEngine
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,451 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @viggle/splat-engine-three
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Gaussian-splat characters for three.js. Load `.vsplat` characters as `Object3D`s, play MetaHuman GLB skeletal animations with crossfade blending, load `.ply` splat scenes, and render them in your own three.js scene under `WebGLRenderer` or `WebGPURenderer`.
|
|
4
|
+
|
|
5
|
+
A port of [`@viggle/splat-engine`](https://www.npmjs.com/package/@viggle/splat-engine) with no PlayCanvas dependency. The rig, animation and bundle-decoding code is shared with that package; the renderer is new.
|
|
6
|
+
|
|
7
|
+
## Install
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npm install @viggle/splat-engine-three three
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Built and tested against three.js r186.
|
|
14
|
+
|
|
15
|
+
## Quick start
|
|
16
|
+
|
|
17
|
+
Characters and splat scenes are ordinary three.js objects: load them with a loader, add them to your scene, move them with `position` / `rotation` / `scale`. Call `character.update(delta)` once per frame - the job `AnimationMixer.update` does for a skinned mesh.
|
|
18
|
+
|
|
19
|
+
```typescript
|
|
20
|
+
import * as THREE from 'three';
|
|
21
|
+
import { VSplatLoader, SplatClipLoader, SplatPLYLoader } from '@viggle/splat-engine-three';
|
|
22
|
+
|
|
23
|
+
const renderer = new THREE.WebGLRenderer({ canvas });
|
|
24
|
+
const scene = new THREE.Scene();
|
|
25
|
+
const camera = new THREE.PerspectiveCamera(45, innerWidth / innerHeight, 0.1, 100);
|
|
26
|
+
|
|
27
|
+
// A splat scene (optional). Many captures are authored Y-down.
|
|
28
|
+
const room = await new SplatPLYLoader().loadAsync('/assets/room.ply');
|
|
29
|
+
room.rotation.x = Math.PI;
|
|
30
|
+
scene.add(room);
|
|
31
|
+
|
|
32
|
+
// A character
|
|
33
|
+
const character = await new VSplatLoader().loadAsync('/assets/character.vsplat');
|
|
34
|
+
character.position.set(0, 0, 1);
|
|
35
|
+
scene.add(character);
|
|
36
|
+
|
|
37
|
+
// Clips
|
|
38
|
+
const clips = new SplatClipLoader();
|
|
39
|
+
const [idle, run] = await Promise.all([
|
|
40
|
+
clips.loadAsync('/assets/idle.glb'),
|
|
41
|
+
clips.loadAsync('/assets/run.glb'),
|
|
42
|
+
]);
|
|
43
|
+
character.play(idle);
|
|
44
|
+
character.crossFadeTo(run, 0.3);
|
|
45
|
+
|
|
46
|
+
const clock = new THREE.Clock();
|
|
47
|
+
renderer.setAnimationLoop(() => {
|
|
48
|
+
character.update(clock.getDelta());
|
|
49
|
+
renderer.render(scene, camera);
|
|
50
|
+
});
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Get character `.vsplat` files and animation `.glb` files from https://viggle.ai/pinoc/app/.
|
|
54
|
+
|
|
55
|
+
## API
|
|
56
|
+
|
|
57
|
+
### Loaders
|
|
58
|
+
|
|
59
|
+
All three extend `THREE.Loader`, so `load` / `loadAsync`, `setPath`, `setRequestHeader`, `setWithCredentials` and a `LoadingManager` work as usual. Each also has `parseAsync(arrayBuffer)` for bytes you already hold.
|
|
60
|
+
|
|
61
|
+
| Loader | Reads | Gives you |
|
|
62
|
+
|---|---|---|
|
|
63
|
+
| `VSplatLoader` | `.vsplat` character bundles | `SplatCharacter` |
|
|
64
|
+
| `SplatClipLoader` | MetaHuman-rig `.glb` animations | `SplatClip` |
|
|
65
|
+
| `SplatPLYLoader` | uncompressed 3D Gaussian Splatting `.ply` | `SplatMesh` |
|
|
66
|
+
|
|
67
|
+
### `SplatCharacter`
|
|
68
|
+
|
|
69
|
+
An `Object3D`. Its `position` is the model origin: a character in its rest pose stands on it.
|
|
70
|
+
|
|
71
|
+
```typescript
|
|
72
|
+
character.play(clip, { loop: false }); // cut to a clip
|
|
73
|
+
character.crossFadeTo(clip, 0.3); // blend to a clip over 0.3 s
|
|
74
|
+
character.stop(0.2); // back to the rest pose
|
|
75
|
+
|
|
76
|
+
character.timeScale = 1.5; // playback speed
|
|
77
|
+
character.paused = true;
|
|
78
|
+
character.time; // seconds into the clip (settable)
|
|
79
|
+
character.duration; // length of the clip in seconds
|
|
80
|
+
character.clip; // what is playing, or null
|
|
81
|
+
|
|
82
|
+
character.addEventListener('finished', (e) => {}); // a non-looping clip ended
|
|
83
|
+
character.addEventListener('loop', (e) => {}); // a looping clip wrapped
|
|
84
|
+
|
|
85
|
+
// Optional for grounded clips such as a dance; pins the lowest splats to y=0.
|
|
86
|
+
// Leave unset for jumps or other intentional airborne motion.
|
|
87
|
+
character.groundLockY = 0;
|
|
88
|
+
|
|
89
|
+
// A second clip on part of the body, over the base clip
|
|
90
|
+
character.playLayer('wave', waveClip, { region: 'upperBody', weight: 0.8 });
|
|
91
|
+
character.stopLayer('wave', 0.2);
|
|
92
|
+
|
|
93
|
+
// Root motion: 'off' plays in place, 'apply' moves the body with the clip,
|
|
94
|
+
// 'extract' plays in place and hands you the travel
|
|
95
|
+
character.rootMotion = 'extract';
|
|
96
|
+
const step = character.consumeRootMotion(); // Vector3, local space, per frame
|
|
97
|
+
|
|
98
|
+
character.setTint('#ff4040', 0.6); // blend toward a colour; 0 clears it
|
|
99
|
+
character.getBoneWorldMatrix('hand_r'); // Matrix4, to attach a prop
|
|
100
|
+
|
|
101
|
+
const twin = character.clone(); // shares the decoded data, not the pose
|
|
102
|
+
character.dispose(); // free GPU resources
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Expression `.vsplat` bundles with 119 bones use their 86 body bones and
|
|
106
|
+
per-character body rest lengths. Their remaining 33 facial influences follow
|
|
107
|
+
the head as a rigid group, keeping the face connected during animation. This
|
|
108
|
+
does not reproduce independent facial expressions; use a 441-bone character
|
|
109
|
+
for those.
|
|
110
|
+
|
|
111
|
+
Animation goes through this class, not `AnimationMixer`: a splat character has no `Bone` hierarchy for a mixer to drive. A `SplatClip` is not tied to a character - play one clip on any number of them, and call `clip.dispose()` when you are done with it.
|
|
112
|
+
|
|
113
|
+
For several copies of one character, load it once and `clone()` it; loading the same file twice decodes it twice.
|
|
114
|
+
|
|
115
|
+
### `SplatMesh`
|
|
116
|
+
|
|
117
|
+
A `Mesh`, so `visible`, `renderOrder`, `layers` and parent transforms behave as they do for any mesh. `SplatPLYLoader` gives it `renderOrder = -1` so it draws before characters; see [Depth and draw order](#depth-and-draw-order). `mesh.splat` holds its render options (`antiAlias`, `minPixelSize`, `setTint`, `setGrade`); `mesh.dispose()` frees it.
|
|
118
|
+
|
|
119
|
+
Splats are not pickable: `Raycaster` reports no hits on them.
|
|
120
|
+
|
|
121
|
+
### Lights
|
|
122
|
+
|
|
123
|
+
Splats carry baked colour and ignore the lights in your scene. To relight them, use the engine's own lights, which multiply that colour: ambient plus sun plus range-attenuated point lights, with no shading by surface direction.
|
|
124
|
+
|
|
125
|
+
```typescript
|
|
126
|
+
import { SplatEngine, SunLight, SkyLight, PointLight } from '@viggle/splat-engine-three';
|
|
127
|
+
|
|
128
|
+
SkyLight.create(SplatEngine.shared, { color: '#8ea8ff', intensity: 0.4 });
|
|
129
|
+
SunLight.create(SplatEngine.shared, { color: '#ffd9a0', intensity: 0.9 });
|
|
130
|
+
PointLight.create(SplatEngine.shared, { position: { x: 1, y: 1.5, z: 1 }, color: '#ff4020', intensity: 2, range: 2.5 });
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
`SplatEngine.shared` is the state loaders use by default - the skeleton library and these lights. With no lights set, splats render as authored.
|
|
134
|
+
|
|
135
|
+
## WebGL or WebGPU
|
|
136
|
+
|
|
137
|
+
The package draws under both `THREE.WebGLRenderer` and `WebGPURenderer`, and picks its implementation from whichever renderer draws a splat. The only thing you choose is the import: applications on `WebGPURenderer` import from `@viggle/splat-engine-three/webgpu`, the same way they import three.js itself from `three/webgpu`.
|
|
138
|
+
|
|
139
|
+
```typescript
|
|
140
|
+
import { WebGPURenderer } from 'three/webgpu';
|
|
141
|
+
import { VSplatLoader, SplatClipLoader, SplatPLYLoader } from '@viggle/splat-engine-three/webgpu';
|
|
142
|
+
|
|
143
|
+
const renderer = new WebGPURenderer({ canvas });
|
|
144
|
+
await renderer.init();
|
|
145
|
+
// ...everything else is identical
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
| | `THREE.WebGLRenderer` | `WebGPURenderer` |
|
|
149
|
+
|---|---|---|
|
|
150
|
+
| Import from | `@viggle/splat-engine-three` | `@viggle/splat-engine-three/webgpu` |
|
|
151
|
+
| Runs on | WebGL2 | WebGPU |
|
|
152
|
+
| Skinning | GPU, vertex stage | GPU, compute |
|
|
153
|
+
| Depth sort | CPU - main thread for characters, a worker for static meshes | GPU, compute |
|
|
154
|
+
| Per-character CPU cost | re-skins every splat centre each frame to feed the sort | a pose evaluation and a palette upload |
|
|
155
|
+
|
|
156
|
+
**Use WebGPU when you can.** Its cost per animated character is about a third of WebGL's, because nothing per-splat happens on the CPU. See [Performance](#performance).
|
|
157
|
+
|
|
158
|
+
`WebGPURenderer` must actually be running on WebGPU. Its WebGL2 fallback (`forceWebGL`, or a browser without WebGPU) has no compute shaders and cannot draw splats; the package logs an error saying so. Check before you build the scene, and use `THREE.WebGLRenderer` on those devices:
|
|
159
|
+
|
|
160
|
+
```typescript
|
|
161
|
+
if (renderer.backend.isWebGPUBackend !== true) { /* use THREE.WebGLRenderer instead */ }
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### Colour and `WebGPURenderer`
|
|
165
|
+
|
|
166
|
+
Splat colours are display-referred (sRGB), and splats are trained to be alpha-blended in that space. `THREE.WebGLRenderer` draws straight to an sRGB canvas, so that is what happens.
|
|
167
|
+
|
|
168
|
+
`WebGPURenderer` with its default `outputColorSpace` renders into a linear buffer first and converts at the end, so splats blend in linear space instead. Soft edges - hair, fabric edges, anything semi-transparent - come out slightly different from the reference look. To get the reference look, have the renderer write its working space straight to the canvas:
|
|
169
|
+
|
|
170
|
+
```typescript
|
|
171
|
+
renderer.outputColorSpace = THREE.LinearSRGBColorSpace;
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
That setting applies to the whole renderer: other materials in the scene are then no longer converted to sRGB for you. If your scene depends on three.js colour management, leave the default and accept the difference on soft edges. Measured against the PlayCanvas build on a face close-up, the default leaves about 10% of pixels off by more than 8/255 (PSNR 33 dB); writing the working space directly leaves 0.02% (PSNR 48 dB).
|
|
175
|
+
|
|
176
|
+
## What is and is not ported
|
|
177
|
+
|
|
178
|
+
Ported, and checked against the PlayCanvas build by the A/B suite (see [Verification](#verification)):
|
|
179
|
+
|
|
180
|
+
- Characters - loading `.vsplat` bundles, position / rotation / scale, tint, colour grade
|
|
181
|
+
- Animation - GLB clips, crossfade, layers with bone masks, root motion
|
|
182
|
+
- Rig - pose evaluation, bone-length scaling (`setBoneStretch`)
|
|
183
|
+
- Splat scenes - uncompressed `.ply`, with position / rotation / scale
|
|
184
|
+
- `SunLight`, `SkyLight`, `PointLight`
|
|
185
|
+
- Render options - anti-aliasing, sub-pixel culling, per-group visibility (`setSplatGroups` / `setVariantMask`)
|
|
186
|
+
- Quality settings - `SplatQuality`, ported from the Unity build of the same engine rather than from the PlayCanvas one. Everything that applies to a renderer with no offscreen splat target; see [Tuning](#tuning) for which of its knobs did not come across and why.
|
|
187
|
+
|
|
188
|
+
Ported from the same source, but not exercised by the suite for want of test assets:
|
|
189
|
+
|
|
190
|
+
- Legacy `.bin` character bundles, and clips embedded in them
|
|
191
|
+
- `.anim` clips and animation events
|
|
192
|
+
- Multi-level character LOD (`Character.load([...])`, `addLodLevel`, `setRenderLod`) and variant splats (`addVariantSplat`)
|
|
193
|
+
- Animation LOD and sort LOD
|
|
194
|
+
|
|
195
|
+
Not ported:
|
|
196
|
+
|
|
197
|
+
- Shadows (`ShadowSystem`)
|
|
198
|
+
- Post effects (`PostFX`, pixel-art, CRT, outline, stained glass) and the VFX system
|
|
199
|
+
- Ragdoll and the physics joints
|
|
200
|
+
- `.spz`, compressed `.ply`, SOG and streamed LOD bundles for environments; environment clip bounds and camera cutaway
|
|
201
|
+
- Presets: `DamageVariant`, `BodyShapeVariant`, `CharacterMotor`, `WasdController`, camera controls, `TileGrid`, grid / sky dome / gizmos. The camera controls and helpers have direct three.js equivalents (`OrbitControls`, `GridHelper`, …).
|
|
202
|
+
|
|
203
|
+
## The engine-level API
|
|
204
|
+
|
|
205
|
+
Underneath the three.js-style API sits a second one, shaped like the PlayCanvas build's: `SplatEngine`, `Character`, `Animation`, `Environment`. It is what the A/B suite compares against that build, and it is how you reach the features the three.js-style API does not surface yet - multi-level character LOD, variant splats, animation and sort LOD, bone-length scaling.
|
|
206
|
+
|
|
207
|
+
`splatCharacter.character` is the engine-level `Character` behind a `SplatCharacter`, and `.armature` its rig, so the two can be mixed:
|
|
208
|
+
|
|
209
|
+
```typescript
|
|
210
|
+
character.armature.setBoneStretch({ thigh_l: { along: 1.3 }, calf_l: { along: 1.3 } });
|
|
211
|
+
await character.character.addLodLevel('/assets/character.l1.vsplat');
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Used on its own it looks like this. Clips are registered and played by name, time is counted in frames, rotations are in degrees, and `engine.update(dt)` advances every character at once:
|
|
215
|
+
|
|
216
|
+
```typescript
|
|
217
|
+
import { SplatEngine, Character, Animation, Environment } from '@viggle/splat-engine-three';
|
|
218
|
+
|
|
219
|
+
const engine = new SplatEngine({ scene, camera }); // everything it loads is added under `scene`
|
|
220
|
+
await Environment.load(engine, '/assets/room.ply', { rotation: { x: 180, y: 0, z: 0 } });
|
|
221
|
+
const character = await Character.load(engine, '/assets/character.vsplat');
|
|
222
|
+
await Animation.loadGlb(engine, '/assets/run.glb', 'run');
|
|
223
|
+
character.playAnimation('run', { loop: true });
|
|
224
|
+
|
|
225
|
+
renderer.setAnimationLoop(() => {
|
|
226
|
+
engine.update(clock.getDelta());
|
|
227
|
+
renderer.render(scene, camera);
|
|
228
|
+
});
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
One difference to know when mixing the two: `Character.setPosition` lowers the model by a per-character ground offset carried over from the PlayCanvas build (about 8 cm on the test characters), to match how characters in that build's older bundle format sit. `SplatCharacter.position` applies no offset.
|
|
232
|
+
|
|
233
|
+
## Differences from the PlayCanvas build
|
|
234
|
+
|
|
235
|
+
- **No `Scene`.** `Scene.create(canvas)` + `scene.start()` become your own renderer, scene and render loop.
|
|
236
|
+
- **Environments always respond to lights.** In the PlayCanvas build an environment only follows `SunLight` / `SkyLight` once a `ShadowSystem` is bound; without one it stays unlit whatever lights exist. Here it follows them from the start.
|
|
237
|
+
- **Character bundles decode on the main thread.** There is no worker file to keep track of, so the bundler notes in that build's README do not apply. A bundle decodes in a few milliseconds.
|
|
238
|
+
- **Under WebGPU, at most 16 point lights and 64 variant groups** are evaluated.
|
|
239
|
+
|
|
240
|
+
### Depth and draw order
|
|
241
|
+
|
|
242
|
+
Splats test against the depth buffer and do not write it. Opaque meshes therefore occlude them correctly with no extra setup, but two splat objects are layered by draw order alone, not by depth. Static meshes from `SplatPLYLoader` draw first (`renderOrder = -1`), so characters always composite on top of the scene they stand in - the same layering the PlayCanvas build uses. A character that walks behind a wall of the splat scene is still drawn over it.
|
|
243
|
+
|
|
244
|
+
## Performance
|
|
245
|
+
|
|
246
|
+
For what to change when these numbers are not good enough on a given device, see [Tuning](#tuning).
|
|
247
|
+
|
|
248
|
+
Mean time per frame, 1920×1080, NVIDIA RTX 3090, Chromium (headless, vsync off), median of five runs. Lower is better. The first two columns are the original PlayCanvas engine on the same scenes, for reference.
|
|
249
|
+
|
|
250
|
+
<!-- BENCH_TABLE -->
|
|
251
|
+
| Scenario | Splats | PlayCanvas, WebGL | PlayCanvas, WebGPU | this package, WebGL | this package, WebGPU |
|
|
252
|
+
|---|--:|--:|--:|--:|--:|
|
|
253
|
+
| 1 character | 46,983 | 1.48 ms | 13.66 ms | 1.42 ms | **0.96 ms** |
|
|
254
|
+
| 8 characters | 356,960 | 11.37 ms | 12.76 ms | 13.44 ms | **4.18 ms** |
|
|
255
|
+
| 32 characters | 1,427,840 | 40.78 ms | 39.58 ms | 45.78 ms | **15.13 ms** |
|
|
256
|
+
| room, still camera | 1,048,576 | 2.73 ms | 15.54 ms | **1.70 ms** | 2.07 ms |
|
|
257
|
+
| room, orbiting camera | 1,048,576 | 3.39 ms | 15.54 ms | **1.86 ms** | 5.28 ms |
|
|
258
|
+
| room + 4 characters, orbiting | 1,227,056 | **6.64 ms** | 22.31 ms | 9.76 ms | 7.23 ms |
|
|
259
|
+
<!-- /BENCH_TABLE -->
|
|
260
|
+
|
|
261
|
+
Reading the table:
|
|
262
|
+
|
|
263
|
+
- The two WebGL columns do the same work per character - both re-skin every splat centre on the CPU to feed the depth sort - and are equal to within the run-to-run noise of the shared machine these were measured on (roughly ±20% on the character rows).
|
|
264
|
+
- The WebGPU backend moves that work to the GPU, which is where the 3x on the character rows comes from. What remains on the CPU is pose evaluation.
|
|
265
|
+
- With a moving camera and nothing animated, the WebGL backend is ahead: it re-sorts the environment in a worker and keeps drawing the previous order meanwhile, while the WebGPU backend re-sorts on the GPU inside the frame. The WebGPU result is never a frame behind; the WebGL one can be.
|
|
266
|
+
- The PlayCanvas WebGPU column sat on a ~13 ms floor in this harness, which drives that engine one `app.tick()` at a time. That was not investigated and may not reflect how it behaves under its own frame loop.
|
|
267
|
+
|
|
268
|
+
Regenerate with `npm run bench -- --backend <webgl|webgpu|webgpu-managed|pc-webgl|pc-webgpu>`, then `node scripts/readme-tables.mjs`.
|
|
269
|
+
|
|
270
|
+
## Tuning
|
|
271
|
+
|
|
272
|
+
A splat frame is bound by one of two things, and that decides which knobs do anything:
|
|
273
|
+
|
|
274
|
+
| Bound by | When | What moves it |
|
|
275
|
+
|---|---|---|
|
|
276
|
+
| **Fragments** (GPU) | WebGPU; anything static; a character filling the screen | `quadAlphaCutoff` |
|
|
277
|
+
| **Splats** (main thread) | WebGL with animated characters - every centre is re-skinned each frame to feed the CPU sort | `minOpacity`, `pruneRatio`, the distance throttles |
|
|
278
|
+
|
|
279
|
+
Measured on 32 animated characters under WebGL: 26.5 ms of main thread per frame, of which 23.7 ms is skinning and 2.8 ms the sort. `node ab/profile-stages.mjs` prints this split for any scene.
|
|
280
|
+
|
|
281
|
+
Settings are module-level, on `SplatQuality`, read per frame (or per decode, where stated). Two ship non-zero.
|
|
282
|
+
|
|
283
|
+
### `quadAlphaCutoff` — default `8 / 255`
|
|
284
|
+
|
|
285
|
+
Sizes each splat's quad to the radius its own opacity reaches, instead of the worst case. Fewer blended fragments; harder splat edges, visible first on hair, fabric and faces. **Takes effect next frame**, reversible, safe to drive from a frame-rate governor. Clamped to `[0, 1]`.
|
|
286
|
+
|
|
287
|
+
| Value | |
|
|
288
|
+
|---|---|
|
|
289
|
+
| `1 / 255` | Identical to the behaviour before this knob existed. A reference, not a setting. |
|
|
290
|
+
| `8 / 255` | **Default.** Also what the Unity build ships. |
|
|
291
|
+
| `12 / 255` | One step down. |
|
|
292
|
+
| `16 / 255` | **Low-end, and the floor.** |
|
|
293
|
+
| above `16 / 255` | Don't. More performance is there, but every silhouette hardens; spend it on `pruneRatio`, pixel ratio or a distance LOD instead. |
|
|
294
|
+
|
|
295
|
+
Cost on a face filling a 760px frame, against `1 / 255`: `8/255` 40.7 dB / 3.1% of pixels off by >8; `12/255` 36.8 dB / 7.8%; `16/255` 34.3 dB / 11.2%. Frame time where fragments bind, against `1 / 255`: `8/255` -29%, `16/255` -42%.
|
|
296
|
+
|
|
297
|
+
### `minOpacity` — default `0.05`, hard cap `0.1`
|
|
298
|
+
|
|
299
|
+
Drops splats whose peak opacity is below the threshold. Saves memory, skinning, sorting and rasterisation together, where `quadAlphaCutoff` saves rasterisation alone.
|
|
300
|
+
|
|
301
|
+
- **Applied during the decode of a character bundle.** Set it before `load()`; changing it later needs a reload.
|
|
302
|
+
- **Does not touch `.ply` scenes** - same scope as the Unity build. `0.05` removes 8.7% of the test character but 41.8% of the 1M-splat test room. `pruneSplats` is exported for scenes.
|
|
303
|
+
- **Changes `Splat.numSplats`.** A `setSplatGroups` array must be built from the loaded splats, not from the file. An array authored offline against the file cannot be truncated to fit - its indices no longer line up.
|
|
304
|
+
- **Values above `0.1` are clamped**, with one console warning. Enforced because every increment buys frame time while the cost arrives gradually and everywhere, so nothing tells a tuning pass to stop.
|
|
305
|
+
|
|
306
|
+
| Value | Cost on a face filling the frame | Splats kept |
|
|
307
|
+
|---|---|---|
|
|
308
|
+
| `0` | none | 100% |
|
|
309
|
+
| `1 / 255` | pixel-identical | 98% |
|
|
310
|
+
| `0.05` | max 20/255, 0.94% of pixels off by >8 | 91% |
|
|
311
|
+
| `0.1` (cap) | max 45/255, 9.3% of pixels off by >8 | 83% |
|
|
312
|
+
|
|
313
|
+
### The rest
|
|
314
|
+
|
|
315
|
+
```ts
|
|
316
|
+
SplatQuality.pruneRatio = 0.3; // 0..1, decode-time, characters only: drop the weakest
|
|
317
|
+
// share of every pruneVoxelSize cell by opacity x footprint
|
|
318
|
+
SplatQuality.pruneVoxelSize = 0.04; // metres; the cell that quota is per
|
|
319
|
+
SplatQuality.screenSizeCull = 0; // pixels; drop splats smaller than this. For distant crowds
|
|
320
|
+
SplatQuality.depthOcclusion = true; // scene geometry hides splats. Correctness, not performance:
|
|
321
|
+
// this renderer gets the hardware depth test for free
|
|
322
|
+
SplatQuality.passTimings = false; // then read SplatQuality.timings after each render
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
`pruneRatio` ranks inside each cell, not globally: a face is many faint splats where a jacket is few strong ones, so a global rank deletes the face first.
|
|
326
|
+
|
|
327
|
+
### Distance throttles, per character
|
|
328
|
+
|
|
329
|
+
```ts
|
|
330
|
+
SplatEngine.shared.camera = camera; // all three measure distance from it; without it they no-op
|
|
331
|
+
|
|
332
|
+
character.setSplatCountLod({ nearDist: 4, farDist: 16, farFraction: 0.25 });
|
|
333
|
+
character.armature.setSortLod({ nearDist: 5, farDist: 15, nearStride: 2, farStride: 4 });
|
|
334
|
+
character.armature.setAnimationLod({ nearDist: 12, farDist: 24, nearStride: 2, farStride: 4 });
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
All off until called. `nearStride` applies between `nearDist` and `farDist`, `farStride` beyond `farDist` - **set both**, or the band most of a crowd is standing in keeps its previous value.
|
|
338
|
+
|
|
339
|
+
`setSplatCountLod` is `pruneRatio` made dynamic: nothing is discarded, and the fraction skinned, sorted and drawn follows camera distance, by the same per-cell importance. WebGL backend only - that is the one paying per splat on the CPU.
|
|
340
|
+
|
|
341
|
+
### Per device
|
|
342
|
+
|
|
343
|
+
Pixel ratio first; it is not in this package and it is larger than everything below it. A phone reports `devicePixelRatio` 3, which is nine times the fragments.
|
|
344
|
+
|
|
345
|
+
```ts
|
|
346
|
+
renderer.setPixelRatio(Math.min(window.devicePixelRatio, 1.5));
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
| Tier | `quadAlphaCutoff` | `minOpacity` | `pruneRatio` | Pixel ratio | Also |
|
|
350
|
+
|---|---|---|---|---|---|
|
|
351
|
+
| Desktop / flagship | `8 / 255` | `0.05` | `0` | ≤ 2 | — |
|
|
352
|
+
| Mid-range phone | `12 / 255` | `0.05` | `0` | ≤ 1.5 | — |
|
|
353
|
+
| Low-end phone | `16 / 255` | `0.05` | `0.3` | ≤ 1.5 | `setSplatCountLod({ nearDist: 3, farDist: 12, farFraction: 0.25 })` |
|
|
354
|
+
| Very low-end, or a crowd | `16 / 255` | `0.05` | `0.5` | 1 | plus `setSortLod` / `setAnimationLod` |
|
|
355
|
+
|
|
356
|
+
On WebGL with several animated characters the cutoff column does nothing; spend the budget to its right.
|
|
357
|
+
|
|
358
|
+
### Measured
|
|
359
|
+
|
|
360
|
+
1920×1080, Apple M5 Pro, headless Chromium, vsync off. Baseline is **every knob off**, including the two that ship on, so the `DEFAULT` rows are what the defaults themselves cost and buy. `~` is inside that column's noise floor.
|
|
361
|
+
|
|
362
|
+
<!-- QUALITY_TABLE -->
|
|
363
|
+
| Setting | Splats | PSNR | Worst pixel | WebGL, 2 close | WebGL, 2 close, GPU only |
|
|
364
|
+
|---|--:|--:|--:|--:|--:|
|
|
365
|
+
| `DEFAULT cutoff 8/255` | - | 53.4 dB | 31/255 | ~ | -22% |
|
|
366
|
+
| `DEFAULT 8/255 + minOp 0.05` | 78,004 | 53.0 dB | 31/255 | -10% | -22% |
|
|
367
|
+
| `cutoff 12/255` | - | 48.5 dB | 47/255 | ~ | -24% |
|
|
368
|
+
| `cutoff 16/255` | - | 45.3 dB | 63/255 | ~ | -29% |
|
|
369
|
+
| `cutoff 16 + minOp 0.05` | 78,004 | 45.3 dB | 63/255 | -12% | -30% |
|
|
370
|
+
| `cutoff 24/255` | - | 40.8 dB | 98/255 | ~ | -34% |
|
|
371
|
+
| `cutoff 32/255` | - | 37.5 dB | 144/255 | ~ | -36% |
|
|
372
|
+
| `minOpacity 1/255` | 87,262 | ∞ | 0/255 | ~ | ~ |
|
|
373
|
+
| `minOpacity 0.05` | 78,004 | 61.7 dB | 14/255 | -11% | ~ |
|
|
374
|
+
| `prune 0.3` | 64,755 | 43.9 dB | 174/255 | -23% | -8% |
|
|
375
|
+
| `prune 0.5` | 45,952 | 37.8 dB | 187/255 | -42% | -20% |
|
|
376
|
+
| `screenSizeCull 1px` | - | ∞ | 0/255 | ~ | +10% |
|
|
377
|
+
| `screenSizeCull 2px` | - | 100.8 dB | 4/255 | ~ | ~ |
|
|
378
|
+
| `cutoff 16 + prune 0.3` | 64,755 | 40.3 dB | 179/255 | -22% | -36% |
|
|
379
|
+
| `cutoff 16 + prune 0.3 + minOp 0.05` | 56,824 | 40.2 dB | 179/255 | -31% | -36% |
|
|
380
|
+
| `countLod 0.5` | - | 35.2 dB | 190/255 | -43% | -17% |
|
|
381
|
+
| `countLod 0.3` | - | 31.1 dB | 196/255 | -41% | -35% |
|
|
382
|
+
| `countLod 0.3 + cutoff 16` | - | 30.4 dB | 201/255 | -57% | -47% |
|
|
383
|
+
| `sortLod stride 2` | - | 48.2 dB | 56/255 | -24% | ~ |
|
|
384
|
+
| `sortLod stride 4` | - | 48.2 dB | 56/255 | -24% | ~ |
|
|
385
|
+
| `animLod stride 2` | - | ∞ | 0/255 | -38% | ~ |
|
|
386
|
+
|
|
387
|
+
Noise floor per column: WebGL, 2 close ±9%, WebGL, 2 close, GPU only ±6%. `~` is a result inside it. Splats are per scene where a setting changes the count, `-` where it does not.
|
|
388
|
+
<!-- /QUALITY_TABLE -->
|
|
389
|
+
|
|
390
|
+
`screenSizeCull` culls nothing in these scenes (frames are bit-identical) yet reads +10% in one column: a single run can still mislead. The throttles read the same because two characters cannot separate skipping half the frames from skipping three quarters; on 32 characters, per-frame main-thread ms:
|
|
391
|
+
|
|
392
|
+
| | frame | update | skin | sort |
|
|
393
|
+
|---|--:|--:|--:|--:|
|
|
394
|
+
| baseline | 29.9 | 26.5 | 23.7 | 2.8 |
|
|
395
|
+
| `sortLod` stride 2 | 16.5 | 14.6 | 11.9 | 1.4 |
|
|
396
|
+
| `sortLod` stride 4 | 9.9 | 8.8 | 6.1 | 0.7 |
|
|
397
|
+
| `countLod 0.3` | 12.2 | 10.1 | 7.3 | 1.7 |
|
|
398
|
+
| `prune 0.5` | 16.7 | 14.7 | 11.9 | 1.4 |
|
|
399
|
+
|
|
400
|
+
With skinning throttled away the floor is pose evaluation, a couple of ms per 32 characters.
|
|
401
|
+
|
|
402
|
+
```bash
|
|
403
|
+
npm run build
|
|
404
|
+
node ab/run-quality-ab.mjs --backend webgl --scenario fill-2 # [--animate 0] for GPU alone
|
|
405
|
+
node ab/profile-stages.mjs --scenario crowd-32 # which stage a setting cut
|
|
406
|
+
node ab/quality-shots.mjs # contact sheets to judge by eye
|
|
407
|
+
node scripts/readme-tables.mjs # rewrite the table above
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
Scenarios: `fill-2`, `crowd-16`, `crowd-32`, `room`, `room-4`.
|
|
411
|
+
|
|
412
|
+
### Not ported
|
|
413
|
+
|
|
414
|
+
`RenderScale`, `LdrRenderTarget`, `FitTargetToCharacters`, `FitCompositeOnly` act on an offscreen splat target. This package draws straight into the camera's target - the mode the Unity build measured as its fastest - so they have nothing to act on. For a lower splat resolution, render the whole frame smaller.
|
|
415
|
+
|
|
416
|
+
`SplitComputeAndDraw` (batching every splat's compute ahead of all draws, worth 11.6% on a Mali-G57) was ported and removed: `WebGPURenderer` already submits each compute call in its own command buffer ahead of the render pass, so there is no interleaving to undo, and it measured 6-18% slower.
|
|
417
|
+
|
|
418
|
+
## Verification
|
|
419
|
+
|
|
420
|
+
`ab/` holds an A/B suite that drives the PlayCanvas build and this package through the same cases - both characters, every supplied clip, crossfades, layers, root motion, transforms, tint, grade, lights, bone stretch, variant masks, and the splat scene - and compares:
|
|
421
|
+
|
|
422
|
+
- **Bone matrices**, numerically. They are bit-identical to the PlayCanvas build in every case.
|
|
423
|
+
- **Rendered frames**, per pixel. See the table below; the PlayCanvas build's own WebGL and WebGPU renderers differ from each other by about as much as this package differs from either.
|
|
424
|
+
|
|
425
|
+
The suite runs this package through the three.js-style API by default, and can run it through the engine-level API instead (`--api engine`); both pass. What has no counterpart in the PlayCanvas build - loaders and the `LoadingManager`, events, time in seconds, `clone`, `dispose` - is covered by `ab/check-api.mjs`.
|
|
426
|
+
|
|
427
|
+
<!-- AB_TABLE -->
|
|
428
|
+
| Compared with the PlayCanvas build (WebGL) | Cases | Bone matrices, max difference | PSNR, median | PSNR, worst | Pixels off by more than 8/255, worst case |
|
|
429
|
+
|---|--:|--:|--:|--:|--:|
|
|
430
|
+
| WebGL backend, characters | 66 | 0 | 55.5 dB | 45.7 dB | 0.050% |
|
|
431
|
+
| WebGL backend, environment | 10 | 0 | 44.1 dB | 42.6 dB | 0.040% |
|
|
432
|
+
| WebGPU backend, characters | 66 | 0 | 55.6 dB | 45.7 dB | 0.048% |
|
|
433
|
+
| WebGPU backend, environment | 10 | 0 | 44.1 dB | 42.6 dB | 0.041% |
|
|
434
|
+
<!-- /AB_TABLE -->
|
|
435
|
+
|
|
436
|
+
```bash
|
|
437
|
+
npm run build
|
|
438
|
+
npm run ab # WebGL vs PlayCanvas (WebGL), three.js-style API
|
|
439
|
+
npm run ab -- --port webgpu # WebGPU
|
|
440
|
+
npm run ab -- --api engine # the engine-level API instead
|
|
441
|
+
node ab/check-api.mjs # the three.js-style API's own contract
|
|
442
|
+
node ab/check-stability.mjs # still frames must be bit-stable
|
|
443
|
+
node ab/check-prune.mjs # the load-time prune keeps every column in lockstep
|
|
444
|
+
node ab/check-animation.mjs # a playing clip must actually reach the screen
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
The suite needs a GPU-backed headless Chromium (`npx playwright install chromium`).
|
|
448
|
+
|
|
449
|
+
## Licence
|
|
450
|
+
|
|
451
|
+
MIT. See [LICENSE](./LICENSE).
|