three-vat 0.1.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +302 -29
- package/dist/chunk-SXTVASKG.js +40 -0
- package/dist/index.d.ts +41 -51
- package/dist/index.js +256 -89
- package/dist/instance-playback-BrGBIKLe.d.ts +109 -0
- package/dist/tsl.d.ts +136 -16
- package/dist/tsl.js +73 -16
- package/dist/webgl.d.ts +64 -14
- package/dist/webgl.js +16 -24
- package/package.json +17 -11
- package/dist/types-DVtQvVjk.d.ts +0 -45
package/README.md
CHANGED
|
@@ -1,10 +1,23 @@
|
|
|
1
1
|
# three-vat
|
|
2
2
|
|
|
3
|
+
[](https://github.com/MikeFernandez-Pro/three-vat/actions/workflows/ci.yml)
|
|
4
|
+
[](https://www.npmjs.com/package/three-vat)
|
|
5
|
+
[](./LICENSE)
|
|
6
|
+
|
|
3
7
|
Bake a glTF `AnimationClip` into GPU textures and animate **hundreds or thousands of instanced characters with zero per-frame CPU** — one draw call, no `SkinnedMesh` per character.
|
|
4
8
|
|
|
5
|
-
VAT (Vertex Animation Texture) is battle-tested in Unity/Unreal but has been a gap on the three.js side: only scattered demos, no maintained package, nothing in drei. `three-vat` bakes the VAT **at runtime, directly from the glTF** — so any Mixamo/Sketchfab asset works with zero pipeline
|
|
9
|
+
VAT (Vertex Animation Texture) is battle-tested in Unity/Unreal but has been a gap on the three.js side: only scattered demos, no maintained package, nothing in drei. `three-vat` bakes the VAT **at runtime, directly from the glTF** — so any Mixamo/Sketchfab asset works with zero pipeline, and there is exactly one way to produce a VAT.
|
|
10
|
+
|
|
11
|
+
## Live demos
|
|
12
|
+
|
|
13
|
+
- **[WebGL crowd](https://mikefernandez-pro.github.io/three-vat/webgl_crowd.html)** — 340 robots, mixed clips, one mesh, through the GLSL decode path.
|
|
14
|
+
- **[WebGPU crowd](https://mikefernandez-pro.github.io/three-vat/webgpu_crowd.html)** — the same crowd through the TSL decode path, on `WebGPURenderer`.
|
|
15
|
+
|
|
16
|
+
Both pages carry a live draw-call counter and a view of the baked textures, with
|
|
17
|
+
cursors on the frame rows each instance is sampling. Source in
|
|
18
|
+
[`examples/`](./examples); they deploy from `main` on every push.
|
|
6
19
|
|
|
7
|
-
> **Status:
|
|
20
|
+
> **Status: `1.0.0`** — the npm badge above reads the registry, so it is the one to trust for what is actually published. The library is **three surfaces**, and all three work: the core baker (`three-vat`), the WebGL/GLSL decode (`three-vat/webgl`), and the WebGPU/TSL decode (`three-vat/tsl`). The two decode paths read one shared instance-playback contract and export the same `createVATMesh`, so nothing documented here is true on one renderer and false on the other. Where the renderers genuinely differ — shadow materials, the `time` clock's type, and what the TSL node builder needs to re-apply instancing — it is called out where it arises. The baker and the WebGL decode are covered by tests; the TSL path is tested structurally, because CI has no GPU, and that the two paths decode *pixel-identically* is a manual release gate ([`pnpm parity`](./docs/releasing.md)). See [`docs/DESIGN.md`](./docs/DESIGN.md) and [`docs/adr/`](./docs/adr) for the full rationale, [What 1.0 does not do](#what-10-does-not-do) for the deferred work, and [`CHANGELOG.md`](./CHANGELOG.md) for release notes.
|
|
8
21
|
|
|
9
22
|
## Install
|
|
10
23
|
|
|
@@ -19,80 +32,340 @@ npm install three-vat three
|
|
|
19
32
|
```ts
|
|
20
33
|
import { bakeVAT } from 'three-vat'
|
|
21
34
|
|
|
22
|
-
// gltf loaded via GLTFLoader
|
|
35
|
+
// gltf loaded via GLTFLoader — pass the subtree root, not a mesh.
|
|
23
36
|
const clips = gltf.animations.filter((c) => c.name !== 'TPose')
|
|
24
|
-
const vat = bakeVAT(gltf.scene,
|
|
25
|
-
// vat: { positionTexture, normalTexture,
|
|
37
|
+
const vat = bakeVAT(gltf.scene, clips, { fps: 30 })
|
|
38
|
+
// vat: { positionTexture, normalTexture, geometry, materials, clips, bounds, ... }
|
|
26
39
|
```
|
|
27
40
|
|
|
28
|
-
|
|
41
|
+
The bake unit is the **whole subtree**, merged into one vertex set and recorded in root space ([ADR-0008](./docs/adr/0008-a-vat-bakes-a-posed-subtree-not-a-skinnedmesh.md)). A VAT only records *where a vertex ended up*, never how it got there, so one call handles a single `SkinnedMesh`, a morph-target mesh, a hierarchy of rigid node-animated parts (three.js `RobotExpressive`), or any mix — with no classification by the caller.
|
|
42
|
+
|
|
43
|
+
Two consequences worth knowing up front:
|
|
44
|
+
|
|
45
|
+
- **Render `vat.geometry`, not your source mesh.** The merged vertex ordering is the baker's, and the textures are indexed by it.
|
|
46
|
+
- **Materials are never merged.** `vat.materials` lines up with `vat.geometry.groups`, giving one draw call per material. VAT collapses *instance* count, not *material* count — a 500-robot crowd with 3 materials is 3 draw calls, not 1 and not 500.
|
|
47
|
+
|
|
48
|
+
The VAT is a flat `vertexCount` × `totalFrames` texture pair, so **both** axes are bounded by the GPU's max texture dimension. The baker is renderer-agnostic and defaults to a conservative `16384`; pass the real limit whenever you have a renderer, or a bake that allocates on desktop can fail on mobile (commonly 4096–8192):
|
|
29
49
|
|
|
30
50
|
```ts
|
|
31
|
-
import {
|
|
51
|
+
import { getMaxTextureSize } from 'three-vat/webgl' // or 'three-vat/tsl'
|
|
32
52
|
|
|
33
|
-
const
|
|
53
|
+
const vat = bakeVAT(gltf.scene, clips, {
|
|
54
|
+
fps: 30,
|
|
55
|
+
maxTextureSize: getMaxTextureSize(renderer),
|
|
56
|
+
})
|
|
57
|
+
```
|
|
34
58
|
|
|
35
|
-
|
|
36
|
-
addInstancedVATAttributes(geometry, instances) // instances: { clip, timeOffset, speed }[]
|
|
37
|
-
geometry.boundingBox = vat.bounds.clone() // union of all frames — avoids culling pops
|
|
38
|
-
geometry.boundingSphere = vat.bounds.getBoundingSphere(new THREE.Sphere())
|
|
59
|
+
## Render a crowd
|
|
39
60
|
|
|
40
|
-
|
|
41
|
-
patchVATMaterial(material, vat, uniforms)
|
|
61
|
+
One call on either renderer. The import line is the only difference:
|
|
42
62
|
|
|
43
|
-
|
|
44
|
-
|
|
63
|
+
```ts
|
|
64
|
+
import { createVATMesh } from 'three-vat/webgl' // WebGLRenderer
|
|
65
|
+
// import { createVATMesh } from 'three-vat/tsl' ← WebGPURenderer: the only line that changes
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
```ts
|
|
69
|
+
// instances: { clip, timeOffset, speed }[] — one per character.
|
|
70
|
+
const { mesh, time } = createVATMesh(vat, instances)
|
|
45
71
|
mesh.castShadow = mesh.receiveShadow = true
|
|
72
|
+
scene.add(mesh)
|
|
73
|
+
|
|
74
|
+
// place the crowd
|
|
75
|
+
for (let i = 0; i < instances.length; i++) mesh.setMatrixAt(i, matrix)
|
|
76
|
+
mesh.instanceMatrix.needsUpdate = true
|
|
77
|
+
mesh.computeBoundingSphere() // or frustumCulled = false, if matrices change every frame
|
|
78
|
+
|
|
79
|
+
// per frame:
|
|
80
|
+
time.value = clock.elapsedTime
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The call clones the baked geometry, writes the per-instance playback attributes, and prepares one material per material group — so a crowd mixes clips, phases and playback rates on either renderer, from the same `instances` array ([ADR-0009](./docs/adr/0009-both-decode-paths-read-one-instance-playback-contract.md)).
|
|
84
|
+
|
|
85
|
+
Shadows are the one place the renderers genuinely differ, and the call absorbs it: on WebGL it attaches the patched depth and distance materials the shadow passes need (miss them by hand and the crowd's body animates while its shadow stays in the bind pose); on TSL it attaches nothing, because `positionNode` already feeds the depth pass. Either way you write `castShadow` and nothing else.
|
|
86
|
+
|
|
87
|
+
Instance matrices and `castShadow`/`receiveShadow` stay yours: only you know where the crowd stands and whether the scene has shadows at all.
|
|
88
|
+
|
|
89
|
+
The returned `time` is the same `{ value }` clock on both paths. The one other place the signatures differ is supplying your own: `options.time` is a `THREE.IUniform` on the WebGL path and a TSL `uniform(0)` on the TSL path, because that is what each renderer's material can read. Let the call make its own — as above — and even that line is identical.
|
|
90
|
+
|
|
91
|
+
<details>
|
|
92
|
+
<summary>By hand on WebGL, when you are not rendering onto a plain <code>InstancedMesh</code></summary>
|
|
93
|
+
|
|
94
|
+
```ts
|
|
95
|
+
import { addVATInstanceAttributes } from 'three-vat'
|
|
96
|
+
import { createVATUniforms, createVATDepthMaterial, patchVATMaterial } from 'three-vat/webgl'
|
|
97
|
+
|
|
98
|
+
const uniforms = createVATUniforms()
|
|
99
|
+
|
|
100
|
+
// vat.geometry already carries the all-frames bounding box/sphere, so instances
|
|
101
|
+
// never cull mid-animation.
|
|
102
|
+
const geometry = vat.geometry.clone()
|
|
103
|
+
// Instance playback — `{ clip, timeOffset, speed }` per instance — is a core
|
|
104
|
+
// contract both decode paths read, not a WebGL-only concept.
|
|
105
|
+
addVATInstanceAttributes(geometry, instances)
|
|
106
|
+
|
|
107
|
+
// One patched material per source material, sharing one clock.
|
|
108
|
+
const materials = vat.materials.map((source) => {
|
|
109
|
+
const material = source.clone()
|
|
110
|
+
patchVATMaterial(material, vat, uniforms)
|
|
111
|
+
return material
|
|
112
|
+
})
|
|
113
|
+
|
|
114
|
+
const mesh = new THREE.InstancedMesh(geometry, materials, instances.length)
|
|
115
|
+
// Correct instanced shadows: depth for directional/spot lights, distance for point lights.
|
|
116
|
+
mesh.customDepthMaterial = createVATDepthMaterial(vat, uniforms)
|
|
117
|
+
mesh.customDistanceMaterial = patchVATMaterial(new THREE.MeshDistanceMaterial(), vat, uniforms)
|
|
46
118
|
|
|
47
119
|
// per frame:
|
|
48
120
|
uniforms.uVatTime.value = clock.elapsedTime
|
|
49
121
|
```
|
|
50
122
|
|
|
51
|
-
|
|
123
|
+
</details>
|
|
124
|
+
|
|
125
|
+
<details>
|
|
126
|
+
<summary>By hand on TSL, and the zero-config default</summary>
|
|
52
127
|
|
|
53
128
|
```ts
|
|
54
129
|
import { MeshStandardNodeMaterial } from 'three/webgpu'
|
|
130
|
+
import { addVATInstanceAttributes } from 'three-vat'
|
|
55
131
|
import { vatNodes } from 'three-vat/tsl'
|
|
56
132
|
|
|
57
|
-
const
|
|
133
|
+
const geometry = vat.geometry.clone()
|
|
134
|
+
addVATInstanceAttributes(geometry, instances)
|
|
135
|
+
|
|
136
|
+
// The mesh is built first, because the decode is built from it.
|
|
58
137
|
const material = new MeshStandardNodeMaterial()
|
|
138
|
+
const mesh = new THREE.InstancedMesh(geometry, material, instances.length)
|
|
139
|
+
|
|
140
|
+
// `geometry`: each instance plays its own clip, phase and rate.
|
|
141
|
+
// `instancedMesh`: the decode re-applies this mesh's instancing itself, because
|
|
142
|
+
// three applies the instance matrix to `positionLocal` *before* it reads
|
|
143
|
+
// `positionNode` — so the delta has to be added in the geometry's own space and
|
|
144
|
+
// instanced afterwards. Omit it only for a single, non-instanced mesh.
|
|
145
|
+
const { positionNode, time } = vatNodes(vat, { geometry, instancedMesh: mesh })
|
|
59
146
|
material.positionNode = positionNode
|
|
60
|
-
material.normalNode = normalNode
|
|
61
147
|
|
|
62
148
|
// per frame:
|
|
63
149
|
time.value = clock.elapsedTime
|
|
64
150
|
```
|
|
65
151
|
|
|
66
|
-
|
|
152
|
+
There is deliberately no `normalNode`: the decode writes `normalLocal` from
|
|
153
|
+
inside the vertex stage, exactly as the GLSL path writes `objectNormal`, and
|
|
154
|
+
three transforms and interpolates it from there. A material's `normalNode` is
|
|
155
|
+
built in the *fragment* stage and expected in view space, which is neither where
|
|
156
|
+
nor what a per-vertex, object-space VAT normal is.
|
|
157
|
+
|
|
158
|
+
Omit `geometry` and you get the zero-config default instead: every instance plays `clipIndex`, phase-desynced by `desync` seconds hashed from `instanceIndex`, with no attributes to write.
|
|
159
|
+
|
|
160
|
+
</details>
|
|
161
|
+
|
|
162
|
+
## Bake cost, and baking in a Web Worker
|
|
163
|
+
|
|
164
|
+
A VAT is produced exactly one way — `bakeVAT` at runtime
|
|
165
|
+
([ADR-0010](./docs/adr/0010-drop-the-offline-format-runtime-bake-is-the-library.md)).
|
|
166
|
+
There is no file format to write or load, so the one cost to budget is the bake
|
|
167
|
+
itself, once at load. Measured on `three@0.185.1` (Node, Apple Silicon, mean of
|
|
168
|
+
3 runs after warm-up):
|
|
169
|
+
|
|
170
|
+
| Asset | Clips | fps | Rows | Bake |
|
|
171
|
+
|---|---|---|---|---|
|
|
172
|
+
| `RobotExpressive` (rigid, 7 214 v) | 3 (the demo) | 30 | 158 | 97 ms |
|
|
173
|
+
| `RobotExpressive` | 5 | 30 | 313 | 178 ms |
|
|
174
|
+
| `RobotExpressive` | 14 (all) | 30 | 585 | 330 ms |
|
|
175
|
+
| `RobotExpressive` | 14 (all) | 60 | 1 168 | 659 ms |
|
|
176
|
+
| `Soldier` (skinned, 7 434 v) | 4 (all) | 30 | 113 | 250 ms |
|
|
177
|
+
| `Soldier` (skinned) | 4 (all) | 60 | 224 | 492 ms |
|
|
178
|
+
|
|
179
|
+
Cost is linear in `vertices × frames`, and **a skinned vertex costs ~4× per
|
|
180
|
+
frame row what a rigid one does** (2.2 ms/row here vs 0.56) — the four-weight
|
|
181
|
+
bone blend is the hot loop. So budget by rows, and halve `fps` before you cut
|
|
182
|
+
clips. A 20k-vertex skinned character with 6 clips at 30 fps extrapolates to
|
|
183
|
+
~1.8 s on this machine and several seconds on a mid-range phone — enough to
|
|
184
|
+
matter, and the point at which the bake belongs off the main thread.
|
|
185
|
+
|
|
186
|
+
It can go there as-is: **the baker is pure CPU and never touches the renderer**,
|
|
187
|
+
so it runs in a Web Worker, with the texel buffers transferred back at no copy
|
|
188
|
+
cost. The worker loads the glTF and bakes; the main thread rebuilds the
|
|
189
|
+
textures and the geometry from plain buffers.
|
|
190
|
+
|
|
191
|
+
```ts
|
|
192
|
+
// bake.worker.ts
|
|
193
|
+
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js'
|
|
194
|
+
import { bakeVAT } from 'three-vat'
|
|
195
|
+
|
|
196
|
+
self.onmessage = async ({ data: { url, fps, maxTextureSize } }) => {
|
|
197
|
+
const gltf = await new GLTFLoader().loadAsync(url)
|
|
198
|
+
const vat = bakeVAT(gltf.scene, gltf.animations, { fps, maxTextureSize })
|
|
67
199
|
|
|
68
|
-
|
|
200
|
+
const position = vat.positionTexture.image.data as Float32Array
|
|
201
|
+
const normal = vat.normalTexture.image.data as Float32Array
|
|
202
|
+
const index = vat.geometry.getIndex()
|
|
203
|
+
|
|
204
|
+
self.postMessage(
|
|
205
|
+
{
|
|
206
|
+
position,
|
|
207
|
+
normal,
|
|
208
|
+
// Every attribute the merge produced — position, normal, and uv/color
|
|
209
|
+
// when the source had them. Carry each itemSize rather than guessing it.
|
|
210
|
+
attributes: Object.fromEntries(
|
|
211
|
+
Object.entries(vat.geometry.attributes).map(([name, a]) => [
|
|
212
|
+
name,
|
|
213
|
+
{ array: a.array, itemSize: a.itemSize },
|
|
214
|
+
]),
|
|
215
|
+
),
|
|
216
|
+
index: index?.array,
|
|
217
|
+
groups: vat.geometry.groups,
|
|
218
|
+
clips: vat.clips,
|
|
219
|
+
bounds: { min: vat.bounds.min.toArray(), max: vat.bounds.max.toArray() },
|
|
220
|
+
vertexCount: vat.vertexCount,
|
|
221
|
+
totalFrames: vat.totalFrames,
|
|
222
|
+
},
|
|
223
|
+
// Transferred, not copied — the texel buffers are the large part.
|
|
224
|
+
[position.buffer, normal.buffer],
|
|
225
|
+
)
|
|
226
|
+
}
|
|
227
|
+
```
|
|
69
228
|
|
|
70
229
|
```ts
|
|
71
|
-
|
|
230
|
+
// main thread
|
|
231
|
+
import * as THREE from 'three'
|
|
232
|
+
import { makeVATTexture } from 'three-vat'
|
|
233
|
+
import { createVATMesh, getMaxTextureSize } from 'three-vat/webgl' // or 'three-vat/tsl'
|
|
234
|
+
import type { VAT } from 'three-vat'
|
|
235
|
+
|
|
236
|
+
// `renderer` is your WebGLRenderer (or WebGPURenderer), already created — only
|
|
237
|
+
// the main thread can ask the GPU for its limits.
|
|
238
|
+
const worker = new Worker(new URL('./bake.worker.ts', import.meta.url), { type: 'module' })
|
|
239
|
+
|
|
240
|
+
function bakeInWorker(url: string, fps = 30): Promise<VAT> {
|
|
241
|
+
return new Promise((resolve) => {
|
|
242
|
+
worker.onmessage = ({ data: d }) => {
|
|
243
|
+
const geometry = new THREE.BufferGeometry()
|
|
244
|
+
for (const [name, { array, itemSize }] of Object.entries(d.attributes)) {
|
|
245
|
+
geometry.setAttribute(name, new THREE.BufferAttribute(array, itemSize))
|
|
246
|
+
}
|
|
247
|
+
if (d.index) geometry.setIndex(new THREE.BufferAttribute(d.index, 1))
|
|
248
|
+
for (const g of d.groups) geometry.addGroup(g.start, g.count, g.materialIndex)
|
|
72
249
|
|
|
73
|
-
const
|
|
74
|
-
|
|
75
|
-
|
|
250
|
+
const bounds = new THREE.Box3(
|
|
251
|
+
new THREE.Vector3(...d.bounds.min),
|
|
252
|
+
new THREE.Vector3(...d.bounds.max),
|
|
253
|
+
)
|
|
254
|
+
// The union-of-all-frames volume, or a deformed crowd culls mid-animation.
|
|
255
|
+
geometry.boundingBox = bounds.clone()
|
|
256
|
+
geometry.boundingSphere = bounds.getBoundingSphere(new THREE.Sphere())
|
|
257
|
+
|
|
258
|
+
resolve({
|
|
259
|
+
positionTexture: makeVATTexture(d.position, d.vertexCount, d.totalFrames),
|
|
260
|
+
normalTexture: makeVATTexture(d.normal, d.vertexCount, d.totalFrames),
|
|
261
|
+
geometry,
|
|
262
|
+
// One per group, in `materialIndex` order — see the note below.
|
|
263
|
+
materials: d.groups.map(() => new THREE.MeshStandardMaterial()),
|
|
264
|
+
clips: d.clips,
|
|
265
|
+
bounds,
|
|
266
|
+
vertexCount: d.vertexCount,
|
|
267
|
+
totalFrames: d.totalFrames,
|
|
268
|
+
encoding: 'delta',
|
|
269
|
+
})
|
|
270
|
+
}
|
|
271
|
+
worker.postMessage({ url, fps, maxTextureSize: getMaxTextureSize(renderer) })
|
|
272
|
+
})
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
const { mesh, time } = createVATMesh(await bakeInWorker('/robot.glb'), instances)
|
|
76
276
|
```
|
|
77
277
|
|
|
78
|
-
|
|
278
|
+
Two things do not cross the wire, both by nature rather than by omission:
|
|
279
|
+
|
|
280
|
+
- **Materials.** A `Material` holds textures and GPU state, so it has to be
|
|
281
|
+
built on the main thread. The placeholder above is a stand-in: `materials`
|
|
282
|
+
must have one entry per `geometry.groups[].materialIndex`, or every group past
|
|
283
|
+
the first renders undefined. For a glTF's real materials, load it a second
|
|
284
|
+
time on the main thread (the browser serves it from cache) and take
|
|
285
|
+
`gltf.scene`'s materials in the same order the merge recorded them.
|
|
286
|
+
- **The renderer's `maxTextureSize`**, which only the main thread can ask for —
|
|
287
|
+
read it there and pass it in, as the snippet does.
|
|
288
|
+
|
|
289
|
+
A `bakeVATInWorker` helper is deferred, for the reason in
|
|
290
|
+
[What 1.0 does not do](#what-10-does-not-do).
|
|
79
291
|
|
|
80
292
|
## Trade-offs
|
|
81
293
|
|
|
82
294
|
- **vs N × `SkinnedMesh`:** N draw calls + per-frame CPU skeletons → VAT is 1 draw call, zero per-frame CPU, 2 texel fetches per vertex. The headline.
|
|
83
295
|
- **vs bone-texture instancing:** smaller textures and supports blending, but more fetches per vertex. VAT also captures morph/non-skeletal deformation for free.
|
|
84
|
-
- **VAT limits:** no runtime IK/blending, discrete frames, memory cost (`verts × frames × 16 B × 2` textures). No clip crossfade
|
|
296
|
+
- **VAT limits:** no runtime IK/blending, discrete frames, memory cost (`verts × frames × 16 B × 2` textures). No clip crossfade — see [What 1.0 does not do](#what-10-does-not-do).
|
|
297
|
+
- **Skinned normals:** positions bake exactly under any rig. Normals reproduce what three's own skinning shader renders — linear-blend skinning transforms a normal by the skin matrix rather than its inverse-transpose, exact for rigid and uniformly-scaled bones, an approximation otherwise. Non-uniform bone scale is where that shows, so `bakeVAT` warns once, naming the bone.
|
|
298
|
+
|
|
299
|
+
## What 1.0 does not do
|
|
300
|
+
|
|
301
|
+
Named rather than left to be discovered. None of these is a known defect; each
|
|
302
|
+
is a decision, with the reasoning recorded where it was made.
|
|
303
|
+
|
|
304
|
+
- **No clip crossfade.** An instance cuts between clips, it does not blend.
|
|
305
|
+
Crossfade doubles the per-vertex texel fetches (2 → 4) and adds per-instance
|
|
306
|
+
transition state, which is not worth spending before the single-clip decode is
|
|
307
|
+
proven on both paths ([ADR-0007](./docs/adr/0007-v1-scope-library-only.md)).
|
|
308
|
+
The instance-attribute layout reserves room for a second clip index, so it
|
|
309
|
+
stays a non-breaking addition.
|
|
310
|
+
- **No LOD.** Every instance samples the VAT at full vertex count, whatever its
|
|
311
|
+
distance.
|
|
312
|
+
- **No `npx vat-bake` CLI, and no file format for it to write.** The offline
|
|
313
|
+
format was removed in 1.0 and the runtime bake is the only way to produce a
|
|
314
|
+
VAT ([ADR-0010](./docs/adr/0010-drop-the-offline-format-runtime-bake-is-the-library.md));
|
|
315
|
+
the design a CLI would build on is kept on record in the superseded
|
|
316
|
+
[ADR-0003](./docs/adr/0003-offline-format-float16-bin-plus-manifest.md), so
|
|
317
|
+
reviving it is a decision rather than a fresh design problem.
|
|
318
|
+
- **No React/drei hook or component.** A downstream contribution rather than a
|
|
319
|
+
library surface, and `createVATMesh` is what makes it thin enough to be one.
|
|
320
|
+
- **No `bakeVATInWorker` helper.** 1.0 ships the [recipe](#bake-cost-and-baking-in-a-web-worker)
|
|
321
|
+
instead: adding a second, async way to bake while the API stabilizes is the
|
|
322
|
+
two-entry-point split [ADR-0008](./docs/adr/0008-a-vat-bakes-a-posed-subtree-not-a-skinnedmesh.md)
|
|
323
|
+
refused, and demand should decide it.
|
|
324
|
+
- **glTF/GLB input only**, and multi-material meshes are rejected rather than
|
|
325
|
+
split by geometry group — `GLTFLoader` emits one mesh per primitive, so the
|
|
326
|
+
case is unreachable through the only input surface there is.
|
|
85
327
|
|
|
86
328
|
## Development
|
|
87
329
|
|
|
88
330
|
```bash
|
|
89
331
|
pnpm install
|
|
90
|
-
pnpm test
|
|
332
|
+
pnpm fetch:test-assets # Soldier.glb — too big for git, so the skinned real-asset tests skip without it
|
|
333
|
+
pnpm test # baker core — pure CPU, no GPU needed
|
|
334
|
+
pnpm test:examples # the demo's own suite (crowd layout, page/bundle shape)
|
|
91
335
|
pnpm typecheck
|
|
336
|
+
pnpm typecheck:examples
|
|
92
337
|
pnpm build
|
|
93
|
-
pnpm example
|
|
338
|
+
pnpm example # serves the demo pages in examples/ (model bundled)
|
|
339
|
+
pnpm parity # cross-path pixel-diff gate — needs a GPU and a WebGPU browser
|
|
94
340
|
```
|
|
95
341
|
|
|
342
|
+
`examples/` is a workspace package, so one `pnpm install` at the root covers
|
|
343
|
+
both it and the library. It is a multi-page app: a landing `index.html` plus one
|
|
344
|
+
page per renderer (`webgl_crowd.html`, `webgpu_crowd.html`), each self-contained
|
|
345
|
+
by design
|
|
346
|
+
([ADR-0011](./docs/adr/0011-one-example-per-renderer-duplicated-on-purpose.md)).
|
|
347
|
+
Both the build entries and the landing page's list are globbed from those HTML
|
|
348
|
+
files, so adding a demo is adding a file. `pnpm build:examples` produces the
|
|
349
|
+
static site that `.github/workflows/pages.yml` publishes from `main`; it builds
|
|
350
|
+
with a relative base, so it also runs from any subpath or a `file://` open.
|
|
351
|
+
|
|
352
|
+
The WebGPU page checks for an adapter before it loads anything else and points
|
|
353
|
+
at the WebGL demo when there is none — `WebGPURenderer` would otherwise fall
|
|
354
|
+
back to its WebGL backend and quietly draw the WebGPU demo through GLSL.
|
|
355
|
+
|
|
356
|
+
`pnpm parity` is the one check that is not in CI and not optional. It renders
|
|
357
|
+
one bake through both decode paths at the same camera, lights and time and
|
|
358
|
+
compares the frames pixel by pixel — the only thing that can catch a decode
|
|
359
|
+
subtly wrong on one path only, and the only thing that needs a real GPU on both
|
|
360
|
+
backends. It is a **required gate before publishing**, not a test; see
|
|
361
|
+
[docs/releasing.md](./docs/releasing.md) for what it checks and how to read a
|
|
362
|
+
failure.
|
|
363
|
+
|
|
364
|
+
The suite is green on a fresh clone with no network: the real-asset tests skip
|
|
365
|
+
when their asset is missing. Run `pnpm fetch:test-assets` before touching the
|
|
366
|
+
baker, so a real skinned character is actually being baked — see
|
|
367
|
+
[docs/test-assets.md](./docs/test-assets.md).
|
|
368
|
+
|
|
96
369
|
## License
|
|
97
370
|
|
|
98
371
|
MIT
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
import { InstancedBufferAttribute } from 'three';
|
|
2
|
+
|
|
3
|
+
// src/instance-playback.ts
|
|
4
|
+
var PLAYBACK_ATTRIBUTES = {
|
|
5
|
+
clipStart: "aClipStart",
|
|
6
|
+
clipFrames: "aClipFrames",
|
|
7
|
+
clipFps: "aClipFps",
|
|
8
|
+
timeOffset: "aTimeOffset",
|
|
9
|
+
speed: "aSpeed"
|
|
10
|
+
};
|
|
11
|
+
function addVATInstanceAttributes(geometry, instances) {
|
|
12
|
+
geometry.morphAttributes = {};
|
|
13
|
+
geometry.morphTargetsRelative = false;
|
|
14
|
+
const n = instances.length;
|
|
15
|
+
const clipStart = new Float32Array(n);
|
|
16
|
+
const clipFrames = new Float32Array(n);
|
|
17
|
+
const clipFps = new Float32Array(n);
|
|
18
|
+
const timeOffset = new Float32Array(n);
|
|
19
|
+
const speed = new Float32Array(n);
|
|
20
|
+
for (let i = 0; i < n; i++) {
|
|
21
|
+
const inst = instances[i];
|
|
22
|
+
clipStart[i] = inst.clip.startFrame;
|
|
23
|
+
clipFrames[i] = inst.clip.frames;
|
|
24
|
+
clipFps[i] = inst.clip.fps;
|
|
25
|
+
timeOffset[i] = inst.timeOffset;
|
|
26
|
+
speed[i] = inst.speed;
|
|
27
|
+
}
|
|
28
|
+
geometry.setAttribute(PLAYBACK_ATTRIBUTES.clipStart, new InstancedBufferAttribute(clipStart, 1));
|
|
29
|
+
geometry.setAttribute(PLAYBACK_ATTRIBUTES.clipFrames, new InstancedBufferAttribute(clipFrames, 1));
|
|
30
|
+
geometry.setAttribute(PLAYBACK_ATTRIBUTES.clipFps, new InstancedBufferAttribute(clipFps, 1));
|
|
31
|
+
geometry.setAttribute(PLAYBACK_ATTRIBUTES.timeOffset, new InstancedBufferAttribute(timeOffset, 1));
|
|
32
|
+
geometry.setAttribute(PLAYBACK_ATTRIBUTES.speed, new InstancedBufferAttribute(speed, 1));
|
|
33
|
+
}
|
|
34
|
+
function createCrowdGeometry(vat, instances) {
|
|
35
|
+
const geometry = vat.geometry.clone();
|
|
36
|
+
addVATInstanceAttributes(geometry, instances);
|
|
37
|
+
return geometry;
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
export { PLAYBACK_ATTRIBUTES, addVATInstanceAttributes, createCrowdGeometry };
|
package/dist/index.d.ts
CHANGED
|
@@ -1,19 +1,51 @@
|
|
|
1
|
-
import { Object3D,
|
|
2
|
-
import { V as VAT
|
|
1
|
+
import { Object3D, AnimationClip, TypedArray, TextureDataType, DataTexture } from 'three';
|
|
2
|
+
import { V as VAT } from './instance-playback-BrGBIKLe.js';
|
|
3
|
+
export { a as VATClip, b as VATClock, c as VATCrowd, d as VATInstance, e as addVATInstanceAttributes } from './instance-playback-BrGBIKLe.js';
|
|
3
4
|
|
|
4
|
-
/**
|
|
5
|
+
/**
|
|
6
|
+
* Conservative fallback texture-dimension cap, used when the caller does not
|
|
7
|
+
* pass `maxTextureSize`. This is the WebGL2 *spec floor for high-end desktop*,
|
|
8
|
+
* not a guarantee — plenty of mobile GPUs report 4096 or 8192. The baker is
|
|
9
|
+
* renderer-agnostic by design (it runs in Node, and in a Web Worker), so it cannot
|
|
10
|
+
* query the real limit itself: pass `getMaxTextureSize(renderer)` from
|
|
11
|
+
* `three-vat/webgl` or `three-vat/tsl` whenever a renderer exists.
|
|
12
|
+
*/
|
|
5
13
|
declare const MAX_TEXTURE_SIZE = 16384;
|
|
6
14
|
interface BakeOptions {
|
|
7
15
|
/** Sample rate in frames per second. Default `30`. */
|
|
8
16
|
fps?: number;
|
|
17
|
+
/**
|
|
18
|
+
* Largest texture dimension the target GPU accepts. Both VAT axes are checked
|
|
19
|
+
* against it: `vertexCount` (width) and `totalFrames` (height). Defaults to
|
|
20
|
+
* {@link MAX_TEXTURE_SIZE}; pass the renderer's real limit to avoid baking a
|
|
21
|
+
* VAT that allocates on your desktop and fails on a phone.
|
|
22
|
+
*/
|
|
23
|
+
maxTextureSize?: number;
|
|
9
24
|
}
|
|
10
25
|
/**
|
|
11
|
-
* Bake
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
26
|
+
* Bake `AnimationClip`s into a VAT by sampling the posed subtree frame by frame
|
|
27
|
+
* on the CPU.
|
|
28
|
+
*
|
|
29
|
+
* The unit of a bake is the whole subtree under `root`, merged into one vertex
|
|
30
|
+
* set and recorded in root space (ADR-0008) — so it handles a single
|
|
31
|
+
* `SkinnedMesh`, a morph-target mesh, a hierarchy of rigid node-animated parts
|
|
32
|
+
* (three.js `RobotExpressive`), or any mix of them, without the caller having
|
|
33
|
+
* to classify the asset. A VAT only records *where a vertex ended up*, never
|
|
34
|
+
* how it got there.
|
|
35
|
+
*
|
|
36
|
+
* Positions are stored as deltas from the merged rest pose; normals absolute.
|
|
37
|
+
* Positions are exact for skinning, morph targets, node animation and any mix
|
|
38
|
+
* of them. A normal follows the same stages its vertex does — morph targets
|
|
39
|
+
* (`morphAttributes.normal`, where the asset carries them), then the skin
|
|
40
|
+
* matrix, then the part matrix. Normals reproduce what three's own skinning
|
|
41
|
+
* shader renders: linear blend skinning transforms a normal by the skin matrix
|
|
42
|
+
* rather than its inverse-transpose, which is exact for rigid and
|
|
43
|
+
* uniformly-scaled bones and an approximation for anything else. Non-uniform bone scale is where that
|
|
44
|
+
* approximation becomes visible, so the bake warns once, naming the bone.
|
|
45
|
+
* Renderer-agnostic — touches no WebGL/WebGPU context — so it runs identically
|
|
46
|
+
* at runtime, in a Web Worker, and in Node.
|
|
15
47
|
*/
|
|
16
|
-
declare function bakeVAT(root: Object3D,
|
|
48
|
+
declare function bakeVAT(root: Object3D, clips: AnimationClip[], { fps, maxTextureSize }?: BakeOptions): VAT;
|
|
17
49
|
/**
|
|
18
50
|
* Build a VAT `DataTexture` with the fixed sampling flags every path relies on:
|
|
19
51
|
* RGBA, nearest filtering, no mipmaps. Frame interpolation is done manually in
|
|
@@ -21,46 +53,4 @@ declare function bakeVAT(root: Object3D, skinnedMesh: SkinnedMesh, clips: Animat
|
|
|
21
53
|
*/
|
|
22
54
|
declare function makeVATTexture(data: TypedArray, width: number, height: number, type?: TextureDataType): DataTexture;
|
|
23
55
|
|
|
24
|
-
type
|
|
25
|
-
/**
|
|
26
|
-
* The versioned descriptor of an offline-baked VAT. The manifest *is* the
|
|
27
|
-
* format — bump `version` on any breaking layout change.
|
|
28
|
-
*/
|
|
29
|
-
interface VATManifest {
|
|
30
|
-
version: 1;
|
|
31
|
-
vertexCount: number;
|
|
32
|
-
totalFrames: number;
|
|
33
|
-
encoding: 'delta';
|
|
34
|
-
precision: VATPrecision;
|
|
35
|
-
clips: VATClip[];
|
|
36
|
-
bounds: {
|
|
37
|
-
min: [number, number, number];
|
|
38
|
-
max: [number, number, number];
|
|
39
|
-
};
|
|
40
|
-
}
|
|
41
|
-
/** A serialized VAT: manifest plus the two raw texel buffers. */
|
|
42
|
-
interface SerializedVAT {
|
|
43
|
-
manifest: VATManifest;
|
|
44
|
-
/** Interleaved RGBA position deltas, `float16` or `float32` per `manifest.precision`. */
|
|
45
|
-
position: ArrayBuffer;
|
|
46
|
-
/** Interleaved RGBA absolute normals, same precision. */
|
|
47
|
-
normal: ArrayBuffer;
|
|
48
|
-
}
|
|
49
|
-
interface SerializeOptions {
|
|
50
|
-
/** On-disk texel precision. Default `'float16'` (half the bytes, uploads directly). */
|
|
51
|
-
precision?: VATPrecision;
|
|
52
|
-
}
|
|
53
|
-
/**
|
|
54
|
-
* Serialize a baked VAT to raw texel buffers + a versioned manifest. This is
|
|
55
|
-
* the on-disk format; the runtime object is always reconstructed with
|
|
56
|
-
* {@link loadVAT}.
|
|
57
|
-
*/
|
|
58
|
-
declare function serializeVAT(vat: VAT, { precision }?: SerializeOptions): SerializedVAT;
|
|
59
|
-
/**
|
|
60
|
-
* Reconstruct a runtime VAT from serialized buffers. `float16` uploads as
|
|
61
|
-
* `HalfFloatType`; `float32` as `FloatType`. The result is interchangeable with
|
|
62
|
-
* a {@link bakeVAT} result.
|
|
63
|
-
*/
|
|
64
|
-
declare function loadVAT(serialized: SerializedVAT): VAT;
|
|
65
|
-
|
|
66
|
-
export { type BakeOptions, MAX_TEXTURE_SIZE, type SerializeOptions, type SerializedVAT, VAT, VATClip, type VATManifest, type VATPrecision, bakeVAT, loadVAT, makeVATTexture, serializeVAT };
|
|
56
|
+
export { type BakeOptions, MAX_TEXTURE_SIZE, VAT, bakeVAT, makeVATTexture };
|