three-vat 0.1.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,10 +1,23 @@
1
1
  # three-vat
2
2
 
3
+ [![CI](https://github.com/MikeFernandez-Pro/three-vat/actions/workflows/ci.yml/badge.svg)](https://github.com/MikeFernandez-Pro/three-vat/actions/workflows/ci.yml)
4
+ [![npm version](https://img.shields.io/npm/v/three-vat.svg)](https://www.npmjs.com/package/three-vat)
5
+ [![license: MIT](https://img.shields.io/npm/l/three-vat.svg)](./LICENSE)
6
+
3
7
  Bake a glTF `AnimationClip` into GPU textures and animate **hundreds or thousands of instanced characters with zero per-frame CPU** — one draw call, no `SkinnedMesh` per character.
4
8
 
5
- VAT (Vertex Animation Texture) is battle-tested in Unity/Unreal but has been a gap on the three.js side: only scattered demos, no maintained package, nothing in drei. `three-vat` bakes the VAT **at runtime, directly from the glTF** — so any Mixamo/Sketchfab asset works with zero pipeline — with an optional offline path that produces the identical texture.
9
+ VAT (Vertex Animation Texture) is battle-tested in Unity/Unreal but has been a gap on the three.js side: only scattered demos, no maintained package, nothing in drei. `three-vat` bakes the VAT **at runtime, directly from the glTF** — so any Mixamo/Sketchfab asset works with zero pipeline, and there is exactly one way to produce a VAT.
10
+
11
+ ## Live demos
12
+
13
+ - **[WebGL crowd](https://mikefernandez-pro.github.io/three-vat/webgl_crowd.html)** — 340 robots, mixed clips, one mesh, through the GLSL decode path.
14
+ - **[WebGPU crowd](https://mikefernandez-pro.github.io/three-vat/webgpu_crowd.html)** — the same crowd through the TSL decode path, on `WebGPURenderer`.
15
+
16
+ Both pages carry a live draw-call counter and a view of the baked textures, with
17
+ cursors on the frame rows each instance is sampling. Source in
18
+ [`examples/`](./examples); they deploy from `main` on every push.
6
19
 
7
- > **Status: v1 / pre-release.** The baker core and WebGL decode are ported from a browser-validated prototype. The TSL path is present but verified visually, not yet by automated tests. See [`docs/DESIGN.md`](./docs/DESIGN.md) and [`docs/adr/`](./docs/adr) for the full rationale.
20
+ > **Status: `1.0.0`** — the npm badge above reads the registry, so it is the one to trust for what is actually published. The library is **three surfaces**, and all three work: the core baker (`three-vat`), the WebGL/GLSL decode (`three-vat/webgl`), and the WebGPU/TSL decode (`three-vat/tsl`). The two decode paths read one shared instance-playback contract and export the same `createVATMesh`, so nothing documented here is true on one renderer and false on the other. Where the renderers genuinely differ — shadow materials, the `time` clock's type, and what the TSL node builder needs to re-apply instancing — it is called out where it arises. The baker and the WebGL decode are covered by tests; the TSL path is tested structurally, because CI has no GPU, and that the two paths decode *pixel-identically* is a manual release gate ([`pnpm parity`](./docs/releasing.md)). See [`docs/DESIGN.md`](./docs/DESIGN.md) and [`docs/adr/`](./docs/adr) for the full rationale, [What 1.0 does not do](#what-10-does-not-do) for the deferred work, and [`CHANGELOG.md`](./CHANGELOG.md) for release notes.
8
21
 
9
22
  ## Install
10
23
 
@@ -19,80 +32,340 @@ npm install three-vat three
19
32
  ```ts
20
33
  import { bakeVAT } from 'three-vat'
21
34
 
22
- // gltf loaded via GLTFLoader; sourceMesh is the SkinnedMesh in it.
35
+ // gltf loaded via GLTFLoader — pass the subtree root, not a mesh.
23
36
  const clips = gltf.animations.filter((c) => c.name !== 'TPose')
24
- const vat = bakeVAT(gltf.scene, sourceMesh, clips, { fps: 30 })
25
- // vat: { positionTexture, normalTexture, clips, bounds, vertexCount, totalFrames, encoding }
37
+ const vat = bakeVAT(gltf.scene, clips, { fps: 30 })
38
+ // vat: { positionTexture, normalTexture, geometry, materials, clips, bounds, ... }
26
39
  ```
27
40
 
28
- ## Render a crowd — WebGL (`WebGLRenderer`)
41
+ The bake unit is the **whole subtree**, merged into one vertex set and recorded in root space ([ADR-0008](./docs/adr/0008-a-vat-bakes-a-posed-subtree-not-a-skinnedmesh.md)). A VAT only records *where a vertex ended up*, never how it got there, so one call handles a single `SkinnedMesh`, a morph-target mesh, a hierarchy of rigid node-animated parts (three.js `RobotExpressive`), or any mix — with no classification by the caller.
42
+
43
+ Two consequences worth knowing up front:
44
+
45
+ - **Render `vat.geometry`, not your source mesh.** The merged vertex ordering is the baker's, and the textures are indexed by it.
46
+ - **Materials are never merged.** `vat.materials` lines up with `vat.geometry.groups`, giving one draw call per material. VAT collapses *instance* count, not *material* count — a 500-robot crowd with 3 materials is 3 draw calls, not 1 and not 500.
47
+
48
+ The VAT is a flat `vertexCount` × `totalFrames` texture pair, so **both** axes are bounded by the GPU's max texture dimension. The baker is renderer-agnostic and defaults to a conservative `16384`; pass the real limit whenever you have a renderer, or a bake that allocates on desktop can fail on mobile (commonly 4096–8192):
29
49
 
30
50
  ```ts
31
- import { addInstancedVATAttributes, createVATUniforms, createVATDepthMaterial, patchVATMaterial } from 'three-vat/webgl'
51
+ import { getMaxTextureSize } from 'three-vat/webgl' // or 'three-vat/tsl'
32
52
 
33
- const uniforms = createVATUniforms()
53
+ const vat = bakeVAT(gltf.scene, clips, {
54
+ fps: 30,
55
+ maxTextureSize: getMaxTextureSize(renderer),
56
+ })
57
+ ```
34
58
 
35
- const geometry = sourceMesh.geometry.clone()
36
- addInstancedVATAttributes(geometry, instances) // instances: { clip, timeOffset, speed }[]
37
- geometry.boundingBox = vat.bounds.clone() // union of all frames — avoids culling pops
38
- geometry.boundingSphere = vat.bounds.getBoundingSphere(new THREE.Sphere())
59
+ ## Render a crowd
39
60
 
40
- const material = sourceMesh.material.clone()
41
- patchVATMaterial(material, vat, uniforms)
61
+ One call on either renderer. The import line is the only difference:
42
62
 
43
- const mesh = new THREE.InstancedMesh(geometry, material, instances.length)
44
- mesh.customDepthMaterial = createVATDepthMaterial(vat, uniforms) // correct instanced shadows
63
+ ```ts
64
+ import { createVATMesh } from 'three-vat/webgl' // WebGLRenderer
65
+ // import { createVATMesh } from 'three-vat/tsl' ← WebGPURenderer: the only line that changes
66
+ ```
67
+
68
+ ```ts
69
+ // instances: { clip, timeOffset, speed }[] — one per character.
70
+ const { mesh, time } = createVATMesh(vat, instances)
45
71
  mesh.castShadow = mesh.receiveShadow = true
72
+ scene.add(mesh)
73
+
74
+ // place the crowd
75
+ for (let i = 0; i < instances.length; i++) mesh.setMatrixAt(i, matrix)
76
+ mesh.instanceMatrix.needsUpdate = true
77
+ mesh.computeBoundingSphere() // or frustumCulled = false, if matrices change every frame
78
+
79
+ // per frame:
80
+ time.value = clock.elapsedTime
81
+ ```
82
+
83
+ The call clones the baked geometry, writes the per-instance playback attributes, and prepares one material per material group — so a crowd mixes clips, phases and playback rates on either renderer, from the same `instances` array ([ADR-0009](./docs/adr/0009-both-decode-paths-read-one-instance-playback-contract.md)).
84
+
85
+ Shadows are the one place the renderers genuinely differ, and the call absorbs it: on WebGL it attaches the patched depth and distance materials the shadow passes need (miss them by hand and the crowd's body animates while its shadow stays in the bind pose); on TSL it attaches nothing, because `positionNode` already feeds the depth pass. Either way you write `castShadow` and nothing else.
86
+
87
+ Instance matrices and `castShadow`/`receiveShadow` stay yours: only you know where the crowd stands and whether the scene has shadows at all.
88
+
89
+ The returned `time` is the same `{ value }` clock on both paths. The one other place the signatures differ is supplying your own: `options.time` is a `THREE.IUniform` on the WebGL path and a TSL `uniform(0)` on the TSL path, because that is what each renderer's material can read. Let the call make its own — as above — and even that line is identical.
90
+
91
+ <details>
92
+ <summary>By hand on WebGL, when you are not rendering onto a plain <code>InstancedMesh</code></summary>
93
+
94
+ ```ts
95
+ import { addVATInstanceAttributes } from 'three-vat'
96
+ import { createVATUniforms, createVATDepthMaterial, patchVATMaterial } from 'three-vat/webgl'
97
+
98
+ const uniforms = createVATUniforms()
99
+
100
+ // vat.geometry already carries the all-frames bounding box/sphere, so instances
101
+ // never cull mid-animation.
102
+ const geometry = vat.geometry.clone()
103
+ // Instance playback — `{ clip, timeOffset, speed }` per instance — is a core
104
+ // contract both decode paths read, not a WebGL-only concept.
105
+ addVATInstanceAttributes(geometry, instances)
106
+
107
+ // One patched material per source material, sharing one clock.
108
+ const materials = vat.materials.map((source) => {
109
+ const material = source.clone()
110
+ patchVATMaterial(material, vat, uniforms)
111
+ return material
112
+ })
113
+
114
+ const mesh = new THREE.InstancedMesh(geometry, materials, instances.length)
115
+ // Correct instanced shadows: depth for directional/spot lights, distance for point lights.
116
+ mesh.customDepthMaterial = createVATDepthMaterial(vat, uniforms)
117
+ mesh.customDistanceMaterial = patchVATMaterial(new THREE.MeshDistanceMaterial(), vat, uniforms)
46
118
 
47
119
  // per frame:
48
120
  uniforms.uVatTime.value = clock.elapsedTime
49
121
  ```
50
122
 
51
- ## Render a crowd — TSL (`WebGPURenderer`)
123
+ </details>
124
+
125
+ <details>
126
+ <summary>By hand on TSL, and the zero-config default</summary>
52
127
 
53
128
  ```ts
54
129
  import { MeshStandardNodeMaterial } from 'three/webgpu'
130
+ import { addVATInstanceAttributes } from 'three-vat'
55
131
  import { vatNodes } from 'three-vat/tsl'
56
132
 
57
- const { positionNode, normalNode, time } = vatNodes(vat, { clipIndex: 0, desync: 10 })
133
+ const geometry = vat.geometry.clone()
134
+ addVATInstanceAttributes(geometry, instances)
135
+
136
+ // The mesh is built first, because the decode is built from it.
58
137
  const material = new MeshStandardNodeMaterial()
138
+ const mesh = new THREE.InstancedMesh(geometry, material, instances.length)
139
+
140
+ // `geometry`: each instance plays its own clip, phase and rate.
141
+ // `instancedMesh`: the decode re-applies this mesh's instancing itself, because
142
+ // three applies the instance matrix to `positionLocal` *before* it reads
143
+ // `positionNode` — so the delta has to be added in the geometry's own space and
144
+ // instanced afterwards. Omit it only for a single, non-instanced mesh.
145
+ const { positionNode, time } = vatNodes(vat, { geometry, instancedMesh: mesh })
59
146
  material.positionNode = positionNode
60
- material.normalNode = normalNode
61
147
 
62
148
  // per frame:
63
149
  time.value = clock.elapsedTime
64
150
  ```
65
151
 
66
- Shadows just work on the TSL path (`positionNode` feeds the depth pass). v1 plays one clip per material with per-instance phase desync; use the WebGL path for mixed-clip crowds.
152
+ There is deliberately no `normalNode`: the decode writes `normalLocal` from
153
+ inside the vertex stage, exactly as the GLSL path writes `objectNormal`, and
154
+ three transforms and interpolates it from there. A material's `normalNode` is
155
+ built in the *fragment* stage and expected in view space, which is neither where
156
+ nor what a per-vertex, object-space VAT normal is.
157
+
158
+ Omit `geometry` and you get the zero-config default instead: every instance plays `clipIndex`, phase-desynced by `desync` seconds hashed from `instanceIndex`, with no attributes to write.
159
+
160
+ </details>
161
+
162
+ ## Bake cost, and baking in a Web Worker
163
+
164
+ A VAT is produced exactly one way — `bakeVAT` at runtime
165
+ ([ADR-0010](./docs/adr/0010-drop-the-offline-format-runtime-bake-is-the-library.md)).
166
+ There is no file format to write or load, so the one cost to budget is the bake
167
+ itself, once at load. Measured on `three@0.185.1` (Node, Apple Silicon, mean of
168
+ 3 runs after warm-up):
169
+
170
+ | Asset | Clips | fps | Rows | Bake |
171
+ |---|---|---|---|---|
172
+ | `RobotExpressive` (rigid, 7 214 v) | 3 (the demo) | 30 | 158 | 97 ms |
173
+ | `RobotExpressive` | 5 | 30 | 313 | 178 ms |
174
+ | `RobotExpressive` | 14 (all) | 30 | 585 | 330 ms |
175
+ | `RobotExpressive` | 14 (all) | 60 | 1 168 | 659 ms |
176
+ | `Soldier` (skinned, 7 434 v) | 4 (all) | 30 | 113 | 250 ms |
177
+ | `Soldier` (skinned) | 4 (all) | 60 | 224 | 492 ms |
178
+
179
+ Cost is linear in `vertices × frames`, and **a skinned vertex costs ~4× per
180
+ frame row what a rigid one does** (2.2 ms/row here vs 0.56) — the four-weight
181
+ bone blend is the hot loop. So budget by rows, and halve `fps` before you cut
182
+ clips. A 20k-vertex skinned character with 6 clips at 30 fps extrapolates to
183
+ ~1.8 s on this machine and several seconds on a mid-range phone — enough to
184
+ matter, and the point at which the bake belongs off the main thread.
185
+
186
+ It can go there as-is: **the baker is pure CPU and never touches the renderer**,
187
+ so it runs in a Web Worker, with the texel buffers transferred back at no copy
188
+ cost. The worker loads the glTF and bakes; the main thread rebuilds the
189
+ textures and the geometry from plain buffers.
190
+
191
+ ```ts
192
+ // bake.worker.ts
193
+ import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js'
194
+ import { bakeVAT } from 'three-vat'
195
+
196
+ self.onmessage = async ({ data: { url, fps, maxTextureSize } }) => {
197
+ const gltf = await new GLTFLoader().loadAsync(url)
198
+ const vat = bakeVAT(gltf.scene, gltf.animations, { fps, maxTextureSize })
67
199
 
68
- ## Offline (bake once, ship the texture)
200
+ const position = vat.positionTexture.image.data as Float32Array
201
+ const normal = vat.normalTexture.image.data as Float32Array
202
+ const index = vat.geometry.getIndex()
203
+
204
+ self.postMessage(
205
+ {
206
+ position,
207
+ normal,
208
+ // Every attribute the merge produced — position, normal, and uv/color
209
+ // when the source had them. Carry each itemSize rather than guessing it.
210
+ attributes: Object.fromEntries(
211
+ Object.entries(vat.geometry.attributes).map(([name, a]) => [
212
+ name,
213
+ { array: a.array, itemSize: a.itemSize },
214
+ ]),
215
+ ),
216
+ index: index?.array,
217
+ groups: vat.geometry.groups,
218
+ clips: vat.clips,
219
+ bounds: { min: vat.bounds.min.toArray(), max: vat.bounds.max.toArray() },
220
+ vertexCount: vat.vertexCount,
221
+ totalFrames: vat.totalFrames,
222
+ },
223
+ // Transferred, not copied — the texel buffers are the large part.
224
+ [position.buffer, normal.buffer],
225
+ )
226
+ }
227
+ ```
69
228
 
70
229
  ```ts
71
- import { serializeVAT, loadVAT } from 'three-vat'
230
+ // main thread
231
+ import * as THREE from 'three'
232
+ import { makeVATTexture } from 'three-vat'
233
+ import { createVATMesh, getMaxTextureSize } from 'three-vat/webgl' // or 'three-vat/tsl'
234
+ import type { VAT } from 'three-vat'
235
+
236
+ // `renderer` is your WebGLRenderer (or WebGPURenderer), already created — only
237
+ // the main thread can ask the GPU for its limits.
238
+ const worker = new Worker(new URL('./bake.worker.ts', import.meta.url), { type: 'module' })
239
+
240
+ function bakeInWorker(url: string, fps = 30): Promise<VAT> {
241
+ return new Promise((resolve) => {
242
+ worker.onmessage = ({ data: d }) => {
243
+ const geometry = new THREE.BufferGeometry()
244
+ for (const [name, { array, itemSize }] of Object.entries(d.attributes)) {
245
+ geometry.setAttribute(name, new THREE.BufferAttribute(array, itemSize))
246
+ }
247
+ if (d.index) geometry.setIndex(new THREE.BufferAttribute(d.index, 1))
248
+ for (const g of d.groups) geometry.addGroup(g.start, g.count, g.materialIndex)
72
249
 
73
- const { manifest, position, normal } = serializeVAT(vat, { precision: 'float16' })
74
- // write manifest as JSON, position/normal as .bin — then later:
75
- const vat = loadVAT({ manifest, position, normal })
250
+ const bounds = new THREE.Box3(
251
+ new THREE.Vector3(...d.bounds.min),
252
+ new THREE.Vector3(...d.bounds.max),
253
+ )
254
+ // The union-of-all-frames volume, or a deformed crowd culls mid-animation.
255
+ geometry.boundingBox = bounds.clone()
256
+ geometry.boundingSphere = bounds.getBoundingSphere(new THREE.Sphere())
257
+
258
+ resolve({
259
+ positionTexture: makeVATTexture(d.position, d.vertexCount, d.totalFrames),
260
+ normalTexture: makeVATTexture(d.normal, d.vertexCount, d.totalFrames),
261
+ geometry,
262
+ // One per group, in `materialIndex` order — see the note below.
263
+ materials: d.groups.map(() => new THREE.MeshStandardMaterial()),
264
+ clips: d.clips,
265
+ bounds,
266
+ vertexCount: d.vertexCount,
267
+ totalFrames: d.totalFrames,
268
+ encoding: 'delta',
269
+ })
270
+ }
271
+ worker.postMessage({ url, fps, maxTextureSize: getMaxTextureSize(renderer) })
272
+ })
273
+ }
274
+
275
+ const { mesh, time } = createVATMesh(await bakeInWorker('/robot.glb'), instances)
76
276
  ```
77
277
 
78
- The canonical format is a raw Float16 `.bin` + versioned JSON manifest (KTX2 rejected as default — its GPU compression doesn't apply to float data). The loaded VAT is interchangeable with a freshly-baked one.
278
+ Two things do not cross the wire, both by nature rather than by omission:
279
+
280
+ - **Materials.** A `Material` holds textures and GPU state, so it has to be
281
+ built on the main thread. The placeholder above is a stand-in: `materials`
282
+ must have one entry per `geometry.groups[].materialIndex`, or every group past
283
+ the first renders undefined. For a glTF's real materials, load it a second
284
+ time on the main thread (the browser serves it from cache) and take
285
+ `gltf.scene`'s materials in the same order the merge recorded them.
286
+ - **The renderer's `maxTextureSize`**, which only the main thread can ask for —
287
+ read it there and pass it in, as the snippet does.
288
+
289
+ A `bakeVATInWorker` helper is deferred, for the reason in
290
+ [What 1.0 does not do](#what-10-does-not-do).
79
291
 
80
292
  ## Trade-offs
81
293
 
82
294
  - **vs N × `SkinnedMesh`:** N draw calls + per-frame CPU skeletons → VAT is 1 draw call, zero per-frame CPU, 2 texel fetches per vertex. The headline.
83
295
  - **vs bone-texture instancing:** smaller textures and supports blending, but more fetches per vertex. VAT also captures morph/non-skeletal deformation for free.
84
- - **VAT limits:** no runtime IK/blending, discrete frames, memory cost (`verts × frames × 16 B × 2` textures). No clip crossfade in v1.
296
+ - **VAT limits:** no runtime IK/blending, discrete frames, memory cost (`verts × frames × 16 B × 2` textures). No clip crossfade — see [What 1.0 does not do](#what-10-does-not-do).
297
+ - **Skinned normals:** positions bake exactly under any rig. Normals reproduce what three's own skinning shader renders — linear-blend skinning transforms a normal by the skin matrix rather than its inverse-transpose, exact for rigid and uniformly-scaled bones, an approximation otherwise. Non-uniform bone scale is where that shows, so `bakeVAT` warns once, naming the bone.
298
+
299
+ ## What 1.0 does not do
300
+
301
+ Named rather than left to be discovered. None of these is a known defect; each
302
+ is a decision, with the reasoning recorded where it was made.
303
+
304
+ - **No clip crossfade.** An instance cuts between clips, it does not blend.
305
+ Crossfade doubles the per-vertex texel fetches (2 → 4) and adds per-instance
306
+ transition state, which is not worth spending before the single-clip decode is
307
+ proven on both paths ([ADR-0007](./docs/adr/0007-v1-scope-library-only.md)).
308
+ The instance-attribute layout reserves room for a second clip index, so it
309
+ stays a non-breaking addition.
310
+ - **No LOD.** Every instance samples the VAT at full vertex count, whatever its
311
+ distance.
312
+ - **No `npx vat-bake` CLI, and no file format for it to write.** The offline
313
+ format was removed in 1.0 and the runtime bake is the only way to produce a
314
+ VAT ([ADR-0010](./docs/adr/0010-drop-the-offline-format-runtime-bake-is-the-library.md));
315
+ the design a CLI would build on is kept on record in the superseded
316
+ [ADR-0003](./docs/adr/0003-offline-format-float16-bin-plus-manifest.md), so
317
+ reviving it is a decision rather than a fresh design problem.
318
+ - **No React/drei hook or component.** A downstream contribution rather than a
319
+ library surface, and `createVATMesh` is what makes it thin enough to be one.
320
+ - **No `bakeVATInWorker` helper.** 1.0 ships the [recipe](#bake-cost-and-baking-in-a-web-worker)
321
+ instead: adding a second, async way to bake while the API stabilizes is the
322
+ two-entry-point split [ADR-0008](./docs/adr/0008-a-vat-bakes-a-posed-subtree-not-a-skinnedmesh.md)
323
+ refused, and demand should decide it.
324
+ - **glTF/GLB input only**, and multi-material meshes are rejected rather than
325
+ split by geometry group — `GLTFLoader` emits one mesh per primitive, so the
326
+ case is unreachable through the only input surface there is.
85
327
 
86
328
  ## Development
87
329
 
88
330
  ```bash
89
331
  pnpm install
90
- pnpm test # baker core — pure CPU, no GPU needed
332
+ pnpm fetch:test-assets # Soldier.glb — too big for git, so the skinned real-asset tests skip without it
333
+ pnpm test # baker core — pure CPU, no GPU needed
334
+ pnpm test:examples # the demo's own suite (crowd layout, page/bundle shape)
91
335
  pnpm typecheck
336
+ pnpm typecheck:examples
92
337
  pnpm build
93
- pnpm example # runs the crowd demo in examples/ (needs a Soldier.glb)
338
+ pnpm example # serves the demo pages in examples/ (model bundled)
339
+ pnpm parity # cross-path pixel-diff gate — needs a GPU and a WebGPU browser
94
340
  ```
95
341
 
342
+ `examples/` is a workspace package, so one `pnpm install` at the root covers
343
+ both it and the library. It is a multi-page app: a landing `index.html` plus one
344
+ page per renderer (`webgl_crowd.html`, `webgpu_crowd.html`), each self-contained
345
+ by design
346
+ ([ADR-0011](./docs/adr/0011-one-example-per-renderer-duplicated-on-purpose.md)).
347
+ Both the build entries and the landing page's list are globbed from those HTML
348
+ files, so adding a demo is adding a file. `pnpm build:examples` produces the
349
+ static site that `.github/workflows/pages.yml` publishes from `main`; it builds
350
+ with a relative base, so it also runs from any subpath or a `file://` open.
351
+
352
+ The WebGPU page checks for an adapter before it loads anything else and points
353
+ at the WebGL demo when there is none — `WebGPURenderer` would otherwise fall
354
+ back to its WebGL backend and quietly draw the WebGPU demo through GLSL.
355
+
356
+ `pnpm parity` is the one check that is not in CI and not optional. It renders
357
+ one bake through both decode paths at the same camera, lights and time and
358
+ compares the frames pixel by pixel — the only thing that can catch a decode
359
+ subtly wrong on one path only, and the only thing that needs a real GPU on both
360
+ backends. It is a **required gate before publishing**, not a test; see
361
+ [docs/releasing.md](./docs/releasing.md) for what it checks and how to read a
362
+ failure.
363
+
364
+ The suite is green on a fresh clone with no network: the real-asset tests skip
365
+ when their asset is missing. Run `pnpm fetch:test-assets` before touching the
366
+ baker, so a real skinned character is actually being baked — see
367
+ [docs/test-assets.md](./docs/test-assets.md).
368
+
96
369
  ## License
97
370
 
98
371
  MIT
@@ -0,0 +1,40 @@
1
+ import { InstancedBufferAttribute } from 'three';
2
+
3
+ // src/instance-playback.ts
4
+ var PLAYBACK_ATTRIBUTES = {
5
+ clipStart: "aClipStart",
6
+ clipFrames: "aClipFrames",
7
+ clipFps: "aClipFps",
8
+ timeOffset: "aTimeOffset",
9
+ speed: "aSpeed"
10
+ };
11
+ function addVATInstanceAttributes(geometry, instances) {
12
+ geometry.morphAttributes = {};
13
+ geometry.morphTargetsRelative = false;
14
+ const n = instances.length;
15
+ const clipStart = new Float32Array(n);
16
+ const clipFrames = new Float32Array(n);
17
+ const clipFps = new Float32Array(n);
18
+ const timeOffset = new Float32Array(n);
19
+ const speed = new Float32Array(n);
20
+ for (let i = 0; i < n; i++) {
21
+ const inst = instances[i];
22
+ clipStart[i] = inst.clip.startFrame;
23
+ clipFrames[i] = inst.clip.frames;
24
+ clipFps[i] = inst.clip.fps;
25
+ timeOffset[i] = inst.timeOffset;
26
+ speed[i] = inst.speed;
27
+ }
28
+ geometry.setAttribute(PLAYBACK_ATTRIBUTES.clipStart, new InstancedBufferAttribute(clipStart, 1));
29
+ geometry.setAttribute(PLAYBACK_ATTRIBUTES.clipFrames, new InstancedBufferAttribute(clipFrames, 1));
30
+ geometry.setAttribute(PLAYBACK_ATTRIBUTES.clipFps, new InstancedBufferAttribute(clipFps, 1));
31
+ geometry.setAttribute(PLAYBACK_ATTRIBUTES.timeOffset, new InstancedBufferAttribute(timeOffset, 1));
32
+ geometry.setAttribute(PLAYBACK_ATTRIBUTES.speed, new InstancedBufferAttribute(speed, 1));
33
+ }
34
+ function createCrowdGeometry(vat, instances) {
35
+ const geometry = vat.geometry.clone();
36
+ addVATInstanceAttributes(geometry, instances);
37
+ return geometry;
38
+ }
39
+
40
+ export { PLAYBACK_ATTRIBUTES, addVATInstanceAttributes, createCrowdGeometry };
package/dist/index.d.ts CHANGED
@@ -1,19 +1,51 @@
1
- import { Object3D, SkinnedMesh, AnimationClip, TypedArray, TextureDataType, DataTexture } from 'three';
2
- import { V as VAT, a as VATClip } from './types-DVtQvVjk.js';
1
+ import { Object3D, AnimationClip, TypedArray, TextureDataType, DataTexture } from 'three';
2
+ import { V as VAT } from './instance-playback-BrGBIKLe.js';
3
+ export { a as VATClip, b as VATClock, c as VATCrowd, d as VATInstance, e as addVATInstanceAttributes } from './instance-playback-BrGBIKLe.js';
3
4
 
4
- /** Maximum texture dimension we support; `vertexCount` must not exceed it. */
5
+ /**
6
+ * Conservative fallback texture-dimension cap, used when the caller does not
7
+ * pass `maxTextureSize`. This is the WebGL2 *spec floor for high-end desktop*,
8
+ * not a guarantee — plenty of mobile GPUs report 4096 or 8192. The baker is
9
+ * renderer-agnostic by design (it runs in Node, and in a Web Worker), so it cannot
10
+ * query the real limit itself: pass `getMaxTextureSize(renderer)` from
11
+ * `three-vat/webgl` or `three-vat/tsl` whenever a renderer exists.
12
+ */
5
13
  declare const MAX_TEXTURE_SIZE = 16384;
6
14
  interface BakeOptions {
7
15
  /** Sample rate in frames per second. Default `30`. */
8
16
  fps?: number;
17
+ /**
18
+ * Largest texture dimension the target GPU accepts. Both VAT axes are checked
19
+ * against it: `vertexCount` (width) and `totalFrames` (height). Defaults to
20
+ * {@link MAX_TEXTURE_SIZE}; pass the renderer's real limit to avoid baking a
21
+ * VAT that allocates on your desktop and fails on a phone.
22
+ */
23
+ maxTextureSize?: number;
9
24
  }
10
25
  /**
11
- * Bake skinned `AnimationClip`s into a VAT by sampling the posed mesh frame by
12
- * frame on the CPU. Positions are stored as deltas from the bind pose; normals
13
- * are stored absolute. Renderer-agnostic — touches no WebGL/WebGPU context — so
14
- * it runs identically at runtime and offline in Node.
26
+ * Bake `AnimationClip`s into a VAT by sampling the posed subtree frame by frame
27
+ * on the CPU.
28
+ *
29
+ * The unit of a bake is the whole subtree under `root`, merged into one vertex
30
+ * set and recorded in root space (ADR-0008) — so it handles a single
31
+ * `SkinnedMesh`, a morph-target mesh, a hierarchy of rigid node-animated parts
32
+ * (three.js `RobotExpressive`), or any mix of them, without the caller having
33
+ * to classify the asset. A VAT only records *where a vertex ended up*, never
34
+ * how it got there.
35
+ *
36
+ * Positions are stored as deltas from the merged rest pose; normals absolute.
37
+ * Positions are exact for skinning, morph targets, node animation and any mix
38
+ * of them. A normal follows the same stages its vertex does — morph targets
39
+ * (`morphAttributes.normal`, where the asset carries them), then the skin
40
+ * matrix, then the part matrix. Normals reproduce what three's own skinning
41
+ * shader renders: linear blend skinning transforms a normal by the skin matrix
42
+ * rather than its inverse-transpose, which is exact for rigid and
43
+ * uniformly-scaled bones and an approximation for anything else. Non-uniform bone scale is where that
44
+ * approximation becomes visible, so the bake warns once, naming the bone.
45
+ * Renderer-agnostic — touches no WebGL/WebGPU context — so it runs identically
46
+ * at runtime, in a Web Worker, and in Node.
15
47
  */
16
- declare function bakeVAT(root: Object3D, skinnedMesh: SkinnedMesh, clips: AnimationClip[], { fps }?: BakeOptions): VAT;
48
+ declare function bakeVAT(root: Object3D, clips: AnimationClip[], { fps, maxTextureSize }?: BakeOptions): VAT;
17
49
  /**
18
50
  * Build a VAT `DataTexture` with the fixed sampling flags every path relies on:
19
51
  * RGBA, nearest filtering, no mipmaps. Frame interpolation is done manually in
@@ -21,46 +53,4 @@ declare function bakeVAT(root: Object3D, skinnedMesh: SkinnedMesh, clips: Animat
21
53
  */
22
54
  declare function makeVATTexture(data: TypedArray, width: number, height: number, type?: TextureDataType): DataTexture;
23
55
 
24
- type VATPrecision = 'float16' | 'float32';
25
- /**
26
- * The versioned descriptor of an offline-baked VAT. The manifest *is* the
27
- * format — bump `version` on any breaking layout change.
28
- */
29
- interface VATManifest {
30
- version: 1;
31
- vertexCount: number;
32
- totalFrames: number;
33
- encoding: 'delta';
34
- precision: VATPrecision;
35
- clips: VATClip[];
36
- bounds: {
37
- min: [number, number, number];
38
- max: [number, number, number];
39
- };
40
- }
41
- /** A serialized VAT: manifest plus the two raw texel buffers. */
42
- interface SerializedVAT {
43
- manifest: VATManifest;
44
- /** Interleaved RGBA position deltas, `float16` or `float32` per `manifest.precision`. */
45
- position: ArrayBuffer;
46
- /** Interleaved RGBA absolute normals, same precision. */
47
- normal: ArrayBuffer;
48
- }
49
- interface SerializeOptions {
50
- /** On-disk texel precision. Default `'float16'` (half the bytes, uploads directly). */
51
- precision?: VATPrecision;
52
- }
53
- /**
54
- * Serialize a baked VAT to raw texel buffers + a versioned manifest. This is
55
- * the on-disk format; the runtime object is always reconstructed with
56
- * {@link loadVAT}.
57
- */
58
- declare function serializeVAT(vat: VAT, { precision }?: SerializeOptions): SerializedVAT;
59
- /**
60
- * Reconstruct a runtime VAT from serialized buffers. `float16` uploads as
61
- * `HalfFloatType`; `float32` as `FloatType`. The result is interchangeable with
62
- * a {@link bakeVAT} result.
63
- */
64
- declare function loadVAT(serialized: SerializedVAT): VAT;
65
-
66
- export { type BakeOptions, MAX_TEXTURE_SIZE, type SerializeOptions, type SerializedVAT, VAT, VATClip, type VATManifest, type VATPrecision, bakeVAT, loadVAT, makeVATTexture, serializeVAT };
56
+ export { type BakeOptions, MAX_TEXTURE_SIZE, VAT, bakeVAT, makeVATTexture };