@lovelace_lol/embody 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +2411 -0
- package/dist/authoring/boneAuthoring.d.ts +132 -0
- package/dist/camera/annotationCameraAngles.d.ts +28 -0
- package/dist/camera/cameraRelativeGaze.d.ts +15 -0
- package/dist/camera/rustAnnotationCameraCore.d.ts +136 -0
- package/dist/cljs.cjs +362 -0
- package/dist/cljs.cjs.map +1 -0
- package/dist/cljs.d.ts +5 -0
- package/dist/cljs.js +359 -0
- package/dist/cljs.js.map +1 -0
- package/dist/core/TsClipCompiler.d.ts +51 -0
- package/dist/core/TsRuntimeCore.d.ts +40 -0
- package/dist/core/WasmRuntimeCore.d.ts +55 -0
- package/dist/core/compositeAxis.d.ts +4 -0
- package/dist/core/contracts/abi.d.ts +70 -0
- package/dist/core/contracts/clip.d.ts +44 -0
- package/dist/core/contracts/frame.d.ts +27 -0
- package/dist/core/contracts/host.d.ts +28 -0
- package/dist/core/contracts/ids.d.ts +15 -0
- package/dist/core/contracts/index.d.ts +10 -0
- package/dist/core/contracts/model.d.ts +31 -0
- package/dist/core/contracts/primitives.d.ts +17 -0
- package/dist/core/contracts/skeletonFit.d.ts +41 -0
- package/dist/core/index.d.ts +8 -0
- package/dist/core/types.d.ts +472 -0
- package/dist/core.cjs +1065 -0
- package/dist/core.cjs.map +1 -0
- package/dist/core.js +1040 -0
- package/dist/core.js.map +1 -0
- package/dist/engines/three/Embody.d.ts +373 -0
- package/dist/engines/three/ThreeAnimationRuntime.d.ts +203 -0
- package/dist/engines/three/ThreeClipAdapter.d.ts +24 -0
- package/dist/engines/three/ThreeFrameApplier.d.ts +68 -0
- package/dist/engines/three/ThreeModelInspector.d.ts +54 -0
- package/dist/engines/three/bakedClipPartitioning.d.ts +16 -0
- package/dist/engines/three/balanceUtils.d.ts +3 -0
- package/dist/engines/three/characterHost.d.ts +80 -0
- package/dist/engines/three/characterScene.d.ts +98 -0
- package/dist/engines/three/hair/HairPhysicsController.d.ts +124 -0
- package/dist/engines/three/index.d.ts +20 -0
- package/dist/engines/three/modelLoader.d.ts +61 -0
- package/dist/engines/three/profileTestScene.d.ts +52 -0
- package/dist/engines/three/types.d.ts +20 -0
- package/dist/hair/appearance.d.ts +29 -0
- package/dist/index.cjs +15431 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.ts +100 -0
- package/dist/index.js +15141 -0
- package/dist/index.js.map +1 -0
- package/dist/interfaces/Animation.d.ts +315 -0
- package/dist/interfaces/EmbodyRuntime.d.ts +90 -0
- package/dist/interfaces/Hair.d.ts +181 -0
- package/dist/interfaces/HairPhysics.d.ts +90 -0
- package/dist/interfaces/index.d.ts +11 -0
- package/dist/mappings/extendPresetWithProfile.d.ts +17 -0
- package/dist/mappings/types.d.ts +309 -0
- package/dist/mappings/visemeSystem.d.ts +26 -0
- package/dist/physics/HairPhysics.d.ts +86 -0
- package/dist/physics/RustHairPhysics.d.ts +18 -0
- package/dist/physics/index.d.ts +6 -0
- package/dist/preset.cjs +2000 -0
- package/dist/preset.cjs.map +1 -0
- package/dist/preset.js +1997 -0
- package/dist/preset.js.map +1 -0
- package/dist/presets/bettaFish.d.ts +328 -0
- package/dist/presets/cc4.d.ts +145 -0
- package/dist/presets/index.d.ts +27 -0
- package/dist/profiles/resolveProfileConfig.d.ts +36 -0
- package/dist/profiles/types.d.ts +118 -0
- package/dist/regions/normalizeRegionTree.d.ts +6 -0
- package/dist/regions/regionMapping.d.ts +17 -0
- package/dist/regions/types.d.ts +122 -0
- package/dist/skeletonTemplates/humanoidSkeletonTemplates.d.ts +32 -0
- package/dist/test/setupWasm.d.ts +1 -0
- package/dist/three.cjs +10115 -0
- package/dist/three.cjs.map +1 -0
- package/dist/three.js +10086 -0
- package/dist/three.js.map +1 -0
- package/dist/validation/analyzeModel.d.ts +68 -0
- package/dist/validation/extractModelData.d.ts +90 -0
- package/dist/validation/generateMappingCorrections.d.ts +32 -0
- package/dist/validation/geometryHelpers.d.ts +66 -0
- package/dist/validation/types.d.ts +10 -0
- package/dist/validation/validateMappings.d.ts +77 -0
- package/dist/wasm/LICENSE +21 -0
- package/dist/wasm/README.md +72 -0
- package/dist/wasm/embody_wasm.d.ts +531 -0
- package/dist/wasm/embody_wasm.js +1635 -0
- package/dist/wasm/embody_wasm_bg.wasm +0 -0
- package/dist/wasm/embody_wasm_bg.wasm.d.ts +107 -0
- package/dist/wasm.cjs +302 -0
- package/dist/wasm.cjs.map +1 -0
- package/dist/wasm.d.ts +43 -0
- package/dist/wasm.js +268 -0
- package/dist/wasm.js.map +1 -0
- package/dist/wasmTypes.d.ts +121 -0
- package/package.json +99 -0
package/README.md
ADDED
|
@@ -0,0 +1,2411 @@
|
|
|
1
|
+
# Embody
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
The missing character controller for Three.js! Embody allows you to bring humanoid and animal characters to life. Embody is based on the Facial Action Coding System (FACS) as the basis of its mappings, providing a morph and bone mapping library for controlling high-definition 3D characters in Three.js.
|
|
5
|
+
|
|
6
|
+
Embody provides mappings that connect [Facial Action Coding System (FACS)](https://en.wikipedia.org/wiki/Facial_Action_Coding_System) Action Units to the morph targets and bone transforms found in 3d character assets. Instead of manually figuring out which blend shapes correspond to which facial movements, you can simply say `setAU(12, 0.8)` and the library handles the rest.
|
|
7
|
+
|
|
8
|
+
> **Note:** If you previously used the `loomlarge` npm package, it has been renamed to `@lovelace_lol/embody`.
|
|
9
|
+
>
|
|
10
|
+
> Continuous PR releases are published via [pkg.pr.new](https://pkg.pr.new/). Stable npm publishing is deferred until we have a release candidate; until then consumers should use git SHA pins or pkg.pr.new preview URLs. Install the [pkg-pr-new GitHub App](https://github.com/apps/pkg-pr-new) on this repository so PR workflows can publish previews.
|
|
11
|
+
|
|
12
|
+

|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## What Embody Covers
|
|
17
|
+
|
|
18
|
+
Embody is broader than a face-controller wrapper. The library spans four practical areas:
|
|
19
|
+
- Runtime control: Action Units, visemes, direct morphs, continuum pairs, composite rotations, transitions, and mixer playback.
|
|
20
|
+
- Rig configuration: built-in presets, profile overrides, preset lookup and extension, name resolution, viseme routing, mix weights, and skeletal-only preset support.
|
|
21
|
+
- Inspection and validation: mesh, morph, and bone discovery; preset-fit checks; correction suggestions; and full model analysis.
|
|
22
|
+
- Runtime tooling: mesh/material debugging, mixer animation clip helpers, hair physics, and region/geometry helpers for annotation or camera tooling.
|
|
23
|
+
|
|
24
|
+
## Reading Paths
|
|
25
|
+
|
|
26
|
+
Use the README in one of these paths:
|
|
27
|
+
- First successful character: [Installation & Setup](#1-installation--setup) -> [Using Presets](#2-using-presets) -> [Preset Selection & Validation](#3-preset-selection--validation) -> [Getting to Know Your Character](#4-getting-to-know-your-character) -> [Action Unit Control](#7-action-unit-control) -> [Viseme System](#12-viseme-system) -> [Transition System](#13-transition-system) -> [Baked Animations](#16-baked-animations)
|
|
28
|
+
- Retargeting an existing rig: [Using Presets](#2-using-presets) -> [Preset Selection & Validation](#3-preset-selection--validation) -> [Getting to Know Your Character](#4-getting-to-know-your-character) -> [Extending & Custom Presets](#5-extending--custom-presets)
|
|
29
|
+
- Skeletal-only character: [Creating Skeletal Animation Presets](#6-creating-skeletal-animation-presets) -> [Baked Animations](#16-baked-animations) -> [Regions & Geometry Helpers](#17-regions--geometry-helpers)
|
|
30
|
+
- Annotation or camera tooling: [Preset Selection & Validation](#3-preset-selection--validation) -> [Getting to Know Your Character](#4-getting-to-know-your-character) -> [Regions & Geometry Helpers](#17-regions--geometry-helpers)
|
|
31
|
+
|
|
32
|
+
## Demo Site Links
|
|
33
|
+
|
|
34
|
+
These demo site links open the LoomLarge drawer on the matching tab. The demo site currently supports stable `drawer` + `tab` deep links, so the README should lean on tab-specific links instead of pretending it can deep-link to a fully reconstructed authoring state.
|
|
35
|
+
|
|
36
|
+
| Goal | Open in LoomLarge |
|
|
37
|
+
|------|-------------------|
|
|
38
|
+
| Start with the main runtime surface | [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation) |
|
|
39
|
+
| Inspect preset and profile settings | [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) |
|
|
40
|
+
| Inspect AU, morph, and bone routing | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings) |
|
|
41
|
+
| Inspect meshes and material state | [Meshes tab](https://www.characterloom.com/?drawer=open&tab=meshes) |
|
|
42
|
+
| Inspect resolved bones | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) |
|
|
43
|
+
| Tune expressions and continuum pairs | [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units) |
|
|
44
|
+
| Inspect lip-sync views | [Visemes tab](https://www.characterloom.com/?drawer=open&tab=visemes) and [Speech tab](https://www.characterloom.com/?drawer=open&tab=speech) |
|
|
45
|
+
| Tune hair behavior | [Hair tab](https://www.characterloom.com/?drawer=open&tab=hair) |
|
|
46
|
+
|
|
47
|
+
Most screenshots below were captured from LoomLarge with the matching tab open so the docs and the live product are easy to compare. The viseme grid image is the main exception: it still shows older labels from the captured UI, so the viseme table later in the README should be treated as the source of truth.
|
|
48
|
+
|
|
49
|
+
## Table of Contents
|
|
50
|
+
|
|
51
|
+
### Foundations
|
|
52
|
+
|
|
53
|
+
1. [Installation & Setup](#1-installation--setup)
|
|
54
|
+
2. [Using Presets](#2-using-presets)
|
|
55
|
+
3. [Preset Selection & Validation](#3-preset-selection--validation)
|
|
56
|
+
4. [Getting to Know Your Character](#4-getting-to-know-your-character)
|
|
57
|
+
|
|
58
|
+
### Advanced Authoring
|
|
59
|
+
|
|
60
|
+
5. [Extending & Custom Presets](#5-extending--custom-presets)
|
|
61
|
+
6. [Creating Skeletal Animation Presets](#6-creating-skeletal-animation-presets)
|
|
62
|
+
|
|
63
|
+
### Runtime Control
|
|
64
|
+
|
|
65
|
+
7. [Action Unit Control](#7-action-unit-control)
|
|
66
|
+
8. [Mix Weight System](#8-mix-weight-system)
|
|
67
|
+
9. [Composite Rotation System](#9-composite-rotation-system)
|
|
68
|
+
10. [Continuum Pairs](#10-continuum-pairs)
|
|
69
|
+
11. [Direct Morph Control](#11-direct-morph-control)
|
|
70
|
+
12. [Viseme System](#12-viseme-system)
|
|
71
|
+
13. [Transition System](#13-transition-system)
|
|
72
|
+
14. [Playback & State Control](#14-playback--state-control)
|
|
73
|
+
15. [Hair Physics](#15-hair-physics)
|
|
74
|
+
16. [Baked Animations](#16-baked-animations)
|
|
75
|
+
|
|
76
|
+
### Tooling & Reference
|
|
77
|
+
|
|
78
|
+
17. [Regions & Geometry Helpers](#17-regions--geometry-helpers)
|
|
79
|
+
18. [API Reference](#18-api-reference)
|
|
80
|
+
|
|
81
|
+
Additional:
|
|
82
|
+
- [Resources](#resources)
|
|
83
|
+
- [License](#license)
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## 1. Installation & Setup
|
|
88
|
+
|
|
89
|
+
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
90
|
+
|
|
91
|
+

|
|
92
|
+
|
|
93
|
+
### Install the package
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
npm install @lovelace_lol/embody
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### Peer dependency
|
|
100
|
+
|
|
101
|
+
Embody requires Three.js as a peer dependency:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
npm install three
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### Rust/Wasm core
|
|
108
|
+
|
|
109
|
+
Embody ships a generated Rust/Wasm core artifact in the npm package. The Three.js
|
|
110
|
+
controller initializes the parts it needs through the package facade, and direct
|
|
111
|
+
core consumers can explicitly initialize the Wasm entrypoint:
|
|
112
|
+
|
|
113
|
+
```typescript
|
|
114
|
+
import { initEmbodyCore } from '@lovelace_lol/embody/wasm';
|
|
115
|
+
|
|
116
|
+
const core = await initEmbodyCore();
|
|
117
|
+
const [left, right] = core.solve_bilateral_values(0.8, 0.25);
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
The Wasm preset registry embeds `cc4` and `fish`. `default` and `human` select
|
|
121
|
+
CC4; `skeletal` selects fish. Hosts can check `core.has_preset(id)` before
|
|
122
|
+
calling `configure_with_preset(id, overrideJson, modelJson)`.
|
|
123
|
+
|
|
124
|
+
Complete custom profiles use
|
|
125
|
+
`configure_exact_profile(profileJson, modelJson)`. Exact mode never merges an
|
|
126
|
+
embedded preset, and it rejects profiles that contain no runtime mappings or
|
|
127
|
+
resolve no bindings against the inspected model. Selecting preset fallback for
|
|
128
|
+
an incomplete legacy profile remains an explicit host decision.
|
|
129
|
+
|
|
130
|
+
Preset/profile JSON crosses the Wasm boundary only during configuration. Rust
|
|
131
|
+
merges overrides, deserializes the runtime projection, compiles model bindings,
|
|
132
|
+
and returns packed numeric frame deltas on the hot path. Three.js object
|
|
133
|
+
inspection and frame application remain host-side concerns.
|
|
134
|
+
|
|
135
|
+
The package exports the Three.js inspector, frame applier, and clip adapter.
|
|
136
|
+
Character host orchestration belongs in Polymer, so Embody no longer exports a
|
|
137
|
+
separate TypeScript `RustEmbodyHost`.
|
|
138
|
+
|
|
139
|
+
### Template skeleton fit metadata
|
|
140
|
+
|
|
141
|
+
Template skeleton fitting metadata is a host-neutral contract for recording how a
|
|
142
|
+
template skeleton should be previewed against a source character. It is metadata
|
|
143
|
+
for placement and QA only. It is not skinning data, bone weights, a rig
|
|
144
|
+
definition, or permission for the host to automatically bind the source mesh to
|
|
145
|
+
the template skeleton.
|
|
146
|
+
|
|
147
|
+
```typescript
|
|
148
|
+
import {
|
|
149
|
+
TEMPLATE_SKELETON_FIT_METADATA_KIND,
|
|
150
|
+
TEMPLATE_SKELETON_FIT_METADATA_VERSION,
|
|
151
|
+
composeTemplateSkeletonFitTransform,
|
|
152
|
+
validateTemplateSkeletonFitMetadata,
|
|
153
|
+
type TemplateSkeletonFitMetadata,
|
|
154
|
+
} from '@lovelace_lol/embody/core/contracts';
|
|
155
|
+
|
|
156
|
+
const fitMetadata: TemplateSkeletonFitMetadata = {
|
|
157
|
+
kind: TEMPLATE_SKELETON_FIT_METADATA_KIND,
|
|
158
|
+
version: TEMPLATE_SKELETON_FIT_METADATA_VERSION,
|
|
159
|
+
templateId: 'humanoid-basic',
|
|
160
|
+
sourceCharacterId: 'character-1',
|
|
161
|
+
verticalAxis: 'y',
|
|
162
|
+
verticalAnchor: 'min',
|
|
163
|
+
fit: {
|
|
164
|
+
scale: 1.2,
|
|
165
|
+
translation: { x: 0.1, y: 0.2, z: -0.1 },
|
|
166
|
+
confidence: 0.75,
|
|
167
|
+
meshHeight: 1.8,
|
|
168
|
+
templateHeight: 1.5,
|
|
169
|
+
crossAxisSpans: { x: 0.6, y: 1.8, z: 0.4 },
|
|
170
|
+
status: 'estimated',
|
|
171
|
+
},
|
|
172
|
+
manualAdjustment: {
|
|
173
|
+
scale: 1.05,
|
|
174
|
+
translation: { x: 0.01, y: -0.02, z: 0.03 },
|
|
175
|
+
},
|
|
176
|
+
};
|
|
177
|
+
|
|
178
|
+
const validation = validateTemplateSkeletonFitMetadata(fitMetadata);
|
|
179
|
+
const transform = composeTemplateSkeletonFitTransform(fitMetadata.fit, fitMetadata.manualAdjustment);
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
The required fields are:
|
|
183
|
+
- `kind`: must be `template-skeleton-fit` so hosts can distinguish this record
|
|
184
|
+
from skinning, rig, animation, or mesh metadata.
|
|
185
|
+
- `version`: must match `TEMPLATE_SKELETON_FIT_METADATA_VERSION`. Unknown
|
|
186
|
+
versions are rejected so migrations are explicit.
|
|
187
|
+
- `templateId` and `sourceCharacterId`: non-empty ids that let the host connect
|
|
188
|
+
the fit to its template registry and source asset record.
|
|
189
|
+
- `verticalAxis`: one of `x`, `y`, or `z`; the source character axis used for
|
|
190
|
+
height alignment.
|
|
191
|
+
- `verticalAnchor`: one of `min`, `center`, or `max`; the height anchor used
|
|
192
|
+
when solving the initial placement.
|
|
193
|
+
- `fit`: the solver result, including positive `scale`, finite `translation`, a
|
|
194
|
+
`confidence` value from `0` to `1`, positive mesh/template heights, non-negative
|
|
195
|
+
cross-axis spans, and a stable status.
|
|
196
|
+
- `manualAdjustment`: optional host-authored corrections. `scale` must be
|
|
197
|
+
positive and `translation` must be finite when present.
|
|
198
|
+
|
|
199
|
+
The stable status values are:
|
|
200
|
+
- `unfitted`: no usable solve exists yet.
|
|
201
|
+
- `estimated`: Embody produced an automatic fit that is ready for host preview.
|
|
202
|
+
- `manual-adjusted`: a user or host tool adjusted the automatic fit.
|
|
203
|
+
- `invalid`: the host should keep the metadata for inspection, but must not use
|
|
204
|
+
it as a usable placement.
|
|
205
|
+
|
|
206
|
+
Hosts should render the composed transform in a preview or setup flow first. The
|
|
207
|
+
host owns any final retargeting, rig binding, or skin-binding decision, and should
|
|
208
|
+
keep those records separate from `TemplateSkeletonFitMetadata`.
|
|
209
|
+
|
|
210
|
+
### Basic setup (default host)
|
|
211
|
+
|
|
212
|
+
Most callers only need a container element and a character URL. Embody creates
|
|
213
|
+
the Three.js scene (renderer, camera, lights, shadow plane), loads the model,
|
|
214
|
+
frames the camera on it, starts a render loop, and binds the runtime:
|
|
215
|
+
|
|
216
|
+
```typescript
|
|
217
|
+
import { createCharacterHost } from '@lovelace_lol/embody';
|
|
218
|
+
// or: import { createCharacterHost } from '@lovelace_lol/embody/three';
|
|
219
|
+
|
|
220
|
+
const host = await createCharacterHost({
|
|
221
|
+
container: document.getElementById('viewport')!,
|
|
222
|
+
character: {
|
|
223
|
+
modelUrl: '/characters/jonathan.glb',
|
|
224
|
+
presetType: 'cc4',
|
|
225
|
+
modelScale: 0.9,
|
|
226
|
+
},
|
|
227
|
+
});
|
|
228
|
+
|
|
229
|
+
// host.engine is a ready Embody instance
|
|
230
|
+
host.engine.setAU(12, 0.8);
|
|
231
|
+
|
|
232
|
+
// later
|
|
233
|
+
host.dispose();
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
`modelScale` is an optional positive finite uniform scale for the loaded model
|
|
237
|
+
root. When omitted, Embody preserves the scale authored in the asset. Embody
|
|
238
|
+
applies model scale before `modelGroundClearance`, so ground placement uses the
|
|
239
|
+
scaled model bounds.
|
|
240
|
+
|
|
241
|
+
Pick a named scene type to change the backdrop, lighting, and shadow defaults
|
|
242
|
+
(`studio` is the default; `showcase`, `inspection`, and `void` are also built in),
|
|
243
|
+
and override any individual option on top:
|
|
244
|
+
|
|
245
|
+
```typescript
|
|
246
|
+
const host = await createCharacterHost({
|
|
247
|
+
container,
|
|
248
|
+
character: { modelUrl: '/characters/jonathan.glb' },
|
|
249
|
+
scene: {
|
|
250
|
+
type: 'showcase', // dark backdrop, contrast lighting
|
|
251
|
+
// background: 0x223344, // per-option overrides still win
|
|
252
|
+
// lightingPreset: 'inspection',
|
|
253
|
+
// shadowPlane: false,
|
|
254
|
+
},
|
|
255
|
+
});
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
The available types are exported as `CHARACTER_SCENE_TYPES` /
|
|
259
|
+
`CHARACTER_SCENE_TYPE_IDS` so UIs can enumerate them.
|
|
260
|
+
|
|
261
|
+
The default host renders on its own (`renderLoop: true`) and auto-frames the
|
|
262
|
+
camera on the model (`autoFrame: true`). Turn either off when you drive the
|
|
263
|
+
camera or render loop yourself (e.g. custom camera controls):
|
|
264
|
+
|
|
265
|
+
```typescript
|
|
266
|
+
const host = await createCharacterHost({
|
|
267
|
+
container,
|
|
268
|
+
character: { modelUrl: '/characters/jonathan.glb' },
|
|
269
|
+
renderLoop: false, // you call renderer.render(scene, camera)
|
|
270
|
+
autoFrame: false, // you position the camera
|
|
271
|
+
});
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
With `external`, both default to `false` — your app owns rendering and camera.
|
|
275
|
+
|
|
276
|
+
### Advanced setup (bring your own Three.js scene)
|
|
277
|
+
|
|
278
|
+
Pass an existing scene/renderer/camera when you already own the WebGL host
|
|
279
|
+
(R3F, custom workbench, etc.). Embody still loads and binds the character:
|
|
280
|
+
|
|
281
|
+
```typescript
|
|
282
|
+
const host = await createCharacterHost({
|
|
283
|
+
container: mountEl,
|
|
284
|
+
character: { modelUrl: '/characters/jonathan.glb', presetType: 'cc4' },
|
|
285
|
+
external: {
|
|
286
|
+
scene,
|
|
287
|
+
renderer,
|
|
288
|
+
camera,
|
|
289
|
+
// mountCanvas: true, // optional
|
|
290
|
+
// manageResize: true, // optional
|
|
291
|
+
},
|
|
292
|
+
});
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
The lowest-level contract remains stable: `new Embody(...)` + `onReady({ meshes, model })`
|
|
296
|
+
if you want to own loading yourself.
|
|
297
|
+
|
|
298
|
+
```typescript
|
|
299
|
+
import * as THREE from 'three';
|
|
300
|
+
import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
|
|
301
|
+
import { Embody, collectMorphMeshes, CC4_PRESET } from '@lovelace_lol/embody';
|
|
302
|
+
|
|
303
|
+
const loom = new Embody({ profile: CC4_PRESET });
|
|
304
|
+
const loader = new GLTFLoader();
|
|
305
|
+
loader.load('/character.glb', (gltf) => {
|
|
306
|
+
scene.add(gltf.scene);
|
|
307
|
+
loom.onReady({ meshes: collectMorphMeshes(gltf.scene), model: gltf.scene });
|
|
308
|
+
});
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
If you’re implementing a custom renderer, target the `Embody` interface exported from `@lovelace_lol/embody` (legacy alias: `LoomLarge`).
|
|
312
|
+
|
|
313
|
+
### Lifecycle and update ownership
|
|
314
|
+
|
|
315
|
+
You have two valid integration patterns:
|
|
316
|
+
- External render loop: call `loom.update(deltaSeconds)` from your app’s main loop.
|
|
317
|
+
- Internal render loop: call `loom.start()` after `onReady()` and let Embody drive its own RAF-based updates.
|
|
318
|
+
|
|
319
|
+
The main lifecycle methods are:
|
|
320
|
+
- `onReady({ meshes, model })`: bind the loaded model to the engine
|
|
321
|
+
- `update(deltaSeconds)`: advance transitions, mixer playback, and runtime systems once
|
|
322
|
+
- `start()` / `stop()`: opt into or out of the internal loop
|
|
323
|
+
- `dispose()`: stop playback and release engine state when the character is torn down
|
|
324
|
+
|
|
325
|
+
### Quick start examples
|
|
326
|
+
|
|
327
|
+
Once your character is loaded, you can control facial expressions immediately:
|
|
328
|
+
|
|
329
|
+
```typescript
|
|
330
|
+
// Make the character smile
|
|
331
|
+
loom.setAU(12, 0.8);
|
|
332
|
+
|
|
333
|
+
// Raise eyebrows
|
|
334
|
+
loom.setAU(1, 0.6);
|
|
335
|
+
loom.setAU(2, 0.6);
|
|
336
|
+
|
|
337
|
+
// Blink
|
|
338
|
+
loom.setAU(45, 1.0);
|
|
339
|
+
|
|
340
|
+
// Open jaw
|
|
341
|
+
loom.setAU(26, 0.5);
|
|
342
|
+
|
|
343
|
+
// Turn head left
|
|
344
|
+
loom.setAU(51, 0.4);
|
|
345
|
+
|
|
346
|
+
// Look up
|
|
347
|
+
loom.setAU(63, 0.6);
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
Animate smoothly with transitions:
|
|
351
|
+
|
|
352
|
+
```typescript
|
|
353
|
+
// Smile over 200ms
|
|
354
|
+
await loom.transitionAU(12, 0.8, 200).promise;
|
|
355
|
+
|
|
356
|
+
// Then fade back to neutral
|
|
357
|
+
await loom.transitionAU(12, 0, 300).promise;
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
### The `collectMorphMeshes` helper
|
|
361
|
+
|
|
362
|
+
This utility function traverses a Three.js scene and returns all meshes that have `morphTargetInfluences` (i.e., blend shapes). It's the recommended way to gather meshes for Embody:
|
|
363
|
+
|
|
364
|
+
```typescript
|
|
365
|
+
import { collectMorphMeshes } from '@lovelace_lol/embody';
|
|
366
|
+
|
|
367
|
+
const meshes = collectMorphMeshes(gltf.scene);
|
|
368
|
+
// Returns: Array of THREE.Mesh objects with morph targets
|
|
369
|
+
```
|
|
370
|
+
|
|
371
|
+

|
|
372
|
+
|
|
373
|
+
---
|
|
374
|
+
|
|
375
|
+
## 2. Using Presets
|
|
376
|
+
|
|
377
|
+
Open in LoomLarge: [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
378
|
+
|
|
379
|
+

|
|
380
|
+
|
|
381
|
+
Presets define how FACS Action Units map to your character's morph targets and bones. Embody ships with `CC4_PRESET` for Character Creator 4 characters.
|
|
382
|
+
|
|
383
|
+
### What's in a preset?
|
|
384
|
+
|
|
385
|
+
```typescript
|
|
386
|
+
import { CC4_PRESET } from '@lovelace_lol/embody';
|
|
387
|
+
|
|
388
|
+
// CC4_PRESET contains:
|
|
389
|
+
{
|
|
390
|
+
auToMorphs: {
|
|
391
|
+
// AU number → morph target names split by side
|
|
392
|
+
1: { left: ['Brow_Raise_Inner_L'], right: ['Brow_Raise_Inner_R'], center: [] },
|
|
393
|
+
12: { left: ['Mouth_Smile_L'], right: ['Mouth_Smile_R'], center: [] },
|
|
394
|
+
45: { left: ['Eye_Blink_L'], right: ['Eye_Blink_R'], center: [] },
|
|
395
|
+
// ... 87 AUs total
|
|
396
|
+
},
|
|
397
|
+
|
|
398
|
+
auToBones: {
|
|
399
|
+
// AU number → array of bone bindings
|
|
400
|
+
51: [{ node: 'HEAD', channel: 'ry', scale: -1, maxDegrees: 30 }],
|
|
401
|
+
61: [{ node: 'EYE_L', channel: 'rz', scale: 1, maxDegrees: 25 }],
|
|
402
|
+
// ... 32 bone bindings
|
|
403
|
+
},
|
|
404
|
+
|
|
405
|
+
boneNodes: {
|
|
406
|
+
// Logical bone name → base node name used with bonePrefix
|
|
407
|
+
'HEAD': 'Head',
|
|
408
|
+
'JAW': 'JawRoot',
|
|
409
|
+
'EYE_L': 'L_Eye',
|
|
410
|
+
'EYE_R': 'R_Eye',
|
|
411
|
+
'TONGUE': 'Tongue01',
|
|
412
|
+
},
|
|
413
|
+
|
|
414
|
+
bonePrefix: 'CC_Base_',
|
|
415
|
+
suffixPattern: '_\\d+$|\\.\\d+$',
|
|
416
|
+
|
|
417
|
+
visemeKeys: [
|
|
418
|
+
// 15 viseme morph names for lip-sync
|
|
419
|
+
'EE', 'Ah', 'Oh', 'OO', 'I',
|
|
420
|
+
'U', 'W', 'L', 'F_V', 'Th',
|
|
421
|
+
'S_Z', 'B_M_P', 'K_G_H_NG', 'AE', 'R'
|
|
422
|
+
],
|
|
423
|
+
|
|
424
|
+
morphToMesh: {
|
|
425
|
+
// Routes morph categories to specific meshes
|
|
426
|
+
'face': ['CC_Base_Body'],
|
|
427
|
+
'viseme': ['CC_Base_Body', 'CC_Base_Body_1'],
|
|
428
|
+
'tongue': ['CC_Base_Tongue'],
|
|
429
|
+
'eye': ['CC_Base_EyeOcclusion_1', 'CC_Base_EyeOcclusion_2'],
|
|
430
|
+
'hair': ['Side_part_wavy_1', 'Side_part_wavy_2'],
|
|
431
|
+
},
|
|
432
|
+
|
|
433
|
+
auMixDefaults: {
|
|
434
|
+
// Default morph/bone blend weights (0 = morph, 1 = bone)
|
|
435
|
+
26: 0.5, // Jaw drop: 50% morph, 50% bone
|
|
436
|
+
51: 0.7, // Head turn: 70% bone
|
|
437
|
+
},
|
|
438
|
+
|
|
439
|
+
auInfo: {
|
|
440
|
+
// Metadata about each AU
|
|
441
|
+
'12': {
|
|
442
|
+
name: 'Lip Corner Puller',
|
|
443
|
+
muscularBasis: 'zygomaticus major',
|
|
444
|
+
faceArea: 'Lower',
|
|
445
|
+
facePart: 'Mouth',
|
|
446
|
+
},
|
|
447
|
+
// ...
|
|
448
|
+
}
|
|
449
|
+
}
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
### Name resolution and profile fields
|
|
453
|
+
|
|
454
|
+
The runtime resolves bone nodes by composing `bonePrefix + boneNodes[key] + boneSuffix`, then falling back to suffix-pattern matching when a model uses numbered exports such as `_01` or `.001`. The same prefix/suffix rules are used by validation and correction helpers, which is why `CC4_PRESET` can keep base bone names like `Head` and `JawRoot` instead of repeating the full node names everywhere.
|
|
455
|
+
|
|
456
|
+
For region and marker configs, `resolveBoneName()` treats any mapped bone name that already contains `_` or `.` as a fully qualified name and skips prefix/suffix composition.
|
|
457
|
+
|
|
458
|
+
Two caveats are worth calling out:
|
|
459
|
+
- `morphPrefix` and `morphSuffix` are part of `Profile`, but morph playback still resolves exact morph keys on the targeted meshes today. They are already used by validation and correction helpers.
|
|
460
|
+
- `leftMorphSuffixes` and `rightMorphSuffixes` are profile metadata for laterality detection in tooling, not core runtime behavior.
|
|
461
|
+
|
|
462
|
+
Other `Profile` fields that are easy to miss:
|
|
463
|
+
- `morphToMesh` routes categories such as `face`, `viseme`, `eye`, `tongue`, and `hair` to specific mesh names.
|
|
464
|
+
- `eyeMeshNodes` provides fallback eye nodes when a rig uses meshes instead of bones for eye control.
|
|
465
|
+
- `auMixDefaults` sets the default morph/bone blend weight per AU.
|
|
466
|
+
- `compositeRotations` defines the per-node pitch/yaw/roll axis layout used by the composite rotation system.
|
|
467
|
+
- `continuumPairs` and `continuumLabels` describe bidirectional AU pairs and their UI labels.
|
|
468
|
+
- `annotationRegions` defines the regions used by the marker and camera tooling, including per-region framing via `paddingFactor`.
|
|
469
|
+
- `hairPhysics` stores the mixer-driven hair defaults, including direction signs and morph target mappings.
|
|
470
|
+
|
|
471
|
+
For `annotationRegions`, `paddingFactor` is the camera framing multiplier for that region:
|
|
472
|
+
- values below `1` zoom in tighter
|
|
473
|
+
- values above `1` pull back to show more surrounding context
|
|
474
|
+
- profile overrides can replace it per region by name without copying the whole preset
|
|
475
|
+
|
|
476
|
+
### Passing a preset to Embody
|
|
477
|
+
|
|
478
|
+
```typescript
|
|
479
|
+
import { Embody, CC4_PRESET } from '@lovelace_lol/embody';
|
|
480
|
+
|
|
481
|
+
const loom = new Embody({ profile: CC4_PRESET });
|
|
482
|
+
```
|
|
483
|
+
|
|
484
|
+
You can also look up presets by name and extend them without cloning the full preset:
|
|
485
|
+
|
|
486
|
+
```typescript
|
|
487
|
+
import { Embody } from '@lovelace_lol/embody';
|
|
488
|
+
|
|
489
|
+
const loom = new Embody({
|
|
490
|
+
presetType: 'cc4',
|
|
491
|
+
profile: {
|
|
492
|
+
auToMorphs: {
|
|
493
|
+
12: { left: ['MySmile_Left'], right: ['MySmile_Right'], center: [] },
|
|
494
|
+
},
|
|
495
|
+
},
|
|
496
|
+
});
|
|
497
|
+
```
|
|
498
|
+
|
|
499
|
+
### Profiles (preset overrides)
|
|
500
|
+
|
|
501
|
+
A **profile** is a partial override object that extends a base preset. Use it to customize a single character without copying the full preset:
|
|
502
|
+
|
|
503
|
+
```typescript
|
|
504
|
+
import type { Profile } from '@lovelace_lol/embody';
|
|
505
|
+
import { Embody } from '@lovelace_lol/embody';
|
|
506
|
+
|
|
507
|
+
const DAISY_PROFILE: Profile = {
|
|
508
|
+
morphToMesh: { face: ['Object_9'] },
|
|
509
|
+
annotationRegions: [
|
|
510
|
+
{ name: 'face', bones: ['CC_Base_Head'] },
|
|
511
|
+
{ name: 'left_eye', paddingFactor: 0.9 },
|
|
512
|
+
{ name: 'right_eye', paddingFactor: 0.9 },
|
|
513
|
+
],
|
|
514
|
+
};
|
|
515
|
+
|
|
516
|
+
const loom = new Embody({
|
|
517
|
+
presetType: 'cc4',
|
|
518
|
+
profile: DAISY_PROFILE,
|
|
519
|
+
});
|
|
520
|
+
```
|
|
521
|
+
|
|
522
|
+
### Annotation configuration
|
|
523
|
+
|
|
524
|
+
`annotationRegions` is the Embody field for camera/marker region defaults and profile overrides.
|
|
525
|
+
|
|
526
|
+
If your app fetches a saved model/profile record from Firestore or another backend, use `extendProfileConfigWithPreset(...)` to build the runtime shape before handing that profile config to camera/marker tooling:
|
|
527
|
+
|
|
528
|
+
```typescript
|
|
529
|
+
import { extendProfileConfigWithPreset } from '@lovelace_lol/embody';
|
|
530
|
+
|
|
531
|
+
const savedConfig = await fetchProfileConfig();
|
|
532
|
+
const runtimeConfig = extendProfileConfigWithPreset({
|
|
533
|
+
...savedConfig,
|
|
534
|
+
profilePresetId: savedConfig.profilePresetId ?? 'cc4',
|
|
535
|
+
});
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
`CharacterConfig`, `auPresetType`, and `extendCharacterConfigWithPreset(...)` are still exported as deprecated compatibility aliases for apps migrating from older LoomLarge-style character records. New Embody integrations should model presets as base profiles, pass profile overrides through `profile`, `annotationRegions`, or other `Profile` fields, and use `profilePresetId` for preset selection.
|
|
539
|
+
|
|
540
|
+
For the current runtime-oriented documentation, including:
|
|
541
|
+
|
|
542
|
+
- `paddingFactor`
|
|
543
|
+
- `cameraAngle`
|
|
544
|
+
- `cameraOffset`
|
|
545
|
+
- `style.lineDirection`
|
|
546
|
+
- the difference between `cameraAngle: 0` and omitting `cameraAngle`
|
|
547
|
+
- runtime compatibility and legacy `config.regions` fallback behavior
|
|
548
|
+
|
|
549
|
+
see [ANNOTATION_CONFIGURATION.md](./ANNOTATION_CONFIGURATION.md).
|
|
550
|
+
|
|
551
|
+

|
|
552
|
+
|
|
553
|
+
---
|
|
554
|
+
|
|
555
|
+
## 3. Preset Selection & Validation
|
|
556
|
+
|
|
557
|
+
Open in LoomLarge: [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings) | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones)
|
|
558
|
+
|
|
559
|
+
Before you tune AUs or hand-edit a profile, confirm that you picked the right preset and that the model actually matches it. Embody exposes a full preset-selection and validation workflow, not just low-level control APIs.
|
|
560
|
+
|
|
561
|
+
### Looking Up and Extending Presets by Type
|
|
562
|
+
|
|
563
|
+
Use preset helpers when you want a stable entry point by model class instead of importing a preset constant directly:
|
|
564
|
+
|
|
565
|
+
```typescript
|
|
566
|
+
import {
|
|
567
|
+
getPreset,
|
|
568
|
+
getPresetWithProfile,
|
|
569
|
+
} from '@lovelace_lol/embody';
|
|
570
|
+
|
|
571
|
+
const preset = getPreset('cc4');
|
|
572
|
+
|
|
573
|
+
const extended = getPresetWithProfile('cc4', {
|
|
574
|
+
morphToMesh: { face: ['Object_9'] },
|
|
575
|
+
});
|
|
576
|
+
```
|
|
577
|
+
|
|
578
|
+
### Validating the config itself
|
|
579
|
+
|
|
580
|
+
`validateMappingConfig()` checks the profile for internal consistency before you even compare it to a model:
|
|
581
|
+
|
|
582
|
+
```typescript
|
|
583
|
+
import { validateMappingConfig } from '@lovelace_lol/embody';
|
|
584
|
+
|
|
585
|
+
const consistency = validateMappingConfig(resolved);
|
|
586
|
+
console.log(consistency.errors, consistency.warnings);
|
|
587
|
+
```
|
|
588
|
+
|
|
589
|
+
### Checking a model against a preset
|
|
590
|
+
|
|
591
|
+
```typescript
|
|
592
|
+
import * as THREE from 'three';
|
|
593
|
+
import {
|
|
594
|
+
validateMappings,
|
|
595
|
+
isPresetCompatible,
|
|
596
|
+
generateMappingCorrections,
|
|
597
|
+
} from '@lovelace_lol/embody';
|
|
598
|
+
|
|
599
|
+
const skinnedMesh = gltf.scene.getObjectByProperty('type', 'SkinnedMesh') as THREE.SkinnedMesh | undefined;
|
|
600
|
+
const skeleton = skinnedMesh?.skeleton ?? null;
|
|
601
|
+
|
|
602
|
+
const validation = validateMappings(meshes, skeleton, resolved, {
|
|
603
|
+
suggestCorrections: true,
|
|
604
|
+
});
|
|
605
|
+
|
|
606
|
+
const compatible = isPresetCompatible(meshes, skeleton, resolved);
|
|
607
|
+
const corrections = generateMappingCorrections(meshes, skeleton, resolved, {
|
|
608
|
+
useResolvedNames: true,
|
|
609
|
+
});
|
|
610
|
+
```
|
|
611
|
+
|
|
612
|
+
### Suggesting the best preset from a candidate set
|
|
613
|
+
|
|
614
|
+
```typescript
|
|
615
|
+
import {
|
|
616
|
+
CC4_PRESET,
|
|
617
|
+
BETTA_FISH_PRESET,
|
|
618
|
+
suggestBestPreset,
|
|
619
|
+
} from '@lovelace_lol/embody';
|
|
620
|
+
|
|
621
|
+
const best = suggestBestPreset(meshes, skeleton, [
|
|
622
|
+
CC4_PRESET,
|
|
623
|
+
BETTA_FISH_PRESET,
|
|
624
|
+
]);
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
### Running a full model analysis
|
|
628
|
+
|
|
629
|
+
```typescript
|
|
630
|
+
import {
|
|
631
|
+
analyzeModel,
|
|
632
|
+
extractFromGLTF,
|
|
633
|
+
extractModelData,
|
|
634
|
+
} from '@lovelace_lol/embody';
|
|
635
|
+
|
|
636
|
+
const extracted = extractFromGLTF(gltf);
|
|
637
|
+
const runtimeData = extractModelData(gltf.scene, meshes, gltf.animations);
|
|
638
|
+
|
|
639
|
+
const report = await analyzeModel({
|
|
640
|
+
source: { type: 'gltf', gltf },
|
|
641
|
+
preset: resolved,
|
|
642
|
+
suggestCorrections: true,
|
|
643
|
+
});
|
|
644
|
+
|
|
645
|
+
console.log(report.summary, report.overallScore);
|
|
646
|
+
```
|
|
647
|
+
|
|
648
|
+
Use this section when you need to:
|
|
649
|
+
- choose between built-in presets before wiring the character into your app
|
|
650
|
+
- lint a preset/profile for broken internal references
|
|
651
|
+
- measure how well a preset matches an imported model
|
|
652
|
+
- generate correction suggestions before building a custom profile
|
|
653
|
+
|
|
654
|
+
---
|
|
655
|
+
|
|
656
|
+
## 4. Getting to Know Your Character
|
|
657
|
+
|
|
658
|
+
Open in LoomLarge: [Meshes tab](https://www.characterloom.com/?drawer=open&tab=meshes) | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
659
|
+
|
|
660
|
+

|
|
661
|
+
|
|
662
|
+
Before customizing presets or extending mappings, it's helpful to understand what's actually in your character model. Embody provides several methods to inspect meshes, morph targets, and bones.
|
|
663
|
+
|
|
664
|
+
### Listing meshes
|
|
665
|
+
|
|
666
|
+
Get all meshes in your character with their visibility and morph target counts:
|
|
667
|
+
|
|
668
|
+
```typescript
|
|
669
|
+
const meshes = loom.getMeshList();
|
|
670
|
+
console.log(meshes);
|
|
671
|
+
// [
|
|
672
|
+
// { name: 'CC_Base_Body', visible: true, morphCount: 142 },
|
|
673
|
+
// { name: 'CC_Base_Tongue', visible: true, morphCount: 12 },
|
|
674
|
+
// { name: 'CC_Base_EyeOcclusion_1', visible: true, morphCount: 8 },
|
|
675
|
+
// { name: 'CC_Base_EyeOcclusion_2', visible: true, morphCount: 8 },
|
|
676
|
+
// { name: 'Male_Bushy_1', visible: true, morphCount: 142 },
|
|
677
|
+
// ...
|
|
678
|
+
// ]
|
|
679
|
+
```
|
|
680
|
+
|
|
681
|
+
### Listing morph targets
|
|
682
|
+
|
|
683
|
+
Get all morph target names grouped by mesh:
|
|
684
|
+
|
|
685
|
+
```typescript
|
|
686
|
+
const morphs = loom.getMorphTargets();
|
|
687
|
+
console.log(morphs);
|
|
688
|
+
// {
|
|
689
|
+
// 'CC_Base_Body': [
|
|
690
|
+
// 'A01_Brow_Inner_Up', 'A02_Brow_Down_Left', 'A02_Brow_Down_Right',
|
|
691
|
+
// 'A04_Brow_Outer_Up_Left', 'A04_Brow_Outer_Up_Right',
|
|
692
|
+
// 'Mouth_Smile_L', 'Mouth_Smile_R', 'Eye_Blink_L', 'Eye_Blink_R',
|
|
693
|
+
// ...
|
|
694
|
+
// ],
|
|
695
|
+
// 'CC_Base_Tongue': [
|
|
696
|
+
// 'V_Tongue_Out', 'V_Tongue_Up', 'V_Tongue_Down', ...
|
|
697
|
+
// ],
|
|
698
|
+
// ...
|
|
699
|
+
// }
|
|
700
|
+
```
|
|
701
|
+
|
|
702
|
+
This is invaluable when creating custom presets—you need to know the exact morph target names your character uses.
|
|
703
|
+
|
|
704
|
+
### Listing bones
|
|
705
|
+
|
|
706
|
+
Get all resolved bones with their current positions and rotations (in degrees):
|
|
707
|
+
|
|
708
|
+
```typescript
|
|
709
|
+
const bones = loom.getBones();
|
|
710
|
+
console.log(bones);
|
|
711
|
+
// {
|
|
712
|
+
// 'HEAD': { position: [0, 156.2, 0], rotation: [0, 0, 0] },
|
|
713
|
+
// 'JAW': { position: [0, 154.1, 2.3], rotation: [0, 0, 0] },
|
|
714
|
+
// 'EYE_L': { position: [-3.2, 160.5, 8.1], rotation: [0, 0, 0] },
|
|
715
|
+
// 'EYE_R': { position: [3.2, 160.5, 8.1], rotation: [0, 0, 0] },
|
|
716
|
+
// 'TONGUE': { position: [0, 152.3, 1.8], rotation: [0, 0, 0] },
|
|
717
|
+
// }
|
|
718
|
+
```
|
|
719
|
+
|
|
720
|
+
### Listing morph target indices
|
|
721
|
+
|
|
722
|
+
Use the index view when you need to work with `setMorphInfluence()` or build tools that operate on morph slots directly:
|
|
723
|
+
|
|
724
|
+
```typescript
|
|
725
|
+
const indices = loom.getMorphTargetIndices();
|
|
726
|
+
console.log(indices);
|
|
727
|
+
// {
|
|
728
|
+
// 'CC_Base_Body': [
|
|
729
|
+
// { index: 0, name: 'A01_Brow_Inner_Up' },
|
|
730
|
+
// { index: 1, name: 'A02_Brow_Down_Left' },
|
|
731
|
+
// ...
|
|
732
|
+
// ],
|
|
733
|
+
// }
|
|
734
|
+
```
|
|
735
|
+
|
|
736
|
+
### Validating and analyzing the model you loaded
|
|
737
|
+
|
|
738
|
+
Use the extraction and validation helpers when inspection needs to turn into preset-fit analysis:
|
|
739
|
+
|
|
740
|
+
```typescript
|
|
741
|
+
import {
|
|
742
|
+
extractFromGLTF,
|
|
743
|
+
extractModelData,
|
|
744
|
+
analyzeModel,
|
|
745
|
+
validateMappings,
|
|
746
|
+
generateMappingCorrections,
|
|
747
|
+
getPreset,
|
|
748
|
+
} from '@lovelace_lol/embody';
|
|
749
|
+
|
|
750
|
+
const preset = getPreset('cc4');
|
|
751
|
+
const modelData = extractModelData(model, meshes, animations);
|
|
752
|
+
const gltfData = extractFromGLTF(gltf); // Same ModelData shape, one-step GLTF wrapper
|
|
753
|
+
|
|
754
|
+
const analysis = await analyzeModel({
|
|
755
|
+
source: { type: 'gltf', gltf },
|
|
756
|
+
preset,
|
|
757
|
+
suggestCorrections: true,
|
|
758
|
+
});
|
|
759
|
+
|
|
760
|
+
// Validate against lower-level mesh + skeleton inputs when you already have them
|
|
761
|
+
const validation = validateMappings(meshes, skeleton, preset, { suggestCorrections: true });
|
|
762
|
+
const corrections = generateMappingCorrections(meshes, skeleton, preset, { useResolvedNames: true });
|
|
763
|
+
```
|
|
764
|
+
|
|
765
|
+
If you already have a `ModelData` bundle, `analyzeModel()` is the higher-level path; `validateMappings()` and `generateMappingCorrections()` are intentionally lower-level mesh/skeleton helpers.
|
|
766
|
+
|
|
767
|
+
Use these helpers to:
|
|
768
|
+
- Extract raw model facts with `extractModelData(model, meshes?, animations?)` or `extractFromGLTF(gltf)`
|
|
769
|
+
- Validate a preset against mesh/skeleton data with `validateMappings(meshes, skeleton, preset, options)`
|
|
770
|
+
- Generate best-effort fixes with `generateMappingCorrections(meshes, skeleton, preset, options)`
|
|
771
|
+
- Run a single end-to-end pass with `analyzeModel({ source, preset, suggestCorrections })`
|
|
772
|
+
|
|
773
|
+
`validateMappings()` returns a `ValidationResult` with:
|
|
774
|
+
- `valid` and `score`
|
|
775
|
+
- `missingMorphs`, `missingBones`, `foundMorphs`, `foundBones`
|
|
776
|
+
- `missingMeshes`, `foundMeshes`, `unmappedMorphs`, `unmappedBones`, `unmappedMeshes`
|
|
777
|
+
- `warnings`
|
|
778
|
+
- optional `suggestedConfig`, `corrections`, and `unresolved` when suggestion mode is enabled
|
|
779
|
+
|
|
780
|
+
`generateMappingCorrections()` returns:
|
|
781
|
+
- `correctedConfig`
|
|
782
|
+
- `corrections`
|
|
783
|
+
- `unresolved`
|
|
784
|
+
|
|
785
|
+
`analyzeModel()` returns a `ModelAnalysisReport` containing the extracted model data, optional validation results, animation summary, `overallScore`, and a plain-language `summary`.
|
|
786
|
+
|
|
787
|
+
### Humanoid skeleton templates
|
|
788
|
+
|
|
789
|
+
Embody owns reusable humanoid skeleton templates and the extraction path for
|
|
790
|
+
turning a skinned character into template data. Host apps can select a template
|
|
791
|
+
by id, derive its rest bounds for fitting, or seed validation/mapping UI with
|
|
792
|
+
the template bone list:
|
|
793
|
+
|
|
794
|
+
```typescript
|
|
795
|
+
import {
|
|
796
|
+
JONATHAN_HUMANOID_SKELETON_TEMPLATE,
|
|
797
|
+
computeHumanoidSkeletonTemplateRestBounds,
|
|
798
|
+
createValidationSkeletonFromHumanoidTemplate,
|
|
799
|
+
getHumanoidSkeletonTemplate,
|
|
800
|
+
} from '@lovelace_lol/embody';
|
|
801
|
+
|
|
802
|
+
const template = getHumanoidSkeletonTemplate('jonathan-cc-base') ?? JONATHAN_HUMANOID_SKELETON_TEMPLATE;
|
|
803
|
+
const templateBounds = computeHumanoidSkeletonTemplateRestBounds(template);
|
|
804
|
+
const validationSkeleton = createValidationSkeletonFromHumanoidTemplate(template);
|
|
805
|
+
```
|
|
806
|
+
|
|
807
|
+
When a new skinned GLB should become a reusable template, extract its skeleton
|
|
808
|
+
into `src/skeletonTemplates/data`:
|
|
809
|
+
|
|
810
|
+
```bash
|
|
811
|
+
npm run extract:humanoid-skeleton-template -- ../LoomLarge/frontend/public/characters/jonathan_new.glb src/skeletonTemplates/data/jonathan-cc-base.json --id jonathan-cc-base --source-character-id jonathan --skin-name Armature
|
|
812
|
+
```
|
|
813
|
+
|
|
814
|
+
The extracted template records joint names, parent links, and local
|
|
815
|
+
translations. It is fit/mapping metadata only; it does not include skin weights,
|
|
816
|
+
inverse bind matrices, or pose retargeting data.
|
|
817
|
+
|
|
818
|
+
### Controlling mesh visibility
|
|
819
|
+
|
|
820
|
+
Hide or show individual meshes:
|
|
821
|
+
|
|
822
|
+
```typescript
|
|
823
|
+
// Hide hair mesh
|
|
824
|
+
loom.setMeshVisible('Side_part_wavy_1', false);
|
|
825
|
+
|
|
826
|
+
// Show it again
|
|
827
|
+
loom.setMeshVisible('Side_part_wavy_1', true);
|
|
828
|
+
```
|
|
829
|
+
|
|
830
|
+
### Highlighting a mesh
|
|
831
|
+
|
|
832
|
+
Use highlighting when you need to confirm which mesh a profile field or morph category is actually targeting:
|
|
833
|
+
|
|
834
|
+
```typescript
|
|
835
|
+
// Highlight one mesh
|
|
836
|
+
loom.highlightMesh('CC_Base_Body');
|
|
837
|
+
|
|
838
|
+
// Clear all highlights
|
|
839
|
+
loom.highlightMesh(null);
|
|
840
|
+
```
|
|
841
|
+
|
|
842
|
+
### Adjusting material properties
|
|
843
|
+
|
|
844
|
+
Fine-tune render order, transparency, and blending for each mesh:
|
|
845
|
+
|
|
846
|
+
```typescript
|
|
847
|
+
// Get current material config
|
|
848
|
+
const config = loom.getMeshMaterialConfig('CC_Base_Body');
|
|
849
|
+
console.log(config);
|
|
850
|
+
// {
|
|
851
|
+
// renderOrder: 0,
|
|
852
|
+
// transparent: false,
|
|
853
|
+
// opacity: 1,
|
|
854
|
+
// depthWrite: true,
|
|
855
|
+
// depthTest: true,
|
|
856
|
+
// blending: 'Normal'
|
|
857
|
+
// }
|
|
858
|
+
|
|
859
|
+
// Set custom material config
|
|
860
|
+
loom.setMeshMaterialConfig('CC_Base_EyeOcclusion_1', {
|
|
861
|
+
renderOrder: 10,
|
|
862
|
+
transparent: true,
|
|
863
|
+
opacity: 0.8,
|
|
864
|
+
blending: 'Normal' // 'Normal', 'Additive', 'Subtractive', 'Multiply', 'None'
|
|
865
|
+
});
|
|
866
|
+
```
|
|
867
|
+
|
|
868
|
+
This is especially useful for:
|
|
869
|
+
- Fixing render order issues (eyebrows behind hair, etc.)
|
|
870
|
+
- Making meshes semi-transparent for debugging
|
|
871
|
+
- Adjusting blending modes for special effects
|
|
872
|
+
|
|
873
|
+

|
|
874
|
+
|
|
875
|
+
---
|
|
876
|
+
|
|
877
|
+
## 5. Extending & Custom Presets
|
|
878
|
+
|
|
879
|
+
Open in LoomLarge: [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
880
|
+
|
|
881
|
+

|
|
882
|
+
|
|
883
|
+
### Extending an existing preset
|
|
884
|
+
|
|
885
|
+
Use `extendPresetWithProfile` to override specific mappings while keeping the rest:
|
|
886
|
+
|
|
887
|
+
```typescript
|
|
888
|
+
import { CC4_PRESET, extendPresetWithProfile } from '@lovelace_lol/embody';
|
|
889
|
+
|
|
890
|
+
const MY_PRESET = extendPresetWithProfile(CC4_PRESET, {
|
|
891
|
+
|
|
892
|
+
// Override AU12 (smile) with custom morph names
|
|
893
|
+
auToMorphs: {
|
|
894
|
+
12: { left: ['MySmile_Left'], right: ['MySmile_Right'], center: [] },
|
|
895
|
+
},
|
|
896
|
+
|
|
897
|
+
// Add a new bone binding
|
|
898
|
+
auToBones: {
|
|
899
|
+
99: [{ node: 'CUSTOM_BONE', channel: 'ry', scale: 1, maxDegrees: 45 }],
|
|
900
|
+
},
|
|
901
|
+
|
|
902
|
+
// Update bone node paths
|
|
903
|
+
boneNodes: {
|
|
904
|
+
'CUSTOM_BONE': 'MyRig_CustomBone',
|
|
905
|
+
},
|
|
906
|
+
});
|
|
907
|
+
|
|
908
|
+
const loom = new Embody({ profile: MY_PRESET });
|
|
909
|
+
```
|
|
910
|
+
|
|
911
|
+
### Creating a preset from scratch
|
|
912
|
+
|
|
913
|
+
```typescript
|
|
914
|
+
import type { Profile } from '@lovelace_lol/embody';
|
|
915
|
+
|
|
916
|
+
const CUSTOM_PRESET: Profile = {
|
|
917
|
+
auToMorphs: {
|
|
918
|
+
1: { left: ['brow_inner_up_L'], right: ['brow_inner_up_R'], center: [] },
|
|
919
|
+
2: { left: ['brow_outer_up_L'], right: ['brow_outer_up_R'], center: [] },
|
|
920
|
+
12: { left: ['mouth_smile_L'], right: ['mouth_smile_R'], center: [] },
|
|
921
|
+
45: { left: ['eye_blink_L'], right: ['eye_blink_R'], center: [] },
|
|
922
|
+
},
|
|
923
|
+
|
|
924
|
+
auToBones: {
|
|
925
|
+
51: [{ node: 'HEAD', channel: 'ry', scale: -1, maxDegrees: 30 }],
|
|
926
|
+
52: [{ node: 'HEAD', channel: 'ry', scale: 1, maxDegrees: 30 }],
|
|
927
|
+
},
|
|
928
|
+
|
|
929
|
+
boneNodes: {
|
|
930
|
+
'HEAD': 'head_bone',
|
|
931
|
+
'JAW': 'jaw_bone',
|
|
932
|
+
},
|
|
933
|
+
|
|
934
|
+
visemeKeys: ['aa', 'ee', 'ih', 'oh', 'oo'],
|
|
935
|
+
|
|
936
|
+
morphToMesh: {
|
|
937
|
+
'face': ['body_mesh'],
|
|
938
|
+
},
|
|
939
|
+
};
|
|
940
|
+
```
|
|
941
|
+
|
|
942
|
+
### Changing presets at runtime
|
|
943
|
+
|
|
944
|
+
```typescript
|
|
945
|
+
// Switch to a different preset
|
|
946
|
+
loom.setProfile(ANOTHER_PRESET);
|
|
947
|
+
|
|
948
|
+
// Get current mappings
|
|
949
|
+
const current = loom.getProfile();
|
|
950
|
+
```
|
|
951
|
+
|
|
952
|
+

|
|
953
|
+
|
|
954
|
+
---
|
|
955
|
+
|
|
956
|
+
## 6. Creating Skeletal Animation Presets
|
|
957
|
+
|
|
958
|
+
Open in LoomLarge: [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) | [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units) | [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
959
|
+
|
|
960
|
+

|
|
961
|
+
|
|
962
|
+
Embody isn't limited to humanoid characters with morph targets. You can create presets for any 3D model that uses skeletal animation, such as fish, animals, or fantasy creatures. This section explains how to create a preset for a betta fish model that has no morph targets—only bone-driven animation.
|
|
963
|
+
|
|
964
|
+
### Understanding skeletal-only models
|
|
965
|
+
|
|
966
|
+
Some models (like fish) rely entirely on bone rotations for animation:
|
|
967
|
+
- **No morph targets:** All movement is skeletal
|
|
968
|
+
- **Hierarchical bones:** Fins and body parts follow parent rotations
|
|
969
|
+
- **Custom "Action Units":** Instead of FACS AUs, you define model-specific actions
|
|
970
|
+
|
|
971
|
+
### Example: Betta Fish Preset
|
|
972
|
+
|
|
973
|
+
Here's a complete example of a preset for a betta fish:
|
|
974
|
+
|
|
975
|
+
```typescript
|
|
976
|
+
import type { BoneBinding, AUInfo, CompositeRotation } from '@lovelace_lol/embody';
|
|
977
|
+
|
|
978
|
+
// Define semantic bone mappings
|
|
979
|
+
export const FISH_BONE_NODES = {
|
|
980
|
+
ROOT: 'Armature_rootJoint',
|
|
981
|
+
BODY_ROOT: 'Bone_Armature',
|
|
982
|
+
HEAD: 'Bone001_Armature',
|
|
983
|
+
BODY_FRONT: 'Bone002_Armature',
|
|
984
|
+
BODY_MID: 'Bone003_Armature',
|
|
985
|
+
BODY_BACK: 'Bone004_Armature',
|
|
986
|
+
TAIL_BASE: 'Bone005_Armature',
|
|
987
|
+
|
|
988
|
+
// Pectoral fins (side fins)
|
|
989
|
+
PECTORAL_L: 'Bone046_Armature',
|
|
990
|
+
PECTORAL_R: 'Bone047_Armature',
|
|
991
|
+
|
|
992
|
+
// Dorsal fin (top fin)
|
|
993
|
+
DORSAL_ROOT: 'Bone006_Armature',
|
|
994
|
+
|
|
995
|
+
// Eyes (single mesh for both)
|
|
996
|
+
EYE_L: 'EYES_0',
|
|
997
|
+
EYE_R: 'EYES_0',
|
|
998
|
+
} as const;
|
|
999
|
+
|
|
1000
|
+
// Define custom fish actions (analogous to FACS AUs)
|
|
1001
|
+
export enum FishAction {
|
|
1002
|
+
// Body orientation
|
|
1003
|
+
TURN_LEFT = 2,
|
|
1004
|
+
TURN_RIGHT = 3,
|
|
1005
|
+
PITCH_UP = 4,
|
|
1006
|
+
PITCH_DOWN = 5,
|
|
1007
|
+
ROLL_LEFT = 6,
|
|
1008
|
+
ROLL_RIGHT = 7,
|
|
1009
|
+
|
|
1010
|
+
// Tail movements
|
|
1011
|
+
TAIL_SWEEP_LEFT = 12,
|
|
1012
|
+
TAIL_SWEEP_RIGHT = 13,
|
|
1013
|
+
TAIL_FIN_SPREAD = 14,
|
|
1014
|
+
TAIL_FIN_CLOSE = 15,
|
|
1015
|
+
|
|
1016
|
+
// Pectoral fins
|
|
1017
|
+
PECTORAL_L_UP = 20,
|
|
1018
|
+
PECTORAL_L_DOWN = 21,
|
|
1019
|
+
PECTORAL_R_UP = 22,
|
|
1020
|
+
PECTORAL_R_DOWN = 23,
|
|
1021
|
+
|
|
1022
|
+
// Eye rotation
|
|
1023
|
+
EYE_LEFT = 61,
|
|
1024
|
+
EYE_RIGHT = 62,
|
|
1025
|
+
EYE_UP = 63,
|
|
1026
|
+
EYE_DOWN = 64,
|
|
1027
|
+
}
|
|
1028
|
+
```
|
|
1029
|
+
|
|
1030
|
+
### Defining bone bindings for movement
|
|
1031
|
+
|
|
1032
|
+
Map each action to bone rotations:
|
|
1033
|
+
|
|
1034
|
+
```typescript
|
|
1035
|
+
export const FISH_BONE_BINDINGS: Record<number, BoneBinding[]> = {
|
|
1036
|
+
// Turn the fish left - affects head, front body, and mid body
|
|
1037
|
+
[FishAction.TURN_LEFT]: [
|
|
1038
|
+
{ node: 'HEAD', channel: 'ry', scale: 1, maxDegrees: 30 },
|
|
1039
|
+
{ node: 'BODY_FRONT', channel: 'ry', scale: 1, maxDegrees: 14 },
|
|
1040
|
+
{ node: 'BODY_MID', channel: 'ry', scale: 1, maxDegrees: 5 },
|
|
1041
|
+
],
|
|
1042
|
+
|
|
1043
|
+
// Tail sweep left - cascading motion through tail bones
|
|
1044
|
+
[FishAction.TAIL_SWEEP_LEFT]: [
|
|
1045
|
+
{ node: 'BODY_BACK', channel: 'rz', scale: 1, maxDegrees: 15 },
|
|
1046
|
+
{ node: 'TAIL_BASE', channel: 'rz', scale: 1, maxDegrees: 30 },
|
|
1047
|
+
{ node: 'TAIL_TOP', channel: 'rz', scale: 1, maxDegrees: 20 },
|
|
1048
|
+
{ node: 'TAIL_MID', channel: 'rz', scale: 1, maxDegrees: 20 },
|
|
1049
|
+
],
|
|
1050
|
+
|
|
1051
|
+
// Pectoral fin movements
|
|
1052
|
+
[FishAction.PECTORAL_L_UP]: [
|
|
1053
|
+
{ node: 'PECTORAL_L', channel: 'rz', scale: 1, maxDegrees: 40 },
|
|
1054
|
+
{ node: 'PECTORAL_L_MID', channel: 'rz', scale: 1, maxDegrees: 20 },
|
|
1055
|
+
],
|
|
1056
|
+
|
|
1057
|
+
// Eye rotation
|
|
1058
|
+
[FishAction.EYE_LEFT]: [
|
|
1059
|
+
{ node: 'EYE_L', channel: 'ry', scale: 1, maxDegrees: 25 },
|
|
1060
|
+
],
|
|
1061
|
+
};
|
|
1062
|
+
```
|
|
1063
|
+
|
|
1064
|
+
### Composite rotations for multi-axis control
|
|
1065
|
+
|
|
1066
|
+
Define how multiple AUs combine for smooth rotation:
|
|
1067
|
+
|
|
1068
|
+
```typescript
|
|
1069
|
+
export const FISH_COMPOSITE_ROTATIONS: CompositeRotation[] = [
|
|
1070
|
+
{
|
|
1071
|
+
node: 'HEAD',
|
|
1072
|
+
pitch: {
|
|
1073
|
+
aus: [FishAction.PITCH_UP, FishAction.PITCH_DOWN],
|
|
1074
|
+
axis: 'rx',
|
|
1075
|
+
negative: FishAction.PITCH_DOWN,
|
|
1076
|
+
positive: FishAction.PITCH_UP
|
|
1077
|
+
},
|
|
1078
|
+
yaw: {
|
|
1079
|
+
aus: [FishAction.TURN_LEFT, FishAction.TURN_RIGHT],
|
|
1080
|
+
axis: 'ry',
|
|
1081
|
+
negative: FishAction.TURN_LEFT,
|
|
1082
|
+
positive: FishAction.TURN_RIGHT
|
|
1083
|
+
},
|
|
1084
|
+
roll: null,
|
|
1085
|
+
},
|
|
1086
|
+
{
|
|
1087
|
+
node: 'TAIL_BASE',
|
|
1088
|
+
pitch: null,
|
|
1089
|
+
yaw: null,
|
|
1090
|
+
roll: {
|
|
1091
|
+
aus: [FishAction.TAIL_SWEEP_LEFT, FishAction.TAIL_SWEEP_RIGHT],
|
|
1092
|
+
axis: 'rz',
|
|
1093
|
+
negative: FishAction.TAIL_SWEEP_RIGHT,
|
|
1094
|
+
positive: FishAction.TAIL_SWEEP_LEFT
|
|
1095
|
+
},
|
|
1096
|
+
},
|
|
1097
|
+
{
|
|
1098
|
+
node: 'EYE_L',
|
|
1099
|
+
pitch: {
|
|
1100
|
+
aus: [FishAction.EYE_UP, FishAction.EYE_DOWN],
|
|
1101
|
+
axis: 'rx',
|
|
1102
|
+
negative: FishAction.EYE_DOWN,
|
|
1103
|
+
positive: FishAction.EYE_UP
|
|
1104
|
+
},
|
|
1105
|
+
yaw: {
|
|
1106
|
+
aus: [FishAction.EYE_LEFT, FishAction.EYE_RIGHT],
|
|
1107
|
+
axis: 'ry',
|
|
1108
|
+
negative: FishAction.EYE_RIGHT,
|
|
1109
|
+
positive: FishAction.EYE_LEFT
|
|
1110
|
+
},
|
|
1111
|
+
roll: null,
|
|
1112
|
+
},
|
|
1113
|
+
];
|
|
1114
|
+
```
|
|
1115
|
+
|
|
1116
|
+
### Action metadata for UI and debugging
|
|
1117
|
+
|
|
1118
|
+
```typescript
|
|
1119
|
+
export const FISH_AU_INFO: Record<string, AUInfo> = {
|
|
1120
|
+
'2': { id: '2', name: 'Turn Left', facePart: 'Body Orientation' },
|
|
1121
|
+
'3': { id: '3', name: 'Turn Right', facePart: 'Body Orientation' },
|
|
1122
|
+
'4': { id: '4', name: 'Pitch Up', facePart: 'Body Orientation' },
|
|
1123
|
+
'5': { id: '5', name: 'Pitch Down', facePart: 'Body Orientation' },
|
|
1124
|
+
'12': { id: '12', name: 'Tail Sweep Left', facePart: 'Tail' },
|
|
1125
|
+
'13': { id: '13', name: 'Tail Sweep Right', facePart: 'Tail' },
|
|
1126
|
+
'20': { id: '20', name: 'Pectoral L Up', facePart: 'Pectoral Fins' },
|
|
1127
|
+
'61': { id: '61', name: 'Eyes Left', facePart: 'Eyes' },
|
|
1128
|
+
// ... more actions
|
|
1129
|
+
};
|
|
1130
|
+
```
|
|
1131
|
+
|
|
1132
|
+
### Continuum pairs for bidirectional sliders
|
|
1133
|
+
|
|
1134
|
+
```typescript
|
|
1135
|
+
export const FISH_CONTINUUM_PAIRS_MAP: Record<number, {
|
|
1136
|
+
pairId: number;
|
|
1137
|
+
isNegative: boolean;
|
|
1138
|
+
axis: 'pitch' | 'yaw' | 'roll';
|
|
1139
|
+
node: string;
|
|
1140
|
+
}> = {
|
|
1141
|
+
[FishAction.TURN_LEFT]: {
|
|
1142
|
+
pairId: FishAction.TURN_RIGHT,
|
|
1143
|
+
isNegative: true,
|
|
1144
|
+
axis: 'yaw',
|
|
1145
|
+
node: 'HEAD'
|
|
1146
|
+
},
|
|
1147
|
+
[FishAction.TURN_RIGHT]: {
|
|
1148
|
+
pairId: FishAction.TURN_LEFT,
|
|
1149
|
+
isNegative: false,
|
|
1150
|
+
axis: 'yaw',
|
|
1151
|
+
node: 'HEAD'
|
|
1152
|
+
},
|
|
1153
|
+
[FishAction.TAIL_SWEEP_LEFT]: {
|
|
1154
|
+
pairId: FishAction.TAIL_SWEEP_RIGHT,
|
|
1155
|
+
isNegative: true,
|
|
1156
|
+
axis: 'roll',
|
|
1157
|
+
node: 'TAIL_BASE'
|
|
1158
|
+
},
|
|
1159
|
+
// ... more pairs
|
|
1160
|
+
};
|
|
1161
|
+
```
|
|
1162
|
+
|
|
1163
|
+
### Creating the final preset config
|
|
1164
|
+
|
|
1165
|
+
```typescript
|
|
1166
|
+
export const FISH_AU_MAPPING_CONFIG = {
|
|
1167
|
+
auToBones: FISH_BONE_BINDINGS,
|
|
1168
|
+
boneNodes: FISH_BONE_NODES,
|
|
1169
|
+
auToMorphs: {} as Record<number, { left: string[]; right: string[]; center: string[] }>, // No morph targets
|
|
1170
|
+
morphToMesh: {} as Record<string, string[]>,
|
|
1171
|
+
visemeKeys: [] as string[], // Fish don't speak!
|
|
1172
|
+
auInfo: FISH_AU_INFO,
|
|
1173
|
+
compositeRotations: FISH_COMPOSITE_ROTATIONS,
|
|
1174
|
+
eyeMeshNodes: { LEFT: 'EYES_0', RIGHT: 'EYES_0' },
|
|
1175
|
+
};
|
|
1176
|
+
```
|
|
1177
|
+
|
|
1178
|
+
### Using the fish preset
|
|
1179
|
+
|
|
1180
|
+
```typescript
|
|
1181
|
+
import { Embody } from '@lovelace_lol/embody';
|
|
1182
|
+
import { FISH_AU_MAPPING_CONFIG, FishAction } from './presets/bettaFish';
|
|
1183
|
+
|
|
1184
|
+
const fishController = new Embody({
|
|
1185
|
+
profile: FISH_AU_MAPPING_CONFIG
|
|
1186
|
+
});
|
|
1187
|
+
|
|
1188
|
+
// Load the fish model
|
|
1189
|
+
loader.load('/characters/betta/scene.gltf', (gltf) => {
|
|
1190
|
+
const meshes = collectMorphMeshes(gltf.scene); // Will be empty for fish
|
|
1191
|
+
fishController.onReady({ meshes, model: gltf.scene });
|
|
1192
|
+
|
|
1193
|
+
// Control the fish!
|
|
1194
|
+
fishController.setAU(FishAction.TURN_LEFT, 0.5); // Turn left
|
|
1195
|
+
fishController.setAU(FishAction.TAIL_SWEEP_LEFT, 0.8); // Sweep tail
|
|
1196
|
+
fishController.setAU(FishAction.PECTORAL_L_UP, 0.6); // Raise left fin
|
|
1197
|
+
|
|
1198
|
+
// Smooth transitions
|
|
1199
|
+
await fishController.transitionAU(FishAction.TURN_RIGHT, 1.0, 500).promise;
|
|
1200
|
+
});
|
|
1201
|
+
```
|
|
1202
|
+
|
|
1203
|
+
### Creating swimming animations
|
|
1204
|
+
|
|
1205
|
+
Use continuum controls for natural swimming motion:
|
|
1206
|
+
|
|
1207
|
+
```typescript
|
|
1208
|
+
// Use setContinuum for paired actions
|
|
1209
|
+
fishController.setContinuum(
|
|
1210
|
+
FishAction.TURN_LEFT,
|
|
1211
|
+
FishAction.TURN_RIGHT,
|
|
1212
|
+
0.3 // Slight turn right
|
|
1213
|
+
);
|
|
1214
|
+
|
|
1215
|
+
// Animate swimming with oscillating tail
|
|
1216
|
+
async function swimCycle() {
|
|
1217
|
+
while (true) {
|
|
1218
|
+
await fishController.transitionContinuum(
|
|
1219
|
+
FishAction.TAIL_SWEEP_LEFT,
|
|
1220
|
+
FishAction.TAIL_SWEEP_RIGHT,
|
|
1221
|
+
0.8, // Sweep right
|
|
1222
|
+
300
|
|
1223
|
+
).promise;
|
|
1224
|
+
|
|
1225
|
+
await fishController.transitionContinuum(
|
|
1226
|
+
FishAction.TAIL_SWEEP_LEFT,
|
|
1227
|
+
FishAction.TAIL_SWEEP_RIGHT,
|
|
1228
|
+
-0.8, // Sweep left
|
|
1229
|
+
300
|
|
1230
|
+
).promise;
|
|
1231
|
+
}
|
|
1232
|
+
}
|
|
1233
|
+
```
|
|
1234
|
+
|
|
1235
|
+

|
|
1236
|
+
|
|
1237
|
+
---
|
|
1238
|
+
|
|
1239
|
+
## 7. Action Unit Control
|
|
1240
|
+
|
|
1241
|
+
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units)
|
|
1242
|
+
|
|
1243
|
+

|
|
1244
|
+
|
|
1245
|
+
Action Units are the core of FACS. Each AU represents a specific muscular movement of the face.
|
|
1246
|
+
|
|
1247
|
+
### Setting an AU immediately
|
|
1248
|
+
|
|
1249
|
+
```typescript
|
|
1250
|
+
// Set AU12 (smile) to 80% intensity
|
|
1251
|
+
loom.setAU(12, 0.8);
|
|
1252
|
+
|
|
1253
|
+
// Set AU45 (blink) to full intensity
|
|
1254
|
+
loom.setAU(45, 1.0);
|
|
1255
|
+
|
|
1256
|
+
// Set to 0 to deactivate
|
|
1257
|
+
loom.setAU(12, 0);
|
|
1258
|
+
```
|
|
1259
|
+
|
|
1260
|
+
### Transitioning an AU over time
|
|
1261
|
+
|
|
1262
|
+
```typescript
|
|
1263
|
+
// Animate AU12 to 0.8 over 200ms
|
|
1264
|
+
const handle = loom.transitionAU(12, 0.8, 200);
|
|
1265
|
+
|
|
1266
|
+
// Wait for completion
|
|
1267
|
+
await handle.promise;
|
|
1268
|
+
|
|
1269
|
+
// Or chain transitions
|
|
1270
|
+
loom.transitionAU(12, 1.0, 200).promise.then(() => {
|
|
1271
|
+
loom.transitionAU(12, 0, 300); // Fade out
|
|
1272
|
+
});
|
|
1273
|
+
```
|
|
1274
|
+
|
|
1275
|
+
### Getting the current AU value
|
|
1276
|
+
|
|
1277
|
+
```typescript
|
|
1278
|
+
const smileAmount = loom.getAU(12);
|
|
1279
|
+
console.log(`Current smile: ${smileAmount}`);
|
|
1280
|
+
```
|
|
1281
|
+
|
|
1282
|
+
### Asymmetric control with balance
|
|
1283
|
+
|
|
1284
|
+
Many AUs have left and right variants (e.g., `Mouth_Smile_L` and `Mouth_Smile_R`). The `balance` parameter lets you control them independently:
|
|
1285
|
+
|
|
1286
|
+
```typescript
|
|
1287
|
+
// Balance range: -1 (left only) to +1 (right only), 0 = both equal
|
|
1288
|
+
|
|
1289
|
+
// Smile on both sides equally
|
|
1290
|
+
loom.setAU(12, 0.8, 0);
|
|
1291
|
+
|
|
1292
|
+
// Smile only on left side
|
|
1293
|
+
loom.setAU(12, 0.8, -1);
|
|
1294
|
+
|
|
1295
|
+
// Smile only on right side
|
|
1296
|
+
loom.setAU(12, 0.8, 1);
|
|
1297
|
+
|
|
1298
|
+
// 70% left, 30% right
|
|
1299
|
+
loom.setAU(12, 0.8, -0.4);
|
|
1300
|
+
```
|
|
1301
|
+
|
|
1302
|
+
### String-based side selection
|
|
1303
|
+
|
|
1304
|
+
You can also specify the side directly in the AU ID:
|
|
1305
|
+
|
|
1306
|
+
```typescript
|
|
1307
|
+
// These are equivalent:
|
|
1308
|
+
loom.setAU('12L', 0.8); // Left side only
|
|
1309
|
+
loom.setAU(12, 0.8, -1); // Left side only
|
|
1310
|
+
|
|
1311
|
+
loom.setAU('12R', 0.8); // Right side only
|
|
1312
|
+
loom.setAU(12, 0.8, 1); // Right side only
|
|
1313
|
+
```
|
|
1314
|
+
|
|
1315
|
+
---
|
|
1316
|
+
|
|
1317
|
+
## 8. Mix Weight System
|
|
1318
|
+
|
|
1319
|
+
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units)
|
|
1320
|
+
|
|
1321
|
+

|
|
1322
|
+
|
|
1323
|
+
Some AUs can be driven by both morph targets (blend shapes) AND bone rotations. The mix weight controls the blend between them.
|
|
1324
|
+
|
|
1325
|
+
### Why mix weights?
|
|
1326
|
+
|
|
1327
|
+
Take jaw opening (AU26) as an example:
|
|
1328
|
+
- **Morph-only (weight 0)**: Vertices deform to show open mouth, but jaw bone doesn't move
|
|
1329
|
+
- **Bone-only (weight 1)**: Jaw bone rotates down, but no soft tissue deformation
|
|
1330
|
+
- **Mixed (weight 0.5)**: Both contribute equally for realistic results
|
|
1331
|
+
|
|
1332
|
+
### Setting mix weights
|
|
1333
|
+
|
|
1334
|
+
```typescript
|
|
1335
|
+
// Get the default mix weight for AU26
|
|
1336
|
+
const weight = loom.getAUMixWeight(26); // e.g., 0.5
|
|
1337
|
+
|
|
1338
|
+
// Set to pure morph
|
|
1339
|
+
loom.setAUMixWeight(26, 0);
|
|
1340
|
+
|
|
1341
|
+
// Set to pure bone
|
|
1342
|
+
loom.setAUMixWeight(26, 1);
|
|
1343
|
+
|
|
1344
|
+
// Set to 70% bone, 30% morph
|
|
1345
|
+
loom.setAUMixWeight(26, 0.7);
|
|
1346
|
+
```
|
|
1347
|
+
|
|
1348
|
+
### Which AUs support mixing?
|
|
1349
|
+
|
|
1350
|
+
Only AUs that have both `auToMorphs` AND `auToBones` entries support mixing. Common examples:
|
|
1351
|
+
- AU26 (Jaw Drop)
|
|
1352
|
+
- AU27 (Mouth Stretch)
|
|
1353
|
+
- AU51-56 (Head movements)
|
|
1354
|
+
- AU61-72 (Shared + independent eye movements)
|
|
1355
|
+
|
|
1356
|
+
```typescript
|
|
1357
|
+
import { isMixedAU } from '@lovelace_lol/embody';
|
|
1358
|
+
|
|
1359
|
+
if (isMixedAU(26)) {
|
|
1360
|
+
console.log('AU26 supports morph/bone mixing');
|
|
1361
|
+
}
|
|
1362
|
+
```
|
|
1363
|
+
|
|
1364
|
+
---
|
|
1365
|
+
|
|
1366
|
+
## 9. Composite Rotation System
|
|
1367
|
+
|
|
1368
|
+
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units) | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones)
|
|
1369
|
+
|
|
1370
|
+

|
|
1371
|
+
|
|
1372
|
+
Bones like the head and eyes need multi-axis rotation (pitch, yaw, roll). The composite rotation system handles this automatically.
|
|
1373
|
+
|
|
1374
|
+
### How it works
|
|
1375
|
+
|
|
1376
|
+
When you set an AU that affects a bone rotation, Embody:
|
|
1377
|
+
1. Queues the rotation update in `pendingCompositeNodes`
|
|
1378
|
+
2. At the end of `update()`, calls `flushPendingComposites()`
|
|
1379
|
+
3. Applies all three axes (pitch, yaw, roll) together to prevent gimbal issues
|
|
1380
|
+
|
|
1381
|
+
### Supported bones and their axes
|
|
1382
|
+
|
|
1383
|
+
| Bone | Pitch (X) | Yaw (Y) | Roll (Z) |
|
|
1384
|
+
|------|-----------|---------|----------|
|
|
1385
|
+
| HEAD | AU53 (up) / AU54 (down) | AU51 (left) / AU52 (right) | AU55 (tilt left) / AU56 (tilt right) |
|
|
1386
|
+
| EYE_L | AU63 (up) / AU64 (down) | AU61 (left) / AU62 (right) | - |
|
|
1387
|
+
| EYE_R | AU63 (up) / AU64 (down) | AU61 (left) / AU62 (right) | - |
|
|
1388
|
+
| JAW | AU25-27 (open) | AU30 (left) / AU35 (right) | - |
|
|
1389
|
+
| TONGUE | AU37 (up) / AU38 (down) | AU39 (left) / AU40 (right) | AU41 / AU42 (tilt) |
|
|
1390
|
+
|
|
1391
|
+
### Example: Moving the head
|
|
1392
|
+
|
|
1393
|
+
```typescript
|
|
1394
|
+
// Turn head left 50%
|
|
1395
|
+
loom.setAU(51, 0.5);
|
|
1396
|
+
|
|
1397
|
+
// Turn head right 50%
|
|
1398
|
+
loom.setAU(52, 0.5);
|
|
1399
|
+
|
|
1400
|
+
// Tilt head up 30%
|
|
1401
|
+
loom.setAU(53, 0.3);
|
|
1402
|
+
|
|
1403
|
+
// Combine: turn left AND tilt up
|
|
1404
|
+
loom.setAU(51, 0.5);
|
|
1405
|
+
loom.setAU(53, 0.3);
|
|
1406
|
+
// Both are applied together in a single composite rotation
|
|
1407
|
+
```
|
|
1408
|
+
|
|
1409
|
+
### Example: Eye gaze
|
|
1410
|
+
|
|
1411
|
+
```typescript
|
|
1412
|
+
// Look left
|
|
1413
|
+
loom.setAU(61, 0.7);
|
|
1414
|
+
|
|
1415
|
+
// Look right
|
|
1416
|
+
loom.setAU(62, 0.7);
|
|
1417
|
+
|
|
1418
|
+
// Look up
|
|
1419
|
+
loom.setAU(63, 0.5);
|
|
1420
|
+
|
|
1421
|
+
// Look down-right (combined)
|
|
1422
|
+
loom.setAU(62, 0.6);
|
|
1423
|
+
loom.setAU(64, 0.4);
|
|
1424
|
+
```
|
|
1425
|
+
|
|
1426
|
+
---
|
|
1427
|
+
|
|
1428
|
+
## 10. Continuum Pairs
|
|
1429
|
+
|
|
1430
|
+
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units)
|
|
1431
|
+
|
|
1432
|
+

|
|
1433
|
+
|
|
1434
|
+
Continuum pairs are bidirectional AU pairs that represent opposite directions on the same axis. They're linked so that activating one should deactivate the other.
|
|
1435
|
+
|
|
1436
|
+
### Pair mappings
|
|
1437
|
+
|
|
1438
|
+
| Pair | Description |
|
|
1439
|
+
|------|-------------|
|
|
1440
|
+
| AU51 ↔ AU52 | Head turn left / right |
|
|
1441
|
+
| AU53 ↔ AU54 | Head up / down |
|
|
1442
|
+
| AU55 ↔ AU56 | Head tilt left / right |
|
|
1443
|
+
| AU61 ↔ AU62 | Eyes look left / right |
|
|
1444
|
+
| AU63 ↔ AU64 | Eyes look up / down |
|
|
1445
|
+
| AU30 ↔ AU35 | Jaw shift left / right |
|
|
1446
|
+
| AU37 ↔ AU38 | Tongue up / down |
|
|
1447
|
+
| AU39 ↔ AU40 | Tongue left / right |
|
|
1448
|
+
| AU73 ↔ AU74 | Tongue narrow / wide |
|
|
1449
|
+
| AU76 ↔ AU77 | Tongue tip up / down |
|
|
1450
|
+
|
|
1451
|
+
### Negative value shorthand (recommended)
|
|
1452
|
+
|
|
1453
|
+
The simplest way to work with continuum pairs is using **negative values**. When you pass a negative value to `setAU()` or `transitionAU()`, the engine automatically activates the paired AU instead:
|
|
1454
|
+
|
|
1455
|
+
```typescript
|
|
1456
|
+
// Head looking left at 50% (AU51 is "head left")
|
|
1457
|
+
loom.setAU(51, 0.5);
|
|
1458
|
+
|
|
1459
|
+
// Head looking right at 50% - just use a negative value!
|
|
1460
|
+
loom.setAU(51, -0.5); // Automatically activates AU52 at 0.5
|
|
1461
|
+
|
|
1462
|
+
// This is equivalent to manually setting the pair:
|
|
1463
|
+
loom.setAU(51, 0);
|
|
1464
|
+
loom.setAU(52, 0.5);
|
|
1465
|
+
```
|
|
1466
|
+
|
|
1467
|
+
This works for transitions too:
|
|
1468
|
+
|
|
1469
|
+
```typescript
|
|
1470
|
+
// Animate head from left to right over 500ms
|
|
1471
|
+
await loom.transitionAU(51, 0.5, 250).promise; // Turn left
|
|
1472
|
+
await loom.transitionAU(51, -0.5, 500).promise; // Turn right (activates AU52)
|
|
1473
|
+
```
|
|
1474
|
+
|
|
1475
|
+
### The setContinuum method
|
|
1476
|
+
|
|
1477
|
+
For explicit continuum control, use `setContinuum()` with a single value from -1 to +1:
|
|
1478
|
+
|
|
1479
|
+
```typescript
|
|
1480
|
+
// setContinuum(negativeAU, positiveAU, value)
|
|
1481
|
+
// value: -1 = full negative, 0 = neutral, +1 = full positive
|
|
1482
|
+
|
|
1483
|
+
// Head centered
|
|
1484
|
+
loom.setContinuum(51, 52, 0);
|
|
1485
|
+
|
|
1486
|
+
// Head 50% left
|
|
1487
|
+
loom.setContinuum(51, 52, -0.5);
|
|
1488
|
+
|
|
1489
|
+
// Head 70% right
|
|
1490
|
+
loom.setContinuum(51, 52, 0.7);
|
|
1491
|
+
```
|
|
1492
|
+
|
|
1493
|
+
With smooth animation:
|
|
1494
|
+
|
|
1495
|
+
```typescript
|
|
1496
|
+
// Animate head from current position to 80% right over 300ms
|
|
1497
|
+
await loom.transitionContinuum(51, 52, 0.8, 300).promise;
|
|
1498
|
+
|
|
1499
|
+
// Animate eyes to look left over 200ms
|
|
1500
|
+
await loom.transitionContinuum(61, 62, -0.6, 200).promise;
|
|
1501
|
+
```
|
|
1502
|
+
|
|
1503
|
+
### Manual pair management
|
|
1504
|
+
|
|
1505
|
+
You can also manually manage pairs by setting each AU individually:
|
|
1506
|
+
|
|
1507
|
+
```typescript
|
|
1508
|
+
// Head looking left at 50%
|
|
1509
|
+
loom.setAU(51, 0.5);
|
|
1510
|
+
loom.setAU(52, 0); // Right should be 0
|
|
1511
|
+
|
|
1512
|
+
// Head looking right at 70%
|
|
1513
|
+
loom.setAU(51, 0); // Left should be 0
|
|
1514
|
+
loom.setAU(52, 0.7);
|
|
1515
|
+
```
|
|
1516
|
+
|
|
1517
|
+
### The CONTINUUM_PAIRS_MAP
|
|
1518
|
+
|
|
1519
|
+
You can access pair information programmatically:
|
|
1520
|
+
|
|
1521
|
+
```typescript
|
|
1522
|
+
import { CONTINUUM_PAIRS_MAP } from '@lovelace_lol/embody';
|
|
1523
|
+
|
|
1524
|
+
const pair = CONTINUUM_PAIRS_MAP[51];
|
|
1525
|
+
// { pairId: 52, isNegative: true, axis: 'yaw', node: 'HEAD' }
|
|
1526
|
+
```
|
|
1527
|
+
|
|
1528
|
+
---
|
|
1529
|
+
|
|
1530
|
+
## 11. Direct Morph Control
|
|
1531
|
+
|
|
1532
|
+
Open in LoomLarge: [Meshes tab](https://www.characterloom.com/?drawer=open&tab=meshes) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
1533
|
+
|
|
1534
|
+

|
|
1535
|
+
|
|
1536
|
+
Sometimes you need to control morph targets directly by name, bypassing the AU system.
|
|
1537
|
+
|
|
1538
|
+
### Setting a morph immediately
|
|
1539
|
+
|
|
1540
|
+
```typescript
|
|
1541
|
+
// Set a specific morph to 50%
|
|
1542
|
+
loom.setMorph('Mouth_Smile_L', 0.5);
|
|
1543
|
+
|
|
1544
|
+
// Set on specific meshes only
|
|
1545
|
+
loom.setMorph('Mouth_Smile_L', 0.5, ['CC_Base_Body']);
|
|
1546
|
+
```
|
|
1547
|
+
|
|
1548
|
+
### Transitioning a morph
|
|
1549
|
+
|
|
1550
|
+
```typescript
|
|
1551
|
+
// Animate morph over 200ms
|
|
1552
|
+
const handle = loom.transitionMorph('Mouth_Smile_L', 0.8, 200);
|
|
1553
|
+
|
|
1554
|
+
// With mesh targeting
|
|
1555
|
+
loom.transitionMorph('Eye_Blink_L', 1.0, 100, ['CC_Base_Body']);
|
|
1556
|
+
|
|
1557
|
+
// Wait for completion
|
|
1558
|
+
await handle.promise;
|
|
1559
|
+
```
|
|
1560
|
+
|
|
1561
|
+
### Resolving current morph targets
|
|
1562
|
+
|
|
1563
|
+
```typescript
|
|
1564
|
+
const targets = loom.resolveMorphTargets('Mouth_Smile_L', ['CC_Base_Body']);
|
|
1565
|
+
const value = targets.length > 0 ? (targets[0].infl[targets[0].idx] ?? 0) : 0;
|
|
1566
|
+
```
|
|
1567
|
+
|
|
1568
|
+
### Adding runtime morph targets
|
|
1569
|
+
|
|
1570
|
+
Generated or sidecar morph targets can be registered after a model loads. Deltas use the same relative `POSITION` format as glTF morph targets: one XYZ delta per base mesh vertex. Optional `normal` and `tangent` deltas can be supplied when available.
|
|
1571
|
+
|
|
1572
|
+
```typescript
|
|
1573
|
+
const index = loom.addMorphTarget({
|
|
1574
|
+
meshName: 'CC_Base_Body',
|
|
1575
|
+
name: 'BodyType_Muscular',
|
|
1576
|
+
position: bodyTypeMuscularDeltas,
|
|
1577
|
+
});
|
|
1578
|
+
|
|
1579
|
+
loom.setMorphInfluence(index, 0.6, ['CC_Base_Body']);
|
|
1580
|
+
loom.setMorph('BodyType_Muscular', 0.6, ['CC_Base_Body']);
|
|
1581
|
+
```
|
|
1582
|
+
|
|
1583
|
+
By default, Embody replaces and disposes the mesh `BufferGeometry` before appending morph attributes. This is intentional: Three.js does not support mutating `geometry.morphAttributes` in place after a geometry has rendered. For pre-render authoring paths, pass `{ forceGeometryReplacement: false }`.
|
|
1584
|
+
|
|
1585
|
+
If you need a named slot before real deltas are available, use `ensureMorphInfluence(meshName, morphName)`. It creates a zero-delta target and returns the assigned `morphTargetInfluences` index. After external code changes morph dictionaries or geometry, call `refreshMorphTargets()` so AU, viseme, hair, and clip-building caches see the updated targets.
|
|
1586
|
+
|
|
1587
|
+
### Morph caching
|
|
1588
|
+
|
|
1589
|
+
Embody caches morph target lookups for performance. The first time you access a morph, it searches all meshes and caches the index. Subsequent accesses are O(1).
|
|
1590
|
+
|
|
1591
|
+
---
|
|
1592
|
+
|
|
1593
|
+
## 12. Viseme System
|
|
1594
|
+
|
|
1595
|
+
Open in LoomLarge: [Visemes tab](https://www.characterloom.com/?drawer=open&tab=visemes) | [Speech tab](https://www.characterloom.com/?drawer=open&tab=speech)
|
|
1596
|
+
|
|
1597
|
+

|
|
1598
|
+
|
|
1599
|
+
This screenshot was captured before the viseme label refresh, so some cards still show legacy names such as `Er`, `IH`, and `W_OO`. Treat the table below as the source of truth for the current exported `VISEME_KEYS` order.
|
|
1600
|
+
|
|
1601
|
+
Visemes are mouth shapes used for lip-sync. Embody includes 15 visemes with automatic jaw coupling.
|
|
1602
|
+
|
|
1603
|
+
### The 15 visemes
|
|
1604
|
+
|
|
1605
|
+
The `VISEME_KEYS` export uses unprefixed keys in this order.
|
|
1606
|
+
|
|
1607
|
+
| Index | Key | Phoneme Example |
|
|
1608
|
+
|-------|-----|-----------------|
|
|
1609
|
+
| 0 | EE | "b**ee**" |
|
|
1610
|
+
| 1 | Ah | "f**a**ther" |
|
|
1611
|
+
| 2 | Oh | "g**o**" |
|
|
1612
|
+
| 3 | OO | "t**oo**" |
|
|
1613
|
+
| 4 | I | "s**i**t" |
|
|
1614
|
+
| 5 | U | "fl**u**te" |
|
|
1615
|
+
| 6 | W | "**w**e" |
|
|
1616
|
+
| 7 | L | "**l**ip" |
|
|
1617
|
+
| 8 | F_V | "**f**un, **v**an" |
|
|
1618
|
+
| 9 | Th | "**th**ink" |
|
|
1619
|
+
| 10 | S_Z | "**s**un, **z**oo" |
|
|
1620
|
+
| 11 | B_M_P | "**b**at, **m**an, **p**op" |
|
|
1621
|
+
| 12 | K_G_H_NG | "**k**ite, **g**o, **h**at, si**ng**" |
|
|
1622
|
+
| 13 | AE | "c**a**t" |
|
|
1623
|
+
| 14 | R | "**r**ed" |
|
|
1624
|
+
|
|
1625
|
+
### Setting a viseme
|
|
1626
|
+
|
|
1627
|
+
```typescript
|
|
1628
|
+
// Set viseme 3 (Ah) to full intensity
|
|
1629
|
+
loom.setViseme(3, 1.0);
|
|
1630
|
+
|
|
1631
|
+
// With jaw scale (0-1, default 1)
|
|
1632
|
+
loom.setViseme(3, 1.0, 0.5); // Half jaw opening
|
|
1633
|
+
```
|
|
1634
|
+
|
|
1635
|
+
### Transitioning visemes
|
|
1636
|
+
|
|
1637
|
+
Viseme transitions default to 80ms and use the standard `easeInOutQuad` easing when no duration is provided.
|
|
1638
|
+
|
|
1639
|
+
```typescript
|
|
1640
|
+
// Animate to a viseme using the default 80ms duration
|
|
1641
|
+
const handle = loom.transitionViseme(3, 1.0);
|
|
1642
|
+
|
|
1643
|
+
// Disable jaw coupling (duration can be omitted to use the 80ms default)
|
|
1644
|
+
loom.transitionViseme(3, 1.0, 80, 0);
|
|
1645
|
+
```
|
|
1646
|
+
|
|
1647
|
+
### Automatic jaw coupling
|
|
1648
|
+
|
|
1649
|
+
Each viseme has a predefined jaw opening amount in the preset. When you set a viseme, the jaw automatically opens proportionally, and the `jawScale` parameter multiplies that amount:
|
|
1650
|
+
- `jawScale = 1.0`: Normal jaw opening
|
|
1651
|
+
- `jawScale = 0.5`: Half jaw opening
|
|
1652
|
+
- `jawScale = 0`: No jaw movement (viseme only)
|
|
1653
|
+
|
|
1654
|
+
### Lip-sync example
|
|
1655
|
+
|
|
1656
|
+
```typescript
|
|
1657
|
+
async function speak(phonemes: number[]) {
|
|
1658
|
+
for (const viseme of phonemes) {
|
|
1659
|
+
// Clear previous viseme
|
|
1660
|
+
for (let i = 0; i < 15; i++) loom.setViseme(i, 0);
|
|
1661
|
+
|
|
1662
|
+
// Transition to new viseme
|
|
1663
|
+
await loom.transitionViseme(viseme, 1.0, 80).promise;
|
|
1664
|
+
|
|
1665
|
+
// Hold briefly
|
|
1666
|
+
await new Promise(r => setTimeout(r, 100));
|
|
1667
|
+
}
|
|
1668
|
+
|
|
1669
|
+
// Return to neutral
|
|
1670
|
+
for (let i = 0; i < 15; i++) loom.setViseme(i, 0);
|
|
1671
|
+
}
|
|
1672
|
+
|
|
1673
|
+
// "Hello" approximation
|
|
1674
|
+
speak([5, 0, 10, 4]);
|
|
1675
|
+
```
|
|
1676
|
+
|
|
1677
|
+
---
|
|
1678
|
+
|
|
1679
|
+
## 13. Transition System
|
|
1680
|
+
|
|
1681
|
+
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
1682
|
+
|
|
1683
|
+

|
|
1684
|
+
|
|
1685
|
+
All animated changes in Embody go through the transition system, which provides smooth interpolation with easing.
|
|
1686
|
+
|
|
1687
|
+
### TransitionHandle
|
|
1688
|
+
|
|
1689
|
+
Every transition method returns a `TransitionHandle`:
|
|
1690
|
+
|
|
1691
|
+
```typescript
|
|
1692
|
+
interface TransitionHandle {
|
|
1693
|
+
promise: Promise<void>; // Resolves when transition completes
|
|
1694
|
+
pause(): void; // Pause this transition
|
|
1695
|
+
resume(): void; // Resume this transition
|
|
1696
|
+
cancel(): void; // Cancel immediately
|
|
1697
|
+
}
|
|
1698
|
+
```
|
|
1699
|
+
|
|
1700
|
+
### Using handles
|
|
1701
|
+
|
|
1702
|
+
```typescript
|
|
1703
|
+
// Start a transition
|
|
1704
|
+
const handle = loom.transitionAU(12, 1.0, 500);
|
|
1705
|
+
|
|
1706
|
+
// Pause it
|
|
1707
|
+
handle.pause();
|
|
1708
|
+
|
|
1709
|
+
// Resume later
|
|
1710
|
+
handle.resume();
|
|
1711
|
+
|
|
1712
|
+
// Or cancel entirely
|
|
1713
|
+
handle.cancel();
|
|
1714
|
+
|
|
1715
|
+
// Wait for completion
|
|
1716
|
+
await handle.promise;
|
|
1717
|
+
```
|
|
1718
|
+
|
|
1719
|
+
### Combining multiple transitions
|
|
1720
|
+
|
|
1721
|
+
When you call `transitionAU`, it may create multiple internal transitions (one per morph target). The returned handle controls all of them:
|
|
1722
|
+
|
|
1723
|
+
```typescript
|
|
1724
|
+
// AU12 might affect Mouth_Smile_L and Mouth_Smile_R
|
|
1725
|
+
const handle = loom.transitionAU(12, 1.0, 200);
|
|
1726
|
+
|
|
1727
|
+
// Pausing the handle pauses both morph transitions
|
|
1728
|
+
handle.pause();
|
|
1729
|
+
```
|
|
1730
|
+
|
|
1731
|
+
### Easing
|
|
1732
|
+
|
|
1733
|
+
The default easing is `easeInOutQuad`. Custom easing can be provided when using the Animation system directly:
|
|
1734
|
+
|
|
1735
|
+
```typescript
|
|
1736
|
+
// The ThreeAnimationRuntime class supports custom easing
|
|
1737
|
+
animation.addTransition(
|
|
1738
|
+
'custom',
|
|
1739
|
+
0,
|
|
1740
|
+
1,
|
|
1741
|
+
200,
|
|
1742
|
+
(v) => console.log(v),
|
|
1743
|
+
(t) => t * t // Custom ease-in quadratic
|
|
1744
|
+
);
|
|
1745
|
+
```
|
|
1746
|
+
|
|
1747
|
+
### Active transition count
|
|
1748
|
+
|
|
1749
|
+
```typescript
|
|
1750
|
+
const count = loom.getActiveTransitionCount();
|
|
1751
|
+
console.log(`${count} transitions in progress`);
|
|
1752
|
+
```
|
|
1753
|
+
|
|
1754
|
+
### Clearing all transitions
|
|
1755
|
+
|
|
1756
|
+
```typescript
|
|
1757
|
+
// Cancel everything immediately
|
|
1758
|
+
loom.clearTransitions();
|
|
1759
|
+
```
|
|
1760
|
+
|
|
1761
|
+
---
|
|
1762
|
+
|
|
1763
|
+
## 14. Playback & State Control
|
|
1764
|
+
|
|
1765
|
+
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
1766
|
+
|
|
1767
|
+

|
|
1768
|
+
|
|
1769
|
+
### Pausing and resuming
|
|
1770
|
+
|
|
1771
|
+
```typescript
|
|
1772
|
+
// Pause all animation updates
|
|
1773
|
+
loom.pause();
|
|
1774
|
+
|
|
1775
|
+
// Check pause state
|
|
1776
|
+
if (loom.getPaused()) {
|
|
1777
|
+
console.log('Animation is paused');
|
|
1778
|
+
}
|
|
1779
|
+
|
|
1780
|
+
// Resume
|
|
1781
|
+
loom.resume();
|
|
1782
|
+
```
|
|
1783
|
+
|
|
1784
|
+
When paused, `loom.update()` stops processing transitions, but you can still call `setAU()` for immediate changes.
|
|
1785
|
+
|
|
1786
|
+
### Resetting to neutral
|
|
1787
|
+
|
|
1788
|
+
```typescript
|
|
1789
|
+
// Reset everything to rest state
|
|
1790
|
+
loom.resetToNeutral();
|
|
1791
|
+
```
|
|
1792
|
+
|
|
1793
|
+
This:
|
|
1794
|
+
- Clears all AU values to 0
|
|
1795
|
+
- Cancels all active transitions
|
|
1796
|
+
- Resets all morph targets to 0
|
|
1797
|
+
- Returns all bones to their original position/rotation
|
|
1798
|
+
|
|
1799
|
+
### Mesh visibility
|
|
1800
|
+
|
|
1801
|
+
```typescript
|
|
1802
|
+
// Get list of all meshes
|
|
1803
|
+
const meshes = loom.getMeshList();
|
|
1804
|
+
// Returns: [{ name: 'CC_Base_Body', visible: true, morphCount: 80 }, ...]
|
|
1805
|
+
|
|
1806
|
+
// Hide a mesh
|
|
1807
|
+
loom.setMeshVisible('CC_Base_Hair', false);
|
|
1808
|
+
|
|
1809
|
+
// Show it again
|
|
1810
|
+
loom.setMeshVisible('CC_Base_Hair', true);
|
|
1811
|
+
```
|
|
1812
|
+
|
|
1813
|
+
### Cleanup
|
|
1814
|
+
|
|
1815
|
+
```typescript
|
|
1816
|
+
// When done, dispose of resources
|
|
1817
|
+
loom.dispose();
|
|
1818
|
+
```
|
|
1819
|
+
|
|
1820
|
+
---
|
|
1821
|
+
|
|
1822
|
+
## 15. Hair Physics
|
|
1823
|
+
|
|
1824
|
+
Open in LoomLarge: [Hair tab](https://www.characterloom.com/?drawer=open&tab=hair)
|
|
1825
|
+
|
|
1826
|
+

|
|
1827
|
+
|
|
1828
|
+
Embody includes a built-in hair physics system that drives morph targets through the AnimationMixer.
|
|
1829
|
+
It is **mixer-only** (no per-frame morph LERP), and it reacts to **head rotation** coming from AUs.
|
|
1830
|
+
|
|
1831
|
+
### How it works
|
|
1832
|
+
|
|
1833
|
+
Hair motion is decomposed into three clip families:
|
|
1834
|
+
|
|
1835
|
+
1. **Idle/Wind loop** - continuous sway and optional wind.
|
|
1836
|
+
2. **Impulse clips** - short oscillations triggered by *changes* in head yaw/pitch.
|
|
1837
|
+
3. **Gravity clip** - a single clip that is **scrubbed** by head pitch (up/down).
|
|
1838
|
+
|
|
1839
|
+
All clips are created with `buildClip` and applied to the mixer.
|
|
1840
|
+
When you update head AUs (e.g. `setAU`, `setContinuum`, `transitionAU`), hair updates automatically.
|
|
1841
|
+
|
|
1842
|
+
### Basic setup
|
|
1843
|
+
|
|
1844
|
+
```typescript
|
|
1845
|
+
const loom = new Embody({ presetType: 'cc4' });
|
|
1846
|
+
|
|
1847
|
+
loader.load('/character.glb', (gltf) => {
|
|
1848
|
+
const meshes = collectMorphMeshes(gltf.scene);
|
|
1849
|
+
loom.onReady({ meshes, model: gltf.scene });
|
|
1850
|
+
|
|
1851
|
+
// Register hair + eyebrow meshes (filters using CC4_MESHES category tags)
|
|
1852
|
+
const allObjects: Object3D[] = [];
|
|
1853
|
+
gltf.scene.traverse((obj) => allObjects.push(obj));
|
|
1854
|
+
loom.registerHairObjects(allObjects);
|
|
1855
|
+
|
|
1856
|
+
// Enable physics (starts idle + gravity + impulse clips)
|
|
1857
|
+
loom.setHairPhysicsEnabled(true);
|
|
1858
|
+
});
|
|
1859
|
+
```
|
|
1860
|
+
|
|
1861
|
+
### Inspecting registered hair objects
|
|
1862
|
+
|
|
1863
|
+
```typescript
|
|
1864
|
+
const hairObjects = loom.getRegisteredHairObjects();
|
|
1865
|
+
console.log(hairObjects.map((mesh) => mesh.name));
|
|
1866
|
+
```
|
|
1867
|
+
|
|
1868
|
+
### Configuration (profile defaults)
|
|
1869
|
+
|
|
1870
|
+
Hair physics defaults live in the preset/profile and are applied automatically at init:
|
|
1871
|
+
|
|
1872
|
+
```typescript
|
|
1873
|
+
import type { Profile } from '@lovelace_lol/embody';
|
|
1874
|
+
|
|
1875
|
+
const profile: Profile = {
|
|
1876
|
+
// ...all your usual AU mappings...
|
|
1877
|
+
hairPhysics: {
|
|
1878
|
+
stiffness: 7.5,
|
|
1879
|
+
damping: 0.18,
|
|
1880
|
+
inertia: 3.5,
|
|
1881
|
+
gravity: 12,
|
|
1882
|
+
responseScale: 2.5,
|
|
1883
|
+
idleSwayAmount: 0.12,
|
|
1884
|
+
idleSwaySpeed: 1.0,
|
|
1885
|
+
windStrength: 0,
|
|
1886
|
+
windDirectionX: 1.0,
|
|
1887
|
+
windDirectionZ: 0,
|
|
1888
|
+
windTurbulence: 0.3,
|
|
1889
|
+
windFrequency: 1.4,
|
|
1890
|
+
idleClipDuration: 10,
|
|
1891
|
+
impulseClipDuration: 1.4,
|
|
1892
|
+
direction: {
|
|
1893
|
+
yawSign: -1,
|
|
1894
|
+
pitchSign: -1,
|
|
1895
|
+
},
|
|
1896
|
+
morphTargets: {
|
|
1897
|
+
swayLeft: { key: 'L_Hair_Left', axis: 'yaw' },
|
|
1898
|
+
swayRight: { key: 'L_Hair_Right', axis: 'yaw' },
|
|
1899
|
+
swayFront: { key: 'L_Hair_Front', axis: 'pitch' },
|
|
1900
|
+
fluffRight: { key: 'Fluffy_Right', axis: 'yaw' },
|
|
1901
|
+
fluffBottom: { key: 'Fluffy_Bottom_ALL', axis: 'pitch' },
|
|
1902
|
+
headUp: {
|
|
1903
|
+
Hairline_High_ALL: { value: 0.45, axis: 'pitch' },
|
|
1904
|
+
Length_Short: { value: 0.65, axis: 'pitch' },
|
|
1905
|
+
},
|
|
1906
|
+
headDown: {
|
|
1907
|
+
L_Hair_Front: { value: 2.0, axis: 'pitch' },
|
|
1908
|
+
Fluffy_Bottom_ALL: { value: 1.0, axis: 'pitch' },
|
|
1909
|
+
},
|
|
1910
|
+
},
|
|
1911
|
+
},
|
|
1912
|
+
};
|
|
1913
|
+
```
|
|
1914
|
+
|
|
1915
|
+
### Configuration (runtime overrides)
|
|
1916
|
+
|
|
1917
|
+
```typescript
|
|
1918
|
+
loom.setHairPhysicsConfig({
|
|
1919
|
+
stiffness: 7.5,
|
|
1920
|
+
damping: 0.18,
|
|
1921
|
+
inertia: 3.5,
|
|
1922
|
+
gravity: 12,
|
|
1923
|
+
responseScale: 2.5,
|
|
1924
|
+
idleSwayAmount: 0.12,
|
|
1925
|
+
idleSwaySpeed: 1.0,
|
|
1926
|
+
windStrength: 0,
|
|
1927
|
+
windDirectionX: 1.0,
|
|
1928
|
+
windDirectionZ: 0,
|
|
1929
|
+
windTurbulence: 0.3,
|
|
1930
|
+
windFrequency: 1.4,
|
|
1931
|
+
idleClipDuration: 10,
|
|
1932
|
+
impulseClipDuration: 1.4,
|
|
1933
|
+
|
|
1934
|
+
// Direction mapping (signs) – adjust if hair goes the wrong way.
|
|
1935
|
+
direction: {
|
|
1936
|
+
yawSign: -1, // hair lags opposite head yaw
|
|
1937
|
+
pitchSign: -1, // head down drives forward hair motion
|
|
1938
|
+
},
|
|
1939
|
+
|
|
1940
|
+
// Morph target mapping (override per character/rig)
|
|
1941
|
+
morphTargets: {
|
|
1942
|
+
swayLeft: 'L_Hair_Left',
|
|
1943
|
+
swayRight: 'L_Hair_Right',
|
|
1944
|
+
swayFront: 'L_Hair_Front',
|
|
1945
|
+
fluffRight: 'Fluffy_Right',
|
|
1946
|
+
fluffBottom: 'Fluffy_Bottom_ALL',
|
|
1947
|
+
headUp: {
|
|
1948
|
+
Hairline_High_ALL: 0.45,
|
|
1949
|
+
Length_Short: 0.65,
|
|
1950
|
+
},
|
|
1951
|
+
headDown: {
|
|
1952
|
+
L_Hair_Front: 2.0,
|
|
1953
|
+
Fluffy_Bottom_ALL: 1.0,
|
|
1954
|
+
},
|
|
1955
|
+
},
|
|
1956
|
+
});
|
|
1957
|
+
```
|
|
1958
|
+
|
|
1959
|
+
### Validation
|
|
1960
|
+
|
|
1961
|
+
```typescript
|
|
1962
|
+
const missing = loom.validateHairMorphTargets();
|
|
1963
|
+
if (missing.length > 0) {
|
|
1964
|
+
console.warn('Missing hair morph targets:', missing);
|
|
1965
|
+
}
|
|
1966
|
+
```
|
|
1967
|
+
|
|
1968
|
+
Embody also logs a warning the first time it encounters a missing hair morph key.
|
|
1969
|
+
|
|
1970
|
+
### Applying styling state
|
|
1971
|
+
|
|
1972
|
+
Use the engine helpers when you want to toggle brows, outlines, or simple per-object visual state from a UI:
|
|
1973
|
+
|
|
1974
|
+
```typescript
|
|
1975
|
+
loom.applyHairStateToObject('Sideburns', {
|
|
1976
|
+
visible: true,
|
|
1977
|
+
outline: { show: true, color: '#7dd3fc', opacity: 0.6 },
|
|
1978
|
+
color: {
|
|
1979
|
+
baseColor: '#8b5e3c',
|
|
1980
|
+
emissive: '#000000',
|
|
1981
|
+
emissiveIntensity: 0,
|
|
1982
|
+
},
|
|
1983
|
+
});
|
|
1984
|
+
```
|
|
1985
|
+
|
|
1986
|
+
### Applying morphs to named hair meshes
|
|
1987
|
+
|
|
1988
|
+
```typescript
|
|
1989
|
+
loom.setMorphOnMeshes(
|
|
1990
|
+
['Side_part_wavy_1', 'Side_part_wavy_2'],
|
|
1991
|
+
'L_Hair_Front',
|
|
1992
|
+
0.35
|
|
1993
|
+
);
|
|
1994
|
+
```
|
|
1995
|
+
|
|
1996
|
+
### Notes
|
|
1997
|
+
|
|
1998
|
+
- **Head rotation input** comes from AUs (e.g. 51/52 yaw, 53/54 pitch).
|
|
1999
|
+
Hair updates when those AUs change.
|
|
2000
|
+
- **Mesh selection** comes from the preset (`CC4_MESHES` categories).
|
|
2001
|
+
Hair morph target *names* live in the preset/profile (`Profile.hairPhysics`) and can be overridden at runtime.
|
|
2002
|
+
- **Direction/morphs are explicit** so you can expose a clean, user-friendly API.
|
|
2003
|
+
|
|
2004
|
+
### Troubleshooting
|
|
2005
|
+
|
|
2006
|
+
- Hair moves the wrong direction → flip `direction.yawSign` or `direction.pitchSign`.
|
|
2007
|
+
- Wrong morphs are moving → override `morphTargets` with your rig’s names.
|
|
2008
|
+
- Need stronger response → increase `responseScale` or the `headDown/headUp` values.
|
|
2009
|
+
|
|
2010
|
+
---
|
|
2011
|
+
|
|
2012
|
+
## 16. Baked Animations
|
|
2013
|
+
|
|
2014
|
+
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
2015
|
+
|
|
2016
|
+
Embody can play baked skeletal animations from your GLB/GLTF files using Three.js AnimationMixer. This allows you to combine pre-made animations (idle, walk, gestures) with real-time facial control.
|
|
2017
|
+
|
|
2018
|
+
### Loading animations
|
|
2019
|
+
|
|
2020
|
+
After loading your model, pass the animations array to Embody:
|
|
2021
|
+
|
|
2022
|
+
```typescript
|
|
2023
|
+
const loader = new GLTFLoader();
|
|
2024
|
+
loader.load('/character.glb', (gltf) => {
|
|
2025
|
+
scene.add(gltf.scene);
|
|
2026
|
+
|
|
2027
|
+
const meshes = collectMorphMeshes(gltf.scene);
|
|
2028
|
+
loom.onReady({ meshes, model: gltf.scene });
|
|
2029
|
+
|
|
2030
|
+
// Load baked animations from the GLB file
|
|
2031
|
+
loom.loadAnimationClips(gltf.animations);
|
|
2032
|
+
|
|
2033
|
+
// Start the internal update loop
|
|
2034
|
+
loom.start();
|
|
2035
|
+
});
|
|
2036
|
+
```
|
|
2037
|
+
|
|
2038
|
+
### Listing available animations
|
|
2039
|
+
|
|
2040
|
+
```typescript
|
|
2041
|
+
const clips = loom.getAnimationClips();
|
|
2042
|
+
console.log(clips);
|
|
2043
|
+
// [
|
|
2044
|
+
// { name: 'Idle', duration: 4.0, trackCount: 52 },
|
|
2045
|
+
// { name: 'Walk', duration: 1.2, trackCount: 52 },
|
|
2046
|
+
// { name: 'Wave', duration: 2.5, trackCount: 24 },
|
|
2047
|
+
// ]
|
|
2048
|
+
```
|
|
2049
|
+
|
|
2050
|
+
### Playing animations
|
|
2051
|
+
|
|
2052
|
+
```typescript
|
|
2053
|
+
// Play an animation with default settings (looping)
|
|
2054
|
+
loom.playAnimation('Idle');
|
|
2055
|
+
|
|
2056
|
+
// Play with options
|
|
2057
|
+
const handle = loom.playAnimation('Wave', {
|
|
2058
|
+
speed: 1.0, // Playback speed (1.0 = normal)
|
|
2059
|
+
intensity: 1.0, // Weight/intensity (0-1)
|
|
2060
|
+
loop: false, // Don't loop
|
|
2061
|
+
loopMode: 'once', // 'repeat', 'pingpong', or 'once'
|
|
2062
|
+
clampWhenFinished: true, // Hold last frame when done
|
|
2063
|
+
startTime: 0, // Start from beginning
|
|
2064
|
+
});
|
|
2065
|
+
|
|
2066
|
+
// Wait for non-looping animation to finish
|
|
2067
|
+
await handle.finished;
|
|
2068
|
+
```
|
|
2069
|
+
|
|
2070
|
+
### Mixer clip playback for curves
|
|
2071
|
+
|
|
2072
|
+
Embody can convert AU/morph curves into AnimationMixer clips for smooth, mixer-only playback. This is the preferred path for high-frequency animation agencies (eye/head tracking, visemes, prosody) because it avoids per-keyframe transitions.
|
|
2073
|
+
|
|
2074
|
+
Key APIs:
|
|
2075
|
+
- `snippetToClip(name, curves, options)` builds an AnimationClip from curves.
|
|
2076
|
+
- `playClip(clip, options)` returns a ClipHandle you can pause/resume/stop.
|
|
2077
|
+
- `clipHandle.subscribe(listener)` streams lifecycle events during mixer updates.
|
|
2078
|
+
- `clipHandle.stop()` now resolves cleanly (no rejected promise).
|
|
2079
|
+
|
|
2080
|
+
```typescript
|
|
2081
|
+
const clip = loom.snippetToClip('gaze', {
|
|
2082
|
+
'61': [{ time: 0, intensity: 0 }, { time: 0.4, intensity: 0.6 }],
|
|
2083
|
+
'62': [{ time: 0, intensity: 0 }, { time: 0.4, intensity: 0 }],
|
|
2084
|
+
}, { loop: false });
|
|
2085
|
+
|
|
2086
|
+
if (clip) {
|
|
2087
|
+
const handle = loom.playClip(clip, { loop: false, speed: 1 });
|
|
2088
|
+
const unsubscribe = handle?.subscribe?.((event) => {
|
|
2089
|
+
if (event.type === 'keyframe') {
|
|
2090
|
+
console.log(event.currentTime, event.keyframeIndex);
|
|
2091
|
+
}
|
|
2092
|
+
});
|
|
2093
|
+
|
|
2094
|
+
await handle.finished;
|
|
2095
|
+
unsubscribe?.();
|
|
2096
|
+
}
|
|
2097
|
+
```
|
|
2098
|
+
|
|
2099
|
+
When the first keyframe has `inherit: true`, Embody anchors that first mixer
|
|
2100
|
+
track value to the character's current live target value instead of treating the
|
|
2101
|
+
placeholder intensity as a real reset. The following keyframes remain authored
|
|
2102
|
+
targets:
|
|
2103
|
+
|
|
2104
|
+
```typescript
|
|
2105
|
+
const handle = loom.buildClip('smile-from-current-pose', {
|
|
2106
|
+
// Starts from the current AU 12/morph/bone state, then moves to 0.7.
|
|
2107
|
+
'12': [{ time: 0, intensity: 0, inherit: true }, { time: 0.25, intensity: 0.7 }],
|
|
2108
|
+
}, { loop: false });
|
|
2109
|
+
```
|
|
2110
|
+
|
|
2111
|
+
Clip stream events are discrete runtime events, not a polling surface:
|
|
2112
|
+
|
|
2113
|
+
- `keyframe` fires when playback crosses an authored keyframe.
|
|
2114
|
+
- `loop` fires when looping playback starts another iteration.
|
|
2115
|
+
- `seek` fires when `setTime()` scrubs the clip.
|
|
2116
|
+
- `completed` fires when non-looping playback reaches its terminal state.
|
|
2117
|
+
|
|
2118
|
+
### Playing a snippet directly
|
|
2119
|
+
|
|
2120
|
+
If you already have a named snippet object, you can skip manual clip creation:
|
|
2121
|
+
|
|
2122
|
+
```typescript
|
|
2123
|
+
const handle = loom.playSnippet({
|
|
2124
|
+
name: 'look-left',
|
|
2125
|
+
curves: {
|
|
2126
|
+
'61': [{ time: 0, intensity: 0 }, { time: 0.25, intensity: 0.7 }],
|
|
2127
|
+
'62': [{ time: 0, intensity: 0 }, { time: 0.25, intensity: 0 }],
|
|
2128
|
+
},
|
|
2129
|
+
}, { loop: false });
|
|
2130
|
+
```
|
|
2131
|
+
|
|
2132
|
+
### Building and updating managed clips
|
|
2133
|
+
|
|
2134
|
+
`buildClip()` keeps a named clip/action around so you can adjust it later without rebuilding your entire animation flow:
|
|
2135
|
+
|
|
2136
|
+
```typescript
|
|
2137
|
+
const clipHandle = loom.buildClip('gaze-loop', {
|
|
2138
|
+
'61': [{ time: 0, intensity: 0 }, { time: 0.3, intensity: 0.6 }],
|
|
2139
|
+
'62': [{ time: 0, intensity: 0.3 }, { time: 0.3, intensity: 0 }],
|
|
2140
|
+
}, {
|
|
2141
|
+
loop: true,
|
|
2142
|
+
loopMode: 'pingpong',
|
|
2143
|
+
});
|
|
2144
|
+
|
|
2145
|
+
loom.updateClipParams('gaze-loop', {
|
|
2146
|
+
weight: 0.7,
|
|
2147
|
+
rate: 1.2,
|
|
2148
|
+
loopMode: 'repeat',
|
|
2149
|
+
});
|
|
2150
|
+
|
|
2151
|
+
clipHandle?.pause();
|
|
2152
|
+
clipHandle?.resume();
|
|
2153
|
+
```
|
|
2154
|
+
|
|
2155
|
+
### Checking curve support
|
|
2156
|
+
|
|
2157
|
+
```typescript
|
|
2158
|
+
const supported = loom.supportsClipCurves({
|
|
2159
|
+
'61': [{ time: 0, intensity: 0 }, { time: 0.2, intensity: 0.4 }],
|
|
2160
|
+
});
|
|
2161
|
+
|
|
2162
|
+
if (!supported) {
|
|
2163
|
+
console.warn('Curves need a fallback playback path');
|
|
2164
|
+
}
|
|
2165
|
+
```
|
|
2166
|
+
|
|
2167
|
+
### Controlling playback
|
|
2168
|
+
|
|
2169
|
+
The handle returned from `playAnimation()` provides full control:
|
|
2170
|
+
|
|
2171
|
+
```typescript
|
|
2172
|
+
const handle = loom.playAnimation('Idle');
|
|
2173
|
+
|
|
2174
|
+
// Pause and resume
|
|
2175
|
+
handle.pause();
|
|
2176
|
+
handle.resume();
|
|
2177
|
+
|
|
2178
|
+
// Adjust speed in real-time
|
|
2179
|
+
handle.setSpeed(0.5); // Half speed
|
|
2180
|
+
handle.setSpeed(2.0); // Double speed
|
|
2181
|
+
|
|
2182
|
+
// Adjust intensity/weight
|
|
2183
|
+
handle.setWeight(0.5); // 50% influence
|
|
2184
|
+
|
|
2185
|
+
// Seek to specific time
|
|
2186
|
+
handle.seekTo(1.5); // Jump to 1.5 seconds
|
|
2187
|
+
|
|
2188
|
+
// Get current state
|
|
2189
|
+
const state = handle.getState();
|
|
2190
|
+
console.log(state);
|
|
2191
|
+
// {
|
|
2192
|
+
// name: 'Idle',
|
|
2193
|
+
// isPlaying: true,
|
|
2194
|
+
// isPaused: false,
|
|
2195
|
+
// time: 1.5,
|
|
2196
|
+
// duration: 4.0,
|
|
2197
|
+
// speed: 1.0,
|
|
2198
|
+
// weight: 1.0,
|
|
2199
|
+
// isLooping: true
|
|
2200
|
+
// }
|
|
2201
|
+
|
|
2202
|
+
// Stop the animation
|
|
2203
|
+
handle.stop();
|
|
2204
|
+
```
|
|
2205
|
+
|
|
2206
|
+
### Crossfading between animations
|
|
2207
|
+
|
|
2208
|
+
Smoothly transition from one animation to another:
|
|
2209
|
+
|
|
2210
|
+
```typescript
|
|
2211
|
+
// Start with idle
|
|
2212
|
+
loom.playAnimation('Idle');
|
|
2213
|
+
|
|
2214
|
+
// Later, crossfade to walk over 0.3 seconds
|
|
2215
|
+
loom.crossfadeTo('Walk', 0.3);
|
|
2216
|
+
|
|
2217
|
+
// Or use the handle
|
|
2218
|
+
const idleHandle = loom.playAnimation('Idle');
|
|
2219
|
+
idleHandle.crossfadeTo('Walk', 0.5);
|
|
2220
|
+
```
|
|
2221
|
+
|
|
2222
|
+
### Global animation control
|
|
2223
|
+
|
|
2224
|
+
Control all animations at once:
|
|
2225
|
+
|
|
2226
|
+
```typescript
|
|
2227
|
+
// Stop all animations
|
|
2228
|
+
loom.stopAllAnimations();
|
|
2229
|
+
|
|
2230
|
+
// Pause all animations
|
|
2231
|
+
loom.pauseAllAnimations();
|
|
2232
|
+
|
|
2233
|
+
// Resume all paused animations
|
|
2234
|
+
loom.resumeAllAnimations();
|
|
2235
|
+
|
|
2236
|
+
// Set global time scale (affects all animations)
|
|
2237
|
+
loom.setAnimationTimeScale(0.5); // Everything at half speed
|
|
2238
|
+
|
|
2239
|
+
// Get all currently playing animations
|
|
2240
|
+
const playing = loom.getPlayingAnimations();
|
|
2241
|
+
```
|
|
2242
|
+
|
|
2243
|
+
### Direct control by name
|
|
2244
|
+
|
|
2245
|
+
You can also control animations directly without using handles:
|
|
2246
|
+
|
|
2247
|
+
```typescript
|
|
2248
|
+
loom.playAnimation('Idle');
|
|
2249
|
+
|
|
2250
|
+
// Later...
|
|
2251
|
+
loom.setAnimationSpeed('Idle', 1.5);
|
|
2252
|
+
loom.setAnimationIntensity('Idle', 0.8);
|
|
2253
|
+
loom.pauseAnimation('Idle');
|
|
2254
|
+
loom.resumeAnimation('Idle');
|
|
2255
|
+
loom.stopAnimation('Idle');
|
|
2256
|
+
|
|
2257
|
+
// Get state of specific animation
|
|
2258
|
+
const state = loom.getAnimationState('Idle');
|
|
2259
|
+
```
|
|
2260
|
+
|
|
2261
|
+
### Combining with facial animation
|
|
2262
|
+
|
|
2263
|
+
Loaded mixer clips and facial AU control work together seamlessly. The AnimationMixer updates automatically when you call `loom.update()` or use `loom.start()`:
|
|
2264
|
+
|
|
2265
|
+
```typescript
|
|
2266
|
+
loom.loadAnimationClips(gltf.animations);
|
|
2267
|
+
loom.start(); // Starts internal RAF loop
|
|
2268
|
+
|
|
2269
|
+
// Play a body animation
|
|
2270
|
+
loom.playAnimation('Idle');
|
|
2271
|
+
|
|
2272
|
+
// Control facial expressions on top
|
|
2273
|
+
loom.setAU(12, 0.8); // Smile
|
|
2274
|
+
loom.transitionAU(45, 1.0, 100); // Blink
|
|
2275
|
+
|
|
2276
|
+
// Both update together - no separate render loop needed
|
|
2277
|
+
```
|
|
2278
|
+
|
|
2279
|
+
### Animation types
|
|
2280
|
+
|
|
2281
|
+
| Option | Type | Default | Description |
|
|
2282
|
+
|--------|------|---------|-------------|
|
|
2283
|
+
| `speed` | number | 1.0 | Playback speed multiplier |
|
|
2284
|
+
| `intensity` | number | 1.0 | Animation weight (0-1) |
|
|
2285
|
+
| `loop` | boolean | true | Whether to loop |
|
|
2286
|
+
| `loopMode` | string | 'repeat' | 'repeat', 'pingpong', or 'once' |
|
|
2287
|
+
| `crossfadeDuration` | number | 0 | Fade in duration (seconds) |
|
|
2288
|
+
| `clampWhenFinished` | boolean | true | Hold last frame when done |
|
|
2289
|
+
| `startTime` | number | 0 | Initial playback position |
|
|
2290
|
+
|
|
2291
|
+
---
|
|
2292
|
+
|
|
2293
|
+
## 17. Regions & Geometry Helpers
|
|
2294
|
+
|
|
2295
|
+
Open in LoomLarge: [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
2296
|
+
|
|
2297
|
+
These helpers are for applications that need semantic face regions, marker anchors, or camera targets in addition to direct animation control.
|
|
2298
|
+
|
|
2299
|
+
### Finding a face center directly from the model
|
|
2300
|
+
|
|
2301
|
+
```typescript
|
|
2302
|
+
import { findFaceCenter } from '@lovelace_lol/embody';
|
|
2303
|
+
|
|
2304
|
+
const result = findFaceCenter(gltf.scene, {
|
|
2305
|
+
headBoneNames: ['CC_Base_Head'],
|
|
2306
|
+
faceMeshNames: ['CC_Base_Body'],
|
|
2307
|
+
});
|
|
2308
|
+
|
|
2309
|
+
console.log(result.center, result.method, result.debugInfo);
|
|
2310
|
+
```
|
|
2311
|
+
|
|
2312
|
+
### Resolving region-driven centers
|
|
2313
|
+
|
|
2314
|
+
```typescript
|
|
2315
|
+
import type { BoneResolutionProfile, Region } from '@lovelace_lol/embody';
|
|
2316
|
+
import { resolveBoneName, resolveBoneNames, resolveFaceCenter } from '@lovelace_lol/embody';
|
|
2317
|
+
|
|
2318
|
+
const region: Region = {
|
|
2319
|
+
name: 'face',
|
|
2320
|
+
bones: ['HEAD'],
|
|
2321
|
+
meshes: ['CC_Base_Body'],
|
|
2322
|
+
};
|
|
2323
|
+
|
|
2324
|
+
const profile: BoneResolutionProfile = {
|
|
2325
|
+
bonePrefix: 'CC_Base_',
|
|
2326
|
+
boneNodes: { HEAD: 'Head' },
|
|
2327
|
+
};
|
|
2328
|
+
|
|
2329
|
+
const headBone = resolveBoneName('HEAD', profile);
|
|
2330
|
+
const resolvedBones = resolveBoneNames(['HEAD'], profile);
|
|
2331
|
+
const faceCenter = resolveFaceCenter(gltf.scene, region, profile);
|
|
2332
|
+
```
|
|
2333
|
+
|
|
2334
|
+
### Working with model orientation
|
|
2335
|
+
|
|
2336
|
+
```typescript
|
|
2337
|
+
import {
|
|
2338
|
+
getModelForwardDirection,
|
|
2339
|
+
detectFacingDirection,
|
|
2340
|
+
} from '@lovelace_lol/embody';
|
|
2341
|
+
|
|
2342
|
+
const forward = getModelForwardDirection(gltf.scene);
|
|
2343
|
+
const facing = detectFacingDirection(gltf.scene);
|
|
2344
|
+
```
|
|
2345
|
+
|
|
2346
|
+
Use these helpers when you need to:
|
|
2347
|
+
- place annotation markers using semantic regions instead of hard-coded coordinates
|
|
2348
|
+
- resolve prefixed/suffixed bone names from a reusable profile or minimal bone-resolution object
|
|
2349
|
+
- derive a face anchor for camera tooling or interaction layers
|
|
2350
|
+
- reason about model orientation before building your own camera or annotation system
|
|
2351
|
+
|
|
2352
|
+
---
|
|
2353
|
+
|
|
2354
|
+
## 18. API Reference
|
|
2355
|
+
|
|
2356
|
+
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
2357
|
+
|
|
2358
|
+
This is a compact reference for the public surface exported by `@lovelace_lol/embody`.
|
|
2359
|
+
|
|
2360
|
+
### Core runtime
|
|
2361
|
+
|
|
2362
|
+
- `Embody` is the main Three.js implementation.
|
|
2363
|
+
- `collectMorphMeshes()` gathers meshes that already expose morph targets.
|
|
2364
|
+
- Lifecycle: `onReady()`, `update()`, `start()`, `stop()`, `dispose()`.
|
|
2365
|
+
- Preset state: `setProfile()`, `getProfile()`.
|
|
2366
|
+
- Control APIs: `setAU()`, `transitionAU()`, `setContinuum()`, `transitionContinuum()`, `setMorph()`, `transitionMorph()`, `setViseme()`, `transitionViseme()`.
|
|
2367
|
+
- Runtime morph authoring: `addMorphTarget()`, `addMorphTargets()`, `ensureMorphInfluence()`, `refreshMorphTargets()`.
|
|
2368
|
+
- Transition state: `pause()`, `resume()`, `getPaused()`, `clearTransitions()`, `getActiveTransitionCount()`, `resetToNeutral()`.
|
|
2369
|
+
|
|
2370
|
+
### Presets and profiles
|
|
2371
|
+
|
|
2372
|
+
- Presets: `CC4_PRESET`, `BETTA_FISH_PRESET`, `getPreset()`, `getPresetWithProfile()`.
|
|
2373
|
+
- Profile composition: `extendPresetWithProfile()`.
|
|
2374
|
+
- CC4 exports: `VISEME_KEYS`, `VISEME_JAW_AMOUNTS`, `CONTINUUM_PAIRS_MAP`, `CONTINUUM_LABELS`, `AU_INFO`, `COMPOSITE_ROTATIONS`, `AU_MIX_DEFAULTS`.
|
|
2375
|
+
- Compatibility helpers: `isMixedAU()`, `hasLeftRightMorphs()`.
|
|
2376
|
+
|
|
2377
|
+
### Validation and inspection
|
|
2378
|
+
|
|
2379
|
+
- Extraction: `extractModelData()`, `extractFromGLTF()`.
|
|
2380
|
+
- Config linting: `validateMappingConfig()`.
|
|
2381
|
+
- Model fit: `validateMappings()`, `isPresetCompatible()`, `suggestBestPreset()`, `generateMappingCorrections()`.
|
|
2382
|
+
- Template skeleton fit metadata: `TEMPLATE_SKELETON_FIT_METADATA_KIND`, `TEMPLATE_SKELETON_FIT_METADATA_VERSION`, `TEMPLATE_SKELETON_FIT_STATUSES`, `validateTemplateSkeletonFitMetadata()`, `composeTemplateSkeletonFitTransform()`.
|
|
2383
|
+
- Unified report: `analyzeModel()`.
|
|
2384
|
+
|
|
2385
|
+
### Runtime tooling
|
|
2386
|
+
|
|
2387
|
+
- Mesh inspection: `getMeshList()`, `getMorphTargets()`, `getMorphTargetIndices()`, `getBones()`.
|
|
2388
|
+
- Mesh debugging: `setMeshVisible()`, `highlightMesh()`, `getMeshMaterialConfig()`, `setMeshMaterialConfig()`.
|
|
2389
|
+
- Hair runtime: `registerHairObjects()`, `getRegisteredHairObjects()`, `setHairPhysicsEnabled()`, `setHairPhysicsConfig()`, `validateHairMorphTargets()`, `applyHairStateToObject()`.
|
|
2390
|
+
- Mixer helpers: `loadAnimationClips()`, `getAnimationClips()`, `playAnimation()`, `pauseAnimation()`, `resumeAnimation()`, `stopAnimation()`, `stopAllAnimations()`, `pauseAllAnimations()`, `resumeAllAnimations()`, `setAnimationSpeed()`, `setAnimationIntensity()`, `setAnimationTimeScale()`, `getAnimationState()`, `getPlayingAnimations()`, `crossfadeTo()`, `snippetToClip()`, `playClip()`, `playSnippet()`, `buildClip()`, `updateClipParams()`, `supportsClipCurves()`.
|
|
2391
|
+
|
|
2392
|
+
### Types and lower-level exports
|
|
2393
|
+
|
|
2394
|
+
- Configuration/types: `Profile`, `MeshInfo`, `BlendingMode`, `TransitionHandle`, `ClipEvent`, `ClipEventListener`, `ClipHandle`, `Snippet`, `AnimationState`, `AnimationClipInfo`.
|
|
2395
|
+
- Standalone implementations: `ThreeAnimationRuntime`, `HairPhysics`, `BLENDING_MODES`.
|
|
2396
|
+
- Region and geometry helpers: `resolveBoneName()`, `resolveBoneNames()`, `resolveFaceCenter()`, `findFaceCenter()`, `getModelForwardDirection()`, `detectFacingDirection()`.
|
|
2397
|
+
|
|
2398
|
+
---
|
|
2399
|
+
|
|
2400
|
+
## Resources
|
|
2401
|
+
|
|
2402
|
+

|
|
2403
|
+
|
|
2404
|
+
- [FACS on Wikipedia](https://en.wikipedia.org/wiki/Facial_Action_Coding_System)
|
|
2405
|
+
- [Paul Ekman Group - FACS](https://www.paulekman.com/facial-action-coding-system/)
|
|
2406
|
+
- [Character Creator 4](https://www.reallusion.com/character-creator/)
|
|
2407
|
+
- [Three.js Documentation](https://threejs.org/docs/)
|
|
2408
|
+
|
|
2409
|
+
## License
|
|
2410
|
+
|
|
2411
|
+
MIT License. Embody is authored by Jonathan Sutton Fields; see [LICENSE](LICENSE) and [AUTHORS.md](AUTHORS.md) for details.
|