@lovelace_lol/embody 0.0.7 → 0.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +541 -2394
- package/dist/index.d.ts +2 -100
- package/dist/index.js +9197 -13851
- package/dist/index.js.map +1 -1
- package/dist/three/annotations/DOMControls.d.ts +73 -0
- package/dist/three/annotations/DPthree.d.ts +113 -0
- package/dist/three/annotations/DPthree3DMarkers.d.ts +319 -0
- package/dist/three/annotations/DPthreeCameraController.d.ts +415 -0
- package/dist/three/annotations/DPthreeHTMLMarkers.d.ts +131 -0
- package/dist/three/annotations/DPthreeMarkers.d.ts +52 -0
- package/dist/three/annotations/adapter.d.ts +157 -0
- package/dist/three/annotations/animationFrameTaskScheduler.d.ts +18 -0
- package/dist/three/annotations/annotationAnchorModel.d.ts +16 -0
- package/dist/three/annotations/annotationCameraCore.d.ts +5 -0
- package/dist/three/annotations/boneResolutionProfile.d.ts +7 -0
- package/dist/three/annotations/index.d.ts +20 -0
- package/dist/three/annotations/loadFrame.d.ts +2 -0
- package/dist/three/annotations/markerSurfaceRaycast.d.ts +11 -0
- package/dist/three/annotations/markerVisibilityLifecycle.d.ts +21 -0
- package/dist/three/annotations/modelRenderPreparation.d.ts +6 -0
- package/dist/three/annotations/runtimeAnnotationPreview.d.ts +15 -0
- package/dist/three/annotations/runtimeAnnotationPreviewSide.d.ts +10 -0
- package/dist/three/annotations/runtimeAnnotationSidePreview.d.ts +5 -0
- package/dist/three/annotations/types.d.ts +262 -0
- package/dist/three/authoring.d.ts +95 -0
- package/dist/three/authoringTypes.d.ts +576 -0
- package/dist/three/character.d.ts +243 -0
- package/dist/three/characterHost.d.ts +48 -0
- package/dist/three/clipBindings.d.ts +6 -0
- package/dist/three/index.d.ts +171 -0
- package/dist/three/model.d.ts +31 -0
- package/dist/three/modelSkinning.d.ts +3 -0
- package/dist/three/nativePlayback.d.ts +121 -0
- package/dist/three/playbackRendering.d.ts +14 -0
- package/dist/three/reference-pose.d.ts +73 -0
- package/dist/three/scene.d.ts +253 -0
- package/dist/three/sceneRenderingBinding.d.ts +22 -0
- package/dist/three.js +9252 -8844
- package/dist/three.js.map +1 -1
- package/dist/wasm/README.md +551 -62
- package/dist/wasm/auto.d.ts +1 -0
- package/dist/wasm/compatibility.d.ts +14 -0
- package/dist/wasm/embody_wasm.d.ts +352 -71
- package/dist/wasm/embody_wasm.js +1266 -37
- package/dist/wasm/embody_wasm_bg.wasm +0 -0
- package/dist/wasm/embody_wasm_bg.wasm.d.ts +113 -54
- package/dist/wasm/index.d.ts +131 -0
- package/dist/wasm.d.ts +1 -43
- package/dist/wasm.js +1100 -239
- package/dist/wasm.js.map +1 -1
- package/package.json +35 -39
- package/service-worker/character-asset-service-worker.js +295 -0
- package/dist/authoring/boneAuthoring.d.ts +0 -132
- package/dist/camera/annotationCameraAngles.d.ts +0 -28
- package/dist/camera/cameraRelativeGaze.d.ts +0 -15
- package/dist/camera/rustAnnotationCameraCore.d.ts +0 -136
- package/dist/cljs.cjs +0 -362
- package/dist/cljs.cjs.map +0 -1
- package/dist/cljs.d.ts +0 -5
- package/dist/cljs.js +0 -359
- package/dist/cljs.js.map +0 -1
- package/dist/core/TsClipCompiler.d.ts +0 -51
- package/dist/core/WasmRuntimeCore.d.ts +0 -41
- package/dist/core/compositeAxis.d.ts +0 -4
- package/dist/core/contracts/abi.d.ts +0 -70
- package/dist/core/contracts/clip.d.ts +0 -44
- package/dist/core/contracts/frame.d.ts +0 -27
- package/dist/core/contracts/host.d.ts +0 -28
- package/dist/core/contracts/ids.d.ts +0 -15
- package/dist/core/contracts/index.d.ts +0 -10
- package/dist/core/contracts/model.d.ts +0 -31
- package/dist/core/contracts/primitives.d.ts +0 -17
- package/dist/core/contracts/skeletonFit.d.ts +0 -41
- package/dist/core/index.d.ts +0 -6
- package/dist/core/types.d.ts +0 -472
- package/dist/core.cjs +0 -343
- package/dist/core.cjs.map +0 -1
- package/dist/core.js +0 -323
- package/dist/core.js.map +0 -1
- package/dist/engines/three/Embody.d.ts +0 -372
- package/dist/engines/three/ThreeAnimationRuntime.d.ts +0 -203
- package/dist/engines/three/ThreeClipAdapter.d.ts +0 -24
- package/dist/engines/three/ThreeFrameApplier.d.ts +0 -68
- package/dist/engines/three/ThreeModelInspector.d.ts +0 -54
- package/dist/engines/three/bakedClipPartitioning.d.ts +0 -16
- package/dist/engines/three/balanceUtils.d.ts +0 -3
- package/dist/engines/three/characterHost.d.ts +0 -80
- package/dist/engines/three/characterScene.d.ts +0 -98
- package/dist/engines/three/hair/HairPhysicsController.d.ts +0 -124
- package/dist/engines/three/index.d.ts +0 -20
- package/dist/engines/three/modelLoader.d.ts +0 -61
- package/dist/engines/three/profileTestScene.d.ts +0 -52
- package/dist/engines/three/types.d.ts +0 -20
- package/dist/hair/appearance.d.ts +0 -29
- package/dist/index.cjs +0 -14871
- package/dist/index.cjs.map +0 -1
- package/dist/interfaces/Animation.d.ts +0 -315
- package/dist/interfaces/EmbodyRuntime.d.ts +0 -90
- package/dist/interfaces/Hair.d.ts +0 -181
- package/dist/interfaces/HairPhysics.d.ts +0 -90
- package/dist/interfaces/index.d.ts +0 -11
- package/dist/mappings/extendPresetWithProfile.d.ts +0 -17
- package/dist/mappings/types.d.ts +0 -309
- package/dist/mappings/visemeSystem.d.ts +0 -26
- package/dist/physics/HairPhysics.d.ts +0 -86
- package/dist/physics/RustHairPhysics.d.ts +0 -18
- package/dist/physics/index.d.ts +0 -6
- package/dist/preset.cjs +0 -2005
- package/dist/preset.cjs.map +0 -1
- package/dist/preset.js +0 -2003
- package/dist/preset.js.map +0 -1
- package/dist/presets/bettaFish.d.ts +0 -328
- package/dist/presets/cc4.d.ts +0 -145
- package/dist/presets/index.d.ts +0 -30
- package/dist/presets/registry.d.ts +0 -57
- package/dist/profiles/resolveProfileConfig.d.ts +0 -36
- package/dist/profiles/rustResolvedProfile.d.ts +0 -31
- package/dist/profiles/types.d.ts +0 -118
- package/dist/regions/normalizeRegionTree.d.ts +0 -6
- package/dist/regions/regionMapping.d.ts +0 -17
- package/dist/regions/types.d.ts +0 -122
- package/dist/skeletonTemplates/humanoidSkeletonTemplates.d.ts +0 -32
- package/dist/test/setupWasm.d.ts +0 -1
- package/dist/three.cjs +0 -9550
- package/dist/three.cjs.map +0 -1
- package/dist/validation/analyzeModel.d.ts +0 -68
- package/dist/validation/extractModelData.d.ts +0 -90
- package/dist/validation/generateMappingCorrections.d.ts +0 -32
- package/dist/validation/geometryHelpers.d.ts +0 -66
- package/dist/validation/types.d.ts +0 -10
- package/dist/validation/validateMappings.d.ts +0 -77
- package/dist/wasm.cjs +0 -302
- package/dist/wasm.cjs.map +0 -1
- package/dist/wasmTypes.d.ts +0 -123
package/README.md
CHANGED
|
@@ -1,2414 +1,561 @@
|
|
|
1
1
|
# Embody
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
Embody
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
3
|
+
Explicit profiles may be empty: `RuntimeCore.configure_with_profile('{}', modelJson)`
|
|
4
|
+
loads a static or not-yet-mapped model without selecting an embedded preset.
|
|
5
|
+
Missing mappings remain an authoring/validation concern; malformed JSON still
|
|
6
|
+
fails configuration. Existing mapped profiles retain their runtime behavior.
|
|
7
|
+
|
|
8
|
+
Embody is a Rust/Wasm character animation core for facial action units,
|
|
9
|
+
visemes, bone controls, baked animation clips, and hair motion. Three.js is a
|
|
10
|
+
host adapter, not the runtime implementation.
|
|
11
|
+
|
|
12
|
+
## Architecture
|
|
13
|
+
|
|
14
|
+
```text
|
|
15
|
+
src/ Rust source (the Wasm crate) — hair, clips, profiles, runtime
|
|
16
|
+
assets/presets/ Embedded typed character profiles
|
|
17
|
+
assets/templates/ Embedded humanoid skeleton templates
|
|
18
|
+
three/ Thin Three.js inspection/application adapter only
|
|
19
|
+
wasm/ Generated-Wasm JS loader + ABI constants
|
|
20
|
+
index.ts Package re-exports only
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
There is no parallel TypeScript hair/physics/runtime core. Hair idle, impulse,
|
|
24
|
+
gravity curves and the spring solver live in `src/hair_curves.rs` /
|
|
25
|
+
`src/hair.rs` and are exported through Wasm. Hosts schedule the resulting
|
|
26
|
+
ClipIR on their mixer; they must not reimplement hair sampling in TS.
|
|
27
|
+
|
|
28
|
+
The Rust core owns:
|
|
29
|
+
|
|
30
|
+
- preset and profile parsing, typed merge rules, validation, and corrections
|
|
31
|
+
- AU, viseme, morph, bone, continuum, and live control state
|
|
32
|
+
- compiling AU/viseme/named-morph snippets into concrete ClipIR
|
|
33
|
+
(`morphTarget` / `boneTransform` tracks) for the host mixer to schedule
|
|
34
|
+
- mesh-category routing and material profile data
|
|
35
|
+
- hair curves, hair physics, and appearance normalization
|
|
36
|
+
- annotation camera/marker math and humanoid template fitting
|
|
37
|
+
- profile-aware screen-space gaze geometry for eye/head AU trajectories
|
|
38
|
+
- renderer-neutral model analysis and packed live frame generation
|
|
39
|
+
|
|
40
|
+
The host animation library (Three `AnimationMixer`, Unity Animator, etc.) owns
|
|
41
|
+
clip playback, lerping, blending, looping, seeking, and crossfades. Rust must
|
|
42
|
+
not sample or lerp clips on the hot path.
|
|
43
|
+
|
|
44
|
+
An AU animation drives its profile's mapped bones and morphs through the same
|
|
45
|
+
Rust evaluation used by live AU controls. For example, speech can send an AU
|
|
46
|
+
103 curve and separate viseme shape curves; the profile decides which bones
|
|
47
|
+
and morphs AU 103 moves. Hosts must not replace that AU with a hardcoded jaw
|
|
48
|
+
rotation or read `auToBones` to construct their own bone tracks. When LipSync
|
|
49
|
+
already supplies a jaw AU curve, its existing `autoVisemeJaw: false` option
|
|
50
|
+
keeps automatic viseme jaw synthesis from replacing that curve; it does not
|
|
51
|
+
disable the AU's mapped morphs or the viseme shapes.
|
|
52
|
+
|
|
53
|
+
The TypeScript adapter is intentionally limited to operations that require
|
|
54
|
+
Three.js objects: scene traversal, ClipIR ↔ `AnimationClip` conversion, frame
|
|
55
|
+
application, material writes, model loading/disposal, and default scene
|
|
56
|
+
construction. The annotation adapter also owns camera controls, marker scene
|
|
57
|
+
objects, labels, selection, and their browser lifecycle; applications provide
|
|
58
|
+
the scene/camera/DOM inputs and render controls through the public API.
|
|
59
|
+
|
|
60
|
+
`ThreeFrameApplier.applyMeshMaterialConfigs(model, profile.meshes)` replays named
|
|
61
|
+
mesh visibility and material overrides when a host binds or replaces a profile.
|
|
62
|
+
It applies explicit `visible: false` and `visible: true` even without a material
|
|
63
|
+
entry, and restores render order, opacity, transparency, depth flags and blending
|
|
64
|
+
from `material`. Omitted fields leave the model's current values unchanged.
|
|
65
|
+
Polymer calls this adapter during model binding and profile replacement; hosts
|
|
66
|
+
do not need a separate mesh-visibility replay loop.
|
|
67
|
+
|
|
68
|
+
### Annotation runtime
|
|
69
|
+
|
|
70
|
+
`DPthreeCameraController` and the `DPthree` convenience facade from
|
|
71
|
+
`@lovelace_lol/embody/three` own the complete camera and annotation runtime.
|
|
72
|
+
`DPthree3DMarkers`, `DPthreeHTMLMarkers`, and the legacy `DPthreeMarkers` adapter
|
|
73
|
+
are available for hosts that need a marker-only layer. Rust owns camera
|
|
74
|
+
framing/flights/orbits, viewport clipping, curves, visibility factors and
|
|
75
|
+
endpoint separation. The Three adapter owns native model inspection,
|
|
76
|
+
raycasting, labels, DOM controls, marker expansion/solo/style state and disposal.
|
|
77
|
+
|
|
78
|
+
Applications pass `AnnotationCharacterConfig`; storage, agency, editor and
|
|
79
|
+
character-asset metadata remain host concerns. `loadRegions` defaults to
|
|
80
|
+
`resolveAnnotationCharacterConfig`, which resolves annotation and AU mapping
|
|
81
|
+
fields from an Embody preset without serializing unrelated host metadata.
|
|
82
|
+
An optional `resolveCharacterConfig` callback supports host profile intake.
|
|
83
|
+
`prepareRegionsAndMarkersForReveal` accepts an already-resolved profile and
|
|
84
|
+
prepares native surface queries while the model pose is still stable.
|
|
85
|
+
Initial queries yield to a browser paint between meshes once a 4 ms slice is
|
|
86
|
+
spent. Each native mesh raycast is indivisible; a large mesh can still cause a
|
|
87
|
+
frame drop, and slicing does not reduce total raycast CPU time. Interactive
|
|
88
|
+
annotation changes retain synchronous queries so a marker samples one pose.
|
|
89
|
+
Replacement, clearing and disposal cancel stale loads before scene commits.
|
|
90
|
+
|
|
91
|
+
For first reveal, call `controller.prepareModelForRender(model, { signal })`
|
|
92
|
+
from the existing `beforeReveal` callback (or directly before attaching the
|
|
93
|
+
model). It compiles the model's visible surfaces, then uses the controller's
|
|
94
|
+
existing render loop for one complete scene preparation draw. No host warmup
|
|
95
|
+
queue, shadow proxies or texture preparation loop is needed. The promise rejects
|
|
96
|
+
on failure or cancellation; retain borrowed model/scene/renderer resources until
|
|
97
|
+
it settles, even after `controller.dispose()`. This prepares visible variants,
|
|
98
|
+
not every future hidden material or animation state. See the
|
|
99
|
+
[model readiness contract](docs/SCENE_READINESS.md#model-readiness).
|
|
100
|
+
|
|
101
|
+
`createMarkerVisibilityLifecycle` owns automatic reveal/hide timers;
|
|
102
|
+
`createRuntimeAnnotationPreviewLifecycle` owns temporary authoring previews.
|
|
103
|
+
React hosts forward user events and subscribe to controller state. They do not
|
|
104
|
+
need local copies of the camera, marker, placement or timer algorithms.
|
|
105
|
+
The convenience facade delegates to one controller-owned marker layer.
|
|
106
|
+
|
|
107
|
+
## Runtime Use
|
|
108
|
+
|
|
109
|
+
Embody is an ES module package. Use `import` / dynamic `import()`; CommonJS
|
|
110
|
+
`require()` is not a supported runtime API. Package subpaths remain resolvable
|
|
111
|
+
through Node for tooling that only needs to locate an exported file.
|
|
112
|
+
|
|
113
|
+
Initialize the Wasm module before constructing a runtime:
|
|
114
|
+
|
|
115
|
+
```ts
|
|
114
116
|
import { initEmbodyCore } from '@lovelace_lol/embody/wasm';
|
|
115
|
-
|
|
116
|
-
const core = await initEmbodyCore();
|
|
117
|
-
const [left, right] = core.solve_bilateral_values(0.8, 0.25);
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
The Wasm preset registry currently embeds the canonical IDs `cc4` and `fish`.
|
|
121
|
-
It does not interpret compatibility aliases such as `default`, `human`, or
|
|
122
|
-
`skeletal`; application compatibility layers must normalize those values before
|
|
123
|
-
calling `configure_with_preset(id, overrideJson, modelJson)`. Hosts can use
|
|
124
|
-
`core.has_preset(id)` to validate a canonical ID.
|
|
125
|
-
|
|
126
|
-
Complete resolved profiles use
|
|
127
|
-
`configure_with_profile(profileJson, modelJson)`. This path never merges an
|
|
128
|
-
embedded preset, and it rejects profiles that contain no runtime mappings or
|
|
129
|
-
resolve no bindings against the inspected model. Selecting preset fallback for
|
|
130
|
-
an incomplete legacy profile remains an explicit host decision.
|
|
131
|
-
|
|
132
|
-
Preset/profile JSON crosses the Wasm boundary only during configuration. Rust
|
|
133
|
-
merges overrides, deserializes the runtime projection, compiles model bindings,
|
|
134
|
-
and returns packed numeric frame deltas on the hot path. Three.js object
|
|
135
|
-
inspection and frame application remain host-side concerns.
|
|
136
|
-
|
|
137
|
-
The package currently exports the Three.js inspector, frame applier, and clip
|
|
138
|
-
adapter. Polymer owns the public CLJS facade during the Rust migration; runtime
|
|
139
|
-
policy and lifecycle are moving into Rust, while engine-object inspection and
|
|
140
|
-
mutation remain in the Three.js adapter.
|
|
141
|
-
|
|
142
|
-
### Template skeleton fit metadata
|
|
143
|
-
|
|
144
|
-
Template skeleton fitting metadata is a host-neutral contract for recording how a
|
|
145
|
-
template skeleton should be previewed against a source character. It is metadata
|
|
146
|
-
for placement and QA only. It is not skinning data, bone weights, a rig
|
|
147
|
-
definition, or permission for the host to automatically bind the source mesh to
|
|
148
|
-
the template skeleton.
|
|
149
|
-
|
|
150
|
-
```typescript
|
|
151
|
-
import {
|
|
152
|
-
TEMPLATE_SKELETON_FIT_METADATA_KIND,
|
|
153
|
-
TEMPLATE_SKELETON_FIT_METADATA_VERSION,
|
|
154
|
-
composeTemplateSkeletonFitTransform,
|
|
155
|
-
validateTemplateSkeletonFitMetadata,
|
|
156
|
-
type TemplateSkeletonFitMetadata,
|
|
157
|
-
} from '@lovelace_lol/embody/core/contracts';
|
|
158
|
-
|
|
159
|
-
const fitMetadata: TemplateSkeletonFitMetadata = {
|
|
160
|
-
kind: TEMPLATE_SKELETON_FIT_METADATA_KIND,
|
|
161
|
-
version: TEMPLATE_SKELETON_FIT_METADATA_VERSION,
|
|
162
|
-
templateId: 'humanoid-basic',
|
|
163
|
-
sourceCharacterId: 'character-1',
|
|
164
|
-
verticalAxis: 'y',
|
|
165
|
-
verticalAnchor: 'min',
|
|
166
|
-
fit: {
|
|
167
|
-
scale: 1.2,
|
|
168
|
-
translation: { x: 0.1, y: 0.2, z: -0.1 },
|
|
169
|
-
confidence: 0.75,
|
|
170
|
-
meshHeight: 1.8,
|
|
171
|
-
templateHeight: 1.5,
|
|
172
|
-
crossAxisSpans: { x: 0.6, y: 1.8, z: 0.4 },
|
|
173
|
-
status: 'estimated',
|
|
174
|
-
},
|
|
175
|
-
manualAdjustment: {
|
|
176
|
-
scale: 1.05,
|
|
177
|
-
translation: { x: 0.01, y: -0.02, z: 0.03 },
|
|
178
|
-
},
|
|
179
|
-
};
|
|
180
|
-
|
|
181
|
-
const validation = validateTemplateSkeletonFitMetadata(fitMetadata);
|
|
182
|
-
const transform = composeTemplateSkeletonFitTransform(fitMetadata.fit, fitMetadata.manualAdjustment);
|
|
183
|
-
```
|
|
184
|
-
|
|
185
|
-
The required fields are:
|
|
186
|
-
- `kind`: must be `template-skeleton-fit` so hosts can distinguish this record
|
|
187
|
-
from skinning, rig, animation, or mesh metadata.
|
|
188
|
-
- `version`: must match `TEMPLATE_SKELETON_FIT_METADATA_VERSION`. Unknown
|
|
189
|
-
versions are rejected so migrations are explicit.
|
|
190
|
-
- `templateId` and `sourceCharacterId`: non-empty ids that let the host connect
|
|
191
|
-
the fit to its template registry and source asset record.
|
|
192
|
-
- `verticalAxis`: one of `x`, `y`, or `z`; the source character axis used for
|
|
193
|
-
height alignment.
|
|
194
|
-
- `verticalAnchor`: one of `min`, `center`, or `max`; the height anchor used
|
|
195
|
-
when solving the initial placement.
|
|
196
|
-
- `fit`: the solver result, including positive `scale`, finite `translation`, a
|
|
197
|
-
`confidence` value from `0` to `1`, positive mesh/template heights, non-negative
|
|
198
|
-
cross-axis spans, and a stable status.
|
|
199
|
-
- `manualAdjustment`: optional host-authored corrections. `scale` must be
|
|
200
|
-
positive and `translation` must be finite when present.
|
|
201
|
-
|
|
202
|
-
The stable status values are:
|
|
203
|
-
- `unfitted`: no usable solve exists yet.
|
|
204
|
-
- `estimated`: Embody produced an automatic fit that is ready for host preview.
|
|
205
|
-
- `manual-adjusted`: a user or host tool adjusted the automatic fit.
|
|
206
|
-
- `invalid`: the host should keep the metadata for inspection, but must not use
|
|
207
|
-
it as a usable placement.
|
|
208
|
-
|
|
209
|
-
Hosts should render the composed transform in a preview or setup flow first. The
|
|
210
|
-
host owns any final retargeting, rig binding, or skin-binding decision, and should
|
|
211
|
-
keep those records separate from `TemplateSkeletonFitMetadata`.
|
|
212
|
-
|
|
213
|
-
### Basic setup (default host)
|
|
214
|
-
|
|
215
|
-
Most callers only need a container element and a character URL. Embody creates
|
|
216
|
-
the Three.js scene (renderer, camera, lights, shadow plane), loads the model,
|
|
217
|
-
frames the camera on it, starts a render loop, and binds the runtime:
|
|
218
|
-
|
|
219
|
-
```typescript
|
|
220
|
-
import { createCharacterHost } from '@lovelace_lol/embody';
|
|
221
|
-
// or: import { createCharacterHost } from '@lovelace_lol/embody/three';
|
|
222
|
-
|
|
223
|
-
const host = await createCharacterHost({
|
|
224
|
-
container: document.getElementById('viewport')!,
|
|
225
|
-
character: {
|
|
226
|
-
modelUrl: '/characters/jonathan.glb',
|
|
227
|
-
presetType: 'cc4',
|
|
228
|
-
modelScale: 0.9,
|
|
229
|
-
},
|
|
230
|
-
});
|
|
231
|
-
|
|
232
|
-
// host.engine is a ready Embody instance
|
|
233
|
-
host.engine.setAU(12, 0.8);
|
|
234
|
-
|
|
235
|
-
// later
|
|
236
|
-
host.dispose();
|
|
237
|
-
```
|
|
238
|
-
|
|
239
|
-
`modelScale` is an optional positive finite uniform scale for the loaded model
|
|
240
|
-
root. When omitted, Embody preserves the scale authored in the asset. Embody
|
|
241
|
-
applies model scale before `modelGroundClearance`, so ground placement uses the
|
|
242
|
-
scaled model bounds.
|
|
243
|
-
|
|
244
|
-
Pick a named scene type to change the backdrop, lighting, and shadow defaults
|
|
245
|
-
(`studio` is the default; `showcase`, `inspection`, and `void` are also built in),
|
|
246
|
-
and override any individual option on top:
|
|
247
|
-
|
|
248
|
-
```typescript
|
|
249
|
-
const host = await createCharacterHost({
|
|
250
|
-
container,
|
|
251
|
-
character: { modelUrl: '/characters/jonathan.glb' },
|
|
252
|
-
scene: {
|
|
253
|
-
type: 'showcase', // dark backdrop, contrast lighting
|
|
254
|
-
// background: 0x223344, // per-option overrides still win
|
|
255
|
-
// lightingPreset: 'inspection',
|
|
256
|
-
// shadowPlane: false,
|
|
257
|
-
},
|
|
258
|
-
});
|
|
259
|
-
```
|
|
260
|
-
|
|
261
|
-
The available types are exported as `CHARACTER_SCENE_TYPES` /
|
|
262
|
-
`CHARACTER_SCENE_TYPE_IDS` so UIs can enumerate them.
|
|
263
|
-
|
|
264
|
-
The default host renders on its own (`renderLoop: true`) and auto-frames the
|
|
265
|
-
camera on the model (`autoFrame: true`). Turn either off when you drive the
|
|
266
|
-
camera or render loop yourself (e.g. custom camera controls):
|
|
267
|
-
|
|
268
|
-
```typescript
|
|
269
|
-
const host = await createCharacterHost({
|
|
270
|
-
container,
|
|
271
|
-
character: { modelUrl: '/characters/jonathan.glb' },
|
|
272
|
-
renderLoop: false, // you call renderer.render(scene, camera)
|
|
273
|
-
autoFrame: false, // you position the camera
|
|
274
|
-
});
|
|
275
|
-
```
|
|
276
|
-
|
|
277
|
-
With `external`, both default to `false` — your app owns rendering and camera.
|
|
278
|
-
|
|
279
|
-
### Advanced setup (bring your own Three.js scene)
|
|
280
|
-
|
|
281
|
-
Pass an existing scene/renderer/camera when you already own the WebGL host
|
|
282
|
-
(R3F, custom workbench, etc.). Embody still loads and binds the character:
|
|
283
|
-
|
|
284
|
-
```typescript
|
|
285
|
-
const host = await createCharacterHost({
|
|
286
|
-
container: mountEl,
|
|
287
|
-
character: { modelUrl: '/characters/jonathan.glb', presetType: 'cc4' },
|
|
288
|
-
external: {
|
|
289
|
-
scene,
|
|
290
|
-
renderer,
|
|
291
|
-
camera,
|
|
292
|
-
// mountCanvas: true, // optional
|
|
293
|
-
// manageResize: true, // optional
|
|
294
|
-
},
|
|
295
|
-
});
|
|
296
|
-
```
|
|
297
|
-
|
|
298
|
-
The lowest-level contract remains stable: `new Embody(...)` + `onReady({ meshes, model })`
|
|
299
|
-
if you want to own loading yourself.
|
|
300
|
-
|
|
301
|
-
```typescript
|
|
302
|
-
import * as THREE from 'three';
|
|
303
|
-
import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
|
|
304
|
-
import { Embody, collectMorphMeshes, CC4_PRESET } from '@lovelace_lol/embody';
|
|
305
|
-
|
|
306
|
-
const loom = new Embody({ profile: CC4_PRESET });
|
|
307
|
-
const loader = new GLTFLoader();
|
|
308
|
-
loader.load('/character.glb', (gltf) => {
|
|
309
|
-
scene.add(gltf.scene);
|
|
310
|
-
loom.onReady({ meshes: collectMorphMeshes(gltf.scene), model: gltf.scene });
|
|
311
|
-
});
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
If you’re implementing a custom renderer, target the `Embody` interface exported from `@lovelace_lol/embody` (legacy alias: `LoomLarge`).
|
|
315
|
-
|
|
316
|
-
### Lifecycle and update ownership
|
|
317
|
-
|
|
318
|
-
You have two valid integration patterns:
|
|
319
|
-
- External render loop: call `loom.update(deltaSeconds)` from your app’s main loop.
|
|
320
|
-
- Internal render loop: call `loom.start()` after `onReady()` and let Embody drive its own RAF-based updates.
|
|
321
|
-
|
|
322
|
-
The main lifecycle methods are:
|
|
323
|
-
- `onReady({ meshes, model })`: bind the loaded model to the engine
|
|
324
|
-
- `update(deltaSeconds)`: advance transitions, mixer playback, and runtime systems once
|
|
325
|
-
- `start()` / `stop()`: opt into or out of the internal loop
|
|
326
|
-
- `dispose()`: stop playback and release engine state when the character is torn down
|
|
327
|
-
|
|
328
|
-
### Quick start examples
|
|
329
|
-
|
|
330
|
-
Once your character is loaded, you can control facial expressions immediately:
|
|
331
|
-
|
|
332
|
-
```typescript
|
|
333
|
-
// Make the character smile
|
|
334
|
-
loom.setAU(12, 0.8);
|
|
335
|
-
|
|
336
|
-
// Raise eyebrows
|
|
337
|
-
loom.setAU(1, 0.6);
|
|
338
|
-
loom.setAU(2, 0.6);
|
|
339
|
-
|
|
340
|
-
// Blink
|
|
341
|
-
loom.setAU(45, 1.0);
|
|
342
|
-
|
|
343
|
-
// Open jaw
|
|
344
|
-
loom.setAU(26, 0.5);
|
|
345
|
-
|
|
346
|
-
// Turn head left
|
|
347
|
-
loom.setAU(51, 0.4);
|
|
348
|
-
|
|
349
|
-
// Look up
|
|
350
|
-
loom.setAU(63, 0.6);
|
|
351
|
-
```
|
|
352
|
-
|
|
353
|
-
Animate smoothly with transitions:
|
|
354
|
-
|
|
355
|
-
```typescript
|
|
356
|
-
// Smile over 200ms
|
|
357
|
-
await loom.transitionAU(12, 0.8, 200).promise;
|
|
358
|
-
|
|
359
|
-
// Then fade back to neutral
|
|
360
|
-
await loom.transitionAU(12, 0, 300).promise;
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
### The `collectMorphMeshes` helper
|
|
364
|
-
|
|
365
|
-
This utility function traverses a Three.js scene and returns all meshes that have `morphTargetInfluences` (i.e., blend shapes). It's the recommended way to gather meshes for Embody:
|
|
366
|
-
|
|
367
|
-
```typescript
|
|
368
|
-
import { collectMorphMeshes } from '@lovelace_lol/embody';
|
|
369
|
-
|
|
370
|
-
const meshes = collectMorphMeshes(gltf.scene);
|
|
371
|
-
// Returns: Array of THREE.Mesh objects with morph targets
|
|
372
|
-
```
|
|
373
|
-
|
|
374
|
-

|
|
375
|
-
|
|
376
|
-
---
|
|
377
|
-
|
|
378
|
-
## 2. Using Presets
|
|
379
|
-
|
|
380
|
-
Open in LoomLarge: [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
381
|
-
|
|
382
|
-

|
|
383
|
-
|
|
384
|
-
Presets define how FACS Action Units map to your character's morph targets and bones. Embody ships with `CC4_PRESET` for Character Creator 4 characters.
|
|
385
|
-
|
|
386
|
-
### What's in a preset?
|
|
387
|
-
|
|
388
|
-
```typescript
|
|
389
|
-
import { CC4_PRESET } from '@lovelace_lol/embody';
|
|
390
|
-
|
|
391
|
-
// CC4_PRESET contains:
|
|
392
|
-
{
|
|
393
|
-
auToMorphs: {
|
|
394
|
-
// AU number → morph target names split by side
|
|
395
|
-
1: { left: ['Brow_Raise_Inner_L'], right: ['Brow_Raise_Inner_R'], center: [] },
|
|
396
|
-
12: { left: ['Mouth_Smile_L'], right: ['Mouth_Smile_R'], center: [] },
|
|
397
|
-
45: { left: ['Eye_Blink_L'], right: ['Eye_Blink_R'], center: [] },
|
|
398
|
-
// ... 87 AUs total
|
|
399
|
-
},
|
|
400
|
-
|
|
401
|
-
auToBones: {
|
|
402
|
-
// AU number → array of bone bindings
|
|
403
|
-
51: [{ node: 'HEAD', channel: 'ry', scale: -1, maxDegrees: 30 }],
|
|
404
|
-
61: [{ node: 'EYE_L', channel: 'rz', scale: 1, maxDegrees: 25 }],
|
|
405
|
-
// ... 32 bone bindings
|
|
406
|
-
},
|
|
407
|
-
|
|
408
|
-
boneNodes: {
|
|
409
|
-
// Logical bone name → base node name used with bonePrefix
|
|
410
|
-
'HEAD': 'Head',
|
|
411
|
-
'JAW': 'JawRoot',
|
|
412
|
-
'EYE_L': 'L_Eye',
|
|
413
|
-
'EYE_R': 'R_Eye',
|
|
414
|
-
'TONGUE': 'Tongue01',
|
|
415
|
-
},
|
|
416
|
-
|
|
417
|
-
bonePrefix: 'CC_Base_',
|
|
418
|
-
suffixPattern: '_\\d+$|\\.\\d+$',
|
|
419
|
-
|
|
420
|
-
visemeKeys: [
|
|
421
|
-
// 15 viseme morph names for lip-sync
|
|
422
|
-
'EE', 'Ah', 'Oh', 'OO', 'I',
|
|
423
|
-
'U', 'W', 'L', 'F_V', 'Th',
|
|
424
|
-
'S_Z', 'B_M_P', 'K_G_H_NG', 'AE', 'R'
|
|
425
|
-
],
|
|
426
|
-
|
|
427
|
-
morphToMesh: {
|
|
428
|
-
// Routes morph categories to specific meshes
|
|
429
|
-
'face': ['CC_Base_Body'],
|
|
430
|
-
'viseme': ['CC_Base_Body', 'CC_Base_Body_1'],
|
|
431
|
-
'tongue': ['CC_Base_Tongue'],
|
|
432
|
-
'eye': ['CC_Base_EyeOcclusion_1', 'CC_Base_EyeOcclusion_2'],
|
|
433
|
-
'hair': ['Side_part_wavy_1', 'Side_part_wavy_2'],
|
|
434
|
-
},
|
|
435
|
-
|
|
436
|
-
auMixDefaults: {
|
|
437
|
-
// Default morph/bone blend weights (0 = morph, 1 = bone)
|
|
438
|
-
26: 0.5, // Jaw drop: 50% morph, 50% bone
|
|
439
|
-
51: 0.7, // Head turn: 70% bone
|
|
440
|
-
},
|
|
441
|
-
|
|
442
|
-
auInfo: {
|
|
443
|
-
// Metadata about each AU
|
|
444
|
-
'12': {
|
|
445
|
-
name: 'Lip Corner Puller',
|
|
446
|
-
muscularBasis: 'zygomaticus major',
|
|
447
|
-
faceArea: 'Lower',
|
|
448
|
-
facePart: 'Mouth',
|
|
449
|
-
},
|
|
450
|
-
// ...
|
|
451
|
-
}
|
|
452
|
-
}
|
|
453
|
-
```
|
|
454
|
-
|
|
455
|
-
### Name resolution and profile fields
|
|
456
|
-
|
|
457
|
-
The runtime resolves bone nodes by composing `bonePrefix + boneNodes[key] + boneSuffix`, then falling back to suffix-pattern matching when a model uses numbered exports such as `_01` or `.001`. The same prefix/suffix rules are used by validation and correction helpers, which is why `CC4_PRESET` can keep base bone names like `Head` and `JawRoot` instead of repeating the full node names everywhere.
|
|
458
|
-
|
|
459
|
-
For region and marker configs, `resolveBoneName()` treats any mapped bone name that already contains `_` or `.` as a fully qualified name and skips prefix/suffix composition.
|
|
460
|
-
|
|
461
|
-
Two caveats are worth calling out:
|
|
462
|
-
- `morphPrefix` and `morphSuffix` are part of `Profile`, but morph playback still resolves exact morph keys on the targeted meshes today. They are already used by validation and correction helpers.
|
|
463
|
-
- `leftMorphSuffixes` and `rightMorphSuffixes` are profile metadata for laterality detection in tooling, not core runtime behavior.
|
|
464
|
-
|
|
465
|
-
Other `Profile` fields that are easy to miss:
|
|
466
|
-
- `morphToMesh` routes categories such as `face`, `viseme`, `eye`, `tongue`, and `hair` to specific mesh names.
|
|
467
|
-
- `eyeMeshNodes` provides fallback eye nodes when a rig uses meshes instead of bones for eye control.
|
|
468
|
-
- `auMixDefaults` sets the default morph/bone blend weight per AU.
|
|
469
|
-
- `compositeRotations` defines the per-node pitch/yaw/roll axis layout used by the composite rotation system.
|
|
470
|
-
- `continuumPairs` and `continuumLabels` describe bidirectional AU pairs and their UI labels.
|
|
471
|
-
- `annotationRegions` defines the regions used by the marker and camera tooling, including per-region framing via `paddingFactor`.
|
|
472
|
-
- `hairPhysics` stores the mixer-driven hair defaults, including direction signs and morph target mappings.
|
|
473
|
-
|
|
474
|
-
For `annotationRegions`, `paddingFactor` is the camera framing multiplier for that region:
|
|
475
|
-
- values below `1` zoom in tighter
|
|
476
|
-
- values above `1` pull back to show more surrounding context
|
|
477
|
-
- profile overrides can replace it per region by name without copying the whole preset
|
|
478
|
-
|
|
479
|
-
### Passing a preset to Embody
|
|
480
|
-
|
|
481
|
-
```typescript
|
|
482
|
-
import { Embody, CC4_PRESET } from '@lovelace_lol/embody';
|
|
483
|
-
|
|
484
|
-
const loom = new Embody({ profile: CC4_PRESET });
|
|
485
|
-
```
|
|
486
|
-
|
|
487
|
-
You can also look up presets by name and extend them without cloning the full preset:
|
|
488
|
-
|
|
489
|
-
```typescript
|
|
490
|
-
import { Embody } from '@lovelace_lol/embody';
|
|
491
|
-
|
|
492
|
-
const loom = new Embody({
|
|
493
|
-
presetType: 'cc4',
|
|
494
|
-
profile: {
|
|
495
|
-
auToMorphs: {
|
|
496
|
-
12: { left: ['MySmile_Left'], right: ['MySmile_Right'], center: [] },
|
|
497
|
-
},
|
|
498
|
-
},
|
|
499
|
-
});
|
|
500
|
-
```
|
|
501
|
-
|
|
502
|
-
### Profiles (preset overrides)
|
|
503
|
-
|
|
504
|
-
A **profile** is a partial override object that extends a base preset. Use it to customize a single character without copying the full preset:
|
|
505
|
-
|
|
506
|
-
```typescript
|
|
507
|
-
import type { Profile } from '@lovelace_lol/embody';
|
|
508
|
-
import { Embody } from '@lovelace_lol/embody';
|
|
509
|
-
|
|
510
|
-
const DAISY_PROFILE: Profile = {
|
|
511
|
-
morphToMesh: { face: ['Object_9'] },
|
|
512
|
-
annotationRegions: [
|
|
513
|
-
{ name: 'face', bones: ['CC_Base_Head'] },
|
|
514
|
-
{ name: 'left_eye', paddingFactor: 0.9 },
|
|
515
|
-
{ name: 'right_eye', paddingFactor: 0.9 },
|
|
516
|
-
],
|
|
517
|
-
};
|
|
518
|
-
|
|
519
|
-
const loom = new Embody({
|
|
520
|
-
presetType: 'cc4',
|
|
521
|
-
profile: DAISY_PROFILE,
|
|
522
|
-
});
|
|
523
|
-
```
|
|
524
|
-
|
|
525
|
-
### Annotation configuration
|
|
526
|
-
|
|
527
|
-
`annotationRegions` is the Embody field for camera/marker region defaults and profile overrides.
|
|
528
|
-
|
|
529
|
-
If your app fetches a saved model/profile record from Firestore or another backend, use `extendProfileConfigWithPreset(...)` to build the runtime shape before handing that profile config to camera/marker tooling:
|
|
530
|
-
|
|
531
|
-
```typescript
|
|
532
|
-
import { extendProfileConfigWithPreset } from '@lovelace_lol/embody';
|
|
533
|
-
|
|
534
|
-
const savedConfig = await fetchProfileConfig();
|
|
535
|
-
const runtimeConfig = extendProfileConfigWithPreset({
|
|
536
|
-
...savedConfig,
|
|
537
|
-
profilePresetId: savedConfig.profilePresetId ?? 'cc4',
|
|
538
|
-
});
|
|
539
|
-
```
|
|
540
|
-
|
|
541
|
-
`CharacterConfig`, `auPresetType`, and `extendCharacterConfigWithPreset(...)` are still exported as deprecated compatibility aliases for apps migrating from older LoomLarge-style character records. New Embody integrations should model presets as base profiles, pass profile overrides through `profile`, `annotationRegions`, or other `Profile` fields, and use `profilePresetId` for preset selection.
|
|
542
|
-
|
|
543
|
-
For the current runtime-oriented documentation, including:
|
|
544
|
-
|
|
545
|
-
- `paddingFactor`
|
|
546
|
-
- `cameraAngle`
|
|
547
|
-
- `cameraOffset`
|
|
548
|
-
- `style.lineDirection`
|
|
549
|
-
- the difference between `cameraAngle: 0` and omitting `cameraAngle`
|
|
550
|
-
- runtime compatibility and legacy `config.regions` fallback behavior
|
|
551
|
-
|
|
552
|
-
see [ANNOTATION_CONFIGURATION.md](./ANNOTATION_CONFIGURATION.md).
|
|
553
|
-
|
|
554
|
-

|
|
555
|
-
|
|
556
|
-
---
|
|
557
|
-
|
|
558
|
-
## 3. Preset Selection & Validation
|
|
559
|
-
|
|
560
|
-
Open in LoomLarge: [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings) | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones)
|
|
561
|
-
|
|
562
|
-
Before you tune AUs or hand-edit a profile, confirm that you picked the right preset and that the model actually matches it. Embody exposes a full preset-selection and validation workflow, not just low-level control APIs.
|
|
563
|
-
|
|
564
|
-
### Looking Up and Extending Presets by Type
|
|
565
|
-
|
|
566
|
-
Use preset helpers when you want a stable entry point by model class instead of importing a preset constant directly:
|
|
567
|
-
|
|
568
|
-
```typescript
|
|
569
|
-
import {
|
|
570
|
-
getPreset,
|
|
571
|
-
getPresetWithProfile,
|
|
572
|
-
} from '@lovelace_lol/embody';
|
|
573
|
-
|
|
574
|
-
const preset = getPreset('cc4');
|
|
575
|
-
|
|
576
|
-
const extended = getPresetWithProfile('cc4', {
|
|
577
|
-
morphToMesh: { face: ['Object_9'] },
|
|
578
|
-
});
|
|
579
|
-
```
|
|
580
|
-
|
|
581
|
-
### Validating the config itself
|
|
582
|
-
|
|
583
|
-
`validateMappingConfig()` checks the profile for internal consistency before you even compare it to a model:
|
|
584
|
-
|
|
585
|
-
```typescript
|
|
586
|
-
import { validateMappingConfig } from '@lovelace_lol/embody';
|
|
587
|
-
|
|
588
|
-
const consistency = validateMappingConfig(resolved);
|
|
589
|
-
console.log(consistency.errors, consistency.warnings);
|
|
590
|
-
```
|
|
591
|
-
|
|
592
|
-
### Checking a model against a preset
|
|
593
|
-
|
|
594
|
-
```typescript
|
|
595
|
-
import * as THREE from 'three';
|
|
596
|
-
import {
|
|
597
|
-
validateMappings,
|
|
598
|
-
isPresetCompatible,
|
|
599
|
-
generateMappingCorrections,
|
|
600
|
-
} from '@lovelace_lol/embody';
|
|
601
|
-
|
|
602
|
-
const skinnedMesh = gltf.scene.getObjectByProperty('type', 'SkinnedMesh') as THREE.SkinnedMesh | undefined;
|
|
603
|
-
const skeleton = skinnedMesh?.skeleton ?? null;
|
|
604
|
-
|
|
605
|
-
const validation = validateMappings(meshes, skeleton, resolved, {
|
|
606
|
-
suggestCorrections: true,
|
|
607
|
-
});
|
|
608
|
-
|
|
609
|
-
const compatible = isPresetCompatible(meshes, skeleton, resolved);
|
|
610
|
-
const corrections = generateMappingCorrections(meshes, skeleton, resolved, {
|
|
611
|
-
useResolvedNames: true,
|
|
612
|
-
});
|
|
613
|
-
```
|
|
614
|
-
|
|
615
|
-
### Suggesting the best preset from a candidate set
|
|
616
|
-
|
|
617
|
-
```typescript
|
|
618
|
-
import {
|
|
619
|
-
CC4_PRESET,
|
|
620
|
-
BETTA_FISH_PRESET,
|
|
621
|
-
suggestBestPreset,
|
|
622
|
-
} from '@lovelace_lol/embody';
|
|
623
|
-
|
|
624
|
-
const best = suggestBestPreset(meshes, skeleton, [
|
|
625
|
-
CC4_PRESET,
|
|
626
|
-
BETTA_FISH_PRESET,
|
|
627
|
-
]);
|
|
628
|
-
```
|
|
629
|
-
|
|
630
|
-
### Running a full model analysis
|
|
631
|
-
|
|
632
|
-
```typescript
|
|
633
|
-
import {
|
|
634
|
-
analyzeModel,
|
|
635
|
-
extractFromGLTF,
|
|
636
|
-
extractModelData,
|
|
637
|
-
} from '@lovelace_lol/embody';
|
|
638
|
-
|
|
639
|
-
const extracted = extractFromGLTF(gltf);
|
|
640
|
-
const runtimeData = extractModelData(gltf.scene, meshes, gltf.animations);
|
|
641
|
-
|
|
642
|
-
const report = await analyzeModel({
|
|
643
|
-
source: { type: 'gltf', gltf },
|
|
644
|
-
preset: resolved,
|
|
645
|
-
suggestCorrections: true,
|
|
646
|
-
});
|
|
647
|
-
|
|
648
|
-
console.log(report.summary, report.overallScore);
|
|
649
|
-
```
|
|
650
|
-
|
|
651
|
-
Use this section when you need to:
|
|
652
|
-
- choose between built-in presets before wiring the character into your app
|
|
653
|
-
- lint a preset/profile for broken internal references
|
|
654
|
-
- measure how well a preset matches an imported model
|
|
655
|
-
- generate correction suggestions before building a custom profile
|
|
656
|
-
|
|
657
|
-
---
|
|
658
|
-
|
|
659
|
-
## 4. Getting to Know Your Character
|
|
660
|
-
|
|
661
|
-
Open in LoomLarge: [Meshes tab](https://www.characterloom.com/?drawer=open&tab=meshes) | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
662
|
-
|
|
663
|
-

|
|
664
|
-
|
|
665
|
-
Before customizing presets or extending mappings, it's helpful to understand what's actually in your character model. Embody provides several methods to inspect meshes, morph targets, and bones.
|
|
666
|
-
|
|
667
|
-
### Listing meshes
|
|
668
|
-
|
|
669
|
-
Get all meshes in your character with their visibility and morph target counts:
|
|
670
|
-
|
|
671
|
-
```typescript
|
|
672
|
-
const meshes = loom.getMeshList();
|
|
673
|
-
console.log(meshes);
|
|
674
|
-
// [
|
|
675
|
-
// { name: 'CC_Base_Body', visible: true, morphCount: 142 },
|
|
676
|
-
// { name: 'CC_Base_Tongue', visible: true, morphCount: 12 },
|
|
677
|
-
// { name: 'CC_Base_EyeOcclusion_1', visible: true, morphCount: 8 },
|
|
678
|
-
// { name: 'CC_Base_EyeOcclusion_2', visible: true, morphCount: 8 },
|
|
679
|
-
// { name: 'Male_Bushy_1', visible: true, morphCount: 142 },
|
|
680
|
-
// ...
|
|
681
|
-
// ]
|
|
682
|
-
```
|
|
683
|
-
|
|
684
|
-
### Listing morph targets
|
|
685
|
-
|
|
686
|
-
Get all morph target names grouped by mesh:
|
|
687
|
-
|
|
688
|
-
```typescript
|
|
689
|
-
const morphs = loom.getMorphTargets();
|
|
690
|
-
console.log(morphs);
|
|
691
|
-
// {
|
|
692
|
-
// 'CC_Base_Body': [
|
|
693
|
-
// 'A01_Brow_Inner_Up', 'A02_Brow_Down_Left', 'A02_Brow_Down_Right',
|
|
694
|
-
// 'A04_Brow_Outer_Up_Left', 'A04_Brow_Outer_Up_Right',
|
|
695
|
-
// 'Mouth_Smile_L', 'Mouth_Smile_R', 'Eye_Blink_L', 'Eye_Blink_R',
|
|
696
|
-
// ...
|
|
697
|
-
// ],
|
|
698
|
-
// 'CC_Base_Tongue': [
|
|
699
|
-
// 'V_Tongue_Out', 'V_Tongue_Up', 'V_Tongue_Down', ...
|
|
700
|
-
// ],
|
|
701
|
-
// ...
|
|
702
|
-
// }
|
|
703
|
-
```
|
|
704
|
-
|
|
705
|
-
This is invaluable when creating custom presets—you need to know the exact morph target names your character uses.
|
|
706
|
-
|
|
707
|
-
### Listing bones
|
|
708
|
-
|
|
709
|
-
Get all resolved bones with their current positions and rotations (in degrees):
|
|
710
|
-
|
|
711
|
-
```typescript
|
|
712
|
-
const bones = loom.getBones();
|
|
713
|
-
console.log(bones);
|
|
714
|
-
// {
|
|
715
|
-
// 'HEAD': { position: [0, 156.2, 0], rotation: [0, 0, 0] },
|
|
716
|
-
// 'JAW': { position: [0, 154.1, 2.3], rotation: [0, 0, 0] },
|
|
717
|
-
// 'EYE_L': { position: [-3.2, 160.5, 8.1], rotation: [0, 0, 0] },
|
|
718
|
-
// 'EYE_R': { position: [3.2, 160.5, 8.1], rotation: [0, 0, 0] },
|
|
719
|
-
// 'TONGUE': { position: [0, 152.3, 1.8], rotation: [0, 0, 0] },
|
|
720
|
-
// }
|
|
721
|
-
```
|
|
722
|
-
|
|
723
|
-
### Listing morph target indices
|
|
724
|
-
|
|
725
|
-
Use the index view when you need to work with `setMorphInfluence()` or build tools that operate on morph slots directly:
|
|
726
|
-
|
|
727
|
-
```typescript
|
|
728
|
-
const indices = loom.getMorphTargetIndices();
|
|
729
|
-
console.log(indices);
|
|
730
|
-
// {
|
|
731
|
-
// 'CC_Base_Body': [
|
|
732
|
-
// { index: 0, name: 'A01_Brow_Inner_Up' },
|
|
733
|
-
// { index: 1, name: 'A02_Brow_Down_Left' },
|
|
734
|
-
// ...
|
|
735
|
-
// ],
|
|
736
|
-
// }
|
|
737
|
-
```
|
|
738
|
-
|
|
739
|
-
### Validating and analyzing the model you loaded
|
|
740
|
-
|
|
741
|
-
Use the extraction and validation helpers when inspection needs to turn into preset-fit analysis:
|
|
742
|
-
|
|
743
|
-
```typescript
|
|
744
117
|
import {
|
|
745
|
-
|
|
746
|
-
|
|
747
|
-
analyzeModel,
|
|
748
|
-
validateMappings,
|
|
749
|
-
generateMappingCorrections,
|
|
750
|
-
getPreset,
|
|
751
|
-
} from '@lovelace_lol/embody';
|
|
752
|
-
|
|
753
|
-
const preset = getPreset('cc4');
|
|
754
|
-
const modelData = extractModelData(model, meshes, animations);
|
|
755
|
-
const gltfData = extractFromGLTF(gltf); // Same ModelData shape, one-step GLTF wrapper
|
|
756
|
-
|
|
757
|
-
const analysis = await analyzeModel({
|
|
758
|
-
source: { type: 'gltf', gltf },
|
|
759
|
-
preset,
|
|
760
|
-
suggestCorrections: true,
|
|
761
|
-
});
|
|
762
|
-
|
|
763
|
-
// Validate against lower-level mesh + skeleton inputs when you already have them
|
|
764
|
-
const validation = validateMappings(meshes, skeleton, preset, { suggestCorrections: true });
|
|
765
|
-
const corrections = generateMappingCorrections(meshes, skeleton, preset, { useResolvedNames: true });
|
|
766
|
-
```
|
|
767
|
-
|
|
768
|
-
If you already have a `ModelData` bundle, `analyzeModel()` is the higher-level path; `validateMappings()` and `generateMappingCorrections()` are intentionally lower-level mesh/skeleton helpers.
|
|
769
|
-
|
|
770
|
-
Use these helpers to:
|
|
771
|
-
- Extract raw model facts with `extractModelData(model, meshes?, animations?)` or `extractFromGLTF(gltf)`
|
|
772
|
-
- Validate a preset against mesh/skeleton data with `validateMappings(meshes, skeleton, preset, options)`
|
|
773
|
-
- Generate best-effort fixes with `generateMappingCorrections(meshes, skeleton, preset, options)`
|
|
774
|
-
- Run a single end-to-end pass with `analyzeModel({ source, preset, suggestCorrections })`
|
|
775
|
-
|
|
776
|
-
`validateMappings()` returns a `ValidationResult` with:
|
|
777
|
-
- `valid` and `score`
|
|
778
|
-
- `missingMorphs`, `missingBones`, `foundMorphs`, `foundBones`
|
|
779
|
-
- `missingMeshes`, `foundMeshes`, `unmappedMorphs`, `unmappedBones`, `unmappedMeshes`
|
|
780
|
-
- `warnings`
|
|
781
|
-
- optional `suggestedConfig`, `corrections`, and `unresolved` when suggestion mode is enabled
|
|
782
|
-
|
|
783
|
-
`generateMappingCorrections()` returns:
|
|
784
|
-
- `correctedConfig`
|
|
785
|
-
- `corrections`
|
|
786
|
-
- `unresolved`
|
|
787
|
-
|
|
788
|
-
`analyzeModel()` returns a `ModelAnalysisReport` containing the extracted model data, optional validation results, animation summary, `overallScore`, and a plain-language `summary`.
|
|
789
|
-
|
|
790
|
-
### Humanoid skeleton templates
|
|
791
|
-
|
|
792
|
-
Embody owns reusable humanoid skeleton templates and the extraction path for
|
|
793
|
-
turning a skinned character into template data. Host apps can select a template
|
|
794
|
-
by id, derive its rest bounds for fitting, or seed validation/mapping UI with
|
|
795
|
-
the template bone list:
|
|
796
|
-
|
|
797
|
-
```typescript
|
|
118
|
+
AnimationMixer,
|
|
119
|
+
} from 'three';
|
|
798
120
|
import {
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
|
|
802
|
-
|
|
803
|
-
|
|
804
|
-
|
|
805
|
-
const
|
|
806
|
-
const
|
|
807
|
-
const
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
811
|
-
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
The extracted template records joint names, parent links, and local
|
|
818
|
-
translations. It is fit/mapping metadata only; it does not include skin weights,
|
|
819
|
-
inverse bind matrices, or pose retargeting data.
|
|
820
|
-
|
|
821
|
-
### Controlling mesh visibility
|
|
822
|
-
|
|
823
|
-
Hide or show individual meshes:
|
|
824
|
-
|
|
825
|
-
```typescript
|
|
826
|
-
// Hide hair mesh
|
|
827
|
-
loom.setMeshVisible('Side_part_wavy_1', false);
|
|
828
|
-
|
|
829
|
-
// Show it again
|
|
830
|
-
loom.setMeshVisible('Side_part_wavy_1', true);
|
|
831
|
-
```
|
|
832
|
-
|
|
833
|
-
### Highlighting a mesh
|
|
834
|
-
|
|
835
|
-
Use highlighting when you need to confirm which mesh a profile field or morph category is actually targeting:
|
|
836
|
-
|
|
837
|
-
```typescript
|
|
838
|
-
// Highlight one mesh
|
|
839
|
-
loom.highlightMesh('CC_Base_Body');
|
|
840
|
-
|
|
841
|
-
// Clear all highlights
|
|
842
|
-
loom.highlightMesh(null);
|
|
843
|
-
```
|
|
844
|
-
|
|
845
|
-
### Adjusting material properties
|
|
846
|
-
|
|
847
|
-
Fine-tune render order, transparency, and blending for each mesh:
|
|
848
|
-
|
|
849
|
-
```typescript
|
|
850
|
-
// Get current material config
|
|
851
|
-
const config = loom.getMeshMaterialConfig('CC_Base_Body');
|
|
852
|
-
console.log(config);
|
|
853
|
-
// {
|
|
854
|
-
// renderOrder: 0,
|
|
855
|
-
// transparent: false,
|
|
856
|
-
// opacity: 1,
|
|
857
|
-
// depthWrite: true,
|
|
858
|
-
// depthTest: true,
|
|
859
|
-
// blending: 'Normal'
|
|
860
|
-
// }
|
|
861
|
-
|
|
862
|
-
// Set custom material config
|
|
863
|
-
loom.setMeshMaterialConfig('CC_Base_EyeOcclusion_1', {
|
|
864
|
-
renderOrder: 10,
|
|
865
|
-
transparent: true,
|
|
866
|
-
opacity: 0.8,
|
|
867
|
-
blending: 'Normal' // 'Normal', 'Additive', 'Subtractive', 'Multiply', 'None'
|
|
868
|
-
});
|
|
869
|
-
```
|
|
870
|
-
|
|
871
|
-
This is especially useful for:
|
|
872
|
-
- Fixing render order issues (eyebrows behind hair, etc.)
|
|
873
|
-
- Making meshes semi-transparent for debugging
|
|
874
|
-
- Adjusting blending modes for special effects
|
|
875
|
-
|
|
876
|
-

|
|
877
|
-
|
|
878
|
-
---
|
|
879
|
-
|
|
880
|
-
## 5. Extending & Custom Presets
|
|
881
|
-
|
|
882
|
-
Open in LoomLarge: [Properties tab](https://www.characterloom.com/?drawer=open&tab=properties) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
883
|
-
|
|
884
|
-

|
|
885
|
-
|
|
886
|
-
### Extending an existing preset
|
|
887
|
-
|
|
888
|
-
Use `extendPresetWithProfile` to override specific mappings while keeping the rest:
|
|
889
|
-
|
|
890
|
-
```typescript
|
|
891
|
-
import { CC4_PRESET, extendPresetWithProfile } from '@lovelace_lol/embody';
|
|
892
|
-
|
|
893
|
-
const MY_PRESET = extendPresetWithProfile(CC4_PRESET, {
|
|
894
|
-
|
|
895
|
-
// Override AU12 (smile) with custom morph names
|
|
896
|
-
auToMorphs: {
|
|
897
|
-
12: { left: ['MySmile_Left'], right: ['MySmile_Right'], center: [] },
|
|
898
|
-
},
|
|
899
|
-
|
|
900
|
-
// Add a new bone binding
|
|
901
|
-
auToBones: {
|
|
902
|
-
99: [{ node: 'CUSTOM_BONE', channel: 'ry', scale: 1, maxDegrees: 45 }],
|
|
903
|
-
},
|
|
904
|
-
|
|
905
|
-
// Update bone node paths
|
|
906
|
-
boneNodes: {
|
|
907
|
-
'CUSTOM_BONE': 'MyRig_CustomBone',
|
|
908
|
-
},
|
|
909
|
-
});
|
|
910
|
-
|
|
911
|
-
const loom = new Embody({ profile: MY_PRESET });
|
|
912
|
-
```
|
|
913
|
-
|
|
914
|
-
### Creating a preset from scratch
|
|
915
|
-
|
|
916
|
-
```typescript
|
|
917
|
-
import type { Profile } from '@lovelace_lol/embody';
|
|
918
|
-
|
|
919
|
-
const CUSTOM_PRESET: Profile = {
|
|
920
|
-
auToMorphs: {
|
|
921
|
-
1: { left: ['brow_inner_up_L'], right: ['brow_inner_up_R'], center: [] },
|
|
922
|
-
2: { left: ['brow_outer_up_L'], right: ['brow_outer_up_R'], center: [] },
|
|
923
|
-
12: { left: ['mouth_smile_L'], right: ['mouth_smile_R'], center: [] },
|
|
924
|
-
45: { left: ['eye_blink_L'], right: ['eye_blink_R'], center: [] },
|
|
925
|
-
},
|
|
926
|
-
|
|
927
|
-
auToBones: {
|
|
928
|
-
51: [{ node: 'HEAD', channel: 'ry', scale: -1, maxDegrees: 30 }],
|
|
929
|
-
52: [{ node: 'HEAD', channel: 'ry', scale: 1, maxDegrees: 30 }],
|
|
930
|
-
},
|
|
931
|
-
|
|
932
|
-
boneNodes: {
|
|
933
|
-
'HEAD': 'head_bone',
|
|
934
|
-
'JAW': 'jaw_bone',
|
|
935
|
-
},
|
|
936
|
-
|
|
937
|
-
visemeKeys: ['aa', 'ee', 'ih', 'oh', 'oo'],
|
|
938
|
-
|
|
939
|
-
morphToMesh: {
|
|
940
|
-
'face': ['body_mesh'],
|
|
941
|
-
},
|
|
942
|
-
};
|
|
943
|
-
```
|
|
944
|
-
|
|
945
|
-
### Changing presets at runtime
|
|
946
|
-
|
|
947
|
-
```typescript
|
|
948
|
-
// Switch to a different preset
|
|
949
|
-
loom.setProfile(ANOTHER_PRESET);
|
|
950
|
-
|
|
951
|
-
// Get current mappings
|
|
952
|
-
const current = loom.getProfile();
|
|
953
|
-
```
|
|
954
|
-
|
|
955
|
-

|
|
956
|
-
|
|
957
|
-
---
|
|
958
|
-
|
|
959
|
-
## 6. Creating Skeletal Animation Presets
|
|
960
|
-
|
|
961
|
-
Open in LoomLarge: [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) | [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units) | [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
962
|
-
|
|
963
|
-

|
|
964
|
-
|
|
965
|
-
Embody isn't limited to humanoid characters with morph targets. You can create presets for any 3D model that uses skeletal animation, such as fish, animals, or fantasy creatures. This section explains how to create a preset for a betta fish model that has no morph targets—only bone-driven animation.
|
|
966
|
-
|
|
967
|
-
### Understanding skeletal-only models
|
|
968
|
-
|
|
969
|
-
Some models (like fish) rely entirely on bone rotations for animation:
|
|
970
|
-
- **No morph targets:** All movement is skeletal
|
|
971
|
-
- **Hierarchical bones:** Fins and body parts follow parent rotations
|
|
972
|
-
- **Custom "Action Units":** Instead of FACS AUs, you define model-specific actions
|
|
973
|
-
|
|
974
|
-
### Example: Betta Fish Preset
|
|
975
|
-
|
|
976
|
-
Here's a complete example of a preset for a betta fish:
|
|
977
|
-
|
|
978
|
-
```typescript
|
|
979
|
-
import type { BoneBinding, AUInfo, CompositeRotation } from '@lovelace_lol/embody';
|
|
980
|
-
|
|
981
|
-
// Define semantic bone mappings
|
|
982
|
-
export const FISH_BONE_NODES = {
|
|
983
|
-
ROOT: 'Armature_rootJoint',
|
|
984
|
-
BODY_ROOT: 'Bone_Armature',
|
|
985
|
-
HEAD: 'Bone001_Armature',
|
|
986
|
-
BODY_FRONT: 'Bone002_Armature',
|
|
987
|
-
BODY_MID: 'Bone003_Armature',
|
|
988
|
-
BODY_BACK: 'Bone004_Armature',
|
|
989
|
-
TAIL_BASE: 'Bone005_Armature',
|
|
990
|
-
|
|
991
|
-
// Pectoral fins (side fins)
|
|
992
|
-
PECTORAL_L: 'Bone046_Armature',
|
|
993
|
-
PECTORAL_R: 'Bone047_Armature',
|
|
994
|
-
|
|
995
|
-
// Dorsal fin (top fin)
|
|
996
|
-
DORSAL_ROOT: 'Bone006_Armature',
|
|
997
|
-
|
|
998
|
-
// Eyes (single mesh for both)
|
|
999
|
-
EYE_L: 'EYES_0',
|
|
1000
|
-
EYE_R: 'EYES_0',
|
|
1001
|
-
} as const;
|
|
1002
|
-
|
|
1003
|
-
// Define custom fish actions (analogous to FACS AUs)
|
|
1004
|
-
export enum FishAction {
|
|
1005
|
-
// Body orientation
|
|
1006
|
-
TURN_LEFT = 2,
|
|
1007
|
-
TURN_RIGHT = 3,
|
|
1008
|
-
PITCH_UP = 4,
|
|
1009
|
-
PITCH_DOWN = 5,
|
|
1010
|
-
ROLL_LEFT = 6,
|
|
1011
|
-
ROLL_RIGHT = 7,
|
|
1012
|
-
|
|
1013
|
-
// Tail movements
|
|
1014
|
-
TAIL_SWEEP_LEFT = 12,
|
|
1015
|
-
TAIL_SWEEP_RIGHT = 13,
|
|
1016
|
-
TAIL_FIN_SPREAD = 14,
|
|
1017
|
-
TAIL_FIN_CLOSE = 15,
|
|
1018
|
-
|
|
1019
|
-
// Pectoral fins
|
|
1020
|
-
PECTORAL_L_UP = 20,
|
|
1021
|
-
PECTORAL_L_DOWN = 21,
|
|
1022
|
-
PECTORAL_R_UP = 22,
|
|
1023
|
-
PECTORAL_R_DOWN = 23,
|
|
121
|
+
ThreeFrameApplier,
|
|
122
|
+
ThreeModelInspector,
|
|
123
|
+
createAnimationClipFromClipIR,
|
|
124
|
+
} from '@lovelace_lol/embody/three';
|
|
125
|
+
|
|
126
|
+
const wasm = await initEmbodyCore();
|
|
127
|
+
const inspector = new ThreeModelInspector();
|
|
128
|
+
const applier = new ThreeFrameApplier();
|
|
129
|
+
const inspection = inspector.inspectModel(model, { profile });
|
|
130
|
+
const runtime = new wasm.RuntimeCore(0);
|
|
131
|
+
const mixer = new AnimationMixer(model);
|
|
132
|
+
|
|
133
|
+
applier.setBindings(inspection);
|
|
134
|
+
runtime.configure_with_preset(
|
|
135
|
+
'cc4',
|
|
136
|
+
JSON.stringify(profileOverrides ?? {}),
|
|
137
|
+
JSON.stringify(inspection.descriptor),
|
|
138
|
+
);
|
|
1024
139
|
|
|
1025
|
-
|
|
1026
|
-
|
|
1027
|
-
|
|
1028
|
-
|
|
1029
|
-
|
|
140
|
+
const clipIR = JSON.parse(runtime.build_clip(
|
|
141
|
+
'smile',
|
|
142
|
+
JSON.stringify(curves),
|
|
143
|
+
JSON.stringify({ intensityScale: 1 }),
|
|
144
|
+
));
|
|
145
|
+
const clip = createAnimationClipFromClipIR(clipIR, inspection);
|
|
146
|
+
mixer.clipAction(clip).play();
|
|
147
|
+
|
|
148
|
+
function update(dtSeconds: number) {
|
|
149
|
+
// Live AU/viseme packed frames, then host mixer owns clip lerp.
|
|
150
|
+
applier.applyPackedMorphFrameDelta(runtime.evaluate_morph_frame_delta());
|
|
151
|
+
applier.applyPackedBoneFrameDelta(runtime.evaluate_bone_frame_delta());
|
|
152
|
+
mixer.update(dtSeconds);
|
|
1030
153
|
}
|
|
1031
154
|
```
|
|
1032
155
|
|
|
1033
|
-
|
|
1034
|
-
|
|
1035
|
-
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1046
|
-
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
|
|
1053
|
-
|
|
1054
|
-
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1060
|
-
|
|
1061
|
-
|
|
1062
|
-
|
|
1063
|
-
|
|
1064
|
-
|
|
1065
|
-
|
|
1066
|
-
|
|
1067
|
-
|
|
1068
|
-
|
|
1069
|
-
|
|
1070
|
-
|
|
1071
|
-
|
|
1072
|
-
|
|
1073
|
-
|
|
1074
|
-
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
|
|
1084
|
-
|
|
1085
|
-
|
|
1086
|
-
|
|
1087
|
-
|
|
1088
|
-
},
|
|
1089
|
-
{
|
|
1090
|
-
node: 'TAIL_BASE',
|
|
1091
|
-
pitch: null,
|
|
1092
|
-
yaw: null,
|
|
1093
|
-
roll: {
|
|
1094
|
-
aus: [FishAction.TAIL_SWEEP_LEFT, FishAction.TAIL_SWEEP_RIGHT],
|
|
1095
|
-
axis: 'rz',
|
|
1096
|
-
negative: FishAction.TAIL_SWEEP_RIGHT,
|
|
1097
|
-
positive: FishAction.TAIL_SWEEP_LEFT
|
|
1098
|
-
},
|
|
1099
|
-
},
|
|
1100
|
-
{
|
|
1101
|
-
node: 'EYE_L',
|
|
1102
|
-
pitch: {
|
|
1103
|
-
aus: [FishAction.EYE_UP, FishAction.EYE_DOWN],
|
|
1104
|
-
axis: 'rx',
|
|
1105
|
-
negative: FishAction.EYE_DOWN,
|
|
1106
|
-
positive: FishAction.EYE_UP
|
|
1107
|
-
},
|
|
1108
|
-
yaw: {
|
|
1109
|
-
aus: [FishAction.EYE_LEFT, FishAction.EYE_RIGHT],
|
|
1110
|
-
axis: 'ry',
|
|
1111
|
-
negative: FishAction.EYE_RIGHT,
|
|
1112
|
-
positive: FishAction.EYE_LEFT
|
|
1113
|
-
},
|
|
1114
|
-
roll: null,
|
|
1115
|
-
},
|
|
156
|
+
Application-facing JavaScript APIs belong in the host package. Polymer owns
|
|
157
|
+
the CLJS character host used by LoomLarge and calls the Wasm exports directly.
|
|
158
|
+
|
|
159
|
+
### Direct viseme snapshots
|
|
160
|
+
|
|
161
|
+
`RuntimeCore.get_viseme(index)` and `get_viseme_jaw_scale(index)` read the
|
|
162
|
+
canonical direct viseme controls without changing a frame or sampling a host
|
|
163
|
+
mixer. Unset, cleared, or out-of-range slots return `0` and `1` respectively.
|
|
164
|
+
Hosts can snapshot these values before a temporary mouth-shape preview and
|
|
165
|
+
restore them with the existing setters; a zero value releases its live override.
|
|
166
|
+
|
|
167
|
+
### Preparing a default scene
|
|
168
|
+
|
|
169
|
+
`createDefaultCharacterSceneRuntime(container, { rendering: { preference: 'auto' } })`
|
|
170
|
+
returns a stable scene, camera, lighting controller and `rendering` settings
|
|
171
|
+
controller immediately. `ready` reports the initial attempt; failure retains the
|
|
172
|
+
controller for recovery through `setSettings`. `getSettings`, `getSnapshot` and
|
|
173
|
+
`subscribe` expose the requested preference, actual backend, status and errors.
|
|
174
|
+
Live changes prepare and replace only backend resources, preserving the character,
|
|
175
|
+
scene, camera, lighting and marker state. `DPthreeCameraController({ rendering,
|
|
176
|
+
scene, camera, domElement: container })` retargets its existing loop and controls;
|
|
177
|
+
hosts persist settings and display status without recreating the scene.
|
|
178
|
+
|
|
179
|
+
Auto tries native WebGPU then falls back only on acquisition/initialization
|
|
180
|
+
failure. Explicit WebGPU remains strict; material/preparation errors retain the
|
|
181
|
+
previous usable renderer. Device loss is observable and recoverable through the
|
|
182
|
+
same controller. `acquireRenderer()` leases protect captures and model mutations;
|
|
183
|
+
release them before requesting a switch. `releaseSceneResources()` drains rendering
|
|
184
|
+
before detaching and freeing borrowed resources, including after device loss or
|
|
185
|
+
scene disposal. Await `runtime.dispose()` before freeing resources on scene teardown.
|
|
186
|
+
|
|
187
|
+
The existing async factory remains available and rejects after cleanup on initial
|
|
188
|
+
failure. Its successful handles expose the same controller and dynamic backend /
|
|
189
|
+
renderer getters. The synchronous factory stays WebGL-specific. See the owning
|
|
190
|
+
contract for latest-request handling, capture/model leases and teardown ordering.
|
|
191
|
+
|
|
192
|
+
Three runtime and types are checked at 0.184.0, with a >=0.184.0 runtime peer.
|
|
193
|
+
Initial scene readiness does not prepare subsequently added models or establish
|
|
194
|
+
first-frame/visual parity. See [renderer selection and scene readiness](docs/SCENE_READINESS.md)
|
|
195
|
+
for ownership, version rationale and remaining real-browser acceptance.
|
|
196
|
+
|
|
197
|
+
### Bilateral snippet channels
|
|
198
|
+
|
|
199
|
+
AU balance uses the character's left/right: `-1` drives the left side,
|
|
200
|
+
`0` drives both, and `1` drives the right. `RuntimeCore.get_au_balance(id)`
|
|
201
|
+
returns the stored live balance, clamped by the existing setters to [-1, 1],
|
|
202
|
+
with neutral zero for an unset id or after `clear()`. Hosts can save it alongside
|
|
203
|
+
`get_au(id)` when serializing a manual pose.
|
|
204
|
+
|
|
205
|
+
Legacy `build_clip` curve maps use
|
|
206
|
+
`options.balanceMap[auId]`, falling back to `options.balance` and then zero.
|
|
207
|
+
For `build_typed_clip`, an AU target's explicit `balance` takes precedence:
|
|
208
|
+
|
|
209
|
+
```js
|
|
210
|
+
const channels = [
|
|
211
|
+
{ target: { type: 'au', id: 43, balance: -1 }, keyframes: leftWink },
|
|
212
|
+
{ target: { type: 'au', id: 43, balance: 1 }, keyframes: rightWink },
|
|
1116
213
|
];
|
|
1117
214
|
```
|
|
1118
215
|
|
|
1119
|
-
|
|
1120
|
-
|
|
1121
|
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
|
|
1129
|
-
|
|
1130
|
-
|
|
1131
|
-
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
1147
|
-
|
|
1148
|
-
|
|
1149
|
-
|
|
1150
|
-
|
|
1151
|
-
|
|
1152
|
-
|
|
1153
|
-
|
|
1154
|
-
|
|
1155
|
-
|
|
1156
|
-
|
|
1157
|
-
|
|
1158
|
-
|
|
1159
|
-
|
|
1160
|
-
|
|
1161
|
-
|
|
1162
|
-
|
|
1163
|
-
|
|
1164
|
-
|
|
1165
|
-
|
|
1166
|
-
|
|
1167
|
-
|
|
1168
|
-
|
|
1169
|
-
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
|
|
1173
|
-
|
|
1174
|
-
|
|
1175
|
-
|
|
1176
|
-
|
|
1177
|
-
|
|
1178
|
-
|
|
1179
|
-
|
|
1180
|
-
|
|
1181
|
-
|
|
1182
|
-
|
|
1183
|
-
|
|
1184
|
-
|
|
1185
|
-
|
|
1186
|
-
|
|
1187
|
-
|
|
1188
|
-
|
|
1189
|
-
|
|
216
|
+
Each channel retains its own keyframe times and intensity scale. Typed AU and
|
|
217
|
+
viseme ids belong to separate namespaces, regardless of `snippetCategory`.
|
|
218
|
+
Duplicate typed channels for the same AU or viseme and concrete morph target
|
|
219
|
+
combine by maximum, including intersections between authored linear curves.
|
|
220
|
+
Independent endpoint-balanced channels keep their own inherited or explicit
|
|
221
|
+
starts; an inactive opposite-side contribution cannot replace an active
|
|
222
|
+
channel's anchor. Composite bone rotations take the maximum AU contribution
|
|
223
|
+
after each channel's balance. Legacy curves and distinct semantic ids that
|
|
224
|
+
share a morph retain their existing separate tracks and host blending.
|
|
225
|
+
|
|
226
|
+
A duplicate typed group with multiple enabled contributions to the same morph
|
|
227
|
+
cannot also contain an inherited start: compilation rejects that combination
|
|
228
|
+
with an error identifying the AU/viseme and destination. Author one curve per
|
|
229
|
+
side or explicit starting values. Previously, duplicate ids silently discarded
|
|
230
|
+
all but the last channel. Inheritance on supported morph tracks is captured
|
|
231
|
+
when the Three clip is constructed. Hosts must construct it again to capture
|
|
232
|
+
a fresh pose; this does not change cached host replay or generated bone
|
|
233
|
+
inheritance behavior.
|
|
234
|
+
|
|
235
|
+
### Gaze geometry contract
|
|
236
|
+
|
|
237
|
+
`solve_profile_screen_space_gaze` and `solve_profile_viewer_space_gaze` return
|
|
238
|
+
14 floats: combined target XY, eye target XY, head target XY, total yaw/pitch,
|
|
239
|
+
camera yaw/pitch, eye-to-camera distance, and viewer world XYZ. XY outputs are
|
|
240
|
+
signed AU intensities in [-1, 1]; positive Y is up and horizontal direction is
|
|
241
|
+
subject-relative. Angular values are degrees.
|
|
242
|
+
|
|
243
|
+
Pass current world camera position/quaternion, character eye midpoint, and
|
|
244
|
+
model quaternion. The viewer solver takes normalized image XY (+Y up) and
|
|
245
|
+
depth behind the virtual camera; depth and all positions must share scene
|
|
246
|
+
units. Convert physical webcam estimates before calling and supply the actual
|
|
247
|
+
source FOV/aspect. Zero viewer XY extends the character-eye-to-camera bearing,
|
|
248
|
+
so the character looks at the viewer even when scene framing puts its eyes
|
|
249
|
+
away from image center. Offsets retain the camera's right/up axes; when the
|
|
250
|
+
camera coincides with the eyes, its local +Z supplies the fallback bearing.
|
|
251
|
+
The depth guard is 0.2–10 scene units. Mouse and webcam can share this contract
|
|
252
|
+
when they use the same calibration.
|
|
253
|
+
|
|
254
|
+
Ordinary head following participates toward the common target from calibrated
|
|
255
|
+
model-neutral +Z: `head_follow_fraction` is a share of the target yaw/pitch,
|
|
256
|
+
defaulting to 0.35 for a non-finite value. Zero prefers neutral, one follows the
|
|
257
|
+
full target, and eye overflow can recruit more head movement within its limits.
|
|
258
|
+
The explicit `lock_head_to_camera` flag instead prefers the camera bearing while
|
|
259
|
+
eyes can reach; a zero follow fraction alone does not select that policy. When
|
|
260
|
+
eyes are disabled the head carries the target regardless of follow or lock.
|
|
261
|
+
This removes the previous implicit camera bias, which could point the head
|
|
262
|
+
opposite the selected target when the camera was off-center. Hosts that intend
|
|
263
|
+
camera-facing behavior must request the explicit viewer-solver lock; Polymer's
|
|
264
|
+
`headFollowEyes: false` maps to zero participation, not an implicit lock.
|
|
265
|
+
Eye angles are solved in the
|
|
266
|
+
rotated head frame, using yaw then pitch to match runtime composition. Limits
|
|
267
|
+
and AU normalization are directional and include each binding's scale. Missing
|
|
268
|
+
or morph-only mappings contribute no inferred angular capacity.
|
|
269
|
+
|
|
270
|
+
The profile must describe the active rig with calibrated semantic axes:
|
|
271
|
+
model +Z is forward, +Y is up. This ABI does not include the skeleton's rest
|
|
272
|
+
frames or optical axes, per-eye origins, current animated head pose, or
|
|
273
|
+
webcam-to-display calibration. It solves an endpoint in the assumed calibrated
|
|
274
|
+
basis; it does not provide binocular convergence or compensate intermediate
|
|
275
|
+
head motion. Shared eye commands use the smaller mapped eye capacity; unequal
|
|
276
|
+
left/right ranges need separate calibration. Hosts must compose the requested
|
|
277
|
+
rotations without averaging independent yaw/pitch clips together.
|
|
278
|
+
|
|
279
|
+
For import-time inspection, `captureModelReferencePose` provides an explicit,
|
|
280
|
+
immutable transform and morph reference that hosts can reuse after playback.
|
|
281
|
+
`extendModelReferencePose` explicitly adds appended skeletons while preserving
|
|
282
|
+
the captured parent transforms. It does not infer a bind pose or perform retargeting. See
|
|
283
|
+
[Animation reference poses](docs/ANIMATION_REFERENCE_POSES.md) for the contract,
|
|
284
|
+
step/linear clip conversion, and remaining import work.
|
|
285
|
+
|
|
286
|
+
## Presets And Profiles
|
|
287
|
+
|
|
288
|
+
### Full humanoid skeleton and body authoring
|
|
289
|
+
|
|
290
|
+
`humanoid.getSpecification` exposes all 55 VRM 1.0 anatomical roles and their
|
|
291
|
+
parent/required rules. CC4 maps all 55 to distinct authored bones and supplies
|
|
292
|
+
58 body controls for fingers, wrists, shoulders, feet, whole body motion, the
|
|
293
|
+
spine, head, eyes and jaw. `profile.setHumanoidRoleBinding` edits or clears a role and
|
|
294
|
+
retargets its existing actuators. The runtime keeps the same bone/morph evaluator;
|
|
295
|
+
other rigs must author their own calibrated axes/ranges and optional muscle
|
|
296
|
+
morph targets. See [Body controls](docs/body-controls.md) for migration, validation
|
|
297
|
+
and role-assignment contracts.
|
|
298
|
+
|
|
299
|
+
|
|
300
|
+
`cc4` and `fish` are embedded in the Wasm binary. `skeletal` is accepted as a
|
|
301
|
+
host-level alias for `fish`; exact custom profiles use
|
|
302
|
+
`RuntimeCore.configure_with_profile` and are never silently merged with CC4.
|
|
303
|
+
|
|
304
|
+
```ts
|
|
305
|
+
const ids = wasm.list_presets();
|
|
306
|
+
const cc4 = JSON.parse(wasm.get_preset_json('cc4'));
|
|
307
|
+
const merged = JSON.parse(wasm.merge_embedded_preset('cc4', JSON.stringify(overrides)));
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
Host-neutral profile authoring operations use one JSON request entry point:
|
|
311
|
+
|
|
312
|
+
```ts
|
|
313
|
+
const result = JSON.parse(wasm.embody_request(JSON.stringify({
|
|
314
|
+
op: 'bone.applyAxisUpdate',
|
|
315
|
+
payload: { profile, boneName: 'CC_Base_Head', axis: 'yaw', update },
|
|
316
|
+
})));
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
This boundary keeps ClojureScript and other hosts independent of Rust struct
|
|
320
|
+
layouts while ensuring profile semantics execute in Rust.
|
|
321
|
+
|
|
322
|
+
Speech articulation data belongs to the rig preset. The CC4 profile keeps two
|
|
323
|
+
ordered tables beside its canonical `visemeSlots`: `visemeJawAmounts` stores
|
|
324
|
+
the jaw opening for each viseme index, and `visemeTongueTargets` stores the AU
|
|
325
|
+
target map for that same index. Empty tongue maps mean that viseme has no
|
|
326
|
+
tongue overlay. Hosts pass these resolved tables to their lip-sync planner;
|
|
327
|
+
they should not recreate rig-specific jaw or tongue values in application code.
|
|
328
|
+
The live runtime also compiles both tables: `setViseme` applies the viseme morph,
|
|
329
|
+
the mapped jaw opening, and the mapped tongue AUs from the same preset entry.
|
|
330
|
+
|
|
331
|
+
## Model Analysis
|
|
332
|
+
|
|
333
|
+
Three.js models are reduced to a renderer-neutral descriptor by
|
|
334
|
+
`ThreeModelInspector`. Rust consumes that descriptor for extraction and
|
|
335
|
+
validation:
|
|
336
|
+
|
|
337
|
+
```ts
|
|
338
|
+
const clips = serializeAnimationClips(model, animations, inspection);
|
|
339
|
+
const report = JSON.parse(wasm.analyze_model_descriptor(
|
|
340
|
+
JSON.stringify(inspection.descriptor),
|
|
341
|
+
JSON.stringify(clips),
|
|
342
|
+
JSON.stringify(profile),
|
|
343
|
+
JSON.stringify({ suggestCorrections: true }),
|
|
344
|
+
));
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
## Pose-aware focus
|
|
348
|
+
|
|
349
|
+
`ThreeGazeFocus` from `@lovelace_lol/embody/three` applies a focus constraint
|
|
350
|
+
inside an animation runtime. It has no clock or target-selection policy:
|
|
351
|
+
call `restore()` before evaluating the base animation, then `apply(request,
|
|
352
|
+
controls)` after it. Restoration compares quaternion orientations independently
|
|
353
|
+
of Float32 sample norms, preserving the exact authored sample and any newer
|
|
354
|
+
authored orientation. Repeated applications between mixer ticks do not accumulate
|
|
355
|
+
the constraint's previous correction. Controls are the current positions of existing motor
|
|
356
|
+
tracks. The optional sampled `controls.headIntensity` and `controls.eyeIntensity`
|
|
357
|
+
override the matching request fields, so the host can animate configuration
|
|
358
|
+
changes on its existing motor tracks. Both mean contribution in [0, 1]: zero
|
|
359
|
+
preserves the evaluated base, one applies the full solved excursion, and legacy
|
|
360
|
+
values above one normalize to one. Intermediate eye contribution proportionally
|
|
361
|
+
reduces the correction; it does not shrink anatomical limits. Reduced eye
|
|
362
|
+
contribution deliberately leaves focus error. These controls do not amplify
|
|
363
|
+
input displacement or control response speed.
|
|
364
|
+
|
|
365
|
+
During travel, the remaining sampled bearing rotates the requested finite
|
|
366
|
+
target. At the endpoint that rotation becomes identity, reaching the exact
|
|
367
|
+
world target continuously. There is no settled-threshold switch between targets
|
|
368
|
+
constructed from the authored and tracking-shifted eye origins; a larger head
|
|
369
|
+
turn must not introduce a final eye snap.
|
|
370
|
+
|
|
371
|
+
Pass `{ referencePose }` as the constructor's third argument, reusing the
|
|
372
|
+
reference captured at import. Without it, construction captures the current
|
|
373
|
+
pose, so callers must construct before playback. The head solves its bounded
|
|
374
|
+
gaze excursion in that reference hierarchy under the model's current scene
|
|
375
|
+
transform, then composes the excursion onto the evaluated authored head pose.
|
|
376
|
+
Head and neck gestures retain their motion instead of being cancelled or
|
|
377
|
+
reversed by tracking. Each eye then solves toward the same finite target from
|
|
378
|
+
its own origin in the final animated hierarchy. Full eye contribution retains
|
|
379
|
+
binocular focus wherever the combined authored motion and gaze are reachable;
|
|
380
|
+
otherwise diagnostics report the remaining angular error and limits. Authored
|
|
381
|
+
gestures are not suppressed to force a target into range. The signed limits
|
|
382
|
+
bound the tracking excursion; they do not clamp the final authored pose plus
|
|
383
|
+
that excursion against a neutral-reference joint range. Authored motion can
|
|
384
|
+
therefore take the combined pose beyond that range or the eyes' reach.
|
|
385
|
+
|
|
386
|
+
`readControlState({worldTarget, headIntensity, eyeIntensity})` provides initial
|
|
387
|
+
motor bearings without changing the rig. It inverts existing focus contributions
|
|
388
|
+
against their saved base, and seeds zero added head excursion when enabling on
|
|
389
|
+
a purely authored pose. When replacing legacy AU controls, also provide the
|
|
390
|
+
evaluated local `headBaseQuaternion` and/or `eyeBaseQuaternions` (a map keyed by
|
|
391
|
+
bone name), with those legacy controls excluded. `headSeedLimited` and
|
|
392
|
+
`eyeSeedLimited` report poses that cannot be represented by the bounded motor
|
|
393
|
+
bearings. Parallel legacy eyes cannot both be preserved exactly by a shared
|
|
394
|
+
finite target; inverse eye contribution avoids double attenuation but does not
|
|
395
|
+
remove that convergence residual. This helper supplies no handoff transition
|
|
396
|
+
clock; the host owns activation policy and movement timing.
|
|
397
|
+
|
|
398
|
+
The optional `profile.gazeCalibration` supplies bone-local
|
|
399
|
+
`head`, `leftEye`, and `rightEye` optical axes and `modelUnitsPerMeter`.
|
|
400
|
+
Without explicit optical axes, signed yaw/pitch bindings determine the optical
|
|
401
|
+
frame. Unresolvable joints remain untouched and are reported in diagnostics.
|
|
402
|
+
|
|
403
|
+
`solve_profile_viewer_space_gaze_scaled` adds `world_units_per_meter` as its
|
|
404
|
+
last argument. Multiply the authored units-per-meter calibration by the
|
|
405
|
+
presentation's uniform root scale. The original viewer solver keeps its
|
|
406
|
+
existing scale-one behavior. Neither solver infers physical webcam/display
|
|
407
|
+
placement from face landmarks.
|
|
408
|
+
|
|
409
|
+
`solve_profile_tracking_gaze` maps normalized display-right/up controls to the
|
|
410
|
+
profile's full signed head AU rotation range. Pass `head_follow_fraction = 1`
|
|
411
|
+
for full participation: on an aligned display axis, an input endpoint of +/-1
|
|
412
|
+
requests the same head rotation as the corresponding AU at intensity 1. Binding
|
|
413
|
+
scale and asymmetric `maxDegrees` remain authoritative. Camera distance and FOV
|
|
414
|
+
do not reduce this range. Intermediate inputs interpolate between the bounded
|
|
415
|
+
camera bearing at zero and the corresponding signed endpoint; camera roll
|
|
416
|
+
rotates the display axes into the model frame. With head tracking disabled,
|
|
417
|
+
the target spans the eye AU range instead.
|
|
418
|
+
|
|
419
|
+
The signature is `(profile_json, tracking_target, camera_position,
|
|
420
|
+
camera_quaternion, gaze_origin, model_quaternion, eyes_enabled, head_enabled,
|
|
421
|
+
head_follow_fraction, lock_head_to_camera, world_units_per_meter)`. It returns
|
|
422
|
+
the same 14-float layout as the existing physical projection solvers. Target Z
|
|
423
|
+
is depth in meters, clamped to 0.2–10 before scene scaling; it sets a finite
|
|
424
|
+
binocular focus distance without attenuating the angular range. X/Y clamp to
|
|
425
|
+
[-1, 1], nonfinite coordinates become zero, invalid positive depth/scale use
|
|
426
|
+
0.8/1 respectively, and absent AU bindings supply no angular capacity.
|
|
427
|
+
|
|
428
|
+
This is an input-range mapping, not a physical reconstruction of the viewer's
|
|
429
|
+
position. The existing screen/viewer projection APIs retain their contracts.
|
|
430
|
+
Consumers must select this entry point and request full head participation to
|
|
431
|
+
use it; adding the export alone does not change LoomLarge tracking. Eye focus
|
|
432
|
+
still uses the resulting finite target and reports residual error when reduced
|
|
433
|
+
strength, authored motion, or rig limits make that target unreachable.
|
|
434
|
+
|
|
435
|
+
## npm releases and PR previews
|
|
436
|
+
|
|
437
|
+
Production consumers install an exact published `@lovelace_lol/embody` version
|
|
438
|
+
from public npm without read credentials and commit their lockfile. The GitHub
|
|
439
|
+
source repository stays private. A stable release contains the compiled JS,
|
|
440
|
+
TypeScript declarations, and Wasm; consumers do not build Embody from Git.
|
|
441
|
+
|
|
442
|
+
The `Verify and publish package` workflow publishes a stable npm release after
|
|
443
|
+
a push to `main`, or a manual run on `main` with the `ref` input left empty.
|
|
444
|
+
It chooses the next patch version, records the source commit in `gitHead`, and
|
|
445
|
+
builds and validates once. Stable version selection reads public metadata from
|
|
446
|
+
`registry.npmjs.org` anonymously; redirects and failed lookups stop the release.
|
|
447
|
+
Metadata reads never transmit publishing credentials. The separate
|
|
448
|
+
publish job first requires `npm access get status` to report the existing package
|
|
449
|
+
as public, then uses the `npm` environment's `NPM_KEY` to publish the packed
|
|
450
|
+
output with public access and creates its Git tag and GitHub release. Stable
|
|
451
|
+
releases are serialized through version selection and publication. Provenance
|
|
452
|
+
remains disabled for the private source repository. Retrying a published commit
|
|
453
|
+
reuses its existing version, and an older unpublished commit cannot replace a
|
|
454
|
+
newer source release.
|
|
455
|
+
|
|
456
|
+
PRs, including drafts, manual runs with an explicit `ref`, and `publish-pkg-pr-new`
|
|
457
|
+
repository dispatches run full verification and publish immutable `pkg.pr.new`
|
|
458
|
+
previews for testing linked upstream changes. An explicit ref remains a preview
|
|
459
|
+
request even when it names `main`; it does not publish to npm. Manual and
|
|
460
|
+
repository dispatch requests may include `upstream_pr` to identify their linked PR.
|
|
461
|
+
|
|
462
|
+
After publication, the producing workflow uploads a JSON notification record
|
|
463
|
+
without a dispatch credential. The separate `Notify package publication`
|
|
464
|
+
`workflow_run` receiver checks out `main`, treats the artifact only as data, and
|
|
465
|
+
validates the successful run attempt, repository, workflow, and source commit.
|
|
466
|
+
Stable records must identify that run's main commit; linked previews must match
|
|
467
|
+
the current open PR head targeting main. The receiver then sends
|
|
468
|
+
`upstream-package-published` to `meekmachine/LoomLarge` using
|
|
469
|
+
`CROSS_REPO_CI_TOKEN` from the `release-coordination` environment. That token must
|
|
470
|
+
not be stored as a repository secret or exposed to a PR build/helper.
|
|
471
|
+
|
|
472
|
+
Stable notifications contain the exact source SHA and npm version. Preview
|
|
473
|
+
notifications contain the full-SHA URL and upstream PR number, or `null` for an
|
|
474
|
+
unlinked manual preview. The receiver checks the canonical preview URL once and
|
|
475
|
+
fails visibly if it is unavailable. LoomLarge owns downstream dependency updates;
|
|
476
|
+
Embody does not edit Polymer or application branches. Missing credentials and
|
|
477
|
+
failed notifications fail the receiver visibly. Fork PRs do not send
|
|
478
|
+
notifications or receive npm/dispatch secrets.
|
|
479
|
+
|
|
480
|
+
GitHub activates the receiver after its workflow is present on `main`. The
|
|
481
|
+
release-stack PR's first preview remains usable by its immutable URL before
|
|
482
|
+
that bootstrap merge; subsequent successful publications notify automatically.
|
|
483
|
+
|
|
484
|
+
Before enabling stable releases, the existing npm package must have public
|
|
485
|
+
visibility and `NPM_KEY` must permit publishing it. No npm read token is required.
|
|
486
|
+
Configure both the `npm` and `release-coordination` GitHub
|
|
487
|
+
environments to allow deployments from `main` only. Store `CROSS_REPO_CI_TOKEN`
|
|
488
|
+
only in `release-coordination`, with permission to dispatch to LoomLarge. These
|
|
489
|
+
account, environment protection, and secret settings are prerequisites; source
|
|
490
|
+
changes do not configure them. Existing versions are never republished or
|
|
491
|
+
overwritten.
|
|
492
|
+
|
|
493
|
+
## Development
|
|
1190
494
|
|
|
1191
|
-
|
|
1192
|
-
|
|
1193
|
-
|
|
1194
|
-
|
|
1195
|
-
|
|
1196
|
-
|
|
1197
|
-
fishController.setAU(FishAction.TURN_LEFT, 0.5); // Turn left
|
|
1198
|
-
fishController.setAU(FishAction.TAIL_SWEEP_LEFT, 0.8); // Sweep tail
|
|
1199
|
-
fishController.setAU(FishAction.PECTORAL_L_UP, 0.6); // Raise left fin
|
|
1200
|
-
|
|
1201
|
-
// Smooth transitions
|
|
1202
|
-
await fishController.transitionAU(FishAction.TURN_RIGHT, 1.0, 500).promise;
|
|
1203
|
-
});
|
|
495
|
+
```bash
|
|
496
|
+
npm ci
|
|
497
|
+
npm run build
|
|
498
|
+
npm test
|
|
499
|
+
npm run test:package
|
|
500
|
+
npm run check:generated
|
|
1204
501
|
```
|
|
1205
502
|
|
|
1206
|
-
|
|
1207
|
-
|
|
1208
|
-
|
|
1209
|
-
|
|
1210
|
-
```typescript
|
|
1211
|
-
// Use setContinuum for paired actions
|
|
1212
|
-
fishController.setContinuum(
|
|
1213
|
-
FishAction.TURN_LEFT,
|
|
1214
|
-
FishAction.TURN_RIGHT,
|
|
1215
|
-
0.3 // Slight turn right
|
|
1216
|
-
);
|
|
1217
|
-
|
|
1218
|
-
// Animate swimming with oscillating tail
|
|
1219
|
-
async function swimCycle() {
|
|
1220
|
-
while (true) {
|
|
1221
|
-
await fishController.transitionContinuum(
|
|
1222
|
-
FishAction.TAIL_SWEEP_LEFT,
|
|
1223
|
-
FishAction.TAIL_SWEEP_RIGHT,
|
|
1224
|
-
0.8, // Sweep right
|
|
1225
|
-
300
|
|
1226
|
-
).promise;
|
|
1227
|
-
|
|
1228
|
-
await fishController.transitionContinuum(
|
|
1229
|
-
FishAction.TAIL_SWEEP_LEFT,
|
|
1230
|
-
FishAction.TAIL_SWEEP_RIGHT,
|
|
1231
|
-
-0.8, // Sweep left
|
|
1232
|
-
300
|
|
1233
|
-
).promise;
|
|
1234
|
-
}
|
|
1235
|
-
}
|
|
1236
|
-
```
|
|
1237
|
-
|
|
1238
|
-

|
|
1239
|
-
|
|
1240
|
-
---
|
|
1241
|
-
|
|
1242
|
-
## 7. Action Unit Control
|
|
1243
|
-
|
|
1244
|
-
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units)
|
|
1245
|
-
|
|
1246
|
-

|
|
1247
|
-
|
|
1248
|
-
Action Units are the core of FACS. Each AU represents a specific muscular movement of the face.
|
|
1249
|
-
|
|
1250
|
-
### Setting an AU immediately
|
|
1251
|
-
|
|
1252
|
-
```typescript
|
|
1253
|
-
// Set AU12 (smile) to 80% intensity
|
|
1254
|
-
loom.setAU(12, 0.8);
|
|
1255
|
-
|
|
1256
|
-
// Set AU45 (blink) to full intensity
|
|
1257
|
-
loom.setAU(45, 1.0);
|
|
1258
|
-
|
|
1259
|
-
// Set to 0 to deactivate
|
|
1260
|
-
loom.setAU(12, 0);
|
|
1261
|
-
```
|
|
1262
|
-
|
|
1263
|
-
### Transitioning an AU over time
|
|
1264
|
-
|
|
1265
|
-
```typescript
|
|
1266
|
-
// Animate AU12 to 0.8 over 200ms
|
|
1267
|
-
const handle = loom.transitionAU(12, 0.8, 200);
|
|
1268
|
-
|
|
1269
|
-
// Wait for completion
|
|
1270
|
-
await handle.promise;
|
|
1271
|
-
|
|
1272
|
-
// Or chain transitions
|
|
1273
|
-
loom.transitionAU(12, 1.0, 200).promise.then(() => {
|
|
1274
|
-
loom.transitionAU(12, 0, 300); // Fade out
|
|
1275
|
-
});
|
|
1276
|
-
```
|
|
1277
|
-
|
|
1278
|
-
### Getting the current AU value
|
|
1279
|
-
|
|
1280
|
-
```typescript
|
|
1281
|
-
const smileAmount = loom.getAU(12);
|
|
1282
|
-
console.log(`Current smile: ${smileAmount}`);
|
|
1283
|
-
```
|
|
1284
|
-
|
|
1285
|
-
### Asymmetric control with balance
|
|
1286
|
-
|
|
1287
|
-
Many AUs have left and right variants (e.g., `Mouth_Smile_L` and `Mouth_Smile_R`). The `balance` parameter lets you control them independently:
|
|
1288
|
-
|
|
1289
|
-
```typescript
|
|
1290
|
-
// Balance range: -1 (left only) to +1 (right only), 0 = both equal
|
|
1291
|
-
|
|
1292
|
-
// Smile on both sides equally
|
|
1293
|
-
loom.setAU(12, 0.8, 0);
|
|
1294
|
-
|
|
1295
|
-
// Smile only on left side
|
|
1296
|
-
loom.setAU(12, 0.8, -1);
|
|
1297
|
-
|
|
1298
|
-
// Smile only on right side
|
|
1299
|
-
loom.setAU(12, 0.8, 1);
|
|
1300
|
-
|
|
1301
|
-
// 70% left, 30% right
|
|
1302
|
-
loom.setAU(12, 0.8, -0.4);
|
|
1303
|
-
```
|
|
1304
|
-
|
|
1305
|
-
### String-based side selection
|
|
1306
|
-
|
|
1307
|
-
You can also specify the side directly in the AU ID:
|
|
1308
|
-
|
|
1309
|
-
```typescript
|
|
1310
|
-
// These are equivalent:
|
|
1311
|
-
loom.setAU('12L', 0.8); // Left side only
|
|
1312
|
-
loom.setAU(12, 0.8, -1); // Left side only
|
|
1313
|
-
|
|
1314
|
-
loom.setAU('12R', 0.8); // Right side only
|
|
1315
|
-
loom.setAU(12, 0.8, 1); // Right side only
|
|
1316
|
-
```
|
|
1317
|
-
|
|
1318
|
-
---
|
|
1319
|
-
|
|
1320
|
-
## 8. Mix Weight System
|
|
1321
|
-
|
|
1322
|
-
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units)
|
|
1323
|
-
|
|
1324
|
-

|
|
1325
|
-
|
|
1326
|
-
Some AUs can be driven by both morph targets (blend shapes) AND bone rotations. The mix weight controls the blend between them.
|
|
1327
|
-
|
|
1328
|
-
### Why mix weights?
|
|
1329
|
-
|
|
1330
|
-
Take jaw opening (AU26) as an example:
|
|
1331
|
-
- **Morph-only (weight 0)**: Vertices deform to show open mouth, but jaw bone doesn't move
|
|
1332
|
-
- **Bone-only (weight 1)**: Jaw bone rotates down, but no soft tissue deformation
|
|
1333
|
-
- **Mixed (weight 0.5)**: Both contribute equally for realistic results
|
|
1334
|
-
|
|
1335
|
-
### Setting mix weights
|
|
1336
|
-
|
|
1337
|
-
```typescript
|
|
1338
|
-
// Get the default mix weight for AU26
|
|
1339
|
-
const weight = loom.getAUMixWeight(26); // e.g., 0.5
|
|
1340
|
-
|
|
1341
|
-
// Set to pure morph
|
|
1342
|
-
loom.setAUMixWeight(26, 0);
|
|
1343
|
-
|
|
1344
|
-
// Set to pure bone
|
|
1345
|
-
loom.setAUMixWeight(26, 1);
|
|
1346
|
-
|
|
1347
|
-
// Set to 70% bone, 30% morph
|
|
1348
|
-
loom.setAUMixWeight(26, 0.7);
|
|
1349
|
-
```
|
|
1350
|
-
|
|
1351
|
-
### Which AUs support mixing?
|
|
1352
|
-
|
|
1353
|
-
Only AUs that have both `auToMorphs` AND `auToBones` entries support mixing. Common examples:
|
|
1354
|
-
- AU26 (Jaw Drop)
|
|
1355
|
-
- AU27 (Mouth Stretch)
|
|
1356
|
-
- AU51-56 (Head movements)
|
|
1357
|
-
- AU61-72 (Shared + independent eye movements)
|
|
1358
|
-
|
|
1359
|
-
```typescript
|
|
1360
|
-
import { isMixedAU } from '@lovelace_lol/embody';
|
|
1361
|
-
|
|
1362
|
-
if (isMixedAU(26)) {
|
|
1363
|
-
console.log('AU26 supports morph/bone mixing');
|
|
1364
|
-
}
|
|
1365
|
-
```
|
|
1366
|
-
|
|
1367
|
-
---
|
|
1368
|
-
|
|
1369
|
-
## 9. Composite Rotation System
|
|
1370
|
-
|
|
1371
|
-
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units) | [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones)
|
|
1372
|
-
|
|
1373
|
-

|
|
1374
|
-
|
|
1375
|
-
Bones like the head and eyes need multi-axis rotation (pitch, yaw, roll). The composite rotation system handles this automatically.
|
|
1376
|
-
|
|
1377
|
-
### How it works
|
|
1378
|
-
|
|
1379
|
-
When you set an AU that affects a bone rotation, Embody:
|
|
1380
|
-
1. Queues the rotation update in `pendingCompositeNodes`
|
|
1381
|
-
2. At the end of `update()`, calls `flushPendingComposites()`
|
|
1382
|
-
3. Applies all three axes (pitch, yaw, roll) together to prevent gimbal issues
|
|
1383
|
-
|
|
1384
|
-
### Supported bones and their axes
|
|
1385
|
-
|
|
1386
|
-
| Bone | Pitch (X) | Yaw (Y) | Roll (Z) |
|
|
1387
|
-
|------|-----------|---------|----------|
|
|
1388
|
-
| HEAD | AU53 (up) / AU54 (down) | AU51 (left) / AU52 (right) | AU55 (tilt left) / AU56 (tilt right) |
|
|
1389
|
-
| EYE_L | AU63 (up) / AU64 (down) | AU61 (left) / AU62 (right) | - |
|
|
1390
|
-
| EYE_R | AU63 (up) / AU64 (down) | AU61 (left) / AU62 (right) | - |
|
|
1391
|
-
| JAW | AU25-27 (open) | AU30 (left) / AU35 (right) | - |
|
|
1392
|
-
| TONGUE | AU37 (up) / AU38 (down) | AU39 (left) / AU40 (right) | AU41 / AU42 (tilt) |
|
|
1393
|
-
|
|
1394
|
-
### Example: Moving the head
|
|
1395
|
-
|
|
1396
|
-
```typescript
|
|
1397
|
-
// Turn head left 50%
|
|
1398
|
-
loom.setAU(51, 0.5);
|
|
1399
|
-
|
|
1400
|
-
// Turn head right 50%
|
|
1401
|
-
loom.setAU(52, 0.5);
|
|
1402
|
-
|
|
1403
|
-
// Tilt head up 30%
|
|
1404
|
-
loom.setAU(53, 0.3);
|
|
1405
|
-
|
|
1406
|
-
// Combine: turn left AND tilt up
|
|
1407
|
-
loom.setAU(51, 0.5);
|
|
1408
|
-
loom.setAU(53, 0.3);
|
|
1409
|
-
// Both are applied together in a single composite rotation
|
|
1410
|
-
```
|
|
1411
|
-
|
|
1412
|
-
### Example: Eye gaze
|
|
1413
|
-
|
|
1414
|
-
```typescript
|
|
1415
|
-
// Look left
|
|
1416
|
-
loom.setAU(61, 0.7);
|
|
1417
|
-
|
|
1418
|
-
// Look right
|
|
1419
|
-
loom.setAU(62, 0.7);
|
|
1420
|
-
|
|
1421
|
-
// Look up
|
|
1422
|
-
loom.setAU(63, 0.5);
|
|
1423
|
-
|
|
1424
|
-
// Look down-right (combined)
|
|
1425
|
-
loom.setAU(62, 0.6);
|
|
1426
|
-
loom.setAU(64, 0.4);
|
|
1427
|
-
```
|
|
1428
|
-
|
|
1429
|
-
---
|
|
1430
|
-
|
|
1431
|
-
## 10. Continuum Pairs
|
|
1432
|
-
|
|
1433
|
-
Open in LoomLarge: [Action Units tab](https://www.characterloom.com/?drawer=open&tab=action-units)
|
|
1434
|
-
|
|
1435
|
-

|
|
1436
|
-
|
|
1437
|
-
Continuum pairs are bidirectional AU pairs that represent opposite directions on the same axis. They're linked so that activating one should deactivate the other.
|
|
1438
|
-
|
|
1439
|
-
### Pair mappings
|
|
1440
|
-
|
|
1441
|
-
| Pair | Description |
|
|
1442
|
-
|------|-------------|
|
|
1443
|
-
| AU51 ↔ AU52 | Head turn left / right |
|
|
1444
|
-
| AU53 ↔ AU54 | Head up / down |
|
|
1445
|
-
| AU55 ↔ AU56 | Head tilt left / right |
|
|
1446
|
-
| AU61 ↔ AU62 | Eyes look left / right |
|
|
1447
|
-
| AU63 ↔ AU64 | Eyes look up / down |
|
|
1448
|
-
| AU30 ↔ AU35 | Jaw shift left / right |
|
|
1449
|
-
| AU37 ↔ AU38 | Tongue up / down |
|
|
1450
|
-
| AU39 ↔ AU40 | Tongue left / right |
|
|
1451
|
-
| AU73 ↔ AU74 | Tongue narrow / wide |
|
|
1452
|
-
| AU76 ↔ AU77 | Tongue tip up / down |
|
|
1453
|
-
|
|
1454
|
-
### Negative value shorthand (recommended)
|
|
1455
|
-
|
|
1456
|
-
The simplest way to work with continuum pairs is using **negative values**. When you pass a negative value to `setAU()` or `transitionAU()`, the engine automatically activates the paired AU instead:
|
|
1457
|
-
|
|
1458
|
-
```typescript
|
|
1459
|
-
// Head looking left at 50% (AU51 is "head left")
|
|
1460
|
-
loom.setAU(51, 0.5);
|
|
1461
|
-
|
|
1462
|
-
// Head looking right at 50% - just use a negative value!
|
|
1463
|
-
loom.setAU(51, -0.5); // Automatically activates AU52 at 0.5
|
|
1464
|
-
|
|
1465
|
-
// This is equivalent to manually setting the pair:
|
|
1466
|
-
loom.setAU(51, 0);
|
|
1467
|
-
loom.setAU(52, 0.5);
|
|
1468
|
-
```
|
|
1469
|
-
|
|
1470
|
-
This works for transitions too:
|
|
1471
|
-
|
|
1472
|
-
```typescript
|
|
1473
|
-
// Animate head from left to right over 500ms
|
|
1474
|
-
await loom.transitionAU(51, 0.5, 250).promise; // Turn left
|
|
1475
|
-
await loom.transitionAU(51, -0.5, 500).promise; // Turn right (activates AU52)
|
|
1476
|
-
```
|
|
1477
|
-
|
|
1478
|
-
### The setContinuum method
|
|
1479
|
-
|
|
1480
|
-
For explicit continuum control, use `setContinuum()` with a single value from -1 to +1:
|
|
1481
|
-
|
|
1482
|
-
```typescript
|
|
1483
|
-
// setContinuum(negativeAU, positiveAU, value)
|
|
1484
|
-
// value: -1 = full negative, 0 = neutral, +1 = full positive
|
|
1485
|
-
|
|
1486
|
-
// Head centered
|
|
1487
|
-
loom.setContinuum(51, 52, 0);
|
|
1488
|
-
|
|
1489
|
-
// Head 50% left
|
|
1490
|
-
loom.setContinuum(51, 52, -0.5);
|
|
1491
|
-
|
|
1492
|
-
// Head 70% right
|
|
1493
|
-
loom.setContinuum(51, 52, 0.7);
|
|
1494
|
-
```
|
|
1495
|
-
|
|
1496
|
-
With smooth animation:
|
|
1497
|
-
|
|
1498
|
-
```typescript
|
|
1499
|
-
// Animate head from current position to 80% right over 300ms
|
|
1500
|
-
await loom.transitionContinuum(51, 52, 0.8, 300).promise;
|
|
1501
|
-
|
|
1502
|
-
// Animate eyes to look left over 200ms
|
|
1503
|
-
await loom.transitionContinuum(61, 62, -0.6, 200).promise;
|
|
1504
|
-
```
|
|
1505
|
-
|
|
1506
|
-
### Manual pair management
|
|
1507
|
-
|
|
1508
|
-
You can also manually manage pairs by setting each AU individually:
|
|
1509
|
-
|
|
1510
|
-
```typescript
|
|
1511
|
-
// Head looking left at 50%
|
|
1512
|
-
loom.setAU(51, 0.5);
|
|
1513
|
-
loom.setAU(52, 0); // Right should be 0
|
|
1514
|
-
|
|
1515
|
-
// Head looking right at 70%
|
|
1516
|
-
loom.setAU(51, 0); // Left should be 0
|
|
1517
|
-
loom.setAU(52, 0.7);
|
|
1518
|
-
```
|
|
1519
|
-
|
|
1520
|
-
### The CONTINUUM_PAIRS_MAP
|
|
1521
|
-
|
|
1522
|
-
You can access pair information programmatically:
|
|
1523
|
-
|
|
1524
|
-
```typescript
|
|
1525
|
-
import { CONTINUUM_PAIRS_MAP } from '@lovelace_lol/embody';
|
|
1526
|
-
|
|
1527
|
-
const pair = CONTINUUM_PAIRS_MAP[51];
|
|
1528
|
-
// { pairId: 52, isNegative: true, axis: 'yaw', node: 'HEAD' }
|
|
1529
|
-
```
|
|
1530
|
-
|
|
1531
|
-
---
|
|
1532
|
-
|
|
1533
|
-
## 11. Direct Morph Control
|
|
1534
|
-
|
|
1535
|
-
Open in LoomLarge: [Meshes tab](https://www.characterloom.com/?drawer=open&tab=meshes) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
1536
|
-
|
|
1537
|
-

|
|
1538
|
-
|
|
1539
|
-
Sometimes you need to control morph targets directly by name, bypassing the AU system.
|
|
1540
|
-
|
|
1541
|
-
### Setting a morph immediately
|
|
1542
|
-
|
|
1543
|
-
```typescript
|
|
1544
|
-
// Set a specific morph to 50%
|
|
1545
|
-
loom.setMorph('Mouth_Smile_L', 0.5);
|
|
1546
|
-
|
|
1547
|
-
// Set on specific meshes only
|
|
1548
|
-
loom.setMorph('Mouth_Smile_L', 0.5, ['CC_Base_Body']);
|
|
1549
|
-
```
|
|
1550
|
-
|
|
1551
|
-
### Transitioning a morph
|
|
1552
|
-
|
|
1553
|
-
```typescript
|
|
1554
|
-
// Animate morph over 200ms
|
|
1555
|
-
const handle = loom.transitionMorph('Mouth_Smile_L', 0.8, 200);
|
|
1556
|
-
|
|
1557
|
-
// With mesh targeting
|
|
1558
|
-
loom.transitionMorph('Eye_Blink_L', 1.0, 100, ['CC_Base_Body']);
|
|
1559
|
-
|
|
1560
|
-
// Wait for completion
|
|
1561
|
-
await handle.promise;
|
|
1562
|
-
```
|
|
1563
|
-
|
|
1564
|
-
### Resolving current morph targets
|
|
1565
|
-
|
|
1566
|
-
```typescript
|
|
1567
|
-
const targets = loom.resolveMorphTargets('Mouth_Smile_L', ['CC_Base_Body']);
|
|
1568
|
-
const value = targets.length > 0 ? (targets[0].infl[targets[0].idx] ?? 0) : 0;
|
|
1569
|
-
```
|
|
1570
|
-
|
|
1571
|
-
### Adding runtime morph targets
|
|
1572
|
-
|
|
1573
|
-
Generated or sidecar morph targets can be registered after a model loads. Deltas use the same relative `POSITION` format as glTF morph targets: one XYZ delta per base mesh vertex. Optional `normal` and `tangent` deltas can be supplied when available.
|
|
1574
|
-
|
|
1575
|
-
```typescript
|
|
1576
|
-
const index = loom.addMorphTarget({
|
|
1577
|
-
meshName: 'CC_Base_Body',
|
|
1578
|
-
name: 'BodyType_Muscular',
|
|
1579
|
-
position: bodyTypeMuscularDeltas,
|
|
1580
|
-
});
|
|
1581
|
-
|
|
1582
|
-
loom.setMorphInfluence(index, 0.6, ['CC_Base_Body']);
|
|
1583
|
-
loom.setMorph('BodyType_Muscular', 0.6, ['CC_Base_Body']);
|
|
1584
|
-
```
|
|
1585
|
-
|
|
1586
|
-
By default, Embody replaces and disposes the mesh `BufferGeometry` before appending morph attributes. This is intentional: Three.js does not support mutating `geometry.morphAttributes` in place after a geometry has rendered. For pre-render authoring paths, pass `{ forceGeometryReplacement: false }`.
|
|
1587
|
-
|
|
1588
|
-
If you need a named slot before real deltas are available, use `ensureMorphInfluence(meshName, morphName)`. It creates a zero-delta target and returns the assigned `morphTargetInfluences` index. After external code changes morph dictionaries or geometry, call `refreshMorphTargets()` so AU, viseme, hair, and clip-building caches see the updated targets.
|
|
1589
|
-
|
|
1590
|
-
### Morph caching
|
|
1591
|
-
|
|
1592
|
-
Embody caches morph target lookups for performance. The first time you access a morph, it searches all meshes and caches the index. Subsequent accesses are O(1).
|
|
1593
|
-
|
|
1594
|
-
---
|
|
1595
|
-
|
|
1596
|
-
## 12. Viseme System
|
|
1597
|
-
|
|
1598
|
-
Open in LoomLarge: [Visemes tab](https://www.characterloom.com/?drawer=open&tab=visemes) | [Speech tab](https://www.characterloom.com/?drawer=open&tab=speech)
|
|
1599
|
-
|
|
1600
|
-

|
|
1601
|
-
|
|
1602
|
-
This screenshot was captured before the viseme label refresh, so some cards still show legacy names such as `Er`, `IH`, and `W_OO`. Treat the table below as the source of truth for the current exported `VISEME_KEYS` order.
|
|
1603
|
-
|
|
1604
|
-
Visemes are mouth shapes used for lip-sync. Embody includes 15 visemes with automatic jaw coupling.
|
|
1605
|
-
|
|
1606
|
-
### The 15 visemes
|
|
1607
|
-
|
|
1608
|
-
The `VISEME_KEYS` export uses unprefixed keys in this order.
|
|
1609
|
-
|
|
1610
|
-
| Index | Key | Phoneme Example |
|
|
1611
|
-
|-------|-----|-----------------|
|
|
1612
|
-
| 0 | EE | "b**ee**" |
|
|
1613
|
-
| 1 | Ah | "f**a**ther" |
|
|
1614
|
-
| 2 | Oh | "g**o**" |
|
|
1615
|
-
| 3 | OO | "t**oo**" |
|
|
1616
|
-
| 4 | I | "s**i**t" |
|
|
1617
|
-
| 5 | U | "fl**u**te" |
|
|
1618
|
-
| 6 | W | "**w**e" |
|
|
1619
|
-
| 7 | L | "**l**ip" |
|
|
1620
|
-
| 8 | F_V | "**f**un, **v**an" |
|
|
1621
|
-
| 9 | Th | "**th**ink" |
|
|
1622
|
-
| 10 | S_Z | "**s**un, **z**oo" |
|
|
1623
|
-
| 11 | B_M_P | "**b**at, **m**an, **p**op" |
|
|
1624
|
-
| 12 | K_G_H_NG | "**k**ite, **g**o, **h**at, si**ng**" |
|
|
1625
|
-
| 13 | AE | "c**a**t" |
|
|
1626
|
-
| 14 | R | "**r**ed" |
|
|
1627
|
-
|
|
1628
|
-
### Setting a viseme
|
|
1629
|
-
|
|
1630
|
-
```typescript
|
|
1631
|
-
// Set viseme 3 (Ah) to full intensity
|
|
1632
|
-
loom.setViseme(3, 1.0);
|
|
1633
|
-
|
|
1634
|
-
// With jaw scale (0-1, default 1)
|
|
1635
|
-
loom.setViseme(3, 1.0, 0.5); // Half jaw opening
|
|
1636
|
-
```
|
|
1637
|
-
|
|
1638
|
-
### Transitioning visemes
|
|
1639
|
-
|
|
1640
|
-
Viseme transitions default to 80ms and use the standard `easeInOutQuad` easing when no duration is provided.
|
|
1641
|
-
|
|
1642
|
-
```typescript
|
|
1643
|
-
// Animate to a viseme using the default 80ms duration
|
|
1644
|
-
const handle = loom.transitionViseme(3, 1.0);
|
|
1645
|
-
|
|
1646
|
-
// Disable jaw coupling (duration can be omitted to use the 80ms default)
|
|
1647
|
-
loom.transitionViseme(3, 1.0, 80, 0);
|
|
1648
|
-
```
|
|
1649
|
-
|
|
1650
|
-
### Automatic jaw coupling
|
|
1651
|
-
|
|
1652
|
-
Each viseme has a predefined jaw opening amount in the preset. When you set a viseme, the jaw automatically opens proportionally, and the `jawScale` parameter multiplies that amount:
|
|
1653
|
-
- `jawScale = 1.0`: Normal jaw opening
|
|
1654
|
-
- `jawScale = 0.5`: Half jaw opening
|
|
1655
|
-
- `jawScale = 0`: No jaw movement (viseme only)
|
|
1656
|
-
|
|
1657
|
-
### Lip-sync example
|
|
1658
|
-
|
|
1659
|
-
```typescript
|
|
1660
|
-
async function speak(phonemes: number[]) {
|
|
1661
|
-
for (const viseme of phonemes) {
|
|
1662
|
-
// Clear previous viseme
|
|
1663
|
-
for (let i = 0; i < 15; i++) loom.setViseme(i, 0);
|
|
1664
|
-
|
|
1665
|
-
// Transition to new viseme
|
|
1666
|
-
await loom.transitionViseme(viseme, 1.0, 80).promise;
|
|
1667
|
-
|
|
1668
|
-
// Hold briefly
|
|
1669
|
-
await new Promise(r => setTimeout(r, 100));
|
|
1670
|
-
}
|
|
1671
|
-
|
|
1672
|
-
// Return to neutral
|
|
1673
|
-
for (let i = 0; i < 15; i++) loom.setViseme(i, 0);
|
|
1674
|
-
}
|
|
1675
|
-
|
|
1676
|
-
// "Hello" approximation
|
|
1677
|
-
speak([5, 0, 10, 4]);
|
|
1678
|
-
```
|
|
1679
|
-
|
|
1680
|
-
---
|
|
1681
|
-
|
|
1682
|
-
## 13. Transition System
|
|
1683
|
-
|
|
1684
|
-
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
1685
|
-
|
|
1686
|
-

|
|
1687
|
-
|
|
1688
|
-
All animated changes in Embody go through the transition system, which provides smooth interpolation with easing.
|
|
1689
|
-
|
|
1690
|
-
### TransitionHandle
|
|
1691
|
-
|
|
1692
|
-
Every transition method returns a `TransitionHandle`:
|
|
1693
|
-
|
|
1694
|
-
```typescript
|
|
1695
|
-
interface TransitionHandle {
|
|
1696
|
-
promise: Promise<void>; // Resolves when transition completes
|
|
1697
|
-
pause(): void; // Pause this transition
|
|
1698
|
-
resume(): void; // Resume this transition
|
|
1699
|
-
cancel(): void; // Cancel immediately
|
|
1700
|
-
}
|
|
1701
|
-
```
|
|
1702
|
-
|
|
1703
|
-
### Using handles
|
|
1704
|
-
|
|
1705
|
-
```typescript
|
|
1706
|
-
// Start a transition
|
|
1707
|
-
const handle = loom.transitionAU(12, 1.0, 500);
|
|
1708
|
-
|
|
1709
|
-
// Pause it
|
|
1710
|
-
handle.pause();
|
|
1711
|
-
|
|
1712
|
-
// Resume later
|
|
1713
|
-
handle.resume();
|
|
1714
|
-
|
|
1715
|
-
// Or cancel entirely
|
|
1716
|
-
handle.cancel();
|
|
1717
|
-
|
|
1718
|
-
// Wait for completion
|
|
1719
|
-
await handle.promise;
|
|
1720
|
-
```
|
|
1721
|
-
|
|
1722
|
-
### Combining multiple transitions
|
|
1723
|
-
|
|
1724
|
-
When you call `transitionAU`, it may create multiple internal transitions (one per morph target). The returned handle controls all of them:
|
|
1725
|
-
|
|
1726
|
-
```typescript
|
|
1727
|
-
// AU12 might affect Mouth_Smile_L and Mouth_Smile_R
|
|
1728
|
-
const handle = loom.transitionAU(12, 1.0, 200);
|
|
1729
|
-
|
|
1730
|
-
// Pausing the handle pauses both morph transitions
|
|
1731
|
-
handle.pause();
|
|
1732
|
-
```
|
|
1733
|
-
|
|
1734
|
-
### Easing
|
|
1735
|
-
|
|
1736
|
-
The default easing is `easeInOutQuad`. Custom easing can be provided when using the Animation system directly:
|
|
1737
|
-
|
|
1738
|
-
```typescript
|
|
1739
|
-
// The ThreeAnimationRuntime class supports custom easing
|
|
1740
|
-
animation.addTransition(
|
|
1741
|
-
'custom',
|
|
1742
|
-
0,
|
|
1743
|
-
1,
|
|
1744
|
-
200,
|
|
1745
|
-
(v) => console.log(v),
|
|
1746
|
-
(t) => t * t // Custom ease-in quadratic
|
|
1747
|
-
);
|
|
1748
|
-
```
|
|
1749
|
-
|
|
1750
|
-
### Active transition count
|
|
1751
|
-
|
|
1752
|
-
```typescript
|
|
1753
|
-
const count = loom.getActiveTransitionCount();
|
|
1754
|
-
console.log(`${count} transitions in progress`);
|
|
1755
|
-
```
|
|
1756
|
-
|
|
1757
|
-
### Clearing all transitions
|
|
1758
|
-
|
|
1759
|
-
```typescript
|
|
1760
|
-
// Cancel everything immediately
|
|
1761
|
-
loom.clearTransitions();
|
|
1762
|
-
```
|
|
1763
|
-
|
|
1764
|
-
---
|
|
1765
|
-
|
|
1766
|
-
## 14. Playback & State Control
|
|
1767
|
-
|
|
1768
|
-
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
1769
|
-
|
|
1770
|
-

|
|
1771
|
-
|
|
1772
|
-
### Pausing and resuming
|
|
1773
|
-
|
|
1774
|
-
```typescript
|
|
1775
|
-
// Pause all animation updates
|
|
1776
|
-
loom.pause();
|
|
1777
|
-
|
|
1778
|
-
// Check pause state
|
|
1779
|
-
if (loom.getPaused()) {
|
|
1780
|
-
console.log('Animation is paused');
|
|
1781
|
-
}
|
|
1782
|
-
|
|
1783
|
-
// Resume
|
|
1784
|
-
loom.resume();
|
|
1785
|
-
```
|
|
1786
|
-
|
|
1787
|
-
When paused, `loom.update()` stops processing transitions, but you can still call `setAU()` for immediate changes.
|
|
1788
|
-
|
|
1789
|
-
### Resetting to neutral
|
|
1790
|
-
|
|
1791
|
-
```typescript
|
|
1792
|
-
// Reset everything to rest state
|
|
1793
|
-
loom.resetToNeutral();
|
|
1794
|
-
```
|
|
1795
|
-
|
|
1796
|
-
This:
|
|
1797
|
-
- Clears all AU values to 0
|
|
1798
|
-
- Cancels all active transitions
|
|
1799
|
-
- Resets all morph targets to 0
|
|
1800
|
-
- Returns all bones to their original position/rotation
|
|
1801
|
-
|
|
1802
|
-
### Mesh visibility
|
|
1803
|
-
|
|
1804
|
-
```typescript
|
|
1805
|
-
// Get list of all meshes
|
|
1806
|
-
const meshes = loom.getMeshList();
|
|
1807
|
-
// Returns: [{ name: 'CC_Base_Body', visible: true, morphCount: 80 }, ...]
|
|
1808
|
-
|
|
1809
|
-
// Hide a mesh
|
|
1810
|
-
loom.setMeshVisible('CC_Base_Hair', false);
|
|
1811
|
-
|
|
1812
|
-
// Show it again
|
|
1813
|
-
loom.setMeshVisible('CC_Base_Hair', true);
|
|
1814
|
-
```
|
|
1815
|
-
|
|
1816
|
-
### Cleanup
|
|
1817
|
-
|
|
1818
|
-
```typescript
|
|
1819
|
-
// When done, dispose of resources
|
|
1820
|
-
loom.dispose();
|
|
1821
|
-
```
|
|
1822
|
-
|
|
1823
|
-
---
|
|
1824
|
-
|
|
1825
|
-
## 15. Hair Physics
|
|
1826
|
-
|
|
1827
|
-
Open in LoomLarge: [Hair tab](https://www.characterloom.com/?drawer=open&tab=hair)
|
|
1828
|
-
|
|
1829
|
-

|
|
1830
|
-
|
|
1831
|
-
Embody includes a built-in hair physics system that drives morph targets through the AnimationMixer.
|
|
1832
|
-
It is **mixer-only** (no per-frame morph LERP), and it reacts to **head rotation** coming from AUs.
|
|
1833
|
-
|
|
1834
|
-
### How it works
|
|
1835
|
-
|
|
1836
|
-
Hair motion is decomposed into three clip families:
|
|
1837
|
-
|
|
1838
|
-
1. **Idle/Wind loop** - continuous sway and optional wind.
|
|
1839
|
-
2. **Impulse clips** - short oscillations triggered by *changes* in head yaw/pitch.
|
|
1840
|
-
3. **Gravity clip** - a single clip that is **scrubbed** by head pitch (up/down).
|
|
1841
|
-
|
|
1842
|
-
All clips are created with `buildClip` and applied to the mixer.
|
|
1843
|
-
When you update head AUs (e.g. `setAU`, `setContinuum`, `transitionAU`), hair updates automatically.
|
|
1844
|
-
|
|
1845
|
-
### Basic setup
|
|
1846
|
-
|
|
1847
|
-
```typescript
|
|
1848
|
-
const loom = new Embody({ presetType: 'cc4' });
|
|
1849
|
-
|
|
1850
|
-
loader.load('/character.glb', (gltf) => {
|
|
1851
|
-
const meshes = collectMorphMeshes(gltf.scene);
|
|
1852
|
-
loom.onReady({ meshes, model: gltf.scene });
|
|
1853
|
-
|
|
1854
|
-
// Register hair + eyebrow meshes (filters using CC4_MESHES category tags)
|
|
1855
|
-
const allObjects: Object3D[] = [];
|
|
1856
|
-
gltf.scene.traverse((obj) => allObjects.push(obj));
|
|
1857
|
-
loom.registerHairObjects(allObjects);
|
|
1858
|
-
|
|
1859
|
-
// Enable physics (starts idle + gravity + impulse clips)
|
|
1860
|
-
loom.setHairPhysicsEnabled(true);
|
|
1861
|
-
});
|
|
1862
|
-
```
|
|
1863
|
-
|
|
1864
|
-
### Inspecting registered hair objects
|
|
1865
|
-
|
|
1866
|
-
```typescript
|
|
1867
|
-
const hairObjects = loom.getRegisteredHairObjects();
|
|
1868
|
-
console.log(hairObjects.map((mesh) => mesh.name));
|
|
1869
|
-
```
|
|
1870
|
-
|
|
1871
|
-
### Configuration (profile defaults)
|
|
1872
|
-
|
|
1873
|
-
Hair physics defaults live in the preset/profile and are applied automatically at init:
|
|
1874
|
-
|
|
1875
|
-
```typescript
|
|
1876
|
-
import type { Profile } from '@lovelace_lol/embody';
|
|
1877
|
-
|
|
1878
|
-
const profile: Profile = {
|
|
1879
|
-
// ...all your usual AU mappings...
|
|
1880
|
-
hairPhysics: {
|
|
1881
|
-
stiffness: 7.5,
|
|
1882
|
-
damping: 0.18,
|
|
1883
|
-
inertia: 3.5,
|
|
1884
|
-
gravity: 12,
|
|
1885
|
-
responseScale: 2.5,
|
|
1886
|
-
idleSwayAmount: 0.12,
|
|
1887
|
-
idleSwaySpeed: 1.0,
|
|
1888
|
-
windStrength: 0,
|
|
1889
|
-
windDirectionX: 1.0,
|
|
1890
|
-
windDirectionZ: 0,
|
|
1891
|
-
windTurbulence: 0.3,
|
|
1892
|
-
windFrequency: 1.4,
|
|
1893
|
-
idleClipDuration: 10,
|
|
1894
|
-
impulseClipDuration: 1.4,
|
|
1895
|
-
direction: {
|
|
1896
|
-
yawSign: -1,
|
|
1897
|
-
pitchSign: -1,
|
|
1898
|
-
},
|
|
1899
|
-
morphTargets: {
|
|
1900
|
-
swayLeft: { key: 'L_Hair_Left', axis: 'yaw' },
|
|
1901
|
-
swayRight: { key: 'L_Hair_Right', axis: 'yaw' },
|
|
1902
|
-
swayFront: { key: 'L_Hair_Front', axis: 'pitch' },
|
|
1903
|
-
fluffRight: { key: 'Fluffy_Right', axis: 'yaw' },
|
|
1904
|
-
fluffBottom: { key: 'Fluffy_Bottom_ALL', axis: 'pitch' },
|
|
1905
|
-
headUp: {
|
|
1906
|
-
Hairline_High_ALL: { value: 0.45, axis: 'pitch' },
|
|
1907
|
-
Length_Short: { value: 0.65, axis: 'pitch' },
|
|
1908
|
-
},
|
|
1909
|
-
headDown: {
|
|
1910
|
-
L_Hair_Front: { value: 2.0, axis: 'pitch' },
|
|
1911
|
-
Fluffy_Bottom_ALL: { value: 1.0, axis: 'pitch' },
|
|
1912
|
-
},
|
|
1913
|
-
},
|
|
1914
|
-
},
|
|
1915
|
-
};
|
|
1916
|
-
```
|
|
1917
|
-
|
|
1918
|
-
### Configuration (runtime overrides)
|
|
1919
|
-
|
|
1920
|
-
```typescript
|
|
1921
|
-
loom.setHairPhysicsConfig({
|
|
1922
|
-
stiffness: 7.5,
|
|
1923
|
-
damping: 0.18,
|
|
1924
|
-
inertia: 3.5,
|
|
1925
|
-
gravity: 12,
|
|
1926
|
-
responseScale: 2.5,
|
|
1927
|
-
idleSwayAmount: 0.12,
|
|
1928
|
-
idleSwaySpeed: 1.0,
|
|
1929
|
-
windStrength: 0,
|
|
1930
|
-
windDirectionX: 1.0,
|
|
1931
|
-
windDirectionZ: 0,
|
|
1932
|
-
windTurbulence: 0.3,
|
|
1933
|
-
windFrequency: 1.4,
|
|
1934
|
-
idleClipDuration: 10,
|
|
1935
|
-
impulseClipDuration: 1.4,
|
|
1936
|
-
|
|
1937
|
-
// Direction mapping (signs) – adjust if hair goes the wrong way.
|
|
1938
|
-
direction: {
|
|
1939
|
-
yawSign: -1, // hair lags opposite head yaw
|
|
1940
|
-
pitchSign: -1, // head down drives forward hair motion
|
|
1941
|
-
},
|
|
1942
|
-
|
|
1943
|
-
// Morph target mapping (override per character/rig)
|
|
1944
|
-
morphTargets: {
|
|
1945
|
-
swayLeft: 'L_Hair_Left',
|
|
1946
|
-
swayRight: 'L_Hair_Right',
|
|
1947
|
-
swayFront: 'L_Hair_Front',
|
|
1948
|
-
fluffRight: 'Fluffy_Right',
|
|
1949
|
-
fluffBottom: 'Fluffy_Bottom_ALL',
|
|
1950
|
-
headUp: {
|
|
1951
|
-
Hairline_High_ALL: 0.45,
|
|
1952
|
-
Length_Short: 0.65,
|
|
1953
|
-
},
|
|
1954
|
-
headDown: {
|
|
1955
|
-
L_Hair_Front: 2.0,
|
|
1956
|
-
Fluffy_Bottom_ALL: 1.0,
|
|
1957
|
-
},
|
|
1958
|
-
},
|
|
1959
|
-
});
|
|
1960
|
-
```
|
|
1961
|
-
|
|
1962
|
-
### Validation
|
|
1963
|
-
|
|
1964
|
-
```typescript
|
|
1965
|
-
const missing = loom.validateHairMorphTargets();
|
|
1966
|
-
if (missing.length > 0) {
|
|
1967
|
-
console.warn('Missing hair morph targets:', missing);
|
|
1968
|
-
}
|
|
1969
|
-
```
|
|
1970
|
-
|
|
1971
|
-
Embody also logs a warning the first time it encounters a missing hair morph key.
|
|
1972
|
-
|
|
1973
|
-
### Applying styling state
|
|
1974
|
-
|
|
1975
|
-
Use the engine helpers when you want to toggle brows, outlines, or simple per-object visual state from a UI:
|
|
1976
|
-
|
|
1977
|
-
```typescript
|
|
1978
|
-
loom.applyHairStateToObject('Sideburns', {
|
|
1979
|
-
visible: true,
|
|
1980
|
-
outline: { show: true, color: '#7dd3fc', opacity: 0.6 },
|
|
1981
|
-
color: {
|
|
1982
|
-
baseColor: '#8b5e3c',
|
|
1983
|
-
emissive: '#000000',
|
|
1984
|
-
emissiveIntensity: 0,
|
|
1985
|
-
},
|
|
1986
|
-
});
|
|
1987
|
-
```
|
|
1988
|
-
|
|
1989
|
-
### Applying morphs to named hair meshes
|
|
1990
|
-
|
|
1991
|
-
```typescript
|
|
1992
|
-
loom.setMorphOnMeshes(
|
|
1993
|
-
['Side_part_wavy_1', 'Side_part_wavy_2'],
|
|
1994
|
-
'L_Hair_Front',
|
|
1995
|
-
0.35
|
|
1996
|
-
);
|
|
1997
|
-
```
|
|
1998
|
-
|
|
1999
|
-
### Notes
|
|
2000
|
-
|
|
2001
|
-
- **Head rotation input** comes from AUs (e.g. 51/52 yaw, 53/54 pitch).
|
|
2002
|
-
Hair updates when those AUs change.
|
|
2003
|
-
- **Mesh selection** comes from the preset (`CC4_MESHES` categories).
|
|
2004
|
-
Hair morph target *names* live in the preset/profile (`Profile.hairPhysics`) and can be overridden at runtime.
|
|
2005
|
-
- **Direction/morphs are explicit** so you can expose a clean, user-friendly API.
|
|
2006
|
-
|
|
2007
|
-
### Troubleshooting
|
|
2008
|
-
|
|
2009
|
-
- Hair moves the wrong direction → flip `direction.yawSign` or `direction.pitchSign`.
|
|
2010
|
-
- Wrong morphs are moving → override `morphTargets` with your rig’s names.
|
|
2011
|
-
- Need stronger response → increase `responseScale` or the `headDown/headUp` values.
|
|
2012
|
-
|
|
2013
|
-
---
|
|
2014
|
-
|
|
2015
|
-
## 16. Baked Animations
|
|
2016
|
-
|
|
2017
|
-
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
2018
|
-
|
|
2019
|
-
Embody can play baked skeletal animations from your GLB/GLTF files using Three.js AnimationMixer. This allows you to combine pre-made animations (idle, walk, gestures) with real-time facial control.
|
|
2020
|
-
|
|
2021
|
-
### Loading animations
|
|
2022
|
-
|
|
2023
|
-
After loading your model, pass the animations array to Embody:
|
|
2024
|
-
|
|
2025
|
-
```typescript
|
|
2026
|
-
const loader = new GLTFLoader();
|
|
2027
|
-
loader.load('/character.glb', (gltf) => {
|
|
2028
|
-
scene.add(gltf.scene);
|
|
2029
|
-
|
|
2030
|
-
const meshes = collectMorphMeshes(gltf.scene);
|
|
2031
|
-
loom.onReady({ meshes, model: gltf.scene });
|
|
2032
|
-
|
|
2033
|
-
// Load baked animations from the GLB file
|
|
2034
|
-
loom.loadAnimationClips(gltf.animations);
|
|
2035
|
-
|
|
2036
|
-
// Start the internal update loop
|
|
2037
|
-
loom.start();
|
|
2038
|
-
});
|
|
2039
|
-
```
|
|
2040
|
-
|
|
2041
|
-
### Listing available animations
|
|
2042
|
-
|
|
2043
|
-
```typescript
|
|
2044
|
-
const clips = loom.getAnimationClips();
|
|
2045
|
-
console.log(clips);
|
|
2046
|
-
// [
|
|
2047
|
-
// { name: 'Idle', duration: 4.0, trackCount: 52 },
|
|
2048
|
-
// { name: 'Walk', duration: 1.2, trackCount: 52 },
|
|
2049
|
-
// { name: 'Wave', duration: 2.5, trackCount: 24 },
|
|
2050
|
-
// ]
|
|
2051
|
-
```
|
|
2052
|
-
|
|
2053
|
-
### Playing animations
|
|
2054
|
-
|
|
2055
|
-
```typescript
|
|
2056
|
-
// Play an animation with default settings (looping)
|
|
2057
|
-
loom.playAnimation('Idle');
|
|
2058
|
-
|
|
2059
|
-
// Play with options
|
|
2060
|
-
const handle = loom.playAnimation('Wave', {
|
|
2061
|
-
speed: 1.0, // Playback speed (1.0 = normal)
|
|
2062
|
-
intensity: 1.0, // Weight/intensity (0-1)
|
|
2063
|
-
loop: false, // Don't loop
|
|
2064
|
-
loopMode: 'once', // 'repeat', 'pingpong', or 'once'
|
|
2065
|
-
clampWhenFinished: true, // Hold last frame when done
|
|
2066
|
-
startTime: 0, // Start from beginning
|
|
2067
|
-
});
|
|
2068
|
-
|
|
2069
|
-
// Wait for non-looping animation to finish
|
|
2070
|
-
await handle.finished;
|
|
2071
|
-
```
|
|
2072
|
-
|
|
2073
|
-
### Mixer clip playback for curves
|
|
2074
|
-
|
|
2075
|
-
Embody can convert AU/morph curves into AnimationMixer clips for smooth, mixer-only playback. This is the preferred path for high-frequency animation agencies (eye/head tracking, visemes, prosody) because it avoids per-keyframe transitions.
|
|
2076
|
-
|
|
2077
|
-
Key APIs:
|
|
2078
|
-
- `snippetToClip(name, curves, options)` builds an AnimationClip from curves.
|
|
2079
|
-
- `playClip(clip, options)` returns a ClipHandle you can pause/resume/stop.
|
|
2080
|
-
- `clipHandle.subscribe(listener)` streams lifecycle events during mixer updates.
|
|
2081
|
-
- `clipHandle.stop()` now resolves cleanly (no rejected promise).
|
|
2082
|
-
|
|
2083
|
-
```typescript
|
|
2084
|
-
const clip = loom.snippetToClip('gaze', {
|
|
2085
|
-
'61': [{ time: 0, intensity: 0 }, { time: 0.4, intensity: 0.6 }],
|
|
2086
|
-
'62': [{ time: 0, intensity: 0 }, { time: 0.4, intensity: 0 }],
|
|
2087
|
-
}, { loop: false });
|
|
2088
|
-
|
|
2089
|
-
if (clip) {
|
|
2090
|
-
const handle = loom.playClip(clip, { loop: false, speed: 1 });
|
|
2091
|
-
const unsubscribe = handle?.subscribe?.((event) => {
|
|
2092
|
-
if (event.type === 'keyframe') {
|
|
2093
|
-
console.log(event.currentTime, event.keyframeIndex);
|
|
2094
|
-
}
|
|
2095
|
-
});
|
|
2096
|
-
|
|
2097
|
-
await handle.finished;
|
|
2098
|
-
unsubscribe?.();
|
|
2099
|
-
}
|
|
2100
|
-
```
|
|
2101
|
-
|
|
2102
|
-
When the first keyframe has `inherit: true`, Embody anchors that first mixer
|
|
2103
|
-
track value to the character's current live target value instead of treating the
|
|
2104
|
-
placeholder intensity as a real reset. The following keyframes remain authored
|
|
2105
|
-
targets:
|
|
2106
|
-
|
|
2107
|
-
```typescript
|
|
2108
|
-
const handle = loom.buildClip('smile-from-current-pose', {
|
|
2109
|
-
// Starts from the current AU 12/morph/bone state, then moves to 0.7.
|
|
2110
|
-
'12': [{ time: 0, intensity: 0, inherit: true }, { time: 0.25, intensity: 0.7 }],
|
|
2111
|
-
}, { loop: false });
|
|
2112
|
-
```
|
|
2113
|
-
|
|
2114
|
-
Clip stream events are discrete runtime events, not a polling surface:
|
|
2115
|
-
|
|
2116
|
-
- `keyframe` fires when playback crosses an authored keyframe.
|
|
2117
|
-
- `loop` fires when looping playback starts another iteration.
|
|
2118
|
-
- `seek` fires when `setTime()` scrubs the clip.
|
|
2119
|
-
- `completed` fires when non-looping playback reaches its terminal state.
|
|
2120
|
-
|
|
2121
|
-
### Playing a snippet directly
|
|
2122
|
-
|
|
2123
|
-
If you already have a named snippet object, you can skip manual clip creation:
|
|
2124
|
-
|
|
2125
|
-
```typescript
|
|
2126
|
-
const handle = loom.playSnippet({
|
|
2127
|
-
name: 'look-left',
|
|
2128
|
-
curves: {
|
|
2129
|
-
'61': [{ time: 0, intensity: 0 }, { time: 0.25, intensity: 0.7 }],
|
|
2130
|
-
'62': [{ time: 0, intensity: 0 }, { time: 0.25, intensity: 0 }],
|
|
2131
|
-
},
|
|
2132
|
-
}, { loop: false });
|
|
2133
|
-
```
|
|
2134
|
-
|
|
2135
|
-
### Building and updating managed clips
|
|
2136
|
-
|
|
2137
|
-
`buildClip()` keeps a named clip/action around so you can adjust it later without rebuilding your entire animation flow:
|
|
2138
|
-
|
|
2139
|
-
```typescript
|
|
2140
|
-
const clipHandle = loom.buildClip('gaze-loop', {
|
|
2141
|
-
'61': [{ time: 0, intensity: 0 }, { time: 0.3, intensity: 0.6 }],
|
|
2142
|
-
'62': [{ time: 0, intensity: 0.3 }, { time: 0.3, intensity: 0 }],
|
|
2143
|
-
}, {
|
|
2144
|
-
loop: true,
|
|
2145
|
-
loopMode: 'pingpong',
|
|
2146
|
-
});
|
|
2147
|
-
|
|
2148
|
-
loom.updateClipParams('gaze-loop', {
|
|
2149
|
-
weight: 0.7,
|
|
2150
|
-
rate: 1.2,
|
|
2151
|
-
loopMode: 'repeat',
|
|
2152
|
-
});
|
|
2153
|
-
|
|
2154
|
-
clipHandle?.pause();
|
|
2155
|
-
clipHandle?.resume();
|
|
2156
|
-
```
|
|
2157
|
-
|
|
2158
|
-
### Checking curve support
|
|
2159
|
-
|
|
2160
|
-
```typescript
|
|
2161
|
-
const supported = loom.supportsClipCurves({
|
|
2162
|
-
'61': [{ time: 0, intensity: 0 }, { time: 0.2, intensity: 0.4 }],
|
|
2163
|
-
});
|
|
2164
|
-
|
|
2165
|
-
if (!supported) {
|
|
2166
|
-
console.warn('Curves need a fallback playback path');
|
|
2167
|
-
}
|
|
2168
|
-
```
|
|
2169
|
-
|
|
2170
|
-
### Controlling playback
|
|
2171
|
-
|
|
2172
|
-
The handle returned from `playAnimation()` provides full control:
|
|
2173
|
-
|
|
2174
|
-
```typescript
|
|
2175
|
-
const handle = loom.playAnimation('Idle');
|
|
2176
|
-
|
|
2177
|
-
// Pause and resume
|
|
2178
|
-
handle.pause();
|
|
2179
|
-
handle.resume();
|
|
2180
|
-
|
|
2181
|
-
// Adjust speed in real-time
|
|
2182
|
-
handle.setSpeed(0.5); // Half speed
|
|
2183
|
-
handle.setSpeed(2.0); // Double speed
|
|
2184
|
-
|
|
2185
|
-
// Adjust intensity/weight
|
|
2186
|
-
handle.setWeight(0.5); // 50% influence
|
|
2187
|
-
|
|
2188
|
-
// Seek to specific time
|
|
2189
|
-
handle.seekTo(1.5); // Jump to 1.5 seconds
|
|
2190
|
-
|
|
2191
|
-
// Get current state
|
|
2192
|
-
const state = handle.getState();
|
|
2193
|
-
console.log(state);
|
|
2194
|
-
// {
|
|
2195
|
-
// name: 'Idle',
|
|
2196
|
-
// isPlaying: true,
|
|
2197
|
-
// isPaused: false,
|
|
2198
|
-
// time: 1.5,
|
|
2199
|
-
// duration: 4.0,
|
|
2200
|
-
// speed: 1.0,
|
|
2201
|
-
// weight: 1.0,
|
|
2202
|
-
// isLooping: true
|
|
2203
|
-
// }
|
|
2204
|
-
|
|
2205
|
-
// Stop the animation
|
|
2206
|
-
handle.stop();
|
|
2207
|
-
```
|
|
2208
|
-
|
|
2209
|
-
### Crossfading between animations
|
|
2210
|
-
|
|
2211
|
-
Smoothly transition from one animation to another:
|
|
2212
|
-
|
|
2213
|
-
```typescript
|
|
2214
|
-
// Start with idle
|
|
2215
|
-
loom.playAnimation('Idle');
|
|
2216
|
-
|
|
2217
|
-
// Later, crossfade to walk over 0.3 seconds
|
|
2218
|
-
loom.crossfadeTo('Walk', 0.3);
|
|
2219
|
-
|
|
2220
|
-
// Or use the handle
|
|
2221
|
-
const idleHandle = loom.playAnimation('Idle');
|
|
2222
|
-
idleHandle.crossfadeTo('Walk', 0.5);
|
|
2223
|
-
```
|
|
2224
|
-
|
|
2225
|
-
### Global animation control
|
|
2226
|
-
|
|
2227
|
-
Control all animations at once:
|
|
2228
|
-
|
|
2229
|
-
```typescript
|
|
2230
|
-
// Stop all animations
|
|
2231
|
-
loom.stopAllAnimations();
|
|
2232
|
-
|
|
2233
|
-
// Pause all animations
|
|
2234
|
-
loom.pauseAllAnimations();
|
|
2235
|
-
|
|
2236
|
-
// Resume all paused animations
|
|
2237
|
-
loom.resumeAllAnimations();
|
|
2238
|
-
|
|
2239
|
-
// Set global time scale (affects all animations)
|
|
2240
|
-
loom.setAnimationTimeScale(0.5); // Everything at half speed
|
|
2241
|
-
|
|
2242
|
-
// Get all currently playing animations
|
|
2243
|
-
const playing = loom.getPlayingAnimations();
|
|
2244
|
-
```
|
|
2245
|
-
|
|
2246
|
-
### Direct control by name
|
|
2247
|
-
|
|
2248
|
-
You can also control animations directly without using handles:
|
|
2249
|
-
|
|
2250
|
-
```typescript
|
|
2251
|
-
loom.playAnimation('Idle');
|
|
2252
|
-
|
|
2253
|
-
// Later...
|
|
2254
|
-
loom.setAnimationSpeed('Idle', 1.5);
|
|
2255
|
-
loom.setAnimationIntensity('Idle', 0.8);
|
|
2256
|
-
loom.pauseAnimation('Idle');
|
|
2257
|
-
loom.resumeAnimation('Idle');
|
|
2258
|
-
loom.stopAnimation('Idle');
|
|
2259
|
-
|
|
2260
|
-
// Get state of specific animation
|
|
2261
|
-
const state = loom.getAnimationState('Idle');
|
|
2262
|
-
```
|
|
2263
|
-
|
|
2264
|
-
### Combining with facial animation
|
|
2265
|
-
|
|
2266
|
-
Loaded mixer clips and facial AU control work together seamlessly. The AnimationMixer updates automatically when you call `loom.update()` or use `loom.start()`:
|
|
2267
|
-
|
|
2268
|
-
```typescript
|
|
2269
|
-
loom.loadAnimationClips(gltf.animations);
|
|
2270
|
-
loom.start(); // Starts internal RAF loop
|
|
2271
|
-
|
|
2272
|
-
// Play a body animation
|
|
2273
|
-
loom.playAnimation('Idle');
|
|
2274
|
-
|
|
2275
|
-
// Control facial expressions on top
|
|
2276
|
-
loom.setAU(12, 0.8); // Smile
|
|
2277
|
-
loom.transitionAU(45, 1.0, 100); // Blink
|
|
2278
|
-
|
|
2279
|
-
// Both update together - no separate render loop needed
|
|
2280
|
-
```
|
|
2281
|
-
|
|
2282
|
-
### Animation types
|
|
2283
|
-
|
|
2284
|
-
| Option | Type | Default | Description |
|
|
2285
|
-
|--------|------|---------|-------------|
|
|
2286
|
-
| `speed` | number | 1.0 | Playback speed multiplier |
|
|
2287
|
-
| `intensity` | number | 1.0 | Animation weight (0-1) |
|
|
2288
|
-
| `loop` | boolean | true | Whether to loop |
|
|
2289
|
-
| `loopMode` | string | 'repeat' | 'repeat', 'pingpong', or 'once' |
|
|
2290
|
-
| `crossfadeDuration` | number | 0 | Fade in duration (seconds) |
|
|
2291
|
-
| `clampWhenFinished` | boolean | true | Hold last frame when done |
|
|
2292
|
-
| `startTime` | number | 0 | Initial playback position |
|
|
2293
|
-
|
|
2294
|
-
---
|
|
2295
|
-
|
|
2296
|
-
## 17. Regions & Geometry Helpers
|
|
2297
|
-
|
|
2298
|
-
Open in LoomLarge: [Bones tab](https://www.characterloom.com/?drawer=open&tab=bones) | [Mappings tab](https://www.characterloom.com/?drawer=open&tab=mappings)
|
|
2299
|
-
|
|
2300
|
-
These helpers are for applications that need semantic face regions, marker anchors, or camera targets in addition to direct animation control.
|
|
2301
|
-
|
|
2302
|
-
### Finding a face center directly from the model
|
|
2303
|
-
|
|
2304
|
-
```typescript
|
|
2305
|
-
import { findFaceCenter } from '@lovelace_lol/embody';
|
|
2306
|
-
|
|
2307
|
-
const result = findFaceCenter(gltf.scene, {
|
|
2308
|
-
headBoneNames: ['CC_Base_Head'],
|
|
2309
|
-
faceMeshNames: ['CC_Base_Body'],
|
|
2310
|
-
});
|
|
2311
|
-
|
|
2312
|
-
console.log(result.center, result.method, result.debugInfo);
|
|
2313
|
-
```
|
|
2314
|
-
|
|
2315
|
-
### Resolving region-driven centers
|
|
2316
|
-
|
|
2317
|
-
```typescript
|
|
2318
|
-
import type { BoneResolutionProfile, Region } from '@lovelace_lol/embody';
|
|
2319
|
-
import { resolveBoneName, resolveBoneNames, resolveFaceCenter } from '@lovelace_lol/embody';
|
|
2320
|
-
|
|
2321
|
-
const region: Region = {
|
|
2322
|
-
name: 'face',
|
|
2323
|
-
bones: ['HEAD'],
|
|
2324
|
-
meshes: ['CC_Base_Body'],
|
|
2325
|
-
};
|
|
2326
|
-
|
|
2327
|
-
const profile: BoneResolutionProfile = {
|
|
2328
|
-
bonePrefix: 'CC_Base_',
|
|
2329
|
-
boneNodes: { HEAD: 'Head' },
|
|
2330
|
-
};
|
|
2331
|
-
|
|
2332
|
-
const headBone = resolveBoneName('HEAD', profile);
|
|
2333
|
-
const resolvedBones = resolveBoneNames(['HEAD'], profile);
|
|
2334
|
-
const faceCenter = resolveFaceCenter(gltf.scene, region, profile);
|
|
2335
|
-
```
|
|
2336
|
-
|
|
2337
|
-
### Working with model orientation
|
|
2338
|
-
|
|
2339
|
-
```typescript
|
|
2340
|
-
import {
|
|
2341
|
-
getModelForwardDirection,
|
|
2342
|
-
detectFacingDirection,
|
|
2343
|
-
} from '@lovelace_lol/embody';
|
|
2344
|
-
|
|
2345
|
-
const forward = getModelForwardDirection(gltf.scene);
|
|
2346
|
-
const facing = detectFacingDirection(gltf.scene);
|
|
2347
|
-
```
|
|
2348
|
-
|
|
2349
|
-
Use these helpers when you need to:
|
|
2350
|
-
- place annotation markers using semantic regions instead of hard-coded coordinates
|
|
2351
|
-
- resolve prefixed/suffixed bone names from a reusable profile or minimal bone-resolution object
|
|
2352
|
-
- derive a face anchor for camera tooling or interaction layers
|
|
2353
|
-
- reason about model orientation before building your own camera or annotation system
|
|
2354
|
-
|
|
2355
|
-
---
|
|
2356
|
-
|
|
2357
|
-
## 18. API Reference
|
|
2358
|
-
|
|
2359
|
-
Open in LoomLarge: [Animation tab](https://www.characterloom.com/?drawer=open&tab=animation)
|
|
2360
|
-
|
|
2361
|
-
This is a compact reference for the public surface exported by `@lovelace_lol/embody`.
|
|
2362
|
-
|
|
2363
|
-
### Core runtime
|
|
2364
|
-
|
|
2365
|
-
- `Embody` is the main Three.js implementation.
|
|
2366
|
-
- `collectMorphMeshes()` gathers meshes that already expose morph targets.
|
|
2367
|
-
- Lifecycle: `onReady()`, `update()`, `start()`, `stop()`, `dispose()`.
|
|
2368
|
-
- Preset state: `setProfile()`, `getProfile()`.
|
|
2369
|
-
- Control APIs: `setAU()`, `transitionAU()`, `setContinuum()`, `transitionContinuum()`, `setMorph()`, `transitionMorph()`, `setViseme()`, `transitionViseme()`.
|
|
2370
|
-
- Runtime morph authoring: `addMorphTarget()`, `addMorphTargets()`, `ensureMorphInfluence()`, `refreshMorphTargets()`.
|
|
2371
|
-
- Transition state: `pause()`, `resume()`, `getPaused()`, `clearTransitions()`, `getActiveTransitionCount()`, `resetToNeutral()`.
|
|
2372
|
-
|
|
2373
|
-
### Presets and profiles
|
|
2374
|
-
|
|
2375
|
-
- Presets: `CC4_PRESET`, `BETTA_FISH_PRESET`, `getPreset()`, `getPresetWithProfile()`.
|
|
2376
|
-
- Profile composition: `extendPresetWithProfile()`.
|
|
2377
|
-
- CC4 exports: `VISEME_KEYS`, `VISEME_JAW_AMOUNTS`, `CONTINUUM_PAIRS_MAP`, `CONTINUUM_LABELS`, `AU_INFO`, `COMPOSITE_ROTATIONS`, `AU_MIX_DEFAULTS`.
|
|
2378
|
-
- Compatibility helpers: `isMixedAU()`, `hasLeftRightMorphs()`.
|
|
2379
|
-
|
|
2380
|
-
### Validation and inspection
|
|
2381
|
-
|
|
2382
|
-
- Extraction: `extractModelData()`, `extractFromGLTF()`.
|
|
2383
|
-
- Config linting: `validateMappingConfig()`.
|
|
2384
|
-
- Model fit: `validateMappings()`, `isPresetCompatible()`, `suggestBestPreset()`, `generateMappingCorrections()`.
|
|
2385
|
-
- Template skeleton fit metadata: `TEMPLATE_SKELETON_FIT_METADATA_KIND`, `TEMPLATE_SKELETON_FIT_METADATA_VERSION`, `TEMPLATE_SKELETON_FIT_STATUSES`, `validateTemplateSkeletonFitMetadata()`, `composeTemplateSkeletonFitTransform()`.
|
|
2386
|
-
- Unified report: `analyzeModel()`.
|
|
2387
|
-
|
|
2388
|
-
### Runtime tooling
|
|
2389
|
-
|
|
2390
|
-
- Mesh inspection: `getMeshList()`, `getMorphTargets()`, `getMorphTargetIndices()`, `getBones()`.
|
|
2391
|
-
- Mesh debugging: `setMeshVisible()`, `highlightMesh()`, `getMeshMaterialConfig()`, `setMeshMaterialConfig()`.
|
|
2392
|
-
- Hair runtime: `registerHairObjects()`, `getRegisteredHairObjects()`, `setHairPhysicsEnabled()`, `setHairPhysicsConfig()`, `validateHairMorphTargets()`, `applyHairStateToObject()`.
|
|
2393
|
-
- Mixer helpers: `loadAnimationClips()`, `getAnimationClips()`, `playAnimation()`, `pauseAnimation()`, `resumeAnimation()`, `stopAnimation()`, `stopAllAnimations()`, `pauseAllAnimations()`, `resumeAllAnimations()`, `setAnimationSpeed()`, `setAnimationIntensity()`, `setAnimationTimeScale()`, `getAnimationState()`, `getPlayingAnimations()`, `crossfadeTo()`, `snippetToClip()`, `playClip()`, `playSnippet()`, `buildClip()`, `updateClipParams()`, `supportsClipCurves()`.
|
|
2394
|
-
|
|
2395
|
-
### Types and lower-level exports
|
|
2396
|
-
|
|
2397
|
-
- Configuration/types: `Profile`, `MeshInfo`, `BlendingMode`, `TransitionHandle`, `ClipEvent`, `ClipEventListener`, `ClipHandle`, `Snippet`, `AnimationState`, `AnimationClipInfo`.
|
|
2398
|
-
- Standalone implementations: `ThreeAnimationRuntime`, `HairPhysics`, `BLENDING_MODES`.
|
|
2399
|
-
- Region and geometry helpers: `resolveBoneName()`, `resolveBoneNames()`, `resolveFaceCenter()`, `findFaceCenter()`, `getModelForwardDirection()`, `detectFacingDirection()`.
|
|
2400
|
-
|
|
2401
|
-
---
|
|
2402
|
-
|
|
2403
|
-
## Resources
|
|
503
|
+
`npm run build` creates the JS adapter bundles, declarations, generated
|
|
504
|
+
wasm-bindgen glue, and `.wasm` binary in `dist/`. Generated output is not source
|
|
505
|
+
and is not committed. Package checks consume that existing build, so run the
|
|
506
|
+
build once before `npm run test:package`.
|
|
2404
507
|
|
|
2405
|
-
|
|
508
|
+
The public Wasm loader returns `EmbodyCore`, derived directly from wasm-bindgen's
|
|
509
|
+
generated declarations, including `RuntimeCore` constructors, methods, and
|
|
510
|
+
numeric helpers. Instance types can also be imported with `import type` from
|
|
511
|
+
`@lovelace_lol/embody/wasm`. Rust exports remain the source of truth for these
|
|
512
|
+
signatures; JSON payload contents are still encoded as strings.
|
|
2406
513
|
|
|
2407
|
-
|
|
2408
|
-
|
|
2409
|
-
|
|
2410
|
-
|
|
514
|
+
Run the build before `npm test` or `npm run typecheck` in a clean checkout, and
|
|
515
|
+
rebuild after changing Rust exports. The build generates the Wasm bindings before
|
|
516
|
+
TypeScript declarations. Neither typechecking nor package verification rebuilds
|
|
517
|
+
the package; CI checks and publishes that same build.
|
|
2411
518
|
|
|
2412
519
|
## License
|
|
2413
520
|
|
|
2414
|
-
MIT
|
|
521
|
+
MIT. See [LICENSE](LICENSE) and [AUTHORS.md](AUTHORS.md).
|
|
522
|
+
|
|
523
|
+
### Semantic poses and generated motion
|
|
524
|
+
|
|
525
|
+
`RuntimeCore.get_semantic_pose_catalog_json()` returns the configured profile's
|
|
526
|
+
motion identifiers, direction labels, VRM roles, resolved bone/morph names and
|
|
527
|
+
per-direction model support and available sides. VRM names anatomy; profile motion IDs name authored
|
|
528
|
+
actions. Hosts should send only supported directions to generation prompts,
|
|
529
|
+
along with their bounds: intensity and morph strength `0..1`, bilateral balance
|
|
530
|
+
`-1` character-left, `0` both, `1` character-right. Amounts are normalized authored
|
|
531
|
+
motion, not radians or world-space rotation.
|
|
532
|
+
|
|
533
|
+
`capture_semantic_pose_json()` returns `{version:1,controls:[{controlId,positive,
|
|
534
|
+
negative?}]}`. Each direction stores intensity, balance and morphStrength
|
|
535
|
+
independently, preserving opposed action values. These are manual runtime
|
|
536
|
+
controls; capturing an arbitrary mixer/baked pose would require pose inference.
|
|
537
|
+
`validate_semantic_pose_json()` fills omitted settings without mutation.
|
|
538
|
+
`apply_semantic_pose_json()` validates the complete input before replacing Body
|
|
539
|
+
actions, preserving unrelated facial state. Either direction may be omitted;
|
|
540
|
+
a control requires at least one. Known missing outputs remain portable to partial
|
|
541
|
+
rigs; unknown IDs, duplicate actions and invalid bounds fail. Applying a semantic pose releases prior direct overrides on its exact mapped
|
|
542
|
+
mesh/target IDs, preserving unrelated facial overrides and same-named targets on
|
|
543
|
+
unmapped meshes.
|
|
544
|
+
|
|
545
|
+
`build_semantic_clip(name, animationJson, optionsJson)` compiles
|
|
546
|
+
`{version:1,durationSeconds,tracks:[{controlId,direction,balance?,morphStrength?,
|
|
547
|
+
keyframes:[{time,intensity}]}]}` through the same AU/bone/morph evaluator.
|
|
548
|
+
Direction is `positive` or `negative`; time is seconds. Unsupported directions,
|
|
549
|
+
unknown controls, duplicate actions and non-increasing/out-of-range keys fail.
|
|
550
|
+
`options.faceCurves` accepts additional non-Body AU curves so face and body compile
|
|
551
|
+
together, including overlapping mapped bones. Compilation preserves live state;
|
|
552
|
+
per-track morph strength changes only the compiled output. The host mixer owns
|
|
553
|
+
playback and interpolation, as with existing snippets.
|
|
554
|
+
|
|
555
|
+
Catalog `availableSides` identifies actual mapped left/right/center outputs.
|
|
556
|
+
Generated animations reject a negative balance without a left output or positive
|
|
557
|
+
balance without a right output; balance zero may drive the remaining side of a
|
|
558
|
+
partial rig. Stored static poses retain their portability across missing outputs.
|
|
559
|
+
`evaluate_semantic_body_bone_frame()` returns exact Body-owned properties including
|
|
560
|
+
neutral values, allowing an explicit semantic pose/reset to replace old raw bone
|
|
561
|
+
edits without touching unrelated properties.
|