@lovelace_lol/embody 0.0.15 → 0.0.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +407 -149
- package/dist/index.js +1320 -381
- package/dist/index.js.map +1 -1
- package/dist/three/annotations/modelObservation.d.ts +9 -7
- package/dist/three/annotations/morphGeometry.d.ts +19 -0
- package/dist/three/authoring.d.ts +18 -18
- package/dist/three/authoringTypes.d.ts +38 -4
- package/dist/three/character.d.ts +58 -14
- package/dist/three/index.d.ts +12 -11
- package/dist/three/model.d.ts +2 -0
- package/dist/three/nativePlayback.d.ts +6 -0
- package/dist/three/playbackRendering.d.ts +1 -1
- package/dist/three/scene.d.ts +2 -0
- package/dist/three/sceneBackground.d.ts +11 -0
- package/dist/three/scenery.d.ts +46 -0
- package/dist/three/vrm-animation.d.ts +70 -0
- package/dist/three.js +1320 -381
- package/dist/three.js.map +1 -1
- package/dist/wasm/README.md +407 -149
- package/dist/wasm/embody_wasm.d.ts +241 -95
- package/dist/wasm/embody_wasm.js +541 -31
- package/dist/wasm/embody_wasm_bg.wasm +0 -0
- package/dist/wasm/embody_wasm_bg.wasm.d.ts +113 -88
- package/dist/wasm/highlighting.d.ts +16 -0
- package/dist/wasm/index.d.ts +10 -0
- package/dist/wasm/savedAnimation.d.ts +4 -0
- package/dist/wasm.js +1123 -2
- package/dist/wasm.js.map +1 -1
- package/model-tools/index.d.ts +57 -0
- package/model-tools/index.mjs +354 -0
- package/model-tools/prepare-actor.mjs +79 -0
- package/package.json +19 -5
package/README.md
CHANGED
|
@@ -9,6 +9,28 @@ Embody is a Rust/Wasm character animation core for facial action units,
|
|
|
9
9
|
visemes, bone controls, baked animation clips, and hair motion. Three.js is a
|
|
10
10
|
host adapter, not the runtime implementation.
|
|
11
11
|
|
|
12
|
+
## Reading historical saved movements
|
|
13
|
+
|
|
14
|
+
`decodeLegacyPose(record, historicalProfile?)` and
|
|
15
|
+
`decodeLegacyAnimation(record, historicalProfile?)` translate retired named
|
|
16
|
+
`semanticPose` / `semanticAnimation` records into existing numeric AU fields.
|
|
17
|
+
The translation and conflict checks live in Rust. Await `initEmbodyCore()` before
|
|
18
|
+
decoding historical records; ordinary records without the retired field return
|
|
19
|
+
unchanged and do not require initialization. Inputs are JSON saved records.
|
|
20
|
+
|
|
21
|
+
Explicit `historicalProfile.bodyControls` metadata is authoritative, including an
|
|
22
|
+
empty map. Only when that metadata is absent does the decoder use the embedded
|
|
23
|
+
historical table. Unresolved names, conflicting authored values, and opposing
|
|
24
|
+
pose directions fail without changing the input. Keyframe seconds, balance,
|
|
25
|
+
morph strength, custom profile IDs, and unrelated saved metadata are preserved.
|
|
26
|
+
The returned copy removes the decoded retired field; this does not write storage.
|
|
27
|
+
|
|
28
|
+
These readers do not consolidate eye mappings, assign FACS/body namespaces,
|
|
29
|
+
convert curve snippets into explicit channels, or change native playback. Those
|
|
30
|
+
migrations require profile-specific equivalence and compiler parity work under
|
|
31
|
+
[Polymer epic #325](https://github.com/meekmachine/Polymer/issues/325). Semantic
|
|
32
|
+
readback and slider ownership remain separate integration work.
|
|
33
|
+
|
|
12
34
|
## Architecture
|
|
13
35
|
|
|
14
36
|
```text
|
|
@@ -16,14 +38,16 @@ src/ Rust source (the Wasm crate) — hair, clips, profiles, run
|
|
|
16
38
|
assets/presets/ Embedded typed character profiles
|
|
17
39
|
assets/templates/ Embedded humanoid skeleton templates
|
|
18
40
|
three/ Thin Three.js inspection/application adapter only
|
|
41
|
+
model-tools/ Browser/Node GLB preparation, public types, and CLI
|
|
19
42
|
wasm/ Generated-Wasm JS loader + ABI constants
|
|
20
43
|
index.ts Package re-exports only
|
|
21
44
|
```
|
|
22
45
|
|
|
23
|
-
There is no parallel TypeScript hair/physics/runtime core. Hair idle, impulse
|
|
24
|
-
gravity
|
|
46
|
+
There is no parallel TypeScript hair/physics/runtime core. Hair idle, impulse
|
|
47
|
+
and gravity curve helpers and the continuous response live in `src/hair_curves.rs` /
|
|
25
48
|
`src/hair.rs` and are exported through Wasm. Hosts schedule the resulting
|
|
26
|
-
ClipIR on their mixer; they must not reimplement hair sampling in TS.
|
|
49
|
+
ClipIR on their mixer; they must not reimplement hair sampling in TS. See
|
|
50
|
+
[Hair motion](#hair-motion) for how the character host drives them.
|
|
27
51
|
|
|
28
52
|
The Rust core owns:
|
|
29
53
|
|
|
@@ -35,8 +59,9 @@ The Rust core owns:
|
|
|
35
59
|
- hair curves, hair physics, and appearance normalization
|
|
36
60
|
- annotation regions, camera gestures/flights/orbits, marker placement and visibility,
|
|
37
61
|
preview/reveal lifecycles, and humanoid template fitting
|
|
38
|
-
-
|
|
62
|
+
- on-demand semantic AU reads of active AU-curve clips (no clock or polling)
|
|
39
63
|
- renderer-neutral model analysis and packed live frame generation
|
|
64
|
+
- transient mesh highlight selection, styling, transitions, and versioned preview snapshots
|
|
40
65
|
|
|
41
66
|
`resolveMorphMeshSelections(profile, categories, availableNames)` resolves editor
|
|
42
67
|
mesh selections in one Rust request. It returns each requested category's effective
|
|
@@ -62,6 +87,38 @@ The host animation library (Three `AnimationMixer`, Unity Animator, etc.) owns
|
|
|
62
87
|
clip playback, lerping, blending, looping, seeking, and crossfades. Rust must
|
|
63
88
|
not sample or lerp clips on the hot path.
|
|
64
89
|
|
|
90
|
+
For portable body clips, Rust converts Mixamo references/keys into standard
|
|
91
|
+
VRM Animation 1.0 `.vrma` GLB files and retargets supported humanoid tracks.
|
|
92
|
+
The Three adapter exposes `convertMixamoFbxToVrmAnimations`,
|
|
93
|
+
`convertMixamoToVrmAnimation`, `inspectVrmAnimation`, and
|
|
94
|
+
`createAnimationClipFromVrmAnimation`. Initialize Wasm first; supply explicit
|
|
95
|
+
units and a target T-pose reference. See [body-animation interchange](docs/ANIMATION_REFERENCE_POSES.md#mixamo-fbx-and-vrm-animation-body-interchange)
|
|
96
|
+
for examples, supported inputs and errors. VRMA joint rotations remain distinct
|
|
97
|
+
from authored semantic action-space controls and facial/morph snippets.
|
|
98
|
+
|
|
99
|
+
The same native `Embody` character exposes `supportsVrmAnimation()`,
|
|
100
|
+
`convertMixamoFbxToVrmAnimations`, `getVrmAnimationInfo`,
|
|
101
|
+
`registerVrmAnimation(name, bytes, { metersPerUnit })` and
|
|
102
|
+
`releaseVrmAnimation(name)`. Registration uses the character's captured reference
|
|
103
|
+
and resolved profile, and does not play or stop an action. Polymer's Animation
|
|
104
|
+
agency schedules later playback. Cleanup removes only the accepted native source
|
|
105
|
+
on the original model, preserving a later same-name replacement; disposal is safe.
|
|
106
|
+
|
|
107
|
+
`Embody` is the character API exposed by the native Three adapter.
|
|
108
|
+
`createRustEmbodyHost`, `createRustEmbodyHostUnbound` and
|
|
109
|
+
`createCharacterHost(...).engine` use that same class. Its profile, mesh,
|
|
110
|
+
material, hair and clip-catalog result types come from `three/authoringTypes.ts`;
|
|
111
|
+
downstream libraries can re-export them without another character interface.
|
|
112
|
+
`getProfile()` returns `Partial<Profile>` because an explicit profile may be
|
|
113
|
+
empty, and `getHairPhysicsConfig()` may be empty before the first model bind.
|
|
114
|
+
`ProfileInput` accepts sparse mappings and app metadata without adding arbitrary
|
|
115
|
+
keys to the output `Profile` declaration. Humanoid queries accept sparse profiles;
|
|
116
|
+
`setHumanoidRoleBinding` returns a sparse profile because Rust omits empty tables.
|
|
117
|
+
The bone editor queries and mutation results follow the same sparse data contract.
|
|
118
|
+
`setAUMorphTargets` preserves its input JSON and guarantees the edited `auToMorphs`
|
|
119
|
+
map. An omitted serialized field is not a request to delete an authored override.
|
|
120
|
+
These type declarations add no playback state, lifecycle or animation clock.
|
|
121
|
+
|
|
65
122
|
An AU animation drives its profile's mapped bones and morphs through the same
|
|
66
123
|
Rust evaluation used by live AU controls. For example, speech can send an AU
|
|
67
124
|
103 curve and separate viseme shape curves; the profile decides which bones
|
|
@@ -78,6 +135,17 @@ construction. Annotation adapters capture DOM input, observe native geometry,
|
|
|
78
135
|
apply Rust camera/marker frames, render labels and release native resources.
|
|
79
136
|
Camera integration, region selection and marker state belong to Rust.
|
|
80
137
|
|
|
138
|
+
`createSceneSceneryRenderer` from `@lovelace_lol/embody/three` applies resolved
|
|
139
|
+
scenery layouts to the existing scene: self-contained GLB placements with independent
|
|
140
|
+
world XYZ positions, equirectangular skyboxes, image backdrops and solid colors.
|
|
141
|
+
Applications supply an abortable authorized byte loader and retain saved layouts,
|
|
142
|
+
asset catalogs and Persona/Character/Interaction selection policy. The adapter
|
|
143
|
+
owns model/texture loading, stale-result cancellation and renderer-drained cleanup;
|
|
144
|
+
it preserves the character, lights and existing render loop. Character scene
|
|
145
|
+
settings remain the inherited background beneath scenery, so clearing an override
|
|
146
|
+
restores the latest character configuration. See the
|
|
147
|
+
[scenery contract](docs/SCENE_READINESS.md#scenery-layouts-and-background-ownership).
|
|
148
|
+
|
|
81
149
|
`ThreeFrameApplier.applyMeshMaterialConfigs(model, profile.meshes)` replays named
|
|
82
150
|
mesh visibility and material overrides when a host binds or replaces a profile.
|
|
83
151
|
It applies explicit `visible: false` and `visible: true` even without a material
|
|
@@ -95,6 +163,88 @@ replacing differently sized geometry. Hiding or changing the hover clears its
|
|
|
95
163
|
sample cache; an explicitly empty mesh selection produces no preview. Applications
|
|
96
164
|
forward hover/show/hide events and dispose the overlay when its scene is released.
|
|
97
165
|
|
|
166
|
+
### Preparing Actor Models
|
|
167
|
+
|
|
168
|
+
`@lovelace_lol/embody/model-tools` exposes `inspectActorModel(bytes)` and
|
|
169
|
+
`prepareActorModel(bytes, options)` to prepare separate GLB files for an Actor.
|
|
170
|
+
This module owns file transformations; LoomLarge owns the guided workflow,
|
|
171
|
+
Model upload, Actor persistence, and Mapping selection. It does not load the
|
|
172
|
+
animation runtime or initialize Embody's runtime Wasm.
|
|
173
|
+
|
|
174
|
+
```js
|
|
175
|
+
import { inspectActorModel, prepareActorModel } from '@lovelace_lol/embody/model-tools';
|
|
176
|
+
|
|
177
|
+
const source = new Uint8Array(await file.arrayBuffer());
|
|
178
|
+
const inspection = await inspectActorModel(source);
|
|
179
|
+
// Show inspection.meshes to the creator before choosing explicit mesh IDs.
|
|
180
|
+
const prepared = await prepareActorModel(source, {
|
|
181
|
+
kind: 'low-resolution',
|
|
182
|
+
targetRatio: 0.5,
|
|
183
|
+
maxError: 0.001,
|
|
184
|
+
signal: abortController.signal,
|
|
185
|
+
});
|
|
186
|
+
// Persist prepared.bytes as a new Model only after the operation succeeds.
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Inspection returns source-scoped mesh IDs, original names, node names,
|
|
190
|
+
materials, vertex/triangle counts, skin and morph presence, total byte length,
|
|
191
|
+
and processing limitations. Preparation returns a new byte array, before/after
|
|
192
|
+
inspection, the removed source mesh IDs, and warnings. It never writes into the
|
|
193
|
+
input array, fetches resources, or persists an Actor.
|
|
194
|
+
|
|
195
|
+
| Kind | File operation |
|
|
196
|
+
| --- | --- |
|
|
197
|
+
| `original` | Copy source bytes exactly. |
|
|
198
|
+
| `optimized` | Compact unused vertex data and remove explicitly selected meshes and orphaned file resources. |
|
|
199
|
+
| `low-resolution` | Reduce triangle count; default target is 50%, limited by geometry, rig protection, and error. |
|
|
200
|
+
| `miniature` | Apply an explicitly supplied uniform `scale` above existing scene roots. This changes physical scale, not texture resolution. |
|
|
201
|
+
| `bust` | Keep explicit `keepMeshIds`; requires separately authored body-part meshes. |
|
|
202
|
+
| `outfit` | Keep/remove explicit mesh IDs for a prepared outfit; never guess garments from names. |
|
|
203
|
+
|
|
204
|
+
`targetRatio` accepts 0.05–1 and `maxError` accepts 0–0.05 relative to each
|
|
205
|
+
primitive's extent. `scale` is required only for miniature and accepts 0.01–0.99.
|
|
206
|
+
Mesh IDs belong to the inspected input; inspect each new file before editing it
|
|
207
|
+
again. Removing a shared mesh removes all its instances. A combined body mesh
|
|
208
|
+
must be split in a modeling tool before a bust or garment can be removed. These
|
|
209
|
+
tools do not generate clothing, transfer garments between rigs, cut surfaces,
|
|
210
|
+
or implement runtime garment switching.
|
|
211
|
+
|
|
212
|
+
Processed variants require self-contained, extension-free GLB. Unsupported
|
|
213
|
+
extensions (including compression and VRM) or resource URIs produce an explicit
|
|
214
|
+
error before rewriting. Original copies retain such payloads and inspection
|
|
215
|
+
reports `canTransform: false`; hosts must still apply their upload policy.
|
|
216
|
+
Textures keep their original bytes and dimensions. Output includes actual byte
|
|
217
|
+
and triangle counts; a requested reduction is not a size guarantee.
|
|
218
|
+
|
|
219
|
+
Triangle reduction locks vertices participating in any morph and vertices at
|
|
220
|
+
skin-influence transitions. It preserves topological borders and weighs normals,
|
|
221
|
+
UVs, and colors. All retained attributes, including joints, weights and every
|
|
222
|
+
morph target, use one vertex remap. Existing mesh/node names, skeleton hierarchy,
|
|
223
|
+
bind matrices and surviving animation targets remain intact. Only morph animation
|
|
224
|
+
channels targeting an explicitly removed mesh are removed, with a warning.
|
|
225
|
+
No node, joint, or surviving morph target is renamed. Hosts must review Mapping
|
|
226
|
+
targets when source names are missing/duplicated or a mapped mesh was removed;
|
|
227
|
+
vertex-index annotations require remapping after compaction.
|
|
228
|
+
|
|
229
|
+
Cancellation is checked between processing stages and primitives. A synchronous
|
|
230
|
+
simplifier call cannot be interrupted halfway through; hosts should use a worker
|
|
231
|
+
when processing large assets must not occupy the UI thread.
|
|
232
|
+
|
|
233
|
+
The installed `embody-prepare-actor` command uses the same implementation:
|
|
234
|
+
|
|
235
|
+
```bash
|
|
236
|
+
embody-prepare-actor --input character.glb --inspect
|
|
237
|
+
embody-prepare-actor --input character.glb --output low.glb --kind low-resolution --ratio 0.5
|
|
238
|
+
embody-prepare-actor --input character.glb --output outfit.glb --kind outfit --remove-mesh mesh:2
|
|
239
|
+
embody-prepare-actor --input character.glb --output bust.glb --kind bust --keep-mesh mesh:0
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
Repeat `--remove-mesh`/`--keep-mesh` for multiple selections. The CLI refuses an
|
|
243
|
+
existing output path, including the source file or a symlink to it. It prints a
|
|
244
|
+
JSON report after creating the new file. In a source checkout, invoke
|
|
245
|
+
`node model-tools/prepare-actor.mjs` with the same arguments. CI runs the
|
|
246
|
+
`test:model-tools` regression source as part of the existing package checks.
|
|
247
|
+
|
|
98
248
|
### Annotation runtime
|
|
99
249
|
|
|
100
250
|
`AnnotationRuntime` in `src/annotation_runtime/` owns the annotation behavior
|
|
@@ -124,11 +274,31 @@ HTML overlays align with the canvas rectangle inside its container, update on
|
|
|
124
274
|
resize/rebind, and restore borrowed parent positioning when the final view is
|
|
125
275
|
disposed. These coordinates assume axis-aligned CSS boxes without transforms.
|
|
126
276
|
|
|
127
|
-
AU morph anchors
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
277
|
+
AU morph anchors follow morph and skeletal animation on each annotation update.
|
|
278
|
+
Rust selects and caches affected vertices and their influence weights during
|
|
279
|
+
marker preparation, using mesh-space geometry so rotating a character cannot
|
|
280
|
+
change the selected semantic band. The adapter evaluates those vertices in the
|
|
281
|
+
current native pose and supplies packed world-space positions; Rust updates the
|
|
282
|
+
existing anchors without restarting reveal or rebuilding marker resources.
|
|
283
|
+
Explicit point and whole-model anchors retain their existing behavior. Model or
|
|
284
|
+
region changes invalidate the plans, and stale or invalid pose observations are
|
|
285
|
+
ignored. Geometry replacement, attribute replacement, morph dictionary edits and
|
|
286
|
+
relative/absolute mode changes are observed on `update()`. For in-place attribute
|
|
287
|
+
edits, set the Three attribute's `needsUpdate = true` (including interleaved
|
|
288
|
+
attributes). Rust retires the affected plans and selects replacements without
|
|
289
|
+
reloading regions or replacing marker resources. When a previously valid
|
|
290
|
+
attachment loses usable position or morph data, its marker becomes hidden and
|
|
291
|
+
`getMarkerPosition()` returns `null` until that data returns. Other attachments
|
|
292
|
+
and current selection remain intact.
|
|
293
|
+
|
|
294
|
+
`getRuntimeAUMorphAnchorPoint` uses the same Rust mesh-space selection and weighted
|
|
295
|
+
pose reduction as live attachments, including rotated and skinned meshes. Native
|
|
296
|
+
adapters only report geometry identities/versions and evaluate selected vertices;
|
|
297
|
+
they do not choose semantic bands or compute attachment weights. Other adapters
|
|
298
|
+
can report monotonically increasing per-object revisions with
|
|
299
|
+
`observe_morph_geometry` before preparing replacement plans, then submit packed
|
|
300
|
+
world-space XYZ samples to `observe_morph_anchor`. Plan IDs are scoped to the
|
|
301
|
+
runtime generation; retired IDs and older geometry revisions are rejected.
|
|
132
302
|
|
|
133
303
|
This is a breaking replacement of the DPthree exports. Use
|
|
134
304
|
`ThreeAnnotationController.create(options)` or `ThreeAnnotations.create(options)`
|
|
@@ -141,16 +311,20 @@ instance. Region and camera snapshots are observations; mutate through controlle
|
|
|
141
311
|
methods. Explicit `undefined` fields in `updateAnnotationRegion` remove overrides.
|
|
142
312
|
`getCharacterConfig()` retains snapshot identity until configuration content or
|
|
143
313
|
the model changes; selection, visibility and camera input do not replace it.
|
|
314
|
+
Authored intro focus targets use the same framing as manual region selection;
|
|
315
|
+
extra eye padding and composition bias apply only to the default region target.
|
|
144
316
|
Call `clearModel()` before releasing a removed model's native resources. The
|
|
145
317
|
controller retains renderer rebinding, canvas input reconnect and resource-drain
|
|
146
318
|
support through the scene's rendering controller.
|
|
147
319
|
Runtime preview region IDs retain the `runtime:annotation:` namespace.
|
|
148
320
|
|
|
149
321
|
When the annotation controller owns the renderer loop, it advances registered
|
|
150
|
-
native character mixers before observing annotation
|
|
322
|
+
native character mixers before observing annotation poses and drawing the
|
|
151
323
|
scene. The first frame establishes the timer origin; skipped asynchronous draws
|
|
152
324
|
do not accumulate playback time. External render loops use the existing
|
|
153
|
-
`createScenePlaybackFrame` integration to advance character playback
|
|
325
|
+
`createScenePlaybackFrame` integration to advance character playback, then call
|
|
326
|
+
`annotations.update()` before drawing. No per-frame application-side anchor
|
|
327
|
+
logic or agency registration is required.
|
|
154
328
|
|
|
155
329
|
```ts
|
|
156
330
|
import { ThreeAnnotations } from '@lovelace_lol/embody/three';
|
|
@@ -209,6 +383,29 @@ React hosts forward user events and subscribe to controller state. They do not
|
|
|
209
383
|
need local copies of the camera, marker, placement or timer algorithms.
|
|
210
384
|
The convenience facade delegates to one controller-owned marker layer.
|
|
211
385
|
|
|
386
|
+
## Character model loading
|
|
387
|
+
|
|
388
|
+
The Three adapter loads GLB/glTF and FBX characters into the same
|
|
389
|
+
`{ model, meshes, animations, gltf }` result. `meshes` contains morph-enabled
|
|
390
|
+
meshes; FBX results have `gltf: null`. Both formats use the same shadow setup,
|
|
391
|
+
model transforms, inspection, and disposal helpers.
|
|
392
|
+
|
|
393
|
+
```ts
|
|
394
|
+
import { loadCharacterModel, parseCharacterModel } from '@lovelace_lol/embody/three';
|
|
395
|
+
|
|
396
|
+
const character = await loadCharacterModel('/characters/avatar.fbx');
|
|
397
|
+
// For blob URLs or endpoints without a file extension, supply the format.
|
|
398
|
+
const uploaded = await loadCharacterModel(objectUrl, { format: 'fbx' });
|
|
399
|
+
// Standard binary/ASCII FBX headers are detected automatically. The resource path resolves
|
|
400
|
+
// external textures referenced by the file.
|
|
401
|
+
const cached = await parseCharacterModel(buffer, '/characters/textures/');
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
Use `format: 'gltf'` or `format: 'fbx'` to override detection. Draco options apply
|
|
405
|
+
to glTF only. FBX rigs, morph names, scale, and external texture paths depend on
|
|
406
|
+
the export; use a matching character profile and model transform as needed.
|
|
407
|
+
Loading a format does not automatically retarget its rig or rename its morphs.
|
|
408
|
+
|
|
212
409
|
## Runtime Use
|
|
213
410
|
|
|
214
411
|
Embody is an ES module package. Use `import` / dynamic `import()`; CommonJS
|
|
@@ -258,8 +455,61 @@ function update(dtSeconds: number) {
|
|
|
258
455
|
}
|
|
259
456
|
```
|
|
260
457
|
|
|
261
|
-
|
|
262
|
-
|
|
458
|
+
Embody owns the native Three character API and mixer binding. Polymer exposes
|
|
459
|
+
that API through its CLJS facade for LoomLarge; the facade does not own another
|
|
460
|
+
playback clock or hair simulation.
|
|
461
|
+
|
|
462
|
+
### Hair motion
|
|
463
|
+
|
|
464
|
+
When the profile enables `hairPhysics` and its registered hair/eyebrow meshes
|
|
465
|
+
contain mapped targets, the Three character host plays one additive native action,
|
|
466
|
+
`__embody_hair_response`. Rust's `HairResponse` owns velocity, spring displacement,
|
|
467
|
+
ambient phase and gravity response. `RuntimeCore.build_hair_response_clip` emits
|
|
468
|
+
zero-neutral morph offsets; the host replaces samples on the existing native
|
|
469
|
+
action when its target bindings stay the same. Numeric tuning and head retargets
|
|
470
|
+
preserve both the action identity and the simulation state.
|
|
471
|
+
|
|
472
|
+
After every mixer has sampled, and before native completion releases its
|
|
473
|
+
resources, `renderScenePlayback` passes the effective mixer delta to the character.
|
|
474
|
+
The character observes the profile's humanoid `head` role in model space relative
|
|
475
|
+
to the captured reference pose, including parent neck/spine motion. It expresses
|
|
476
|
+
that delta in the bound local reference before passing primitive quaternions to
|
|
477
|
+
Rust. The model root's placement rotation is excluded. Snippets, baked clips,
|
|
478
|
+
live setters and external bone writers therefore drive the same response.
|
|
479
|
+
A profile without a head role still supports idle and wind with no invented rig
|
|
480
|
+
mapping. Rebinding primes head history only after restored live controls have
|
|
481
|
+
sampled, avoiding a false rest-to-pose impulse.
|
|
482
|
+
|
|
483
|
+
Rust combines observed angular velocity, idle/wind, and smoothed pitch gravity
|
|
484
|
+
using `inertia`, `stiffness`, `damping`, `gravity`, `responseScale` and `direction`.
|
|
485
|
+
Directional mapping approaches neutral with zero slope. Contributions sharing a
|
|
486
|
+
concrete morph add before a soft limit keeps that response within `[0, 2]`.
|
|
487
|
+
The next native mixer sample applies the resulting offsets, so output trails the
|
|
488
|
+
observed head by one render step. There is no second mixer update or hair timer.
|
|
489
|
+
Playing, replaying or retargeting a caller clip performs no hair work.
|
|
490
|
+
|
|
491
|
+
`Embody.stop()` suspends response sampling with rendering. Mixer time scale zero
|
|
492
|
+
freezes physics; reversed native playback still advances physics forward while
|
|
493
|
+
the actual reversed pose determines motion direction. A renderer gap longer than
|
|
494
|
+
250 ms re-primes head velocity and integrates at most 250 ms of settling. Numeric
|
|
495
|
+
configuration edits preserve momentum and ambient phase. Clearing mappings releases
|
|
496
|
+
native output without erasing simulation state; disabling hair resets dynamics.
|
|
497
|
+
Registration, model rebind and profile replacement start a new response binding.
|
|
498
|
+
Disposal removes its render hook and releases native output.
|
|
499
|
+
|
|
500
|
+
Hair actions have explicit character ownership. `stopAllAnimations`, `crossfadeTo`,
|
|
501
|
+
`pauseAllAnimations` and `resumeAllAnimations` leave them running; Stop All and
|
|
502
|
+
crossfade also leave live setter targets intact. Aggregate pause/resume continues
|
|
503
|
+
to pause/resume live setter transitions as well as caller playback. Explicit
|
|
504
|
+
`pauseAnimation('__embody_hair_response')` freezes the simulation, and that pause
|
|
505
|
+
survives cleared mappings and disable/re-enable until the matching
|
|
506
|
+
`resumeAnimation` call. Profile replacement starts a new configuration.
|
|
507
|
+
|
|
508
|
+
`playHairImpulse(horizontal, vertical)` retains its existing separately controllable
|
|
509
|
+
one-shot native handle. It is never triggered by caller clip admission. The lower
|
|
510
|
+
level `RuntimeCore.add_hair_response_impulse` instead injects velocity into the
|
|
511
|
+
continuous response without creating an action; neither path replaces head history.
|
|
512
|
+
Legacy idle/impulse/gravity curve exports remain available to explicit consumers.
|
|
263
513
|
|
|
264
514
|
### Direct viseme snapshots
|
|
265
515
|
|
|
@@ -337,50 +587,6 @@ when the Three clip is constructed. Hosts must construct it again to capture
|
|
|
337
587
|
a fresh pose; this does not change cached host replay or generated bone
|
|
338
588
|
inheritance behavior.
|
|
339
589
|
|
|
340
|
-
### Gaze geometry contract
|
|
341
|
-
|
|
342
|
-
`solve_profile_screen_space_gaze` and `solve_profile_viewer_space_gaze` return
|
|
343
|
-
14 floats: combined target XY, eye target XY, head target XY, total yaw/pitch,
|
|
344
|
-
camera yaw/pitch, eye-to-camera distance, and viewer world XYZ. XY outputs are
|
|
345
|
-
signed AU intensities in [-1, 1]; positive Y is up and horizontal direction is
|
|
346
|
-
subject-relative. Angular values are degrees.
|
|
347
|
-
|
|
348
|
-
Pass current world camera position/quaternion, character eye midpoint, and
|
|
349
|
-
model quaternion. The viewer solver takes normalized image XY (+Y up) and
|
|
350
|
-
depth behind the virtual camera; depth and all positions must share scene
|
|
351
|
-
units. Convert physical webcam estimates before calling and supply the actual
|
|
352
|
-
source FOV/aspect. Zero viewer XY extends the character-eye-to-camera bearing,
|
|
353
|
-
so the character looks at the viewer even when scene framing puts its eyes
|
|
354
|
-
away from image center. Offsets retain the camera's right/up axes; when the
|
|
355
|
-
camera coincides with the eyes, its local +Z supplies the fallback bearing.
|
|
356
|
-
The depth guard is 0.2–10 scene units. Mouse and webcam can share this contract
|
|
357
|
-
when they use the same calibration.
|
|
358
|
-
|
|
359
|
-
Ordinary head following participates toward the common target from calibrated
|
|
360
|
-
model-neutral +Z: `head_follow_fraction` is a share of the target yaw/pitch,
|
|
361
|
-
defaulting to 0.35 for a non-finite value. Zero prefers neutral, one follows the
|
|
362
|
-
full target, and eye overflow can recruit more head movement within its limits.
|
|
363
|
-
The explicit `lock_head_to_camera` flag instead prefers the camera bearing while
|
|
364
|
-
eyes can reach; a zero follow fraction alone does not select that policy. When
|
|
365
|
-
eyes are disabled the head carries the target regardless of follow or lock.
|
|
366
|
-
This removes the previous implicit camera bias, which could point the head
|
|
367
|
-
opposite the selected target when the camera was off-center. Hosts that intend
|
|
368
|
-
camera-facing behavior must request the explicit viewer-solver lock; Polymer's
|
|
369
|
-
`headFollowEyes: false` maps to zero participation, not an implicit lock.
|
|
370
|
-
Eye angles are solved in the
|
|
371
|
-
rotated head frame, using yaw then pitch to match runtime composition. Limits
|
|
372
|
-
and AU normalization are directional and include each binding's scale. Missing
|
|
373
|
-
or morph-only mappings contribute no inferred angular capacity.
|
|
374
|
-
|
|
375
|
-
The profile must describe the active rig with calibrated semantic axes:
|
|
376
|
-
model +Z is forward, +Y is up. This ABI does not include the skeleton's rest
|
|
377
|
-
frames or optical axes, per-eye origins, current animated head pose, or
|
|
378
|
-
webcam-to-display calibration. It solves an endpoint in the assumed calibrated
|
|
379
|
-
basis; it does not provide binocular convergence or compensate intermediate
|
|
380
|
-
head motion. Shared eye commands use the smaller mapped eye capacity; unequal
|
|
381
|
-
left/right ranges need separate calibration. Hosts must compose the requested
|
|
382
|
-
rotations without averaging independent yaw/pitch clips together.
|
|
383
|
-
|
|
384
590
|
For import-time inspection, `captureModelReferencePose` provides an explicit,
|
|
385
591
|
immutable transform and morph reference that hosts can reuse after playback.
|
|
386
592
|
`extendModelReferencePose` explicitly adds appended skeletons while preserving
|
|
@@ -449,94 +655,6 @@ const report = JSON.parse(wasm.analyze_model_descriptor(
|
|
|
449
655
|
));
|
|
450
656
|
```
|
|
451
657
|
|
|
452
|
-
## Pose-aware focus
|
|
453
|
-
|
|
454
|
-
`ThreeGazeFocus` from `@lovelace_lol/embody/three` applies a focus constraint
|
|
455
|
-
inside an animation runtime. It has no clock or target-selection policy:
|
|
456
|
-
call `restore()` before evaluating the base animation, then `apply(request,
|
|
457
|
-
controls)` after it. Restoration compares quaternion orientations independently
|
|
458
|
-
of Float32 sample norms, preserving the exact authored sample and any newer
|
|
459
|
-
authored orientation. Repeated applications between mixer ticks do not accumulate
|
|
460
|
-
the constraint's previous correction. Controls are the current positions of existing motor
|
|
461
|
-
tracks. The optional sampled `controls.headIntensity` and `controls.eyeIntensity`
|
|
462
|
-
override the matching request fields, so the host can animate configuration
|
|
463
|
-
changes on its existing motor tracks. Both mean contribution in [0, 1]: zero
|
|
464
|
-
preserves the evaluated base, one applies the full solved excursion, and legacy
|
|
465
|
-
values above one normalize to one. Intermediate eye contribution proportionally
|
|
466
|
-
reduces the correction; it does not shrink anatomical limits. Reduced eye
|
|
467
|
-
contribution deliberately leaves focus error. These controls do not amplify
|
|
468
|
-
input displacement or control response speed.
|
|
469
|
-
|
|
470
|
-
During travel, the remaining sampled bearing rotates the requested finite
|
|
471
|
-
target. At the endpoint that rotation becomes identity, reaching the exact
|
|
472
|
-
world target continuously. There is no settled-threshold switch between targets
|
|
473
|
-
constructed from the authored and tracking-shifted eye origins; a larger head
|
|
474
|
-
turn must not introduce a final eye snap.
|
|
475
|
-
|
|
476
|
-
Pass `{ referencePose }` as the constructor's third argument, reusing the
|
|
477
|
-
reference captured at import. Without it, construction captures the current
|
|
478
|
-
pose, so callers must construct before playback. The head solves its bounded
|
|
479
|
-
gaze excursion in that reference hierarchy under the model's current scene
|
|
480
|
-
transform, then composes the excursion onto the evaluated authored head pose.
|
|
481
|
-
Head and neck gestures retain their motion instead of being cancelled or
|
|
482
|
-
reversed by tracking. Each eye then solves toward the same finite target from
|
|
483
|
-
its own origin in the final animated hierarchy. Full eye contribution retains
|
|
484
|
-
binocular focus wherever the combined authored motion and gaze are reachable;
|
|
485
|
-
otherwise diagnostics report the remaining angular error and limits. Authored
|
|
486
|
-
gestures are not suppressed to force a target into range. The signed limits
|
|
487
|
-
bound the tracking excursion; they do not clamp the final authored pose plus
|
|
488
|
-
that excursion against a neutral-reference joint range. Authored motion can
|
|
489
|
-
therefore take the combined pose beyond that range or the eyes' reach.
|
|
490
|
-
|
|
491
|
-
`readControlState({worldTarget, headIntensity, eyeIntensity})` provides initial
|
|
492
|
-
motor bearings without changing the rig. It inverts existing focus contributions
|
|
493
|
-
against their saved base, and seeds zero added head excursion when enabling on
|
|
494
|
-
a purely authored pose. When replacing legacy AU controls, also provide the
|
|
495
|
-
evaluated local `headBaseQuaternion` and/or `eyeBaseQuaternions` (a map keyed by
|
|
496
|
-
bone name), with those legacy controls excluded. `headSeedLimited` and
|
|
497
|
-
`eyeSeedLimited` report poses that cannot be represented by the bounded motor
|
|
498
|
-
bearings. Parallel legacy eyes cannot both be preserved exactly by a shared
|
|
499
|
-
finite target; inverse eye contribution avoids double attenuation but does not
|
|
500
|
-
remove that convergence residual. This helper supplies no handoff transition
|
|
501
|
-
clock; the host owns activation policy and movement timing.
|
|
502
|
-
|
|
503
|
-
The optional `profile.gazeCalibration` supplies bone-local
|
|
504
|
-
`head`, `leftEye`, and `rightEye` optical axes and `modelUnitsPerMeter`.
|
|
505
|
-
Without explicit optical axes, signed yaw/pitch bindings determine the optical
|
|
506
|
-
frame. Unresolvable joints remain untouched and are reported in diagnostics.
|
|
507
|
-
|
|
508
|
-
`solve_profile_viewer_space_gaze_scaled` adds `world_units_per_meter` as its
|
|
509
|
-
last argument. Multiply the authored units-per-meter calibration by the
|
|
510
|
-
presentation's uniform root scale. The original viewer solver keeps its
|
|
511
|
-
existing scale-one behavior. Neither solver infers physical webcam/display
|
|
512
|
-
placement from face landmarks.
|
|
513
|
-
|
|
514
|
-
`solve_profile_tracking_gaze` maps normalized display-right/up controls to the
|
|
515
|
-
profile's full signed head AU rotation range. Pass `head_follow_fraction = 1`
|
|
516
|
-
for full participation: on an aligned display axis, an input endpoint of +/-1
|
|
517
|
-
requests the same head rotation as the corresponding AU at intensity 1. Binding
|
|
518
|
-
scale and asymmetric `maxDegrees` remain authoritative. Camera distance and FOV
|
|
519
|
-
do not reduce this range. Intermediate inputs interpolate between the bounded
|
|
520
|
-
camera bearing at zero and the corresponding signed endpoint; camera roll
|
|
521
|
-
rotates the display axes into the model frame. With head tracking disabled,
|
|
522
|
-
the target spans the eye AU range instead.
|
|
523
|
-
|
|
524
|
-
The signature is `(profile_json, tracking_target, camera_position,
|
|
525
|
-
camera_quaternion, gaze_origin, model_quaternion, eyes_enabled, head_enabled,
|
|
526
|
-
head_follow_fraction, lock_head_to_camera, world_units_per_meter)`. It returns
|
|
527
|
-
the same 14-float layout as the existing physical projection solvers. Target Z
|
|
528
|
-
is depth in meters, clamped to 0.2–10 before scene scaling; it sets a finite
|
|
529
|
-
binocular focus distance without attenuating the angular range. X/Y clamp to
|
|
530
|
-
[-1, 1], nonfinite coordinates become zero, invalid positive depth/scale use
|
|
531
|
-
0.8/1 respectively, and absent AU bindings supply no angular capacity.
|
|
532
|
-
|
|
533
|
-
This is an input-range mapping, not a physical reconstruction of the viewer's
|
|
534
|
-
position. The existing screen/viewer projection APIs retain their contracts.
|
|
535
|
-
Consumers must select this entry point and request full head participation to
|
|
536
|
-
use it; adding the export alone does not change LoomLarge tracking. Eye focus
|
|
537
|
-
still uses the resulting finite target and reports residual error when reduced
|
|
538
|
-
strength, authored motion, or rig limits make that target unreachable.
|
|
539
|
-
|
|
540
658
|
## npm releases and PR previews
|
|
541
659
|
|
|
542
660
|
Production consumers install an exact published `@lovelace_lol/embody` version
|
|
@@ -550,9 +668,11 @@ It chooses the next patch version, records the source commit in `gitHead`, and
|
|
|
550
668
|
builds and validates once. Stable version selection reads public metadata from
|
|
551
669
|
`registry.npmjs.org` anonymously; redirects and failed lookups stop the release.
|
|
552
670
|
Metadata reads never transmit publishing credentials. The separate
|
|
553
|
-
publish job first requires
|
|
554
|
-
|
|
555
|
-
|
|
671
|
+
publish job first requires the npm visibility endpoint (the endpoint used by
|
|
672
|
+
`npm access get status`) to report the existing package as public. It reads this
|
|
673
|
+
anonymously without npm token configuration, then uses GitHub OIDC trusted
|
|
674
|
+
publishing in the `npm` environment
|
|
675
|
+
to publish the packed output with public access and create its Git tag and GitHub release. Stable
|
|
556
676
|
releases are serialized through version selection and publication. Provenance
|
|
557
677
|
remains disabled for the private source repository. Retrying a published commit
|
|
558
678
|
reuses its existing version, and an older unpublished commit cannot replace a
|
|
@@ -564,6 +684,16 @@ previews for testing linked upstream changes. An explicit ref remains a preview
|
|
|
564
684
|
request even when it names `main`; it does not publish to npm. Manual and
|
|
565
685
|
repository dispatch requests may include `upstream_pr` to identify their linked PR.
|
|
566
686
|
|
|
687
|
+
For an Embody → Polymer → LoomLarge feature stack, open every draft from its own
|
|
688
|
+
current `main` without waiting for publication. The Polymer PR body declares
|
|
689
|
+
`Depends-on: https://github.com/meekmachine/embody/pull/<number>` and the LoomLarge
|
|
690
|
+
PR body declares `Depends-on: https://github.com/meekmachine/Polymer/pull/<number>`.
|
|
691
|
+
CI resolves the compiled previews in temporary test checkouts. Committed
|
|
692
|
+
manifests and lockfiles retain exact stable npm versions; the release coordinator
|
|
693
|
+
adopts a stable release containing the change after upstream merge and publication.
|
|
694
|
+
Agents do not add preview URLs before opening downstream drafts.
|
|
695
|
+
Documentation-only companion PRs use ordinary reciprocal links instead.
|
|
696
|
+
|
|
567
697
|
After publication, the producing workflow uploads a JSON notification record
|
|
568
698
|
without a dispatch credential. The separate `Notify package publication`
|
|
569
699
|
`workflow_run` receiver checks out `main`, treats the artifact only as data, and
|
|
@@ -587,7 +717,8 @@ release-stack PR's first preview remains usable by its immutable URL before
|
|
|
587
717
|
that bootstrap merge; subsequent successful publications notify automatically.
|
|
588
718
|
|
|
589
719
|
Before enabling stable releases, the existing npm package must have public
|
|
590
|
-
visibility and
|
|
720
|
+
visibility and its npm trusted publisher must match the configuration below.
|
|
721
|
+
No npm publishing secret or read token is required.
|
|
591
722
|
Configure both the `npm` and `release-coordination` GitHub
|
|
592
723
|
environments to allow deployments from `main` only. Store `CROSS_REPO_CI_TOKEN`
|
|
593
724
|
only in `release-coordination`, with permission to dispatch to LoomLarge. These
|
|
@@ -595,6 +726,52 @@ account, environment protection, and secret settings are prerequisites; source
|
|
|
595
726
|
changes do not configure them. Existing versions are never republished or
|
|
596
727
|
overwritten.
|
|
597
728
|
|
|
729
|
+
### Configure and recover stable publication
|
|
730
|
+
|
|
731
|
+
The npm package settings must register this GitHub Actions trusted publisher:
|
|
732
|
+
|
|
733
|
+
| Setting | Value |
|
|
734
|
+
| --- | --- |
|
|
735
|
+
| Package | `@lovelace_lol/embody` |
|
|
736
|
+
| Organization or user | `meekmachine` |
|
|
737
|
+
| Repository | `embody` |
|
|
738
|
+
| Workflow filename | `pr-checks.yml` |
|
|
739
|
+
| Environment | `npm` |
|
|
740
|
+
| Publishing permission | **Allow npm publish** |
|
|
741
|
+
|
|
742
|
+
New npm trusted publishers default to staging only; explicitly allow publishing.
|
|
743
|
+
The package's `repository.url` must match `git+https://github.com/meekmachine/embody.git`.
|
|
744
|
+
These settings live on npm and cannot be configured by merging workflow YAML.
|
|
745
|
+
Retain account 2FA and GitHub environment protections. The job uses Node 24 and
|
|
746
|
+
requires npm >=11.5.1 (with Node >=22.14.0), publishes without `NPM_KEY` or
|
|
747
|
+
`NODE_AUTH_TOKEN`, and never grants an OIDC token to PR or preview jobs.
|
|
748
|
+
Provenance remains disabled for this private source repository.
|
|
749
|
+
|
|
750
|
+
A green `pkg.pr.new` preview confirms a preview, not stable publication. The
|
|
751
|
+
main-only publish job writes a **Stable npm publication confirmed** summary only
|
|
752
|
+
after an anonymous exact-version registry read confirms the version and `gitHead`, and
|
|
753
|
+
records the release's SHA-512 integrity. New uploads must match the verified
|
|
754
|
+
tarball's integrity. Retries confirm an existing immutable release without
|
|
755
|
+
assuming that rebuilding the same source reproduces every archive byte. A failed publish or conflicting registry result
|
|
756
|
+
fails the job and blocks downstream notification. npm authentication failures show
|
|
757
|
+
the error code and exact publisher settings to repair, without echoing raw npm
|
|
758
|
+
output or tokens. The publisher checks the exact-version endpoint before any
|
|
759
|
+
upload, with `Cache-Control: no-cache`, so a stale package index cannot cause a
|
|
760
|
+
duplicate upload. Only HTTP 404 means absent; other lookup failures stop publication.
|
|
761
|
+
A new upload gets one exact-version confirmation lookup, without polling. A transient registry failure can leave an upload present
|
|
762
|
+
but unconfirmed; rerun the main release after investigation rather than upload a
|
|
763
|
+
replacement version blindly.
|
|
764
|
+
|
|
765
|
+
After repairing configuration, run this workflow on current `main` with its `ref`
|
|
766
|
+
input empty. A successful current-main stable release contains all source changes
|
|
767
|
+
already merged into that repository; individual failed intermediate releases do
|
|
768
|
+
not need replaying. Explicit refs and preview dispatches publish previews only.
|
|
769
|
+
An existing matching release is confirmed without publishing it again; source or
|
|
770
|
+
dependency conflicts and new-upload archive conflicts stop instead of overwriting it. The receipt
|
|
771
|
+
establishes npm publication, while the LoomLarge coordinator separately opens or
|
|
772
|
+
updates reviewed stable dependency PRs and lockfiles. It does not establish that
|
|
773
|
+
Polymer or the application adopted the release or deployed it.
|
|
774
|
+
|
|
598
775
|
## Development
|
|
599
776
|
|
|
600
777
|
```bash
|
|
@@ -643,6 +820,31 @@ Reset an authored control by setting its mapped AU values to zero. Hosts select
|
|
|
643
820
|
which mapped AU IDs to reset; unrelated face state and saved mix strengths remain
|
|
644
821
|
under their existing owners.
|
|
645
822
|
|
|
823
|
+
### Reading current AU state
|
|
824
|
+
|
|
825
|
+
Rust records each admitted AU-curve clip's authored keys, resolved inherited
|
|
826
|
+
start, strength and easing. Those records change only at admission or a
|
|
827
|
+
strength/easing edit; nothing ticks them. Reads evaluate them on demand at the
|
|
828
|
+
native action's time and effective weight, so a fading slot reports
|
|
829
|
+
value × weight:
|
|
830
|
+
|
|
831
|
+
- `readAUContribution(name)` returns one slot's own `{auId: value}`, or `null`
|
|
832
|
+
when it is not playing or was not an AU-curve clip.
|
|
833
|
+
- `readAUState()` returns `{live, animated, total, unreadable}`: live control
|
|
834
|
+
values, active AU-curve clip contributions and their sum. Baked/native and
|
|
835
|
+
typed clips animate rig outputs directly and cannot be inverted to AU values,
|
|
836
|
+
so they are listed in `unreadable` rather than estimated.
|
|
837
|
+
- `measureAUGoal(name, destinations)` returns `{distance, reversal, current}`:
|
|
838
|
+
the largest travel from the slot's current contribution to `{auId: value}`
|
|
839
|
+
destinations, measured along profile `continuumPairs` as `positive − negative`
|
|
840
|
+
(crossing neutral from 0.3 negative to 0.3 positive is 0.6), and whether it
|
|
841
|
+
reverses the slot's unfinished travel. Callers apply their own timing policy
|
|
842
|
+
and send an ordinary inherited clip with the duration they choose.
|
|
843
|
+
|
|
844
|
+
Each call does a small amount of work in Rust; read when deciding, not every
|
|
845
|
+
frame. See [inherited clip starts](docs/INHERITED_CLIP_STARTS.md) for how
|
|
846
|
+
inherited starts and weights resolve.
|
|
847
|
+
|
|
646
848
|
Named semantic pose/animation APIs and profile `bodyControls` are retired.
|
|
647
849
|
Persisted named records must be decoded at the storage boundary to numeric AU
|
|
648
850
|
values/curves, using their original profile mappings. `LEGACY_BODY_CONTROL_AUS`
|
|
@@ -650,6 +852,62 @@ is a deprecated reader-only historical identity snapshot for old CC4 records;
|
|
|
650
852
|
it is never consulted for current controls, actuation or profile compilation.
|
|
651
853
|
Unknown legacy names must be reported rather than silently omitted.
|
|
652
854
|
|
|
855
|
+
### Mesh highlight previews
|
|
856
|
+
|
|
857
|
+
Rust owns transient mesh highlight state and its renderer-neutral output in
|
|
858
|
+
`src/highlighting.rs`, exposed through `RuntimeCore`. This preview is independent
|
|
859
|
+
of AU/morph/bone actuation, clip playback, and persisted profile data. Rust resolves
|
|
860
|
+
mesh names (including duplicate names) against the configured model, chooses the
|
|
861
|
+
default gold color and intensity, computes darker edges in linear color space,
|
|
862
|
+
and advances a 400 ms eased transition. Interrupted fades resume from the current
|
|
863
|
+
strength; changing targets removes the old preview immediately. Clearing or an
|
|
864
|
+
unknown name fades out the current selection. Repeated identical requests do not
|
|
865
|
+
restart a fade. Model reconfiguration clears transient highlights.
|
|
866
|
+
|
|
867
|
+
| Core operation | Contract |
|
|
868
|
+
| --- | --- |
|
|
869
|
+
| `request_mesh_highlight(name?, color?, intensity?)` | Request a mesh name or omit it to clear. Optional color is packed 24-bit sRGB; intensity is clamped to 0–1, with nonfinite values using the default. |
|
|
870
|
+
| `evaluate_mesh_highlight_frame()` | Return the current complete JSON snapshot without advancing time. |
|
|
871
|
+
| `advance_mesh_highlight(delta_seconds)` | Advance preview time using the host render clock and return the snapshot. Negative/nonfinite deltas do not advance time. |
|
|
872
|
+
|
|
873
|
+
Snapshot version 1 contains `highlights`, an array of writes with `meshId`,
|
|
874
|
+
`fillColor`, `fillIntensity`, `edgeColor`, `edgeMode: "triangles"`, and `edgeOpacity`.
|
|
875
|
+
Colors are linear RGB triplets, strength/opacity are in 0–1, and IDs refer to the
|
|
876
|
+
configured model descriptor. The fill is an emissive surface preview and the
|
|
877
|
+
edges show triangle topology. An omitted mesh ID means restore its original
|
|
878
|
+
appearance and release its preview resources. `{ "version": 1, "highlights": [] }`
|
|
879
|
+
clears every preview. The TypeScript contract is exported from
|
|
880
|
+
`@lovelace_lol/embody/wasm` without importing a renderer.
|
|
881
|
+
|
|
882
|
+
Another renderer configures the same `RuntimeCore` from its own model descriptor,
|
|
883
|
+
keeps a map of mesh IDs to native surfaces, sends requests, and applies snapshots
|
|
884
|
+
from its existing render tick. It does not traverse by name, derive edge colors,
|
|
885
|
+
or reproduce the easing curve. For example, after configuring `core`:
|
|
886
|
+
|
|
887
|
+
```ts
|
|
888
|
+
core.request_mesh_highlight('Face');
|
|
889
|
+
// Apply the initial snapshot immediately to remove obsolete selections.
|
|
890
|
+
applyNativePreview(JSON.parse(core.evaluate_mesh_highlight_frame()));
|
|
891
|
+
// Inside the host's existing render tick (seconds, independent of clip speed):
|
|
892
|
+
applyNativePreview(JSON.parse(core.advance_mesh_highlight(deltaSeconds)));
|
|
893
|
+
```
|
|
894
|
+
|
|
895
|
+
`applyNativePreview` is the renderer's effect boundary: resolve `meshId`, apply
|
|
896
|
+
emission and triangle edges from the supplied values, and remove omitted previews.
|
|
897
|
+
The host must advance the core once per frame, even with no playing clips. Pausing
|
|
898
|
+
its render clock pauses preview progression; no additional animation loop exists.
|
|
899
|
+
|
|
900
|
+
For Three, `character.highlightMesh(name, color?, intensity?)` sends the core
|
|
901
|
+
request; the existing scene playback binding supplies elapsed seconds and calls
|
|
902
|
+
`ThreeFrameApplier.applyMeshHighlightFrame`. The applier only binds mesh IDs and
|
|
903
|
+
manages native materials/overlays, deformation, picking exclusion and disposal.
|
|
904
|
+
It does not choose colors, resolve names, or sample fades. Temporary materials
|
|
905
|
+
are isolated from shared source materials; geometry and skeletons remain owned
|
|
906
|
+
by the source model. Native overlays follow source geometry, morph weights and
|
|
907
|
+
skeletons. Character rebind/disposal removes them. Direct applier consumers must
|
|
908
|
+
call `clearMeshHighlights()` on disposal. Other renderers implement their own
|
|
909
|
+
snapshot application; this change does not supply a second native renderer.
|
|
910
|
+
|
|
653
911
|
### Character-derived studio background
|
|
654
912
|
|
|
655
913
|
The Three `createCharacterHost` paints its container with soft gradients sampled
|