@lovelace_lol/embody 0.0.15 → 0.0.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -9,6 +9,28 @@ Embody is a Rust/Wasm character animation core for facial action units,
9
9
  visemes, bone controls, baked animation clips, and hair motion. Three.js is a
10
10
  host adapter, not the runtime implementation.
11
11
 
12
+ ## Reading historical saved movements
13
+
14
+ `decodeLegacyPose(record, historicalProfile?)` and
15
+ `decodeLegacyAnimation(record, historicalProfile?)` translate retired named
16
+ `semanticPose` / `semanticAnimation` records into existing numeric AU fields.
17
+ The translation and conflict checks live in Rust. Await `initEmbodyCore()` before
18
+ decoding historical records; ordinary records without the retired field return
19
+ unchanged and do not require initialization. Inputs are JSON saved records.
20
+
21
+ Explicit `historicalProfile.bodyControls` metadata is authoritative, including an
22
+ empty map. Only when that metadata is absent does the decoder use the embedded
23
+ historical table. Unresolved names, conflicting authored values, and opposing
24
+ pose directions fail without changing the input. Keyframe seconds, balance,
25
+ morph strength, custom profile IDs, and unrelated saved metadata are preserved.
26
+ The returned copy removes the decoded retired field; this does not write storage.
27
+
28
+ These readers do not consolidate eye mappings, assign FACS/body namespaces,
29
+ convert curve snippets into explicit channels, or change native playback. Those
30
+ migrations require profile-specific equivalence and compiler parity work under
31
+ [Polymer epic #325](https://github.com/meekmachine/Polymer/issues/325). Semantic
32
+ readback and slider ownership remain separate integration work.
33
+
12
34
  ## Architecture
13
35
 
14
36
  ```text
@@ -16,14 +38,16 @@ src/ Rust source (the Wasm crate) — hair, clips, profiles, run
16
38
  assets/presets/ Embedded typed character profiles
17
39
  assets/templates/ Embedded humanoid skeleton templates
18
40
  three/ Thin Three.js inspection/application adapter only
41
+ model-tools/ Browser/Node GLB preparation, public types, and CLI
19
42
  wasm/ Generated-Wasm JS loader + ABI constants
20
43
  index.ts Package re-exports only
21
44
  ```
22
45
 
23
- There is no parallel TypeScript hair/physics/runtime core. Hair idle, impulse,
24
- gravity curves and the spring solver live in `src/hair_curves.rs` /
46
+ There is no parallel TypeScript hair/physics/runtime core. Hair idle, impulse
47
+ and gravity curve helpers and the continuous response live in `src/hair_curves.rs` /
25
48
  `src/hair.rs` and are exported through Wasm. Hosts schedule the resulting
26
- ClipIR on their mixer; they must not reimplement hair sampling in TS.
49
+ ClipIR on their mixer; they must not reimplement hair sampling in TS. See
50
+ [Hair motion](#hair-motion) for how the character host drives them.
27
51
 
28
52
  The Rust core owns:
29
53
 
@@ -35,8 +59,9 @@ The Rust core owns:
35
59
  - hair curves, hair physics, and appearance normalization
36
60
  - annotation regions, camera gestures/flights/orbits, marker placement and visibility,
37
61
  preview/reveal lifecycles, and humanoid template fitting
38
- - profile-aware screen-space gaze geometry for eye/head AU trajectories
62
+ - on-demand semantic AU reads of active AU-curve clips (no clock or polling)
39
63
  - renderer-neutral model analysis and packed live frame generation
64
+ - transient mesh highlight selection, styling, transitions, and versioned preview snapshots
40
65
 
41
66
  `resolveMorphMeshSelections(profile, categories, availableNames)` resolves editor
42
67
  mesh selections in one Rust request. It returns each requested category's effective
@@ -62,6 +87,38 @@ The host animation library (Three `AnimationMixer`, Unity Animator, etc.) owns
62
87
  clip playback, lerping, blending, looping, seeking, and crossfades. Rust must
63
88
  not sample or lerp clips on the hot path.
64
89
 
90
+ For portable body clips, Rust converts Mixamo references/keys into standard
91
+ VRM Animation 1.0 `.vrma` GLB files and retargets supported humanoid tracks.
92
+ The Three adapter exposes `convertMixamoFbxToVrmAnimations`,
93
+ `convertMixamoToVrmAnimation`, `inspectVrmAnimation`, and
94
+ `createAnimationClipFromVrmAnimation`. Initialize Wasm first; supply explicit
95
+ units and a target T-pose reference. See [body-animation interchange](docs/ANIMATION_REFERENCE_POSES.md#mixamo-fbx-and-vrm-animation-body-interchange)
96
+ for examples, supported inputs and errors. VRMA joint rotations remain distinct
97
+ from authored semantic action-space controls and facial/morph snippets.
98
+
99
+ The same native `Embody` character exposes `supportsVrmAnimation()`,
100
+ `convertMixamoFbxToVrmAnimations`, `getVrmAnimationInfo`,
101
+ `registerVrmAnimation(name, bytes, { metersPerUnit })` and
102
+ `releaseVrmAnimation(name)`. Registration uses the character's captured reference
103
+ and resolved profile, and does not play or stop an action. Polymer's Animation
104
+ agency schedules later playback. Cleanup removes only the accepted native source
105
+ on the original model, preserving a later same-name replacement; disposal is safe.
106
+
107
+ `Embody` is the character API exposed by the native Three adapter.
108
+ `createRustEmbodyHost`, `createRustEmbodyHostUnbound` and
109
+ `createCharacterHost(...).engine` use that same class. Its profile, mesh,
110
+ material, hair and clip-catalog result types come from `three/authoringTypes.ts`;
111
+ downstream libraries can re-export them without another character interface.
112
+ `getProfile()` returns `Partial<Profile>` because an explicit profile may be
113
+ empty, and `getHairPhysicsConfig()` may be empty before the first model bind.
114
+ `ProfileInput` accepts sparse mappings and app metadata without adding arbitrary
115
+ keys to the output `Profile` declaration. Humanoid queries accept sparse profiles;
116
+ `setHumanoidRoleBinding` returns a sparse profile because Rust omits empty tables.
117
+ The bone editor queries and mutation results follow the same sparse data contract.
118
+ `setAUMorphTargets` preserves its input JSON and guarantees the edited `auToMorphs`
119
+ map. An omitted serialized field is not a request to delete an authored override.
120
+ These type declarations add no playback state, lifecycle or animation clock.
121
+
65
122
  An AU animation drives its profile's mapped bones and morphs through the same
66
123
  Rust evaluation used by live AU controls. For example, speech can send an AU
67
124
  103 curve and separate viseme shape curves; the profile decides which bones
@@ -78,6 +135,17 @@ construction. Annotation adapters capture DOM input, observe native geometry,
78
135
  apply Rust camera/marker frames, render labels and release native resources.
79
136
  Camera integration, region selection and marker state belong to Rust.
80
137
 
138
+ `createSceneSceneryRenderer` from `@lovelace_lol/embody/three` applies resolved
139
+ scenery layouts to the existing scene: self-contained GLB placements with independent
140
+ world XYZ positions, equirectangular skyboxes, image backdrops and solid colors.
141
+ Applications supply an abortable authorized byte loader and retain saved layouts,
142
+ asset catalogs and Persona/Character/Interaction selection policy. The adapter
143
+ owns model/texture loading, stale-result cancellation and renderer-drained cleanup;
144
+ it preserves the character, lights and existing render loop. Character scene
145
+ settings remain the inherited background beneath scenery, so clearing an override
146
+ restores the latest character configuration. See the
147
+ [scenery contract](docs/SCENE_READINESS.md#scenery-layouts-and-background-ownership).
148
+
81
149
  `ThreeFrameApplier.applyMeshMaterialConfigs(model, profile.meshes)` replays named
82
150
  mesh visibility and material overrides when a host binds or replaces a profile.
83
151
  It applies explicit `visible: false` and `visible: true` even without a material
@@ -95,6 +163,88 @@ replacing differently sized geometry. Hiding or changing the hover clears its
95
163
  sample cache; an explicitly empty mesh selection produces no preview. Applications
96
164
  forward hover/show/hide events and dispose the overlay when its scene is released.
97
165
 
166
+ ### Preparing Actor Models
167
+
168
+ `@lovelace_lol/embody/model-tools` exposes `inspectActorModel(bytes)` and
169
+ `prepareActorModel(bytes, options)` to prepare separate GLB files for an Actor.
170
+ This module owns file transformations; LoomLarge owns the guided workflow,
171
+ Model upload, Actor persistence, and Mapping selection. It does not load the
172
+ animation runtime or initialize Embody's runtime Wasm.
173
+
174
+ ```js
175
+ import { inspectActorModel, prepareActorModel } from '@lovelace_lol/embody/model-tools';
176
+
177
+ const source = new Uint8Array(await file.arrayBuffer());
178
+ const inspection = await inspectActorModel(source);
179
+ // Show inspection.meshes to the creator before choosing explicit mesh IDs.
180
+ const prepared = await prepareActorModel(source, {
181
+ kind: 'low-resolution',
182
+ targetRatio: 0.5,
183
+ maxError: 0.001,
184
+ signal: abortController.signal,
185
+ });
186
+ // Persist prepared.bytes as a new Model only after the operation succeeds.
187
+ ```
188
+
189
+ Inspection returns source-scoped mesh IDs, original names, node names,
190
+ materials, vertex/triangle counts, skin and morph presence, total byte length,
191
+ and processing limitations. Preparation returns a new byte array, before/after
192
+ inspection, the removed source mesh IDs, and warnings. It never writes into the
193
+ input array, fetches resources, or persists an Actor.
194
+
195
+ | Kind | File operation |
196
+ | --- | --- |
197
+ | `original` | Copy source bytes exactly. |
198
+ | `optimized` | Compact unused vertex data and remove explicitly selected meshes and orphaned file resources. |
199
+ | `low-resolution` | Reduce triangle count; default target is 50%, limited by geometry, rig protection, and error. |
200
+ | `miniature` | Apply an explicitly supplied uniform `scale` above existing scene roots. This changes physical scale, not texture resolution. |
201
+ | `bust` | Keep explicit `keepMeshIds`; requires separately authored body-part meshes. |
202
+ | `outfit` | Keep/remove explicit mesh IDs for a prepared outfit; never guess garments from names. |
203
+
204
+ `targetRatio` accepts 0.05–1 and `maxError` accepts 0–0.05 relative to each
205
+ primitive's extent. `scale` is required only for miniature and accepts 0.01–0.99.
206
+ Mesh IDs belong to the inspected input; inspect each new file before editing it
207
+ again. Removing a shared mesh removes all its instances. A combined body mesh
208
+ must be split in a modeling tool before a bust or garment can be removed. These
209
+ tools do not generate clothing, transfer garments between rigs, cut surfaces,
210
+ or implement runtime garment switching.
211
+
212
+ Processed variants require self-contained, extension-free GLB. Unsupported
213
+ extensions (including compression and VRM) or resource URIs produce an explicit
214
+ error before rewriting. Original copies retain such payloads and inspection
215
+ reports `canTransform: false`; hosts must still apply their upload policy.
216
+ Textures keep their original bytes and dimensions. Output includes actual byte
217
+ and triangle counts; a requested reduction is not a size guarantee.
218
+
219
+ Triangle reduction locks vertices participating in any morph and vertices at
220
+ skin-influence transitions. It preserves topological borders and weighs normals,
221
+ UVs, and colors. All retained attributes, including joints, weights and every
222
+ morph target, use one vertex remap. Existing mesh/node names, skeleton hierarchy,
223
+ bind matrices and surviving animation targets remain intact. Only morph animation
224
+ channels targeting an explicitly removed mesh are removed, with a warning.
225
+ No node, joint, or surviving morph target is renamed. Hosts must review Mapping
226
+ targets when source names are missing/duplicated or a mapped mesh was removed;
227
+ vertex-index annotations require remapping after compaction.
228
+
229
+ Cancellation is checked between processing stages and primitives. A synchronous
230
+ simplifier call cannot be interrupted halfway through; hosts should use a worker
231
+ when processing large assets must not occupy the UI thread.
232
+
233
+ The installed `embody-prepare-actor` command uses the same implementation:
234
+
235
+ ```bash
236
+ embody-prepare-actor --input character.glb --inspect
237
+ embody-prepare-actor --input character.glb --output low.glb --kind low-resolution --ratio 0.5
238
+ embody-prepare-actor --input character.glb --output outfit.glb --kind outfit --remove-mesh mesh:2
239
+ embody-prepare-actor --input character.glb --output bust.glb --kind bust --keep-mesh mesh:0
240
+ ```
241
+
242
+ Repeat `--remove-mesh`/`--keep-mesh` for multiple selections. The CLI refuses an
243
+ existing output path, including the source file or a symlink to it. It prints a
244
+ JSON report after creating the new file. In a source checkout, invoke
245
+ `node model-tools/prepare-actor.mjs` with the same arguments. CI runs the
246
+ `test:model-tools` regression source as part of the existing package checks.
247
+
98
248
  ### Annotation runtime
99
249
 
100
250
  `AnnotationRuntime` in `src/annotation_runtime/` owns the annotation behavior
@@ -124,11 +274,31 @@ HTML overlays align with the canvas rectangle inside its container, update on
124
274
  resize/rebind, and restore borrowed parent positioning when the final view is
125
275
  disposed. These coordinates assume axis-aligned CSS boxes without transforms.
126
276
 
127
- AU morph anchors currently sample native deformed vertices during marker
128
- preparation, then follow the selected object's transform. They do not yet
129
- resample morph weights or skinning on subsequent frames. Animated surface
130
- tracking requires cached vertex candidates and per-frame pose observations;
131
- ordinary object-transform tracking does not provide that behavior.
277
+ AU morph anchors follow morph and skeletal animation on each annotation update.
278
+ Rust selects and caches affected vertices and their influence weights during
279
+ marker preparation, using mesh-space geometry so rotating a character cannot
280
+ change the selected semantic band. The adapter evaluates those vertices in the
281
+ current native pose and supplies packed world-space positions; Rust updates the
282
+ existing anchors without restarting reveal or rebuilding marker resources.
283
+ Explicit point and whole-model anchors retain their existing behavior. Model or
284
+ region changes invalidate the plans, and stale or invalid pose observations are
285
+ ignored. Geometry replacement, attribute replacement, morph dictionary edits and
286
+ relative/absolute mode changes are observed on `update()`. For in-place attribute
287
+ edits, set the Three attribute's `needsUpdate = true` (including interleaved
288
+ attributes). Rust retires the affected plans and selects replacements without
289
+ reloading regions or replacing marker resources. When a previously valid
290
+ attachment loses usable position or morph data, its marker becomes hidden and
291
+ `getMarkerPosition()` returns `null` until that data returns. Other attachments
292
+ and current selection remain intact.
293
+
294
+ `getRuntimeAUMorphAnchorPoint` uses the same Rust mesh-space selection and weighted
295
+ pose reduction as live attachments, including rotated and skinned meshes. Native
296
+ adapters only report geometry identities/versions and evaluate selected vertices;
297
+ they do not choose semantic bands or compute attachment weights. Other adapters
298
+ can report monotonically increasing per-object revisions with
299
+ `observe_morph_geometry` before preparing replacement plans, then submit packed
300
+ world-space XYZ samples to `observe_morph_anchor`. Plan IDs are scoped to the
301
+ runtime generation; retired IDs and older geometry revisions are rejected.
132
302
 
133
303
  This is a breaking replacement of the DPthree exports. Use
134
304
  `ThreeAnnotationController.create(options)` or `ThreeAnnotations.create(options)`
@@ -141,16 +311,20 @@ instance. Region and camera snapshots are observations; mutate through controlle
141
311
  methods. Explicit `undefined` fields in `updateAnnotationRegion` remove overrides.
142
312
  `getCharacterConfig()` retains snapshot identity until configuration content or
143
313
  the model changes; selection, visibility and camera input do not replace it.
314
+ Authored intro focus targets use the same framing as manual region selection;
315
+ extra eye padding and composition bias apply only to the default region target.
144
316
  Call `clearModel()` before releasing a removed model's native resources. The
145
317
  controller retains renderer rebinding, canvas input reconnect and resource-drain
146
318
  support through the scene's rendering controller.
147
319
  Runtime preview region IDs retain the `runtime:annotation:` namespace.
148
320
 
149
321
  When the annotation controller owns the renderer loop, it advances registered
150
- native character mixers before observing annotation transforms and drawing the
322
+ native character mixers before observing annotation poses and drawing the
151
323
  scene. The first frame establishes the timer origin; skipped asynchronous draws
152
324
  do not accumulate playback time. External render loops use the existing
153
- `createScenePlaybackFrame` integration to advance character playback.
325
+ `createScenePlaybackFrame` integration to advance character playback, then call
326
+ `annotations.update()` before drawing. No per-frame application-side anchor
327
+ logic or agency registration is required.
154
328
 
155
329
  ```ts
156
330
  import { ThreeAnnotations } from '@lovelace_lol/embody/three';
@@ -209,6 +383,29 @@ React hosts forward user events and subscribe to controller state. They do not
209
383
  need local copies of the camera, marker, placement or timer algorithms.
210
384
  The convenience facade delegates to one controller-owned marker layer.
211
385
 
386
+ ## Character model loading
387
+
388
+ The Three adapter loads GLB/glTF and FBX characters into the same
389
+ `{ model, meshes, animations, gltf }` result. `meshes` contains morph-enabled
390
+ meshes; FBX results have `gltf: null`. Both formats use the same shadow setup,
391
+ model transforms, inspection, and disposal helpers.
392
+
393
+ ```ts
394
+ import { loadCharacterModel, parseCharacterModel } from '@lovelace_lol/embody/three';
395
+
396
+ const character = await loadCharacterModel('/characters/avatar.fbx');
397
+ // For blob URLs or endpoints without a file extension, supply the format.
398
+ const uploaded = await loadCharacterModel(objectUrl, { format: 'fbx' });
399
+ // Standard binary/ASCII FBX headers are detected automatically. The resource path resolves
400
+ // external textures referenced by the file.
401
+ const cached = await parseCharacterModel(buffer, '/characters/textures/');
402
+ ```
403
+
404
+ Use `format: 'gltf'` or `format: 'fbx'` to override detection. Draco options apply
405
+ to glTF only. FBX rigs, morph names, scale, and external texture paths depend on
406
+ the export; use a matching character profile and model transform as needed.
407
+ Loading a format does not automatically retarget its rig or rename its morphs.
408
+
212
409
  ## Runtime Use
213
410
 
214
411
  Embody is an ES module package. Use `import` / dynamic `import()`; CommonJS
@@ -258,8 +455,61 @@ function update(dtSeconds: number) {
258
455
  }
259
456
  ```
260
457
 
261
- Application-facing JavaScript APIs belong in the host package. Polymer owns
262
- the CLJS character host used by LoomLarge and calls the Wasm exports directly.
458
+ Embody owns the native Three character API and mixer binding. Polymer exposes
459
+ that API through its CLJS facade for LoomLarge; the facade does not own another
460
+ playback clock or hair simulation.
461
+
462
+ ### Hair motion
463
+
464
+ When the profile enables `hairPhysics` and its registered hair/eyebrow meshes
465
+ contain mapped targets, the Three character host plays one additive native action,
466
+ `__embody_hair_response`. Rust's `HairResponse` owns velocity, spring displacement,
467
+ ambient phase and gravity response. `RuntimeCore.build_hair_response_clip` emits
468
+ zero-neutral morph offsets; the host replaces samples on the existing native
469
+ action when its target bindings stay the same. Numeric tuning and head retargets
470
+ preserve both the action identity and the simulation state.
471
+
472
+ After every mixer has sampled, and before native completion releases its
473
+ resources, `renderScenePlayback` passes the effective mixer delta to the character.
474
+ The character observes the profile's humanoid `head` role in model space relative
475
+ to the captured reference pose, including parent neck/spine motion. It expresses
476
+ that delta in the bound local reference before passing primitive quaternions to
477
+ Rust. The model root's placement rotation is excluded. Snippets, baked clips,
478
+ live setters and external bone writers therefore drive the same response.
479
+ A profile without a head role still supports idle and wind with no invented rig
480
+ mapping. Rebinding primes head history only after restored live controls have
481
+ sampled, avoiding a false rest-to-pose impulse.
482
+
483
+ Rust combines observed angular velocity, idle/wind, and smoothed pitch gravity
484
+ using `inertia`, `stiffness`, `damping`, `gravity`, `responseScale` and `direction`.
485
+ Directional mapping approaches neutral with zero slope. Contributions sharing a
486
+ concrete morph add before a soft limit keeps that response within `[0, 2]`.
487
+ The next native mixer sample applies the resulting offsets, so output trails the
488
+ observed head by one render step. There is no second mixer update or hair timer.
489
+ Playing, replaying or retargeting a caller clip performs no hair work.
490
+
491
+ `Embody.stop()` suspends response sampling with rendering. Mixer time scale zero
492
+ freezes physics; reversed native playback still advances physics forward while
493
+ the actual reversed pose determines motion direction. A renderer gap longer than
494
+ 250 ms re-primes head velocity and integrates at most 250 ms of settling. Numeric
495
+ configuration edits preserve momentum and ambient phase. Clearing mappings releases
496
+ native output without erasing simulation state; disabling hair resets dynamics.
497
+ Registration, model rebind and profile replacement start a new response binding.
498
+ Disposal removes its render hook and releases native output.
499
+
500
+ Hair actions have explicit character ownership. `stopAllAnimations`, `crossfadeTo`,
501
+ `pauseAllAnimations` and `resumeAllAnimations` leave them running; Stop All and
502
+ crossfade also leave live setter targets intact. Aggregate pause/resume continues
503
+ to pause/resume live setter transitions as well as caller playback. Explicit
504
+ `pauseAnimation('__embody_hair_response')` freezes the simulation, and that pause
505
+ survives cleared mappings and disable/re-enable until the matching
506
+ `resumeAnimation` call. Profile replacement starts a new configuration.
507
+
508
+ `playHairImpulse(horizontal, vertical)` retains its existing separately controllable
509
+ one-shot native handle. It is never triggered by caller clip admission. The lower
510
+ level `RuntimeCore.add_hair_response_impulse` instead injects velocity into the
511
+ continuous response without creating an action; neither path replaces head history.
512
+ Legacy idle/impulse/gravity curve exports remain available to explicit consumers.
263
513
 
264
514
  ### Direct viseme snapshots
265
515
 
@@ -337,50 +587,6 @@ when the Three clip is constructed. Hosts must construct it again to capture
337
587
  a fresh pose; this does not change cached host replay or generated bone
338
588
  inheritance behavior.
339
589
 
340
- ### Gaze geometry contract
341
-
342
- `solve_profile_screen_space_gaze` and `solve_profile_viewer_space_gaze` return
343
- 14 floats: combined target XY, eye target XY, head target XY, total yaw/pitch,
344
- camera yaw/pitch, eye-to-camera distance, and viewer world XYZ. XY outputs are
345
- signed AU intensities in [-1, 1]; positive Y is up and horizontal direction is
346
- subject-relative. Angular values are degrees.
347
-
348
- Pass current world camera position/quaternion, character eye midpoint, and
349
- model quaternion. The viewer solver takes normalized image XY (+Y up) and
350
- depth behind the virtual camera; depth and all positions must share scene
351
- units. Convert physical webcam estimates before calling and supply the actual
352
- source FOV/aspect. Zero viewer XY extends the character-eye-to-camera bearing,
353
- so the character looks at the viewer even when scene framing puts its eyes
354
- away from image center. Offsets retain the camera's right/up axes; when the
355
- camera coincides with the eyes, its local +Z supplies the fallback bearing.
356
- The depth guard is 0.2–10 scene units. Mouse and webcam can share this contract
357
- when they use the same calibration.
358
-
359
- Ordinary head following participates toward the common target from calibrated
360
- model-neutral +Z: `head_follow_fraction` is a share of the target yaw/pitch,
361
- defaulting to 0.35 for a non-finite value. Zero prefers neutral, one follows the
362
- full target, and eye overflow can recruit more head movement within its limits.
363
- The explicit `lock_head_to_camera` flag instead prefers the camera bearing while
364
- eyes can reach; a zero follow fraction alone does not select that policy. When
365
- eyes are disabled the head carries the target regardless of follow or lock.
366
- This removes the previous implicit camera bias, which could point the head
367
- opposite the selected target when the camera was off-center. Hosts that intend
368
- camera-facing behavior must request the explicit viewer-solver lock; Polymer's
369
- `headFollowEyes: false` maps to zero participation, not an implicit lock.
370
- Eye angles are solved in the
371
- rotated head frame, using yaw then pitch to match runtime composition. Limits
372
- and AU normalization are directional and include each binding's scale. Missing
373
- or morph-only mappings contribute no inferred angular capacity.
374
-
375
- The profile must describe the active rig with calibrated semantic axes:
376
- model +Z is forward, +Y is up. This ABI does not include the skeleton's rest
377
- frames or optical axes, per-eye origins, current animated head pose, or
378
- webcam-to-display calibration. It solves an endpoint in the assumed calibrated
379
- basis; it does not provide binocular convergence or compensate intermediate
380
- head motion. Shared eye commands use the smaller mapped eye capacity; unequal
381
- left/right ranges need separate calibration. Hosts must compose the requested
382
- rotations without averaging independent yaw/pitch clips together.
383
-
384
590
  For import-time inspection, `captureModelReferencePose` provides an explicit,
385
591
  immutable transform and morph reference that hosts can reuse after playback.
386
592
  `extendModelReferencePose` explicitly adds appended skeletons while preserving
@@ -449,94 +655,6 @@ const report = JSON.parse(wasm.analyze_model_descriptor(
449
655
  ));
450
656
  ```
451
657
 
452
- ## Pose-aware focus
453
-
454
- `ThreeGazeFocus` from `@lovelace_lol/embody/three` applies a focus constraint
455
- inside an animation runtime. It has no clock or target-selection policy:
456
- call `restore()` before evaluating the base animation, then `apply(request,
457
- controls)` after it. Restoration compares quaternion orientations independently
458
- of Float32 sample norms, preserving the exact authored sample and any newer
459
- authored orientation. Repeated applications between mixer ticks do not accumulate
460
- the constraint's previous correction. Controls are the current positions of existing motor
461
- tracks. The optional sampled `controls.headIntensity` and `controls.eyeIntensity`
462
- override the matching request fields, so the host can animate configuration
463
- changes on its existing motor tracks. Both mean contribution in [0, 1]: zero
464
- preserves the evaluated base, one applies the full solved excursion, and legacy
465
- values above one normalize to one. Intermediate eye contribution proportionally
466
- reduces the correction; it does not shrink anatomical limits. Reduced eye
467
- contribution deliberately leaves focus error. These controls do not amplify
468
- input displacement or control response speed.
469
-
470
- During travel, the remaining sampled bearing rotates the requested finite
471
- target. At the endpoint that rotation becomes identity, reaching the exact
472
- world target continuously. There is no settled-threshold switch between targets
473
- constructed from the authored and tracking-shifted eye origins; a larger head
474
- turn must not introduce a final eye snap.
475
-
476
- Pass `{ referencePose }` as the constructor's third argument, reusing the
477
- reference captured at import. Without it, construction captures the current
478
- pose, so callers must construct before playback. The head solves its bounded
479
- gaze excursion in that reference hierarchy under the model's current scene
480
- transform, then composes the excursion onto the evaluated authored head pose.
481
- Head and neck gestures retain their motion instead of being cancelled or
482
- reversed by tracking. Each eye then solves toward the same finite target from
483
- its own origin in the final animated hierarchy. Full eye contribution retains
484
- binocular focus wherever the combined authored motion and gaze are reachable;
485
- otherwise diagnostics report the remaining angular error and limits. Authored
486
- gestures are not suppressed to force a target into range. The signed limits
487
- bound the tracking excursion; they do not clamp the final authored pose plus
488
- that excursion against a neutral-reference joint range. Authored motion can
489
- therefore take the combined pose beyond that range or the eyes' reach.
490
-
491
- `readControlState({worldTarget, headIntensity, eyeIntensity})` provides initial
492
- motor bearings without changing the rig. It inverts existing focus contributions
493
- against their saved base, and seeds zero added head excursion when enabling on
494
- a purely authored pose. When replacing legacy AU controls, also provide the
495
- evaluated local `headBaseQuaternion` and/or `eyeBaseQuaternions` (a map keyed by
496
- bone name), with those legacy controls excluded. `headSeedLimited` and
497
- `eyeSeedLimited` report poses that cannot be represented by the bounded motor
498
- bearings. Parallel legacy eyes cannot both be preserved exactly by a shared
499
- finite target; inverse eye contribution avoids double attenuation but does not
500
- remove that convergence residual. This helper supplies no handoff transition
501
- clock; the host owns activation policy and movement timing.
502
-
503
- The optional `profile.gazeCalibration` supplies bone-local
504
- `head`, `leftEye`, and `rightEye` optical axes and `modelUnitsPerMeter`.
505
- Without explicit optical axes, signed yaw/pitch bindings determine the optical
506
- frame. Unresolvable joints remain untouched and are reported in diagnostics.
507
-
508
- `solve_profile_viewer_space_gaze_scaled` adds `world_units_per_meter` as its
509
- last argument. Multiply the authored units-per-meter calibration by the
510
- presentation's uniform root scale. The original viewer solver keeps its
511
- existing scale-one behavior. Neither solver infers physical webcam/display
512
- placement from face landmarks.
513
-
514
- `solve_profile_tracking_gaze` maps normalized display-right/up controls to the
515
- profile's full signed head AU rotation range. Pass `head_follow_fraction = 1`
516
- for full participation: on an aligned display axis, an input endpoint of +/-1
517
- requests the same head rotation as the corresponding AU at intensity 1. Binding
518
- scale and asymmetric `maxDegrees` remain authoritative. Camera distance and FOV
519
- do not reduce this range. Intermediate inputs interpolate between the bounded
520
- camera bearing at zero and the corresponding signed endpoint; camera roll
521
- rotates the display axes into the model frame. With head tracking disabled,
522
- the target spans the eye AU range instead.
523
-
524
- The signature is `(profile_json, tracking_target, camera_position,
525
- camera_quaternion, gaze_origin, model_quaternion, eyes_enabled, head_enabled,
526
- head_follow_fraction, lock_head_to_camera, world_units_per_meter)`. It returns
527
- the same 14-float layout as the existing physical projection solvers. Target Z
528
- is depth in meters, clamped to 0.2–10 before scene scaling; it sets a finite
529
- binocular focus distance without attenuating the angular range. X/Y clamp to
530
- [-1, 1], nonfinite coordinates become zero, invalid positive depth/scale use
531
- 0.8/1 respectively, and absent AU bindings supply no angular capacity.
532
-
533
- This is an input-range mapping, not a physical reconstruction of the viewer's
534
- position. The existing screen/viewer projection APIs retain their contracts.
535
- Consumers must select this entry point and request full head participation to
536
- use it; adding the export alone does not change LoomLarge tracking. Eye focus
537
- still uses the resulting finite target and reports residual error when reduced
538
- strength, authored motion, or rig limits make that target unreachable.
539
-
540
658
  ## npm releases and PR previews
541
659
 
542
660
  Production consumers install an exact published `@lovelace_lol/embody` version
@@ -550,9 +668,11 @@ It chooses the next patch version, records the source commit in `gitHead`, and
550
668
  builds and validates once. Stable version selection reads public metadata from
551
669
  `registry.npmjs.org` anonymously; redirects and failed lookups stop the release.
552
670
  Metadata reads never transmit publishing credentials. The separate
553
- publish job first requires `npm access get status` to report the existing package
554
- as public, then uses the `npm` environment's `NPM_KEY` to publish the packed
555
- output with public access and creates its Git tag and GitHub release. Stable
671
+ publish job first requires the npm visibility endpoint (the endpoint used by
672
+ `npm access get status`) to report the existing package as public. It reads this
673
+ anonymously without npm token configuration, then uses GitHub OIDC trusted
674
+ publishing in the `npm` environment
675
+ to publish the packed output with public access and create its Git tag and GitHub release. Stable
556
676
  releases are serialized through version selection and publication. Provenance
557
677
  remains disabled for the private source repository. Retrying a published commit
558
678
  reuses its existing version, and an older unpublished commit cannot replace a
@@ -564,6 +684,16 @@ previews for testing linked upstream changes. An explicit ref remains a preview
564
684
  request even when it names `main`; it does not publish to npm. Manual and
565
685
  repository dispatch requests may include `upstream_pr` to identify their linked PR.
566
686
 
687
+ For an Embody → Polymer → LoomLarge feature stack, open every draft from its own
688
+ current `main` without waiting for publication. The Polymer PR body declares
689
+ `Depends-on: https://github.com/meekmachine/embody/pull/<number>` and the LoomLarge
690
+ PR body declares `Depends-on: https://github.com/meekmachine/Polymer/pull/<number>`.
691
+ CI resolves the compiled previews in temporary test checkouts. Committed
692
+ manifests and lockfiles retain exact stable npm versions; the release coordinator
693
+ adopts a stable release containing the change after upstream merge and publication.
694
+ Agents do not add preview URLs before opening downstream drafts.
695
+ Documentation-only companion PRs use ordinary reciprocal links instead.
696
+
567
697
  After publication, the producing workflow uploads a JSON notification record
568
698
  without a dispatch credential. The separate `Notify package publication`
569
699
  `workflow_run` receiver checks out `main`, treats the artifact only as data, and
@@ -587,7 +717,8 @@ release-stack PR's first preview remains usable by its immutable URL before
587
717
  that bootstrap merge; subsequent successful publications notify automatically.
588
718
 
589
719
  Before enabling stable releases, the existing npm package must have public
590
- visibility and `NPM_KEY` must permit publishing it. No npm read token is required.
720
+ visibility and its npm trusted publisher must match the configuration below.
721
+ No npm publishing secret or read token is required.
591
722
  Configure both the `npm` and `release-coordination` GitHub
592
723
  environments to allow deployments from `main` only. Store `CROSS_REPO_CI_TOKEN`
593
724
  only in `release-coordination`, with permission to dispatch to LoomLarge. These
@@ -595,6 +726,52 @@ account, environment protection, and secret settings are prerequisites; source
595
726
  changes do not configure them. Existing versions are never republished or
596
727
  overwritten.
597
728
 
729
+ ### Configure and recover stable publication
730
+
731
+ The npm package settings must register this GitHub Actions trusted publisher:
732
+
733
+ | Setting | Value |
734
+ | --- | --- |
735
+ | Package | `@lovelace_lol/embody` |
736
+ | Organization or user | `meekmachine` |
737
+ | Repository | `embody` |
738
+ | Workflow filename | `pr-checks.yml` |
739
+ | Environment | `npm` |
740
+ | Publishing permission | **Allow npm publish** |
741
+
742
+ New npm trusted publishers default to staging only; explicitly allow publishing.
743
+ The package's `repository.url` must match `git+https://github.com/meekmachine/embody.git`.
744
+ These settings live on npm and cannot be configured by merging workflow YAML.
745
+ Retain account 2FA and GitHub environment protections. The job uses Node 24 and
746
+ requires npm >=11.5.1 (with Node >=22.14.0), publishes without `NPM_KEY` or
747
+ `NODE_AUTH_TOKEN`, and never grants an OIDC token to PR or preview jobs.
748
+ Provenance remains disabled for this private source repository.
749
+
750
+ A green `pkg.pr.new` preview confirms a preview, not stable publication. The
751
+ main-only publish job writes a **Stable npm publication confirmed** summary only
752
+ after an anonymous exact-version registry read confirms the version and `gitHead`, and
753
+ records the release's SHA-512 integrity. New uploads must match the verified
754
+ tarball's integrity. Retries confirm an existing immutable release without
755
+ assuming that rebuilding the same source reproduces every archive byte. A failed publish or conflicting registry result
756
+ fails the job and blocks downstream notification. npm authentication failures show
757
+ the error code and exact publisher settings to repair, without echoing raw npm
758
+ output or tokens. The publisher checks the exact-version endpoint before any
759
+ upload, with `Cache-Control: no-cache`, so a stale package index cannot cause a
760
+ duplicate upload. Only HTTP 404 means absent; other lookup failures stop publication.
761
+ A new upload gets one exact-version confirmation lookup, without polling. A transient registry failure can leave an upload present
762
+ but unconfirmed; rerun the main release after investigation rather than upload a
763
+ replacement version blindly.
764
+
765
+ After repairing configuration, run this workflow on current `main` with its `ref`
766
+ input empty. A successful current-main stable release contains all source changes
767
+ already merged into that repository; individual failed intermediate releases do
768
+ not need replaying. Explicit refs and preview dispatches publish previews only.
769
+ An existing matching release is confirmed without publishing it again; source or
770
+ dependency conflicts and new-upload archive conflicts stop instead of overwriting it. The receipt
771
+ establishes npm publication, while the LoomLarge coordinator separately opens or
772
+ updates reviewed stable dependency PRs and lockfiles. It does not establish that
773
+ Polymer or the application adopted the release or deployed it.
774
+
598
775
  ## Development
599
776
 
600
777
  ```bash
@@ -643,6 +820,31 @@ Reset an authored control by setting its mapped AU values to zero. Hosts select
643
820
  which mapped AU IDs to reset; unrelated face state and saved mix strengths remain
644
821
  under their existing owners.
645
822
 
823
+ ### Reading current AU state
824
+
825
+ Rust records each admitted AU-curve clip's authored keys, resolved inherited
826
+ start, strength and easing. Those records change only at admission or a
827
+ strength/easing edit; nothing ticks them. Reads evaluate them on demand at the
828
+ native action's time and effective weight, so a fading slot reports
829
+ value × weight:
830
+
831
+ - `readAUContribution(name)` returns one slot's own `{auId: value}`, or `null`
832
+ when it is not playing or was not an AU-curve clip.
833
+ - `readAUState()` returns `{live, animated, total, unreadable}`: live control
834
+ values, active AU-curve clip contributions and their sum. Baked/native and
835
+ typed clips animate rig outputs directly and cannot be inverted to AU values,
836
+ so they are listed in `unreadable` rather than estimated.
837
+ - `measureAUGoal(name, destinations)` returns `{distance, reversal, current}`:
838
+ the largest travel from the slot's current contribution to `{auId: value}`
839
+ destinations, measured along profile `continuumPairs` as `positive − negative`
840
+ (crossing neutral from 0.3 negative to 0.3 positive is 0.6), and whether it
841
+ reverses the slot's unfinished travel. Callers apply their own timing policy
842
+ and send an ordinary inherited clip with the duration they choose.
843
+
844
+ Each call does a small amount of work in Rust; read when deciding, not every
845
+ frame. See [inherited clip starts](docs/INHERITED_CLIP_STARTS.md) for how
846
+ inherited starts and weights resolve.
847
+
646
848
  Named semantic pose/animation APIs and profile `bodyControls` are retired.
647
849
  Persisted named records must be decoded at the storage boundary to numeric AU
648
850
  values/curves, using their original profile mappings. `LEGACY_BODY_CONTROL_AUS`
@@ -650,6 +852,62 @@ is a deprecated reader-only historical identity snapshot for old CC4 records;
650
852
  it is never consulted for current controls, actuation or profile compilation.
651
853
  Unknown legacy names must be reported rather than silently omitted.
652
854
 
855
+ ### Mesh highlight previews
856
+
857
+ Rust owns transient mesh highlight state and its renderer-neutral output in
858
+ `src/highlighting.rs`, exposed through `RuntimeCore`. This preview is independent
859
+ of AU/morph/bone actuation, clip playback, and persisted profile data. Rust resolves
860
+ mesh names (including duplicate names) against the configured model, chooses the
861
+ default gold color and intensity, computes darker edges in linear color space,
862
+ and advances a 400 ms eased transition. Interrupted fades resume from the current
863
+ strength; changing targets removes the old preview immediately. Clearing or an
864
+ unknown name fades out the current selection. Repeated identical requests do not
865
+ restart a fade. Model reconfiguration clears transient highlights.
866
+
867
+ | Core operation | Contract |
868
+ | --- | --- |
869
+ | `request_mesh_highlight(name?, color?, intensity?)` | Request a mesh name or omit it to clear. Optional color is packed 24-bit sRGB; intensity is clamped to 0–1, with nonfinite values using the default. |
870
+ | `evaluate_mesh_highlight_frame()` | Return the current complete JSON snapshot without advancing time. |
871
+ | `advance_mesh_highlight(delta_seconds)` | Advance preview time using the host render clock and return the snapshot. Negative/nonfinite deltas do not advance time. |
872
+
873
+ Snapshot version 1 contains `highlights`, an array of writes with `meshId`,
874
+ `fillColor`, `fillIntensity`, `edgeColor`, `edgeMode: "triangles"`, and `edgeOpacity`.
875
+ Colors are linear RGB triplets, strength/opacity are in 0–1, and IDs refer to the
876
+ configured model descriptor. The fill is an emissive surface preview and the
877
+ edges show triangle topology. An omitted mesh ID means restore its original
878
+ appearance and release its preview resources. `{ "version": 1, "highlights": [] }`
879
+ clears every preview. The TypeScript contract is exported from
880
+ `@lovelace_lol/embody/wasm` without importing a renderer.
881
+
882
+ Another renderer configures the same `RuntimeCore` from its own model descriptor,
883
+ keeps a map of mesh IDs to native surfaces, sends requests, and applies snapshots
884
+ from its existing render tick. It does not traverse by name, derive edge colors,
885
+ or reproduce the easing curve. For example, after configuring `core`:
886
+
887
+ ```ts
888
+ core.request_mesh_highlight('Face');
889
+ // Apply the initial snapshot immediately to remove obsolete selections.
890
+ applyNativePreview(JSON.parse(core.evaluate_mesh_highlight_frame()));
891
+ // Inside the host's existing render tick (seconds, independent of clip speed):
892
+ applyNativePreview(JSON.parse(core.advance_mesh_highlight(deltaSeconds)));
893
+ ```
894
+
895
+ `applyNativePreview` is the renderer's effect boundary: resolve `meshId`, apply
896
+ emission and triangle edges from the supplied values, and remove omitted previews.
897
+ The host must advance the core once per frame, even with no playing clips. Pausing
898
+ its render clock pauses preview progression; no additional animation loop exists.
899
+
900
+ For Three, `character.highlightMesh(name, color?, intensity?)` sends the core
901
+ request; the existing scene playback binding supplies elapsed seconds and calls
902
+ `ThreeFrameApplier.applyMeshHighlightFrame`. The applier only binds mesh IDs and
903
+ manages native materials/overlays, deformation, picking exclusion and disposal.
904
+ It does not choose colors, resolve names, or sample fades. Temporary materials
905
+ are isolated from shared source materials; geometry and skeletons remain owned
906
+ by the source model. Native overlays follow source geometry, morph weights and
907
+ skeletons. Character rebind/disposal removes them. Direct applier consumers must
908
+ call `clearMeshHighlights()` on disposal. Other renderers implement their own
909
+ snapshot application; this change does not supply a second native renderer.
910
+
653
911
  ### Character-derived studio background
654
912
 
655
913
  The Three `createCharacterHost` paints its container with soft gradients sampled