@convai/embody 1.0.0-beta.2 → 1.0.1-beta.0

package/CHANGELOG.md

@@ -1,5 +1,45 @@
 # @convai/embody
 
+## 1.0.1-beta.0
+
+### Patch Changes
+
+- Fix: `ConvaiCharacter` and `CharacterModel` call `onReady` (and `onError`) once per character. An inline `onReady={(c) => …}` used to run again on every parent re-render, repeating whatever setup the app did there, such as adding a `speakingChange` listener.
+
+  Docs: two new guides, "Add your own animations to a character" (the `animations` option, `idleClip`, `crossfadeTo`, a talking clip on `speakingChange`, `register` at runtime) and "Use your own 3D model" (a resolver for a MetaHuman GLB you host, the rig requirements, Full and Lite files, `connect: false` previews). The `createCharacter` reference gains `modelVariant`, `character.model`, and sections for `AnimationController` and the `CharacterBundle` manifest; the React reference says which props reload the character. The README's Lite count no longer says 113.
+
+## 1.0.0
+
+### Major Changes
+
+- 078036f: **1.0.0: embody is the 3D character only, and it works with `@convai/web-sdk` 2.x.** This is the first `latest` release since 0.4.2. It carries everything from 0.5.0 and 0.6.0, which were staging-only builds.
+
+  Migration from 0.4.2:
+
+  | Was (0.4.2)                                                          | Now (1.0.0)                                                                                                                                                                                      |
+  | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+  | `import { ConvaiWidget } from '@convai/embody/react'`                | `import { ConvaiWidget } from '@convai/web-widget/react'` (install `@convai/web-widget`), fed by `useWidgetClient(character.client)` so the chat and the body share one connection               |
+  | `import { AudioRenderer, AudioContext } from '@convai/embody/react'` | Gone. `ConvaiCharacter` and `createCharacter` play the character's audio themselves. For custom playback, `@convai/web-sdk/vanilla` has a headless `AudioRenderer` class (not a React component) |
+  | `@convai/web-sdk` peer `>=1.8.0-beta.4`                              | `>=2.0.1 <3`: web-sdk 2.x. 1.9.x is no longer maintained; upgrade the SDK with embody                                                                                                            |
+  | `react` / `@react-three/fiber` / `three` peers unbounded             | `react >=18 <20`, `@react-three/fiber >=8 <10`, `three >=0.160 <1`                                                                                                                               |
+
+  Nothing else needs a code change: every other export and option is unchanged or new (corrective maps, Full/Lite bodies, default-body face fixes).
+
+### Minor Changes
+
+- 6527d85: Full and Lite bodies: when a body ships both GLBs, embody loads Lite (113 blendshapes) on phones, on tablets and on GPUs too small for Full's 251, and Full everywhere else. The new `modelVariant` option (`'auto'` | `'full'` | `'lite'`) forces a choice, and `character.model` reports which file loaded. `pickModelVariant` and `detectDevice` are exported for apps that build their own loader. Bodies with one GLB behave as before.
+
+### Patch Changes
+
+- 86a6659: Face fixes for the convai-mha-1 default bodies (Formal Women, Activity Convai):
+
+  - Lipsync: the bilabial seal is off by default (it held the lips half shut on most speaking frames of the MHA stream), lower-lip gain is the SDK's 0.45, teeth follow a rest offset plus a share of the jaw, frames are consumed against the wall clock so the lips cannot fall behind the voice, and a stalled stream relaxes the mouth instead of freezing it. `teethLowerTuck` is deprecated and read as `teethLowerRest`.
+  - Gaze: the eye ease no longer speeds up on high-refresh displays.
+  - Materials: export defects are repaired by pattern: self-lit skin, hair and clothes lose the emissive copy of their base colour (eyes keep a quarter), out-of-range roughness and specular are clamped, the head loses its glossy scalp-edge roughness map and its alpha mask, and `Hide` helpers are hidden. `ConvaiCharacter` adds a neutral room environment for the now physically shaded skin and eyes.
+
+- 0a8ce1b: Docs: the quick start and both guides now say that a character with Long-Term Memory needs `session.endUserId`; without it, connecting fails with "Missing end_user_id".
+- 298e121: `@convai/web-sdk` peer is now `>=2.0.1 <3`. web-sdk 1.9.x is no longer maintained, so embody 1.0 targets 2.x only.
+
 ## 1.0.0-beta.2
 
 ### Patch Changes

package/README.md

@@ -217,7 +217,8 @@ profile, so a new rig is a value rather than a code change.
 ### Full and Lite bodies
 
 A body can ship two GLBs of the same rig: **Full**, with all 251 MetaHuman
-blendshapes, and **Lite**, with 113. Phones need Lite. three.js spends one
+blendshapes, and **Lite**, with a reduced set (the art team's Lite exports
+carry 103; keep it at 192 or fewer). Phones need Lite. three.js spends one
 vertex-shader uniform per blendshape, many mobile GPUs allow 256 in all, and
 past that the head fails to compile and is not drawn.
 
@@ -229,6 +230,35 @@ gets Full. Pass `modelVariant: 'full'` or `'lite'` to force one;
 file everywhere. Blendshapes Lite drops are skipped by lipsync, blink and
 eye gaze rather than erroring.
 
+## Your own animations and models
+
+**Animations.** `animations` maps a clip name to a GLB holding an animation of
+that name, merged over the body's clips by name (the key must equal the
+animation's name inside the file). `character.animations` then plays them:
+
+```ts
+const character = await createCharacter({
+  token, characterId,
+  animations: { Wave: '/animations/gestures.glb' },
+})
+character.animations.crossfadeTo('Wave', { once: true, thenBackTo: 'Idle' })
+
+// embody moves the mouth while the character speaks, never the body clip:
+character.client?.on('speakingChange', (s) => character.animations.crossfadeTo(s ? 'Talk' : 'Idle'))
+```
+
+Clips must be authored on the MetaHuman skeleton; nothing is retargeted, and
+tracks for bones the model lacks are dropped with a console warning.
+
+**Models.** A `resolver` returning a `CharacterBundle` for files you host
+replaces the body while the conversation still comes from the character's
+Convai session. The model must be on `convai-mha-1`: MetaHuman bone names
+(`head`, `neck_01`, `neck_02`, `spine_02`, `spine_04`, …) and `CTRL_expressions_*`
+blendshapes. Other rigs load with a still head and face.
+
+Step-by-step guides: [Add your own animations](./docs/site/convai-embody/add-your-own-animations.md),
+[Use your own 3D model](./docs/site/convai-embody/use-your-own-3d-model.md).
+
 ## Escape hatches
 
 The defaults cover the normal case; each of these exists for a specific one.
@@ -272,6 +302,8 @@ Convai Embody*. Its source is [`docs/site/`](./docs/site) in this repository:
 | [Add a 3D character to a React app](./docs/site/convai-embody/add-a-3d-character-to-a-react-app.md) | install, render, verify, troubleshoot |
 | [Add a 3D character to a three.js scene](./docs/site/convai-embody/add-a-3d-character-to-a-three-js-scene.md) | the framework-free API |
 | [Add a chat widget to a character](./docs/site/convai-embody/add-a-chat-widget-to-a-character.md) | one connection for body and chat |
+| [Add your own animations to a character](./docs/site/convai-embody/add-your-own-animations.md) | load, crossfade, talking clip, runtime clips |
+| [Use your own 3D model](./docs/site/convai-embody/use-your-own-3d-model.md) | a MetaHuman GLB you host, Full and Lite, rig requirements |
 | [React API reference](./docs/site/convai-embody/react-api-reference.md) | every prop and default |
 | [createCharacter reference](./docs/site/convai-embody/createcharacter-reference.md) | every option, the handle, the errors |
 | [How Convai Embody works](./docs/site/convai-embody/how-convai-embody-works.md) | resolution, the frame pipeline, environments |

package/dist/react/CharacterModel.js

@@ -19,20 +19,27 @@ import { jsx as _jsx } from "react/jsx-runtime";
  * (which delegates cancellation to `../character/lifecycle`). This adds
  * nothing but the `<primitive>` render and the onReady/onError plumbing.
  */
-import { useEffect } from 'react';
+import { useEffect, useRef } from 'react';
 import { useCharacter } from './useCharacter.js';
 export function CharacterModel({ onReady, onError, ...opts }) {
     const { character, error } = useCharacter(opts);
+    // The callbacks are read through refs so that announcing is keyed on the
+    // character (or error) alone. Apps pass them inline, `onReady={(c) => …}`,
+    // which is a new function on every render; with the callback as an effect
+    // dependency, each parent re-render re-announced the same character and
+    // re-ran whatever setup the app did there.
+    const onReadyRef = useRef(onReady);
+    const onErrorRef = useRef(onError);
+    onReadyRef.current = onReady;
+    onErrorRef.current = onError;
     useEffect(() => {
-        if (character && onReady)
-            onReady(character);
-        // eslint-disable-next-line react-hooks/exhaustive-deps
-    }, [character, onReady]);
+        if (character)
+            onReadyRef.current?.(character);
+    }, [character]);
     useEffect(() => {
-        if (error && onError)
-            onError(error);
-        // eslint-disable-next-line react-hooks/exhaustive-deps
-    }, [error, onError]);
+        if (error)
+            onErrorRef.current?.(error);
+    }, [error]);
     if (!character)
         return null;
     return _jsx("primitive", { object: character.root });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@convai/embody",
-  "version": "1.0.0-beta.2",
+  "version": "1.0.1-beta.0",
   "description": "Give a Convai character a body: resolve 3D characters by characterId and drive them with lipsync and animation. Layered on @convai/web-sdk.",
   "license": "Apache-2.0",
   "author": "Convai <support@convai.com>",