@convai/embody 0.6.0 → 1.0.0-beta.1

package/CHANGELOG.md

@@ -1,5 +1,42 @@
 # @convai/embody
 
+## 1.0.0-beta.1
+
+### Patch Changes
+
+- `@convai/web-sdk` peer is now `>=2.0.1 <3`. web-sdk 1.9.x is no longer maintained, so embody 1.0 targets 2.x only.
+
+## 1.0.0-beta.0
+
+### Major Changes
+
+- **1.0.0: embody is the 3D character only, and it works with `@convai/web-sdk` 2.x.** This is the first `latest` release since 0.4.2. It carries everything from 0.5.0 and 0.6.0, which were staging-only builds.
+
+  Migration from 0.4.2:
+
+  | Was (0.4.2)                                                          | Now (1.0.0)                                                                                                                                                                                      |
+  | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+  | `import { ConvaiWidget } from '@convai/embody/react'`                | `import { ConvaiWidget } from '@convai/web-widget/react'` (install `@convai/web-widget`), fed by `useWidgetClient(character.client)` so the chat and the body share one connection               |
+  | `import { AudioRenderer, AudioContext } from '@convai/embody/react'` | Gone. `ConvaiCharacter` and `createCharacter` play the character's audio themselves. For custom playback, `@convai/web-sdk/vanilla` has a headless `AudioRenderer` class (not a React component) |
+  | `@convai/web-sdk` peer `>=1.8.0-beta.4`                              | `>=1.9.4 <3`: 1.9.4 or any 2.x                                                                                                                                                                   |
+  | `react` / `@react-three/fiber` / `three` peers unbounded             | `react >=18 <20`, `@react-three/fiber >=8 <10`, `three >=0.160 <1`                                                                                                                               |
+
+  Nothing else needs a code change: every other export and option is unchanged or new (corrective maps, Full/Lite bodies, default-body face fixes).
+
+### Minor Changes
+
+- 6527d85: Full and Lite bodies: when a body ships both GLBs, embody loads Lite (113 blendshapes) on phones, on tablets and on GPUs too small for Full's 251, and Full everywhere else. The new `modelVariant` option (`'auto'` | `'full'` | `'lite'`) forces a choice, and `character.model` reports which file loaded. `pickModelVariant` and `detectDevice` are exported for apps that build their own loader. Bodies with one GLB behave as before.
+
+### Patch Changes
+
+- 86a6659: Face fixes for the convai-mha-1 default bodies (Formal Women, Activity Convai):
+
+  - Lipsync: the bilabial seal is off by default (it held the lips half shut on most speaking frames of the MHA stream), lower-lip gain is the SDK's 0.45, teeth follow a rest offset plus a share of the jaw, frames are consumed against the wall clock so the lips cannot fall behind the voice, and a stalled stream relaxes the mouth instead of freezing it. `teethLowerTuck` is deprecated and read as `teethLowerRest`.
+  - Gaze: the eye ease no longer speeds up on high-refresh displays.
+  - Materials: export defects are repaired by pattern: self-lit skin, hair and clothes lose the emissive copy of their base colour (eyes keep a quarter), out-of-range roughness and specular are clamped, the head loses its glossy scalp-edge roughness map and its alpha mask, and `Hide` helpers are hidden. `ConvaiCharacter` adds a neutral room environment for the now physically shaded skin and eyes.
+
+> 0.5.0 and 0.6.0 were published only to the `stg` npm tag, as staging-backend builds. Their changes reach `latest` in 1.0.0.
+
 ## 0.6.0
 
 ### Minor Changes
@@ -20,6 +57,32 @@
 
 - 249a96f: Optional corrective map for one-morph-per-control bodies. A table fitted from the body's MetaHuman DNA re-weights each lipsync frame toward the full rig (PSD correctives, joint arcs). Pass `correctiveMap` (from `compileCorrectiveMap`) to `createCharacter` or the React components; it is off by default and swaps live through the prop or `character.lipsync.correctiveMap`.
 
+## 0.4.2
+
+### Patch Changes
+
+- 532c49e: Swapping the body under a fixed character id now reaches the screen.
+
+  `useCharacter` rebuilt the character only when the character id, credentials
+  or base url changed. The **resolver** was not among them, and the effect read
+  its options from a ref — so an authoring tool that holds one avatar id and
+  swaps the block underneath (preview release 3, then release 4) got nothing:
+  the old GLB stayed on the canvas and it read as a stubborn cache. Nothing was
+  cached. The loader was never asked for the new url.
+
+  The resolver decides WHICH body a character id resolves to, so it belongs in
+  the rebuild set, and now is.
+
+  This makes the resolver's identity part of the contract: **memoize it**, and
+  change it exactly when the body should change. A resolver rebuilt inline on
+  every render would rebuild the character on every render. The README's new
+  "Swapping the body under a fixed character id" section shows the shape.
+
+  Also documented, unchanged in behaviour: the `orbit` prop (shipped in 0.4.0
+  and previously absent from the README), which backend each npm tag talks to,
+  and the correct release-branch names — the table named three branches that do
+  not exist.
+
 ## 0.4.1
 
 ### Patch Changes

package/README.md

@@ -2,7 +2,7 @@
 
 **Give a Convai character a body.**
 
-[![npm](https://img.shields.io/npm/v/@convai/embody/stg?label=npm%20stg&color=blue)](https://www.npmjs.com/package/@convai/embody)
+[![npm](https://img.shields.io/npm/v/@convai/embody/latest?label=npm&color=blue)](https://www.npmjs.com/package/@convai/embody)
 [![TypeScript](https://img.shields.io/badge/TypeScript-first-3178C6)](https://www.typescriptlang.org/)
 [![License](https://img.shields.io/badge/license-Apache--2.0-green)](./LICENSE)
 
@@ -12,13 +12,21 @@ lipsync, gaze, breathing and blink. Layered on
 
 ---
 
-> **Branch: `stg`** — this branch defaults to the staging backend, which has
-> its own character database. Use `main` or `prod` for production.
+> **Branch: `main`** — this branch defaults to the production backend and
+> publishes the `latest` tag. `stg` targets staging, which has its own
+> character database.
 >
 > **Status: early release.** The front door — a credential plus a
 > `characterId` — is settled; what is still moving is listed in
 > [Roadmap](#roadmap).
 
+> **Which backend a build talks to is fixed at publish time.** The `latest`
+> and `beta` tags point at `https://api.convai.com`; the `stg` tag points at
+> `https://api-stg.convai.com`. Character ids are per-environment and not
+> interchangeable — a staging id against production returns *"Character not
+> found or doesn't belong to user"*, which reads like a permissions problem
+> and is a wrong-host one. Override with `baseUrl` only if you know why.
+
 ## Install
 
 ```bash
@@ -63,6 +71,32 @@ For anything user-facing, mint a short-lived auth token server-side and pass
 <ConvaiCharacter token={token} characterId={characterId} />
 ```
 
+### Moving the camera
+
+Drag to turn the character, scroll to move in and out. On by default — a 3D
+character you cannot walk around is a picture of one.
+
+```tsx
+<ConvaiCharacter apiKey={apiKey} characterId={characterId} orbit={false} />
+```
+
+Pass an object to tune the limits instead of turning it off:
+
+```tsx
+<ConvaiCharacter
+  apiKey={apiKey}
+  characterId={characterId}
+  orbit={{ maxDistance: 6, autoRotate: true, autoRotateSpeed: 0.8 }}
+/>
+```
+
+`minDistance`, `maxDistance`, `minPolarAngle`, `maxPolarAngle`, `enablePan`,
+`enableZoom`, `autoRotate`, `autoRotateSpeed`. The defaults are tuned for one
+subject — a standing human — not for a general scene viewer: panning is off
+because it only ever slides the character out of frame, and the vertical sweep
+stops just below eye level because orbiting under the floor shows the inside
+of a head.
+
 ### In a scene you already have
 
 `ConvaiCharacter` brings its own canvas. When the character should join a
@@ -85,12 +119,11 @@ a second one. Lift it out with `onReady`; chat UI is DOM, so it lives outside
 the canvas:
 
 ```tsx
-import { ConvaiCharacter, useWidgetClient, type ConvaiClient } from '@convai/embody/react'
+import { ConvaiCharacter, type ConvaiClient } from '@convai/embody/react'
 import { ConvaiWidget } from '@convai/web-widget/react'
 
 function App() {
   const [client, setClient] = useState<ConvaiClient | null>(null)
-  const widgetClient = useWidgetClient(client)
   return (
     <>
       <ConvaiCharacter
@@ -98,7 +131,7 @@ function App() {
         characterId={characterId}
         onReady={(c) => setClient(c.client)}
       />
-      {widgetClient && <ConvaiWidget convaiClient={widgetClient} />}
+      {client && <ConvaiWidget convaiClient={client} />}
     </>
   )
 }
@@ -106,10 +139,15 @@ function App() {
 
 embody is the 3D character only; the chat panel is
 [`@convai/web-widget`](https://www.npmjs.com/package/@convai/web-widget)
-(`npm install @convai/web-widget`). `useWidgetClient` adapts
-embody's client to the widget's props — the SDK's own `useConvaiClient` would
-*construct a second client*, and so open a second connection, which is exactly
-what embody exists to avoid.
+(`npm install @convai/web-widget`, 0.3.0 or later). Hand it the character's own
+client — the widget subscribes to it, so chat and lip-sync share one
+connection. Do not use the SDK's `useConvaiClient` for this: it *constructs a
+second client*, and so opens a second connection, which is exactly what embody
+exists to avoid.
+
+`useWidgetClient` is still exported for `@convai/web-widget` before 0.3.0,
+whose `ConvaiWidget` accepted only `useConvaiClient`'s result; on 0.3.0 and
+later it is unnecessary.
 
 ## Without React
 
@@ -166,6 +204,21 @@ load-bearing, not incidental:
 ships but is deliberately not registered. Bone and morph names are data on the
 profile, so a new rig is a value rather than a code change.
 
+### Full and Lite bodies
+
+A body can ship two GLBs of the same rig: **Full**, with all 251 MetaHuman
+blendshapes, and **Lite**, with 113. Phones need Lite. three.js spends one
+vertex-shader uniform per blendshape, many mobile GPUs allow 256 in all, and
+past that the head fails to compile and is not drawn.
+
+embody picks the file per device (`modelVariant: 'auto'`, the default):
+phones and tablets get Lite, and so does any GPU whose
+`MAX_VERTEX_UNIFORM_VECTORS` cannot hold Full's blendshapes. Everything else
+gets Full. Pass `modelVariant: 'full'` or `'lite'` to force one;
+`character.model` reports which file loaded. A body with one GLB loads that
+file everywhere. Blendshapes Lite drops are skipped by lipsync, blink and
+eye gaze rather than erroring.
+
 ## Escape hatches
 
 The defaults cover the normal case; each of these exists for a specific one.
@@ -177,9 +230,28 @@ The defaults cover the normal case; each of these exists for a specific one.
 | `connect: false` | A body with no mind — thumbnails, previews, asset review tools. `character.client` is then `null`. |
 | `session` | Extra SDK options (voice, memory, transport, `endUserId`). Credentials and `characterId` are not repeated here. |
 | `baseUrl` | A different backend environment. Defaults per release channel. |
+| `modelVariant` | Force the Full or Lite GLB (`'full'` / `'lite'`) instead of choosing per device, e.g. to preview Lite on a desktop. |
 | `queue` on `update()` | One client driving several bodies, when you are deciding which owns the current turn. |
 | `correctiveMap` | A table fitted from the body's MetaHuman DNA (`compileCorrectiveMap(json)`) that re-weights each lipsync frame toward what the full rig shows: PSD correctives and joint arcs that one-morph-per-control bodies lack. Off by default. Changing the prop, or `character.lipsync.correctiveMap`, swaps it live. In `pnpm dev`, the demo lists `*.map.json` tables from `../open-rig-dna/maps` (or `$EMBODY_CORRECTIVE_MAPS`) for A/B: **C** toggles, **F** frames the face. |
 
+### Swapping the body under a fixed character id
+
+`resolver` is how an authoring tool previews a body that is not published — or
+not even saved. Its **identity is part of the rebuild contract**: memoize it,
+and change it exactly when the body should change.
+
+```tsx
+const resolver = useMemo(
+  () => ({ async resolve() { return bundle } }),
+  [bundle],
+)
+
+<ConvaiCharacter characterId={avatarId} resolver={resolver} connect={false} />
+```
+
+Changing the resolver rebuilds the character; keeping it stable does not. A
+resolver rebuilt inline on every render would reload the GLB on every render.
+
 ## Documentation
 
 **User documentation** is published on docs.convai.com under *Web Plugins →

package/dist/character/createCharacter.d.ts

@@ -3,7 +3,8 @@
  * framework-free handle the app ticks once per frame.
  */
 import * as THREE from 'three';
-import { type CharacterResolver, type CharacterBundle, type Representation } from '../resolve/resolver.js';
+import { type CharacterResolver, type CharacterBundle, type Representation, type ModelVariant } from '../resolve/resolver.js';
+import { type ModelVariantPreference } from '../resolve/modelVariant.js';
 import { type SessionOptions } from '../connect/session.js';
 import type { ConvaiClient } from '@convai/web-sdk/core';
 import { type UnitNormalizer } from '../rig/units.js';
@@ -85,6 +86,12 @@ export interface CreateCharacterOptions {
     dracoDecoderPath?: string;
     /** Rig profiles this build understands, in preference order. */
     supportedRigs?: string[];
+    /**
+     * Which model file to load when the body ships more than one. `auto`
+     * (default) loads Lite on phones and tablets and on any GPU too small for
+     * Full's blendshapes; `full` and `lite` force one. See `pickModelVariant`.
+     */
+    modelVariant?: ModelVariantPreference;
     targetHeight?: number;
     groundY?: number;
     mouth?: MouthTuning;
@@ -134,6 +141,8 @@ export interface CharacterHandle {
     root: THREE.Object3D;
     bundle: CharacterBundle;
     representation: Representation;
+    /** The model file that was loaded — `representation.model` lists every one on offer. */
+    model: ModelVariant;
     animations: AnimationController;
     lipsync: LipsyncSystem;
     rig: ResolvedRig;

package/dist/character/createCharacter.js

@@ -1,6 +1,7 @@
 import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';
 import { DRACOLoader } from 'three/examples/jsm/loaders/DRACOLoader.js';
 import { pickRepresentation, httpResolver, MISSING_CREDENTIAL } from '../resolve/resolver.js';
+import { pickModelVariant } from '../resolve/modelVariant.js';
 import { createSession, adoptSession } from '../connect/session.js';
 import { createUnitNormalizer } from '../rig/units.js';
 import { AnimationController } from '../clips/animations.js';
@@ -174,7 +175,8 @@ export async function createCharacter(opts) {
     const draco = new DRACOLoader();
     draco.setDecoderPath(opts.dracoDecoderPath ?? DEFAULT_DRACO_DECODER_PATH);
     loader.setDRACOLoader(draco);
-    const modelUrl = rep.model[0].url;
+    const model = pickModelVariant(rep.model, opts.modelVariant);
+    const modelUrl = model.url;
     // clip sources: manifest clips, with local overrides winning by name
     const clipUrls = new Map();
     for (const [name, c] of Object.entries(rep.clips ?? {}))
@@ -254,7 +256,7 @@ export async function createCharacter(opts) {
         throw sessionError;
     }
     const handle = {
-        root, bundle, representation: rep, animations, lipsync, rig, pipeline, gaze,
+        root, bundle, representation: rep, model, animations, lipsync, rig, pipeline, gaze,
         client: session?.client ?? null,
         camera: undefined,
         update(delta, queue) {

package/dist/index.d.ts

@@ -22,6 +22,8 @@ export type { SessionOptions, SessionHandle } from './connect/session.js';
 export { httpResolver, pickRepresentation, DEFAULT_BASE_URL } from './resolve/resolver.js';
 export type { HttpResolverOptions } from './resolve/resolver.js';
 export type { CharacterResolver, CharacterBundle, Representation, ModelVariant } from './resolve/resolver.js';
+export { pickModelVariant, detectDevice, MORPH_UNIFORM_RESERVE } from './resolve/modelVariant.js';
+export type { ModelVariantPreference, DeviceProfile } from './resolve/modelVariant.js';
 export { createCharacter } from './character/createCharacter.js';
 export type { CharacterHandle, CreateCharacterOptions } from './character/createCharacter.js';
 export { AnimationController } from './clips/animations.js';

package/dist/index.js

@@ -22,6 +22,7 @@
 export { ConvaiClient, BlendshapeQueue, AudioManager, VideoManager, ScreenShareManager, MemoryManager, getDisconnectReasonMessage, } from '@convai/web-sdk/core';
 export { createSession, adoptSession } from './connect/session.js';
 export { httpResolver, pickRepresentation, DEFAULT_BASE_URL } from './resolve/resolver.js';
+export { pickModelVariant, detectDevice, MORPH_UNIFORM_RESERVE } from './resolve/modelVariant.js';
 export { createCharacter } from './character/createCharacter.js';
 export { AnimationController } from './clips/animations.js';
 export { LipsyncSystem } from './speech/lipsync.js';

package/dist/motion/eyeGaze.d.ts

@@ -37,12 +37,12 @@ export interface EyeGazeConfig {
  * (useEyeTracking.ts:538-562) — the bone-rotation path is CC4-only and out
  * of scope here.
  *
- * The channel names are cross-named: driving `eyeLookLeft*` rotates the eyes
- * toward the character's RIGHT (useEyeTracking.ts:156-159, verified live on
- * the UE rig by driving each morph in isolation). `CONVAI_MHA_1` encodes
- * that as in/out morph roles (`eyeLookInL`/`eyeLookOutL`/`eyeLookInR`/
- * `eyeLookOutR`); this system only ever speaks in in/out terms and never
- * references a literal left/right channel name.
+ * Driving `eyeLookLeft*` turns the eyes to the character's own LEFT (each
+ * morph driven alone, in Chrome, on three convai-mha-1 heads, 2026-10-14).
+ * `CONVAI_MHA_1` maps its in/out morph roles (`eyeLookInL`/`eyeLookOutL`/
+ * `eyeLookInR`/`eyeLookOutR`) in the frame of the rig basis rather than the
+ * anatomy; see the profile. This system only ever speaks in in/out terms and
+ * never references a literal left/right channel name.
  *
  * DEFERRED (RULING R48 — logged for final review, deliberately NOT built
  * here):
@@ -61,5 +61,5 @@ export declare class EyeGazeSystem implements System {
     private h;
     private v;
     constructor(gaze: GazeSource, cfg?: EyeGazeConfig);
-    update(_dt: number, ctx: FrameContext): void;
+    update(dt: number, ctx: FrameContext): void;
 }

package/dist/motion/eyeGaze.js

@@ -7,12 +7,12 @@ import { writeMorph } from '../rig/resolve.js';
  * (useEyeTracking.ts:538-562) — the bone-rotation path is CC4-only and out
  * of scope here.
  *
- * The channel names are cross-named: driving `eyeLookLeft*` rotates the eyes
- * toward the character's RIGHT (useEyeTracking.ts:156-159, verified live on
- * the UE rig by driving each morph in isolation). `CONVAI_MHA_1` encodes
- * that as in/out morph roles (`eyeLookInL`/`eyeLookOutL`/`eyeLookInR`/
- * `eyeLookOutR`); this system only ever speaks in in/out terms and never
- * references a literal left/right channel name.
+ * Driving `eyeLookLeft*` turns the eyes to the character's own LEFT (each
+ * morph driven alone, in Chrome, on three convai-mha-1 heads, 2026-10-14).
+ * `CONVAI_MHA_1` maps its in/out morph roles (`eyeLookInL`/`eyeLookOutL`/
+ * `eyeLookInR`/`eyeLookOutR`) in the frame of the rig basis rather than the
+ * anatomy; see the profile. This system only ever speaks in in/out terms and
+ * never references a literal left/right channel name.
  *
  * DEFERRED (RULING R48 — logged for final review, deliberately NOT built
  * here):
@@ -32,7 +32,7 @@ export class EyeGazeSystem {
         this.h = 0;
         this.v = 0;
     }
-    update(_dt, ctx) {
+    update(dt, ctx) {
         const lerpSpeed = this.cfg.lerpSpeed ?? 0.2;
         const verticalOffset = this.cfg.verticalOffset ?? -0.15;
         const gain = this.cfg.morphGain ?? 0.65;
@@ -47,12 +47,17 @@ export class EyeGazeSystem {
         const targetV = ctx.reducedMotion
             ? 0
             : THREE.MathUtils.clamp(d.vertical + verticalOffset, -1, 1);
-        this.h += (targetH - this.h) * lerpSpeed;
-        this.v += (targetV - this.v) * lerpSpeed;
+        // lerpSpeed is a per-frame rate tuned at 60 fps. Applied per frame as is,
+        // the eyes moved twice as fast on a 120 Hz display; this is the same
+        // ease, made independent of the frame rate.
+        const k = 1 - Math.pow(1 - lerpSpeed, Math.max(0, dt) * 60);
+        this.h += (targetH - this.h) * k;
+        this.v += (targetV - this.v) * k;
         const h = this.h * gain;
         const v = this.v * gain;
-        // Positive h = looking to the character's RIGHT. That is "in" (nose-ward)
-        // for the LEFT eye and "out" for the RIGHT eye.
+        // Positive h follows the rig basis' `right` axis. On convai-mha-1 that
+        // axis is the character's LEFT, and the profile's in/out roles are mapped
+        // to match, so the eyes turn toward the target; see the profile.
         writeMorph(ctx.rig, 'eyeLookInL', Math.max(0, h));
         writeMorph(ctx.rig, 'eyeLookOutL', Math.max(0, -h));
         writeMorph(ctx.rig, 'eyeLookInR', Math.max(0, -h));

package/dist/react/ConvaiCharacter.js

@@ -25,6 +25,7 @@ import { Suspense } from 'react';
 import { Canvas } from '@react-three/fiber';
 import { CharacterModel } from './CharacterModel.js';
 import { OrbitRig } from './OrbitRig.js';
+import { RoomLight } from './RoomLight.js';
 const DEFAULT_POSITION = [0, 1.45, 3.6];
 const DEFAULT_TARGET = [0, 0.92, 0];
 export function ConvaiCharacter({ style, className, background = '#17181c', camera, orbit = true, fallback = null, ...character }) {
@@ -38,6 +39,6 @@ export function ConvaiCharacter({ style, className, background = '#17181c', came
                 // r3f points the camera at [0,0,0] otherwise. Harmless when orbit is
                 // on — OrbitRig takes the same target and drives the camera from
                 // there — and load-bearing when it is off.
-                onCreated: ({ camera: cam }) => cam.lookAt(...target), children: [background !== null && _jsx("color", { attach: "background", args: [background] }), _jsx("hemisphereLight", { args: [0xffffff, 0x334455, 1.4] }), _jsx("directionalLight", { position: [2, 4, 3], intensity: 2.2 }), orbit !== false && (_jsx(OrbitRig, { target: target, options: orbit === true ? undefined : orbit })), _jsx(Suspense, { fallback: null, children: _jsx(CharacterModel, { ...character }) })] }), fallback] }));
+                onCreated: ({ camera: cam }) => cam.lookAt(...target), children: [background !== null && _jsx("color", { attach: "background", args: [background] }), _jsx("hemisphereLight", { args: [0xffffff, 0x334455, 1.4] }), _jsx("directionalLight", { position: [2, 4, 3], intensity: 2.2 }), _jsx(RoomLight, {}), orbit !== false && (_jsx(OrbitRig, { target: target, options: orbit === true ? undefined : orbit })), _jsx(Suspense, { fallback: null, children: _jsx(CharacterModel, { ...character }) })] }), fallback] }));
 }
 export default ConvaiCharacter;

package/dist/react/RoomLight.d.ts

@@ -0,0 +1,3 @@
+export declare function RoomLight({ intensity }: {
+    intensity?: number;
+}): null;

package/dist/react/RoomLight.js

@@ -0,0 +1,41 @@
+/**
+ * A neutral image-based light for `ConvaiCharacter`'s own canvas.
+ *
+ * The default bodies are physically shaded: glossy eyes, skin with a real
+ * specular lobe. With only a hemisphere and a key light there is nothing for
+ * that gloss to reflect, so the eyes read as black marbles and the skin as
+ * flat. Their exports used to hide this by wiring the base colour into the
+ * emissive slot, which made every surface glow instead; `applyFaceMaterials`
+ * now removes that, and this supplies the light the surfaces were missing.
+ *
+ * Built once per renderer from three's own RoomEnvironment (no asset to
+ * fetch), and removed again on unmount. A scene that sets its own
+ * `environment` keeps it.
+ */
+import { useEffect } from 'react';
+import { useThree } from '@react-three/fiber';
+import * as THREE from 'three';
+import { RoomEnvironment } from 'three/examples/jsm/environments/RoomEnvironment.js';
+export function RoomLight({ intensity = 0.6 }) {
+    const gl = useThree((s) => s.gl);
+    const scene = useThree((s) => s.scene);
+    useEffect(() => {
+        if (scene.environment)
+            return;
+        const pmrem = new THREE.PMREMGenerator(gl);
+        const room = new RoomEnvironment();
+        const target = pmrem.fromScene(room, 0.04);
+        room.dispose?.();
+        pmrem.dispose();
+        scene.environment = target.texture;
+        const previous = scene.environmentIntensity;
+        scene.environmentIntensity = intensity;
+        return () => {
+            if (scene.environment === target.texture)
+                scene.environment = null;
+            scene.environmentIntensity = previous;
+            target.dispose();
+        };
+    }, [gl, scene, intensity]);
+    return null;
+}

package/dist/react/index.d.ts

@@ -11,7 +11,8 @@
  *
  * embody is the 3D character only. For a chat panel, install
  * `@convai/web-widget` and hand it the character's own client — one session,
- * not two. `useWidgetClient` adapts that client to the widget's props.
+ * not two. `ConvaiWidget` (web-widget 0.3.0+) takes that client directly;
+ * `useWidgetClient` adapts it for older widgets.
  *
  * Chat UI is DOM and the character is a `<Canvas>` child, so lift the client
  * out rather than rendering the widget from inside the scene:
@@ -20,12 +21,11 @@
  * import { ConvaiWidget } from '@convai/web-widget/react'
  *
  * const [client, setClient] = useState<ConvaiClient | null>(null)
- * const widgetClient = useWidgetClient(client)
  * <Canvas>
  *   <ConvaiCharacter apiKey={apiKey} characterId={characterId}
  *                    onReady={(c) => setClient(c.client)} />
  * </Canvas>
- * {widgetClient && <ConvaiWidget convaiClient={widgetClient} />}
+ * {client && <ConvaiWidget convaiClient={client} />}
  * ```
  */
 export { useCharacter } from './useCharacter.js';

package/dist/react/index.js

@@ -11,7 +11,8 @@
  *
  * embody is the 3D character only. For a chat panel, install
  * `@convai/web-widget` and hand it the character's own client — one session,
- * not two. `useWidgetClient` adapts that client to the widget's props.
+ * not two. `ConvaiWidget` (web-widget 0.3.0+) takes that client directly;
+ * `useWidgetClient` adapts it for older widgets.
  *
  * Chat UI is DOM and the character is a `<Canvas>` child, so lift the client
  * out rather than rendering the widget from inside the scene:
@@ -20,12 +21,11 @@
  * import { ConvaiWidget } from '@convai/web-widget/react'
  *
  * const [client, setClient] = useState<ConvaiClient | null>(null)
- * const widgetClient = useWidgetClient(client)
  * <Canvas>
  *   <ConvaiCharacter apiKey={apiKey} characterId={characterId}
  *                    onReady={(c) => setClient(c.client)} />
  * </Canvas>
- * {widgetClient && <ConvaiWidget convaiClient={widgetClient} />}
+ * {client && <ConvaiWidget convaiClient={client} />}
  * ```
  */
 export { useCharacter } from './useCharacter.js';

package/dist/react/useCharacter.d.ts

@@ -7,6 +7,12 @@ export interface UseCharacterOptions extends CreateCharacterOptions {
      */
     queue?: BlendshapeQueue;
 }
+/**
+ * NOTE ON `resolver` (inherited from CreateCharacterOptions): its identity is
+ * part of this hook's dependency set. Memoize it, and change it precisely
+ * when the body should change — an inline `{ resolve() {...} }` is a new
+ * object every render and would rebuild the character every render.
+ */
 export interface UseCharacterResult {
     character: CharacterHandle | null;
     /**

package/dist/react/useCharacter.js

@@ -35,11 +35,26 @@ export function useCharacter(opts) {
             setError(null);
         }, (c) => setCharacter(c), (e) => setError(e));
         return () => lifecycle.cancel();
-        // Re-create when the identity of the character OR the credentials change
-        // — a new key/token means a different session, and reusing the old one
-        // would leave the body talking over a connection the app has revoked.
+        // Re-create when the identity of the character, the credentials, OR the
+        // resolver change.
+        //
+        // The resolver belongs here because it decides WHICH BODY this character
+        // id resolves to. An authoring tool holds the character id fixed and
+        // swaps the block underneath — preview release 3, then release 4 — and
+        // without this dependency none of that reached the screen: the effect
+        // never re-ran, so the old GLB stayed on the canvas and the change looked
+        // like a stubborn cache. Nothing was cached; the loader was simply never
+        // asked for the new url.
+        //
+        // This makes the resolver's IDENTITY part of the contract: pass a stable
+        // one (`useMemo`) and change it exactly when the body should change. A
+        // resolver rebuilt inline on every render would rebuild the character on
+        // every render.
+        //
+        // `modelVariant` too: it picks which of the body's files (Full or Lite)
+        // to load.
         // eslint-disable-next-line react-hooks/exhaustive-deps
-    }, [opts.characterId, opts.apiKey, opts.authToken, opts.token, opts.baseUrl]);
+    }, [opts.characterId, opts.apiKey, opts.authToken, opts.token, opts.baseUrl, opts.resolver, opts.modelVariant]);
     useEffect(() => {
         if (character)
             character.camera = camera;

package/dist/render/faceMaterials.d.ts

@@ -9,7 +9,7 @@ import * as THREE from 'three';
  * neither is sufficient alone, and the teeth in particular need both:
  *
  *   SHAPE — the arches must retract and tuck as the jaw moves. That lives in
- *           the lipsync layer (teethRetract / teethUpperLift / teethLowerTuck)
+ *           the lipsync layer (teethRetract / teethUpper* / teethLower*)
  *           and is already here.
  *   SHADE — the arches must be textured and lit like a mouth interior. That is
  *           this file, and without it the mouth renders as one flat slab.
@@ -70,6 +70,10 @@ export interface FaceMaterialReport {
     eyes: number;
     teeth: number;
     occlusion: number;
+    /** Materials whose base colour was also wired up as their emissive map. */
+    selfLit: number;
+    /** Hidden `Hide` helper materials. */
+    hidden: number;
 }
 /**
  * Pull this model's teeth albedo out of the file, bound or not.

package/dist/render/faceMaterials.js

@@ -1,4 +1,88 @@
 import * as THREE from 'three';
+const isEye = (name) => /^Eyes?(?!occlusion)/i.test(name);
+/**
+ * Repair what the convai-mha-1 Blender exports get wrong, matched on the
+ * defect rather than the material name wherever the defect is detectable, so
+ * a re-export with renamed materials (`EyeSofia`, `Head.001`, `TeethSofia`)
+ * is repaired the same way. Found on three heads in Chrome and, independently,
+ * on the PlayCanvas templates.
+ *
+ *  · SELF-LIT SURFACES. Skin, hair, brows, clothes ship their base colour a
+ *    second time as the emissive map at full strength (the male head at 1.24
+ *    via KHR_materials_emissive_strength). Everything glows: oily skin, white
+ *    hair strands, no shading. The emissive is dropped. Eyes are the one
+ *    exception and keep a quarter of it, so the iris stays readable instead of
+ *    reflecting a dark room as a black marble.
+ *  · OUT-OF-RANGE FACTORS. roughness up to 2.9 and specular colour 2.0.
+ *    Clamped to 1.
+ *  · THE HEAD. Its roughness map is near-glossy along the painted scalp edge,
+ *    which renders as a grey fringe under the hair (specular, not alpha), and
+ *    it is alpha-masked, which cuts holes at the hairline. Constant roughness,
+ *    opaque.
+ *  · `Hide` helper geometry, which is never meant to render.
+ */
+function repairExport(mat, found) {
+    const m = mat;
+    const name = m.name || '';
+    let changed = false;
+    if (/^Hide/i.test(name)) {
+        m.visible = false;
+        found.hidden++;
+        return;
+    }
+    const emissive = m.emissive;
+    const emissiveMap = m.emissiveMap;
+    if (emissive && emissiveMap) {
+        const sameImage = !!m.map && (m.map === emissiveMap || m.map.source === emissiveMap.source);
+        const lit = emissive.r + emissive.g + emissive.b > 0;
+        if (isEye(name)) {
+            // Unlit eyes are a deliberate export choice (see `tune`); lit ones get a
+            // floor of their own albedo.
+            if ('roughness' in m) {
+                if (!lit)
+                    emissive.setScalar(0.25);
+                else
+                    emissive.multiplyScalar(0.25);
+                if ('emissiveIntensity' in m)
+                    m.emissiveIntensity = 1;
+                changed = true;
+            }
+        }
+        else if (sameImage && lit) {
+            m.emissiveMap = null;
+            emissive.setScalar(0);
+            found.selfLit++;
+            changed = true;
+        }
+    }
+    else if (isEye(name) && emissive && m.map && 'roughness' in m) {
+        m.emissiveMap = m.map;
+        emissive.setScalar(0.25);
+        changed = true;
+    }
+    if (typeof m.roughness === 'number' && m.roughness > 1) {
+        m.roughness = 1;
+        changed = true;
+    }
+    const spec = m.specularColor;
+    if (spec && (spec.r > 1 || spec.g > 1 || spec.b > 1)) {
+        spec.setRGB(Math.min(1, spec.r), Math.min(1, spec.g), Math.min(1, spec.b));
+        changed = true;
+    }
+    if (/^(Base)?Head/i.test(name) && 'roughness' in m) {
+        if (m.roughnessMap) {
+            m.roughnessMap = null;
+            m.roughness = 0.62;
+        }
+        if ('specularIntensity' in m)
+            m.specularIntensity = 0.35;
+        if (m.alphaTest > 0 && !m.transparent)
+            m.alphaTest = 0;
+        changed = true;
+    }
+    if (changed)
+        m.needsUpdate = true;
+}
 /**
  * Pull this model's teeth albedo out of the file, bound or not.
  *
@@ -59,7 +143,7 @@ export async function loadTeethTexture(parser) {
 function tune(mat, teethScale, found) {
     const m = mat;
     const name = m.name || '';
-    if (/^Eyes/i.test(name)) {
+    if (isEye(name)) {
         // ONLY on a material that has a lighting model.
         //
         // Measured on the live convai-mha-1 body: its eye materials arrive as
@@ -145,7 +229,7 @@ function referenceCount(root, target) {
  */
 export function applyFaceMaterials(root, tuning = {}, parser) {
     const teethScale = tuning.teethScale ?? 1;
-    const found = { eyes: 0, teeth: 0, occlusion: 0 };
+    const found = { eyes: 0, teeth: 0, occlusion: 0, selfLit: 0, hidden: 0 };
     /** Every teeth material on THIS model, for the map to land on once loaded. */
     const mouths = [];
     root.traverse((obj) => {
@@ -153,8 +237,10 @@ export function applyFaceMaterials(root, tuning = {}, parser) {
         if (!mesh.isMesh || !mesh.material)
             return;
         const mats = Array.isArray(mesh.material) ? mesh.material : [mesh.material];
-        for (const m of mats)
+        for (const m of mats) {
+            repairExport(m, found);
             tune(m, teethScale, found);
+        }
         const teeth = mats.find((mm) => /^Teeth/i.test(mm.name || ''));
         if (!teeth)
             return;

package/dist/resolve/modelVariant.d.ts

@@ -0,0 +1,44 @@
+/**
+ * Choosing which of a body's model files to load on this device.
+ *
+ * A body can ship more than one GLB of the same rig. The art team makes two
+ * per MetaHuman: Full, with all 251 facial blendshapes, and Lite, with 113.
+ * They differ only in blendshape count, and that count is what phones cannot
+ * afford:
+ *
+ * - three.js spends one vertex-shader uniform slot per blendshape
+ *   (`uniform float morphTargetInfluences[MORPHTARGETS_COUNT]`). Many mobile
+ *   GPUs allow 256 slots in all, so 251 plus three's own matrices overflows,
+ *   the program fails to link, and the head is simply not drawn.
+ * - Morph data lives in a texture sized vertices × blendshapes, so Full also
+ *   costs over twice Lite's GPU memory on the device least able to spare it.
+ *
+ * In the manifest, `lod` orders the files: 0 is the most detailed, higher is
+ * lighter. `morphTargets` is the most blendshapes on any one mesh in the file,
+ * which is the number the uniform limit applies to; the console records it at
+ * upload.
+ */
+import type { ModelVariant } from './resolver.js';
+/** `auto` picks per device; `full` and `lite` force the most and least detailed file. */
+export type ModelVariantPreference = 'auto' | 'full' | 'lite';
+export interface DeviceProfile {
+    /** `MAX_VERTEX_UNIFORM_VECTORS` of a WebGL2 context, or null when none could be made. */
+    maxVertexUniformVectors: number | null;
+    /** A phone or tablet: Lite regardless of what the GPU reports. */
+    mobile: boolean;
+}
+/**
+ * Uniform slots three.js uses besides the blendshape weights: model, view,
+ * projection, normal and bind matrices plus a few scalars, about 35 for a
+ * skinned mesh. Kept generous so a material with extra uniforms still links.
+ */
+export declare const MORPH_UNIFORM_RESERVE = 64;
+/**
+ * The model file to load. Pure: the device is an argument, so the rule is
+ * testable and an app can apply it to a profile of its own.
+ */
+export declare function pickModelVariant(models: readonly ModelVariant[], prefer?: ModelVariantPreference, device?: DeviceProfile): ModelVariant;
+/** Profile this device once. Safe to call during SSR, where it reports a desktop with no GPU. */
+export declare function detectDevice(): DeviceProfile;
+/** Test seam: forget the cached profile. */
+export declare function resetDeviceProfileForTests(): void;

package/dist/resolve/modelVariant.js

@@ -0,0 +1,68 @@
+/**
+ * Uniform slots three.js uses besides the blendshape weights: model, view,
+ * projection, normal and bind matrices plus a few scalars, about 35 for a
+ * skinned mesh. Kept generous so a material with extra uniforms still links.
+ */
+export const MORPH_UNIFORM_RESERVE = 64;
+function byDetail(models) {
+    return [...models].sort((a, b) => (a.lod ?? 0) - (b.lod ?? 0));
+}
+/**
+ * The model file to load. Pure: the device is an argument, so the rule is
+ * testable and an app can apply it to a profile of its own.
+ */
+export function pickModelVariant(models, prefer = 'auto', device = detectDevice()) {
+    if (models.length === 0)
+        throw new Error('representation has no model entries');
+    const ordered = byDetail(models);
+    const lightest = ordered[ordered.length - 1];
+    if (prefer === 'full')
+        return ordered[0];
+    if (prefer === 'lite' || ordered.length === 1)
+        return lightest;
+    if (device.mobile)
+        return lightest;
+    // Desktop: the most detailed file whose blendshapes fit this GPU. An
+    // unknown count or budget is assumed to fit; desktop budgets are 1024+.
+    const budget = device.maxVertexUniformVectors;
+    const fits = (m) => m.morphTargets === undefined || budget === null || m.morphTargets + MORPH_UNIFORM_RESERVE <= budget;
+    return ordered.find(fits) ?? lightest;
+}
+let cached;
+/** Profile this device once. Safe to call during SSR, where it reports a desktop with no GPU. */
+export function detectDevice() {
+    if (cached)
+        return cached;
+    if (typeof window === 'undefined' || typeof document === 'undefined') {
+        return { maxVertexUniformVectors: null, mobile: false };
+    }
+    cached = { maxVertexUniformVectors: probeVertexUniforms(), mobile: isMobile() };
+    return cached;
+}
+function isMobile() {
+    const uaData = navigator.userAgentData;
+    if (uaData?.mobile)
+        return true;
+    // Safari has no userAgentData, and iPadOS reports a desktop user agent. A
+    // touch-first device with no hover is a phone or tablet whichever it is.
+    const mq = (q) => typeof window.matchMedia === 'function' && window.matchMedia(q).matches;
+    return mq('(pointer: coarse)') && !mq('(hover: hover)');
+}
+function probeVertexUniforms() {
+    try {
+        const gl = document.createElement('canvas').getContext('webgl2');
+        if (!gl)
+            return null;
+        const value = gl.getParameter(gl.MAX_VERTEX_UNIFORM_VECTORS);
+        // Browsers cap live contexts (about 16); give this one back at once.
+        gl.getExtension('WEBGL_lose_context')?.loseContext();
+        return typeof value === 'number' ? value : null;
+    }
+    catch {
+        return null;
+    }
+}
+/** Test seam: forget the cached profile. */
+export function resetDeviceProfileForTests() {
+    cached = undefined;
+}

package/dist/resolve/resolver.d.ts

@@ -7,9 +7,12 @@
  * the entire reason the seam exists.
  */
 export interface ModelVariant {
+    /** 0 is the most detailed file; higher is lighter (Lite is 1). Absent means 0. */
     lod?: number;
     url: string;
     size?: number;
+    /** The most blendshapes on any one mesh in the file — what mobile GPUs limit. */
+    morphTargets?: number;
     encodings?: Record<string, string>;
 }
 export interface Representation {
@@ -53,7 +56,7 @@ export interface CharacterResolver {
  * not pass `baseUrl` to the wrong backend, which is why it is checked rather
  * than trusted.
  */
-export declare const DEFAULT_BASE_URL = "https://api-stg.convai.com";
+export declare const DEFAULT_BASE_URL = "https://api.convai.com";
 export interface HttpResolverOptions {
     /** Base url of the embodiment backend. Defaults to the release branch's host. */
     baseUrl?: string;

package/dist/resolve/resolver.js

@@ -20,7 +20,7 @@
  * not pass `baseUrl` to the wrong backend, which is why it is checked rather
  * than trusted.
  */
-export const DEFAULT_BASE_URL = 'https://api-stg.convai.com';
+export const DEFAULT_BASE_URL = 'https://api.convai.com';
 /** Shared so `createCharacter` and the resolver cannot drift apart. */
 export const MISSING_CREDENTIAL = '@convai/embody: a credential is required — pass `token` (recommended for ' +
     'browsers) or `apiKey`. There is no anonymous access to Convai.';

package/dist/rig/profile.js

@@ -46,9 +46,15 @@ export const CONVAI_MHA_1 = {
         blinkR: ['CTRL_expressions_eyeBlinkR', 'EyeBlinkRight'],
         relax: ['CTRL_expressions_eyeRelaxL', 'CTRL_expressions_eyeRelaxR'],
         lowerLidUp: ['CTRL_expressions_eyeLowerLidUpL', 'CTRL_expressions_eyeLowerLidUpR'],
-        // Cross-named on purpose — verified in useEyeTracking. eyeLookLeft*
-        // rotates toward the character's RIGHT, so it is the "in" channel for the
-        // LEFT eye and the "out" channel for the right.
+        // In/out here are in the frame of EyeGazeSystem's residual, not the
+        // anatomy. eyeLookLeft* turns the eyes to the character's own LEFT
+        // (verified 2026-10-14 in Chrome, each morph alone, on three convai-mha-1
+        // heads). This basis's `right` axis, (0, 0, -1) on the UE head bone, is
+        // also the character's left, so a positive residual means "toward the
+        // character's left" and these mappings send the eyes there. Behaviour is
+        // right end to end (camera at the character's left -> eyeLookLeft*);
+        // only the old "cross-named" explanation was wrong. Swap these two only
+        // together with the basis.
         eyeLookUpL: ['CTRL_expressions_eyeLookUpL', 'EyeLookUpLeft'],
         eyeLookDownL: ['CTRL_expressions_eyeLookDownL', 'EyeLookDownLeft'],
         eyeLookInL: ['CTRL_expressions_eyeLookLeftL', 'EyeLookInLeft'],

package/dist/speech/lipsync.d.ts

@@ -20,29 +20,59 @@ import { type CorrectiveMap } from './correctiveMap.js';
 export interface MouthTuning {
     /** Hard cap on jawOpen; peak excursions read exaggerated on camera. */
     jawCap?: number;
-    /** Lip-seal floor ceiling and onset for bilabials (m/b/p). */
+    /**
+     * Lip-seal floor ceiling and onset for bilabials (m/b/p). Off by default
+     * (ceiling 0): the MHA stream keeps jawOpen low (median 0.04 measured on a
+     * live reply) and opens the mouth through the lip channels, so a seal keyed
+     * on the raw jaw fired on 74% of speaking frames and held the lips half
+     * shut against them. Raise the ceiling for a stream that drives the jaw.
+     */
     sealCeiling?: number;
     sealOnset?: number;
-    /** Teeth follow rules. */
+    /**
+     * Teeth follow the shaped jaw, as weights on the teeth morphs:
+     *   teethBackU/D = teethRetract × jaw
+     *   teethUpU     = teethUpperRest + teethUpperLift × jaw
+     *   teethDownD   = teethLowerRest + teethLowerDrop × jaw
+     * The rest terms keep the teeth behind the lips at a closed mouth; the
+     * lower teeth then drop with the jaw rather than snapping open.
+     */
     teethRetract?: number;
+    teethUpperRest?: number;
     teethUpperLift?: number;
+    teethLowerRest?: number;
+    teethLowerDrop?: number;
+    /**
+     * @deprecated Read as `teethLowerRest` when that is not set. It used to tuck
+     * the lower teeth only while the mouth was closed and release them as it
+     * opened, which is backwards: the lower teeth should sit low and drop.
+     */
     teethLowerTuck?: number;
     /** Gain multipliers by MHA channel name, forwarded to the processor. */
     gainOverrides?: Record<string, number>;
     /** Seconds of visual lead granted at speech start. */
     visualLeadSeconds?: number;
+    /**
+     * Seconds without a fresh frame, while the bot still reads as speaking,
+     * after which the mouth relaxes instead of holding its last shape: a stream
+     * stall, or a speaking flag that outlives the voice.
+     */
+    staleReleaseSeconds?: number;
 }
-declare const DEFAULTS: Required<Omit<MouthTuning, 'gainOverrides'>> & {
+type Tuning = Required<Omit<MouthTuning, 'gainOverrides' | 'teethLowerTuck'>> & {
     gainOverrides: Record<string, number>;
 };
 export declare class LipsyncSystem {
-    readonly t: typeof DEFAULTS;
+    readonly t: Tuning;
     private readonly processor;
     private bindings;
     private accum;
     private lastFrame;
     private wasSpeaking;
     private env;
+    /** Seconds since a fresh frame came off the queue while speaking. */
+    private sinceFresh;
+    private lastClock;
     /** 0–1 speech energy; gaze layers read it. */
     energy: number;
     /** True while the bot owns the mouth; body layers read it. */
@@ -54,6 +84,7 @@ export declare class LipsyncSystem {
      */
     correctiveMap: CorrectiveMap | null;
     private mapped;
+    private unpinned;
     private readonly ch;
     constructor(tuning?: MouthTuning, correctiveMap?: CorrectiveMap | null);
     /** Bind every morph-bearing mesh under root. Returns matched channel count. */

package/dist/speech/lipsync.js

@@ -1,15 +1,26 @@
 import { createLipsyncProcessor, mhaChannelIndex, } from '@convai/web-sdk/lipsync-helpers';
 import { applyCorrectiveMap } from './correctiveMap.js';
+/**
+ * Teeth values are the ones approved on a convai-mha-1 head (2026-10-13) and
+ * re-checked on the two default bodies: the teeth stay behind the lips at
+ * rest and never show more than the lip opening allows.
+ */
 const DEFAULTS = {
     jawCap: 0.55,
-    sealCeiling: 0.5,
+    sealCeiling: 0,
     sealOnset: 0.45,
-    teethRetract: 0.55,
-    teethUpperLift: 0.3,
-    teethLowerTuck: 0.34,
+    teethRetract: 0.12,
+    teethUpperRest: 0.03,
+    teethUpperLift: 0.2,
+    teethLowerRest: 0.18,
+    teethLowerDrop: 0.35,
     visualLeadSeconds: 0.1,
-    gainOverrides: { mouthLowerLipDepressL: 0.28, mouthLowerLipDepressR: 0.28 },
+    staleReleaseSeconds: 0.35,
+    // 0.45 is the SDK's recommended gain; 0.28 damped articulation.
+    gainOverrides: { mouthLowerLipDepressL: 0.45, mouthLowerLipDepressR: 0.45 },
 };
+/** Wall-clock seconds, for catching up when a caller caps or skips its delta. */
+const nowSeconds = () => (typeof performance !== 'undefined' ? performance.now() : Date.now()) / 1000;
 export class LipsyncSystem {
     constructor(tuning = {}, correctiveMap = null) {
         this.bindings = [];
@@ -17,13 +28,23 @@ export class LipsyncSystem {
         this.lastFrame = null;
         this.wasSpeaking = false;
         this.env = 0;
+        /** Seconds since a fresh frame came off the queue while speaking. */
+        this.sinceFresh = 0;
+        this.lastClock = null;
         /** 0–1 speech energy; gaze layers read it. */
         this.energy = 0;
         /** True while the bot owns the mouth; body layers read it. */
         this.speaking = false;
         this.mapped = null;
+        this.unpinned = null;
         this.correctiveMap = correctiveMap;
-        this.t = { ...DEFAULTS, ...tuning, gainOverrides: { ...DEFAULTS.gainOverrides, ...tuning.gainOverrides } };
+        const { teethLowerTuck, ...rest } = tuning;
+        this.t = {
+            ...DEFAULTS,
+            ...(teethLowerTuck !== undefined && rest.teethLowerRest === undefined ? { teethLowerRest: teethLowerTuck } : {}),
+            ...rest,
+            gainOverrides: { ...DEFAULTS.gainOverrides, ...tuning.gainOverrides },
+        };
         this.processor = createLipsyncProcessor('mha', {
             skipEyeChannels: true,
             gainOverrides: this.t.gainOverrides,
@@ -39,6 +60,7 @@ export class LipsyncSystem {
             teethBackLower: idx('teethBackD'),
             teethUpUpper: idx('teethUpU'),
             teethDownLower: idx('teethDownD'),
+            noseUpper: idxs('noseWrinkleUpperL', 'noseWrinkleUpperR'),
         };
     }
     /** Bind every morph-bearing mesh under root. Returns matched channel count. */
@@ -47,6 +69,21 @@ export class LipsyncSystem {
         return this.bindings.reduce((s, b) => s + b.matched, 0);
     }
     correct(frame) {
+        // The MHA stream sends noseWrinkleUpperL/R as exactly 1.0 on every frame
+        // of every reply (a server artifact, confirmed against staging and
+        // production), which reads as a frozen sneer. Treat a pinned value as
+        // unset, as the SDK's MHA processor now does. Copied, never written back:
+        // the queue's own frame is shared.
+        const nose = this.ch.noseUpper;
+        if (nose.some((i) => frame[i] >= 0.999)) {
+            if (this.unpinned?.length !== frame.length)
+                this.unpinned = new Float32Array(frame.length);
+            this.unpinned.set(frame);
+            for (const i of nose)
+                if (this.unpinned[i] >= 0.999)
+                    this.unpinned[i] = 0;
+            frame = this.unpinned;
+        }
         const map = this.correctiveMap;
         if (!map)
             return frame;
@@ -58,6 +95,14 @@ export class LipsyncSystem {
     update(delta, q) {
         if (!this.bindings.length)
             return;
+        // The queue is paced by the voice, which keeps playing whatever the
+        // render loop does. A host that caps its delta (PlayCanvas caps at 0.1 s)
+        // or loses frames would otherwise leave the lips behind the voice for the
+        // rest of the reply, so frames are consumed against the wall clock when it
+        // has moved further than the delta says.
+        const clock = nowSeconds();
+        const elapsed = this.lastClock === null ? delta : Math.min(30, Math.max(delta, clock - this.lastClock));
+        this.lastClock = clock;
         if (!q) {
             // Not holding the queue means not speaking — and that must be SAID,
             // or the body layer holds "talking" forever.
@@ -69,12 +114,14 @@ export class LipsyncSystem {
         const fps = q.getPlaybackFps?.() || 60;
         const frameTime = 1 / fps;
         const speaking = q.isBotSpeaking?.() ?? true;
-        if (speaking && !this.wasSpeaking)
+        if (speaking && !this.wasSpeaking) {
             this.accum = this.t.visualLeadSeconds;
+            this.sinceFresh = 0;
+        }
         this.wasSpeaking = speaking;
         let fresh = null;
         if (speaking) {
-            this.accum += delta;
+            this.accum += elapsed;
             while (this.accum >= frameTime && q.hasFrames()) {
                 const frame = q.getFrameWithAlpha(0);
                 q.consumeFrames(1);
@@ -84,20 +131,28 @@ export class LipsyncSystem {
             }
             if (this.accum > 0.25)
                 this.accum = 0.25; // a starved queue banks no debt
-            if (fresh)
+            if (fresh) {
                 this.lastFrame = this.correct(fresh);
+                this.sinceFresh = 0;
+            }
+            else {
+                this.sinceFresh += elapsed;
+            }
         }
         else {
             this.accum = 0;
         }
-        const raw = speaking ? this.lastFrame : null;
+        // Holding the last shape through a stall reads as a frozen mouth; let the
+        // envelope settle it instead, and pick up again on the next frame.
+        const stale = this.sinceFresh > this.t.staleReleaseSeconds;
+        const raw = speaking && !stale ? this.lastFrame : null;
         const out = this.processor.process(raw, delta);
         // mirror the processor's envelope so the seal fades with speech
         const target = raw ? 1 : 0;
         const tau = target > this.env ? 0.09 : 0.18;
         this.env += (target - this.env) * (1 - Math.exp(-delta / tau));
         if (out) {
-            const { jawCap, sealCeiling, sealOnset, teethRetract, teethUpperLift, teethLowerTuck } = this.t;
+            const { jawCap, sealCeiling, sealOnset, teethRetract, teethUpperRest, teethUpperLift, teethLowerRest, teethLowerDrop } = this.t;
             const ch = this.ch;
             // hard jaw cap, lower lip scaled by the same ratio so the cap changes
             // the size of the opening without changing its shape
@@ -110,7 +165,7 @@ export class LipsyncSystem {
                 for (const i of ch.lowerLipDepress)
                     out[i] *= capRatio;
             // bilabial closure floor, driven from the RAW jaw
-            if (raw && ch.jawOpen >= 0 && ch.lipsTogether.length) {
+            if (sealCeiling > 0 && raw && ch.jawOpen >= 0 && ch.lipsTogether.length) {
                 const jawRaw = raw[ch.jawOpen];
                 const bilabial = 1 - Math.min(1, Math.max(0, (jawRaw - 0.02) / 0.16));
                 if (bilabial > sealOnset) {
@@ -127,12 +182,9 @@ export class LipsyncSystem {
             if (ch.teethBackLower >= 0)
                 out[ch.teethBackLower] = jaw * teethRetract;
             if (ch.teethUpUpper >= 0)
-                out[ch.teethUpUpper] = jaw * teethUpperLift;
-            if (ch.teethDownLower >= 0) {
-                // normalised against the CAP, not 1 — lands on exactly 0 at widest
-                const open = jawCap > 0 ? Math.min(1, jaw / jawCap) : 0;
-                out[ch.teethDownLower] = teethLowerTuck * (1 - open);
-            }
+                out[ch.teethUpUpper] = teethUpperRest + jaw * teethUpperLift;
+            if (ch.teethDownLower >= 0)
+                out[ch.teethDownLower] = teethLowerRest + jaw * teethLowerDrop;
             // speech energy: fast attack, slow release
             let mag = jaw;
             for (const i of ch.funnel)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@convai/embody",
-  "version": "0.6.0",
+  "version": "1.0.0-beta.1",
   "description": "Give a Convai character a body: resolve 3D characters by characterId and drive them with lipsync and animation. Layered on @convai/web-sdk.",
   "license": "Apache-2.0",
   "author": "Convai <support@convai.com>",
@@ -58,10 +58,10 @@
     "access": "public"
   },
   "peerDependencies": {
-    "@convai/web-sdk": ">=1.8.0-beta.4 || >=2.0.0-beta.0",
-    "@react-three/fiber": ">=8",
-    "react": ">=18",
-    "three": ">=0.160"
+    "@convai/web-sdk": ">=2.0.1 <3",
+    "@react-three/fiber": ">=8 <10",
+    "react": ">=18 <20",
+    "three": ">=0.160 <1"
   },
   "peerDependenciesMeta": {
     "react": {
@@ -73,8 +73,8 @@
   },
   "devDependencies": {
     "@changesets/cli": "^2.27.0",
-    "@convai/web-sdk": "2.0.0-beta.0",
-    "@convai/web-widget": "0.2.1",
+    "@convai/web-sdk": "2.0.1",
+    "@convai/web-widget": "0.3.0",
     "@react-three/fiber": "^8.18.0",
     "@types/react": "^18.3.31",
     "@types/react-dom": "^19.2.7",