@vincentt-xr/sdk 2.0.0-alpha.8 → 2.0.0-alpha.9

package/docs/api/face-instructions.mdx

@@ -0,0 +1,183 @@
+---
+title: "Face Instructions"
+description: "Ask for a face expression with a 3D hint face and your own copy, detect it from the face blendshapes, and complete when it is held."
+sidebarTitle: "Face Instructions"
+icon: "face-smile-wink"
+---
+
+`FaceInstruction` asks the person for a face expression and waits for it. A cute, translucent 3D face
+loops into the expression, your copy sits under it, and holding the expression for `holdMs` completes
+it. It is the face counterpart of [Hand Instructions](/api/hand-instructions), with the same layout
+and feedback. The pieces underneath are exported too: `useFaceExpression` (the detection alone) and
+`FaceExpressionHint` (the face alone).
+
+Import from `@vincentt-xr/sdk/instructions`. Everything is customisable: your own expressions,
+detection logic, hint look and motion, text style, or a fully custom prompt. See
+[Instructions](/api/instructions).
+
+## Quick start
+
+```tsx
+import { useLoader } from "@react-three/fiber";
+import { MeshoptDecoder } from "three/examples/jsm/libs/meshopt_decoder.module.js";
+import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js";
+import { FaceInstruction } from "@vincentt-xr/sdk/instructions";
+
+const SmileToStart = ({ onStart }: { onStart: () => void }) => {
+  // face-v1.glb is meshopt-compressed, so the loader needs the meshopt decoder.
+  const face = useLoader(GLTFLoader, faceUrl, (loader) => loader.setMeshoptDecoder(MeshoptDecoder));
+  return (
+    <FaceInstruction
+      expression="smile"
+      text="Smile to start"
+      model={face.scene}
+      onComplete={onStart}
+    />
+  );
+};
+```
+
+Mount it inside `XRScene`. It acquires the face model itself, so you don't need a `FaceTracker`, and
+draws over the AR content in its own screen-space layer (`INSTRUCTION_RENDER_ORDER` = 2000 and up).
+Detection reads MediaPipe's face blendshapes.
+
+## `FaceInstruction`
+
+| Prop             | Default                          | What it does                                                                                            |
+| ---------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------- |
+| `expression`     | —                                | What to ask for: a `FACE_EXPRESSIONS` name or your own `defineFaceExpression(…)`.                       |
+| `model`          | —                                | The hint face: morph targets named like the expressions (e.g. `face-v1.glb`'s scene).                   |
+| `text`           | the expression's `label`         | Your copy, in any language. Long text shrinks to fit.                                                   |
+| `doneText`       | keep `text`                      | Copy shown once completed, while it fades out.                                                          |
+| `nudgeText`      | `"Show your face to the camera"` | Second line after `nudgeAfterMs` with no face in view; the hint pulses. `null` turns it off.            |
+| `nudgeAfterMs`   | `3000`                           | How long without a face before the nudge.                                                               |
+| `faceIndex`      | `0`                              | Which detected face to read.                                                                            |
+| `holdMs`         | `400`                            | How long the expression must be held.                                                                   |
+| `graceMs`        | `150`                            | Dropouts shorter than this don't restart the hold.                                                      |
+| `size`           | `"md"`                           | `"sm"`, `"md"`, `"lg"`: 10 / 14 / 19 % of the screen height; or a fraction of it.                       |
+| `placement`      | `"center"`                       | `"top"`, `"center"` or `"bottom"`.                                                                      |
+| `offset`         | `[0, 0]`                         | Shift from the placement, in half-screens.                                                              |
+| `orbit`          | `30`                             | The hint's auto orbit, degrees to each side. `0` holds it still.                                        |
+| `look`           | `FACE_HINT_LOOK`                 | The hint's look, merged over the default: ghost, per-part tints, halo, glow, material.                  |
+| `loop`           | `FACE_HINT_LOOP`                 | The hint's loop: `seconds`, `hold`, `from`, `easing`.                                                   |
+| `color`          | `"#ffffff"`                      | Text colour, with a soft dark shadow so it reads on any camera feed.                                    |
+| `textStyle` …    | —                                | `textStyle`, `nudgeTextStyle`, `renderText`, `motion`, `layout`: see [Instructions](/api/instructions). |
+| `onComplete`     | —                                | Called once when the expression is held long enough.                                                    |
+| `onStatusChange` | —                                | Called once with the first status, then on every change.                                                |
+| `hideOnComplete` | `true`                           | Fade out after completing. A small pop and a white ring play either way.                                |
+| `active`         | `true`                           | `false` hides it and pauses detection.                                                                  |
+
+`onStatusChange` reports an `InstructionStatus`:
+
+- `"waiting"`: no face in view.
+- `"detected"`: a face in view, not making the expression.
+- `"holding"`: making it; the hold is running.
+- `"done"`: completed.
+
+## Expressions
+
+| Name         | Default copy       | Recognised when (MediaPipe blendshapes, 0..1)                                 |
+| ------------ | ------------------ | ----------------------------------------------------------------------------- |
+| `smile`      | Give a big smile   | `mouthSmileLeft` and `mouthSmileRight` ≥ 0.45                                 |
+| `wink`       | Wink at the camera | one `eyeBlink` ≥ 0.55 and the other ≤ 0.3 (either eye)                        |
+| `surprised`  | Look surprised     | `jawOpen` ≥ 0.35 and `browInnerUp` ≥ 0.4                                      |
+| `kiss`       | Blow a kiss        | `mouthPucker` ≥ 0.5 and `jawOpen` ≤ 0.3                                       |
+| `puffCheeks` | Puff your cheeks   | `cheekPuff` ≥ 0.3                                                             |
+| `frown`      | Frown              | `browDownLeft` and `browDownRight` ≥ 0.4                                      |
+| `scrunch`    | Scrunch your face  | both `noseSneer` ≥ 0.3, or both `eyeSquint` ≥ 0.45 with both `browDown` ≥ 0.3 |
+
+Several can match at once: a scrunch usually also counts as a frown.
+
+## Detection only: `useFaceExpression`
+
+```tsx
+import { useFaceExpression } from "@vincentt-xr/sdk/instructions";
+
+const { status, holdRef, matchedRef, faceRef, isInView, reset } = useFaceExpression("wink", {
+  holdMs: 400,
+  onComplete: () => unlock(),
+});
+```
+
+`status` (`"waiting"` → `"holding"` → `"done"`) re-renders only when it changes. Read
+`holdRef.current.progress` (0..1) and `faceRef.current` (the latest `FaceInfo`, or `null`) inside
+`useFrame`. `reset()` asks again. The result is an `InstructionDetector`, so it can drive a custom
+`<Instruction>`.
+
+A rule, a match function `(blendshapes, face) => boolean` or a definition works anywhere an
+expression name does. A `FaceExpressionRule` has:
+
+- `atLeast`: per blendshape, the minimum score.
+- `atMost`: per blendshape, the maximum score.
+- `anyOf`: sub-rules, one of which must pass.
+
+```tsx
+useFaceExpression({ atLeast: { jawOpen: 0.6 }, atMost: { mouthSmileLeft: 0.2 } });
+useFaceExpression((b) => (b.mouthSmileLeft ?? 0) - (b.mouthSmileRight ?? 0) > 0.4); // a half smile
+```
+
+The checks are pure: `matchFaceExpression(face.blendshapes, nameOrRule)` and
+`matchingFaceExpressions(face.blendshapes)`.
+
+## The hint face: `FaceExpressionHint`
+
+```tsx
+<FaceExpressionHint expression="surprised" model={face.scene} height={0.4} />
+```
+
+The face alone, for your own layout. It is fitted to `height` (parent units) and plays the
+expression's morph target (`FACE_EXPRESSIONS[name].hint.target`, or pass `expression={{ target }}`,
+or a weighted mix `expression={{ targets: { surprised: 1, wink: 1 } }}`)
+from the idle face and back, while slowly orbiting side to side. The default look:
+
+- **Ghost:** translucent white with a brighter rim, the same as the hand hint.
+- **Tinted features:** each part is tinted by its material name (`iris`, `brow`, `lash`, `lip`,
+  `mouth_inside`, `tongue`, `sparkle`, `eye_white`), so the expression reads at small size.
+- **Halo:** a soft dark edge so it reads on light backgrounds (`look.halo`).
+- **Glow:** a soft light from behind the head, brightening as the expression forms (`look.glow`). It
+  is a separate disc behind the face, so it never shows as inner layers.
+
+`look` is merged over `FACE_HINT_LOOK` (retint a part with `look.parts.iris`, drop a tint with
+`null`, `false` turns a layer off, `look.material` replaces the ghost). A custom face works if its meshes carry morph targets named like the
+expressions and it faces +Z.
+
+## The example model
+
+`assets-src/faces/face-v1.blend` and `build_face.py` (README beside them) build the face from scratch,
+with no third-party asset. The model is one mesh with nine materials and one fixed morph target per
+expression: `smile`, `wink`, `surprised`, `kiss`, `puffCheeks`, `frown`, `scrunch`. The idle face is
+the base shape, and the cheek blush is in `COLOR_0`.
+
+Each expression is computed exactly rather than mixed from smaller keys, so the eyelids never cut
+through the eyes. The raw export is ~1.8 MB; `gltf-transform meshopt` brings it to ~255 KB (~105 KB
+gzipped) with every shape within 0.02 mm.
+
+## Example
+
+`/face-instructions` covers:
+
+- every expression as a chip, and a sequence mode that moves to the next expression when one
+  completes;
+- text, nudge, size, placement and hold-time controls;
+- a live readout of the status, the blendshapes the rules read, and the matching expressions.
+
+## Manual visual test plan
+
+Open `/face-instructions`.
+
+1. **Every hint reads**: step through all seven chips. The face loops into each expression: closed
+   happy eyes and an open smile, a left wink, wide eyes with an O, a pucker, puffed cheeks (clearest
+   mid-orbit), slanted brows with a downturned mouth, squinted slits. It stays inside its slot through
+   the orbit.
+2. **Looks**: the ghost is translucent with no back layers showing through, the eyes, brows and
+   mouth stay clear, and the halo separates it from a white shirt.
+3. **Detection**: on the webcam, make each expression. The readout's matches list it and the status
+   goes `holding` → `done`. A resting face matches nothing; a wink with either eye works; blinking
+   both eyes is not a wink.
+4. **Completion**: on done, a small pop and a white ring play. With sequence on, it fades out and the
+   next expression comes in.
+5. **Nudge**: turn away for 3 s. The second line fades in and the hint pulses; it fades out when your
+   face is back. With the nudge off it never appears.
+6. **Copy and layout**: type custom text, switch sizes and top / center / bottom placement. Nothing
+   overlaps the screen edge.
+7. **Hands unchanged**: `/hand-instructions` looks and behaves as before (both use the same layout).

package/docs/api/face-liquify.mdx

@@ -0,0 +1,196 @@
+---
+title: "Face Liquify"
+description: "2D liquify on the live face: bigger eyes, a slimmer jaw, a smile, a swirl. Eleven presets by weight, your own warps on named face points, every tracked face, with makeup, glasses and hats warping along."
+sidebarTitle: "Face Liquify"
+icon: "face-grin-stars"
+---
+
+`FaceLiquify` reshapes faces on screen with local 2D warps, like a liquify brush that follows the
+face.
+
+- **Everything on the face warps together.** It warps the picture after everything 3D is drawn: the
+  camera, makeup and lips, face effects, glasses, hats and earrings.
+- **Screen UI stays sharp.** `ScreenImage`, `ScreenText` and other screen UI draw after it.
+- **Weight 0 is the untouched image**, byte for byte.
+
+Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { VideoBackground, XRScene } from "@vincentt-xr/sdk";
+import { FaceLiquify } from "@vincentt-xr/sdk/face-effects";
+
+<XRScene>
+  <VideoBackground />
+  <FaceLiquify weights={{ bigEyes: 1, slimJaw: 1 }} />
+</XRScene>;
+```
+
+It starts the face model itself; no `FaceTracker` is needed.
+
+## FaceLiquify or FaceWarp3D?
+
+|           | `FaceLiquify`                                                     | [`FaceWarp3D`](/api/face-warp-3d)                       |
+| --------- | ----------------------------------------------------------------- | ------------------------------------------------------- |
+| How       | 2D warps of the screen image around the face                      | Blend shapes on the face mesh, wearing the camera image |
+| Warps     | Everything drawn on the face: makeup, face effects, accessories   | The face surface; makeup nested in it rides it          |
+| Authoring | Presets and warps in code, no model                               | Shape keys sculpted in Blender                          |
+| Best for  | Beauty tweaks, cartoon eyes and mouths, swirls, quick fun filters | Character proportions that follow the face in 3D        |
+
+They combine: the liquify warps the `FaceWarp3D` face too.
+
+## Presets and weights
+
+`weights` sets each preset from 0 to 3:
+
+- **1** is a subtle, natural look.
+- **2** is clearly stylised.
+- **3** is the strongest cartoon look before the image would fold over itself.
+
+There are no negative weights; out-of-range values clamp.
+
+| Preset         | Does                             |
+| -------------- | -------------------------------- |
+| `bigEyes`      | Magnifies both eyes              |
+| `smallNose`    | Shrinks the nose                 |
+| `bigMouth`     | Magnifies the mouth              |
+| `smile`        | Lifts both mouth corners         |
+| `chubbyCheeks` | Puffs both cheeks                |
+| `slimJaw`      | Pushes both jaw angles in and up |
+| `slimFace`     | Pushes both sides of the face in |
+| `longChin`     | Pulls the chin down              |
+| `bigForehead`  | Magnifies the forehead           |
+| `bigHead`      | Magnifies the whole face         |
+| `swirl`        | Twists the face about its centre |
+
+`weight` (0..1) fades everything at once, linearly. Pass a ref to animate it without re-rendering:
+
+```tsx
+const amount = useRef(0);
+useFrame(({ clock }) => {
+  amount.current = (1 - Math.cos(clock.elapsedTime * 1.6)) / 2;
+});
+<FaceLiquify weights={{ bigEyes: 3 }} weight={amount} />;
+```
+
+`weights` takes a ref too.
+
+## Your own warps
+
+`warps` adds warps by name; set their weight in `weights` (a custom warp without a weight draws as
+authored, at weight 3).
+
+```tsx
+<FaceLiquify
+  warps={{
+    pointyChin: { kind: "stretch", at: "chin", radius: 0.6, strength: 0.8, direction: [0, -1] },
+    elfEars: {
+      kind: "push",
+      at: "leftFaceSide",
+      offset: [0.2, 0.6],
+      radius: 0.4,
+      strength: 0.7,
+      direction: [0.6, 1],
+      mirror: true,
+    },
+  }}
+  weights={{ pointyChin: 2, elfEars: 3 }}
+/>
+```
+
+| Field       | Meaning                                                                                                              |
+| ----------- | -------------------------------------------------------------------------------------------------------------------- |
+| `kind`      | `bulge` (magnify), `pinch` (shrink), `push` (move along `direction`), `stretch` (magnify along `direction`), `swirl` |
+| `at`        | A named point (below) or `{ landmark: n }`, a MediaPipe face landmark                                                |
+| `offset`    | From `at`, in eye distances: x toward the person's own left, y up the face                                           |
+| `radius`    | The circle it changes, in eye distances (the 3D distance between the eye centres)                                    |
+| `strength`  | 0..1: the share of the kind's limit at weight 3. At 1 it is the most that kind can do without folding                |
+| `direction` | `push` and `stretch`: the way the content moves, in face axes. Default up the face                                   |
+| `clockwise` | `swirl`: turn the other way                                                                                          |
+| `mirror`    | Also apply it on the other side of the face                                                                          |
+
+Named points: `leftEye`, `rightEye`, `noseTip`, `noseBridge`, `mouth`, `leftMouthCorner`,
+`rightMouthCorner`, `chin`, `leftJaw`, `rightJaw`, `leftCheek`, `rightCheek`, `leftFaceSide`,
+`rightFaceSide`, `forehead`, `faceCenter`. Left and right are the person's own sides on every feed.
+
+Warps apply one after another, so together they never fold either. A face holds at most 32 warps.
+Memoise the `warps` object: a new object re-resolves the warps.
+
+## Faces, fades and turned heads
+
+- **Every tracked face** warps, up to four, each its own pass. `faceIndex` picks one; `numFaces` sets
+  how many faces the face model tracks.
+- **Fades:** a face fades in over 150 ms when found, holds through a dropout of up to 250 ms, then
+  fades out over 120 ms.
+- **Turned heads:** a warp on one side of the face fades as that side turns away, so the far cheek
+  never smears over the nose.
+- **Smoothing:** landmarks are smoothed as face swap's are (`smoothing`, default 0.35; 0 is raw).
+
+## Hands in front (experimental)
+
+With `HandCutout` in the scene, a hand over the face stays sharp: hand pixels are never warped, and a
+warp never pulls a hand into the face.
+
+```tsx
+import { HandCutout, HandTracker } from "@vincentt-xr/sdk/tracking";
+
+<>
+  <HandTracker>
+    <HandCutout />
+  </HandTracker>
+  <FaceLiquify weights={{ bigMouth: 2 }} />
+</>;
+```
+
+Without `HandCutout`, a hand over the face warps with it.
+
+## Props
+
+| Prop          | Default            | Meaning                                                                       |
+| ------------- | ------------------ | ----------------------------------------------------------------------------- |
+| `weights`     | `{}` (nothing)     | Preset and custom-warp weights by name, 0..3, or a ref read every frame       |
+| `warps`       | none               | Your own warps by name                                                        |
+| `weight`      | 1                  | Overall fade 0..1, or a ref                                                   |
+| `faceIndex`   | every tracked face | Only this face (default: the enclosing `FaceTracker`'s face, else every face) |
+| `numFaces`    | 1                  | Faces the face model tracks when this starts it                               |
+| `smoothing`   | 0.35               | Landmark smoothing 0..1                                                       |
+| `active`      | true               | False: no model, no copy, nothing drawn                                       |
+| `debug`       | false              | Draws each warp's circle and direction, and the face's points                 |
+| `renderOrder` | 998                | The pass's render order                                                       |
+
+## Limits
+
+- **Draw order.** The warp draws at render order 998. Content drawn after it stays sharp: screen UI
+  (999+), a transparent mesh placed directly inside a tracker (wrap it in a `<group>` to warp it),
+  your own content at 998 or above.
+- **Opaque screen UI** (`transparent={false}`) draws before it and warps.
+- **Render targets.** Multisampled or float targets (effect composers), WebXR and multi-target passes
+  are drawn unwarped, with one console warning.
+- **Cost.** Each face copies and redraws only its warp area. Check phone performance before shipping a
+  multi-face effect.
+
+## Example
+
+`/face-warp` in the SDK playground, mode "2D liquify", has the preset sliders, a custom-warp editor
+that shows the code to paste, an animate toggle, one or two faces, hands in front and the debug
+circles. Settings are in the URL: `?lq=bigEyes:1,smile:2`, `?lqdebug=1`, `?lqanim=1`,
+`?lqfaces=2`, `?hands=0`. Mode "both" adds the 3D shape keys under the liquify.
+
+## Manual visual test plan
+
+Open `/face-warp` on the webcam.
+
+1. **Weight 0** (all presets off): the image is untouched. No edge, band or shimmer around the face.
+2. **Each preset at 1, then 3**: 1 looks natural, 3 is a strong cartoon. No tearing, no fold, no hard
+   circle edge.
+3. **Sides**: with debug on, the circles sit on your own eyes, mouth corners, jaw and chin, and follow
+   tilt and turn.
+4. **Turned head**: turn far to one side with `chubbyCheeks` at 3. The far cheek fades instead of
+   smearing over the nose.
+5. **Dropout**: cover the camera for a moment. The warp holds briefly, then fades out; it fades back
+   in when you reappear.
+6. **Hands**: with hands in front, put a hand over your mouth with `bigMouth` at 3. The hand keeps its
+   real size, with no hand colour spreading into the face. Show a hand away from your face: no hole
+   appears in the warp.
+7. **Two faces**: faces "two" on the two-people clip warps both people.

package/docs/api/face-mesh.mdx

@@ -108,14 +108,13 @@ subtracts Y while adding X/Z. `DEFAULT_FACE_MESH_TRANSFORM.position.y` is
 backwards compatibility. New scene-object work should use `FaceMeshRenderer`.
 
 ```tsx
-import { CanonicalFaceMeshRenderer, FACE_WARP_3D_PRESET } from "@vincentt-xr/sdk/face-effects";
+import { CanonicalFaceMeshRenderer } from "@vincentt-xr/sdk/face-effects";
 
 <CanonicalFaceMeshRenderer
   materialType="shader"
   shaderPreset="originalFace"
   textureMap="none"
   alphaMap="canonical"
-  faceWarp3D={FACE_WARP_3D_PRESET}
 />;
 ```
 
@@ -126,8 +125,6 @@ import { CanonicalFaceMeshRenderer, FACE_WARP_3D_PRESET } from "@vincentt-xr/sdk
 
 ## FaceWarp3D
 
-`FACE_WARP_3D_PRESET`, `DEFAULT_FACE_WARP_3D_SETTINGS`, and
-`FaceWarp3DSettings` cover face, forehead, cheek, eye, nose, mouth, jaw, and
-chin proportions with normalized values around zero. Use `faceWarp3D` on
-`CanonicalFaceMeshRenderer` or `FaceMeshRenderer` settings instead of writing a
-second landmark deformation path.
+To reshape the face (bigger eyes, a longer nose), use [`FaceWarp3D`](/api/face-warp-3d). It applies
+blend shapes on the live face, with the camera image itself as the texture. It replaces the old
+proportion sliders and `CanonicalFaceMeshRenderer`'s `faceWarp3D` prop.

package/docs/api/face-morph.mdx

@@ -0,0 +1,117 @@
+---
+title: "Face Morph"
+description: "The person's face morphs into another shape (a sphere, a cube, a character), wearing their live camera face all the way."
+sidebarTitle: "Face Morph"
+icon: "wand-magic-sparkles"
+---
+
+`FaceMorph` morphs the person's face into another shape: a sphere, a cube, or a character. The shape
+wears their live camera face all the way, as in Snap Lens Studio's Face Morph.
+
+- `weight` 0 is the live face itself, so the image is unchanged.
+- `weight` 1 is the shape, which still moves with the person's expression.
+
+Animate the weight for the transformation.
+
+It is part of the face-character pipeline ([#194](https://github.com/vincentt-xr/sdk/issues/194)).
+Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { FaceMorph } from "@vincentt-xr/sdk/face-effects";
+import { FaceTracker } from "@vincentt-xr/sdk/tracking";
+
+<FaceTracker>
+  <FaceMorph shape="sphere" weight={0.8} />
+</FaceTracker>;
+```
+
+## FaceMorph or FaceWarp3D?
+
+`FaceMorph` turns the face into another object (a ball, a box, a stylised character) that wears the camera
+face. To reshape the person's real face instead (bigger eyes, a longer nose, a cartoon's proportions,
+photoreal, with their own hair and skin), use [`FaceWarp3D`](/api/face-warp-3d). On a realistic head model,
+`FaceMorph` shows the model's own material on the skull and ears, while `FaceWarp3D` keeps the person's
+real hair. Use one face effect per face.
+
+## Shapes
+
+- **Built-in `"sphere"` and `"cube"`.** These are the face itself, with the eyes and mouth closed
+  over, subdivided and pushed out onto the shape. They fade out at the edge of the face.
+- **Your own model**, authored in the canonical head frame (centimetres, as for
+  [`ExternalFaceMesh`](/api/external-face-mesh)). Two things matter:
+
+  - its **base positions**: where it sits on a face. These bind it.
+  - a **shape key** named `"morph"` (`morphKey`): what it becomes.
+
+  In Blender, start from a mesh fitted to the face (the basis), sculpt the shape key, and export
+  glTF. A model without the key is its own base. For example, facecap morphs into its own shape.
+
+Every vertex is bound through its base pose to its closest point on the face, within `falloff` cm:
+
+- A vertex beyond that keeps to the shape, so a skull or the back of a head stays put.
+- Parts named `eye…` wear your eyes.
+- Teeth keep their own material.
+
+## Props
+
+| Prop                                | Default   | What it does                                                                                         |
+| ----------------------------------- | --------- | ---------------------------------------------------------------------------------------------------- |
+| `shape`                             | —         | `"sphere"`, `"cube"`, or a model (`THREE.Object3D`).                                                 |
+| `weight`                            | `1`       | 0 = the live face, 1 = the shape. Also a ref (`LiveValue`), read every frame.                        |
+| `expression`                        | `1`       | How strongly the shape takes the expression (Snap's Expression Multiplier). Also a ref.              |
+| `scale`                             | `1`       | Scale of the shape around the head (Snap's External Scale).                                          |
+| `morphKey`                          | `"morph"` | The shape key a model morphs into.                                                                   |
+| `faceTexture`                       | `1`       | How much of the camera face it wears: 0 shows the model's own material (a character's skin).         |
+| `region`                            | `"face"`  | `"face"`, `"eyes"`, `"mouth"` or `"eyesAndMouth"` (see [Face Reprojection](/api/face-reprojection)). |
+| `shading`                           | `"unlit"` | `"unlit"`: the camera's colours. `"lit"`: shaded by the scene lights.                                |
+| `color`                             | `#f1c6a8` | A built-in shape's colour where it shows no face.                                                    |
+| `subdivisions`                      | `2`       | Smoothness of a built-in shape.                                                                      |
+| `falloff`                           | `[1, 3]`  | How far from the face (cm, base pose) a vertex still morphs and wears it.                            |
+| `facing`, `fps`, `unwrapResolution` | —         | As for Face Reprojection.                                                                            |
+| rig props                           | —         | `faceIndex`, `smoothing`, `holdMs`, `neutral` … (when it runs its own `FaceRig`).                    |
+
+## How it works
+
+1. **Live face.** Every frame, the live face (`FaceRigState.local`) and its expression
+   (`residual`) are read on the filled canonical face. The eye and mouth centres are the
+   averages of their rings.
+2. **Blend.** Each vertex blends from its bound point on the live face to its shape position plus
+   the expression, by `weight` and by how close it is to the face.
+3. **Texture.** The face texture is the reprojector's unwrap, the camera face in the canonical
+   layout. Each vertex reads it through its `faceUv` / `faceCoverage` attributes, so no remap pass
+   is needed.
+
+**Pure pieces:** `createFaceMorphShape`, `createFaceMorphLive`, `writeFaceMorphLive` and
+`stepFaceMorph`.
+
+## Example
+
+`/face-morph` offers four shapes:
+
+- the sphere;
+- the cube;
+- a **Lens Studio-style head**, sculpted from Snap's face + skull + ears mesh (a "chibi" shape key
+  with a bigger skull, puffed cheeks, a small chin and bigger ears);
+- the **facecap** character.
+
+The last two are local, git-ignored files. The page has controls for weight (or a looping
+animation), expression, scale, face texture, region and shading.
+
+## Manual visual test plan
+
+Open `/face-morph` on the webcam.
+
+1. **Weight 0**: nothing changes. The mesh is exactly your face, with no edge or band.
+2. **Sphere / cube at 1**: your face is on a ball or a box, sharp and centred. The edges fade into
+   the camera image.
+3. **Animate**: the face swells into the shape and back, smoothly and without popping.
+4. **Expressions**: at weight 1, open your mouth, smile, blink and raise your brows. The shape does
+   too, and `expression` scales it.
+5. **Lens Studio head**: the skull, cheeks and ears grow as the weight rises. Your face stays on it
+   and the ears stay on the head.
+6. **facecap + face texture**: lower "face texture" and your face fades into facecap's skin. Your
+   eyes stay on its eyeballs until it is fully faded.
+7. **Head turn and dropout**: the shape turns with the head, holds through a short dropout, hides
+   when the face is lost, and comes back.

package/docs/api/face-painting.mdx

@@ -31,3 +31,10 @@ inspector controls to show for the active material state.
 Geometry visibility is grouped by FaceMesh part: left eye, right eye, mouth,
 and nose. Use `CANONICAL_FACE_MESH_GEOMETRY_SPECS` when an inspector needs the
 runtime group order instead of hardcoding another list.
+
+The `showLeftEye…` / `showRightEye…` settings are the person's own eyes on
+every feed (GROUNDING "Left and right"). The mesh's `"left-eye"` /
+`"right-eye"` groups are screen sides (`"left-eye"` is always the eye on the
+screen's left); the renderer maps the settings onto them with
+`resolveFaceMeshEyeVisibility(person, mirrored)`, which swaps them on
+unmirrored feeds.