@vincentt-xr/sdk 2.0.0-alpha.8 → 2.0.0-alpha.9

package/docs/api/body-triggers.mdx

@@ -0,0 +1,234 @@
+---
+title: "Body Triggers"
+description: "Fire on full-body poses (T-pose, hands up, squat) and moves over time (jump, squat reps, wave, clap, punch, arm flap, kick, step, march) with begin / end events."
+sidebarTitle: "Body Triggers"
+icon: "person-running"
+---
+
+`BodyTrigger` fires when the person holds a body pose or performs a body move. It reads the body
+model's 33 pose landmarks (it acquires the model itself; no `BodyTracker` needed) and runs the same
+lifecycle as [`useGesture`](/api/gestures): begin / end events, hold, cooldown, loss grace and a
+count.
+
+Import from `@vincentt-xr/sdk/tracking`.
+
+```tsx
+import { BodyTrigger, useBodyTrigger } from "@vincentt-xr/sdk/tracking";
+
+// Declarative: mount inside the scene, act on the events.
+<BodyTrigger pose="tPose" onBegin={() => takeOff()} onEnd={() => land()} />;
+<BodyTrigger move="jump" onBegin={() => hop()} />;
+<BodyTrigger move="punch" side="right" onBegin={(state) => hit(state.side)} />;
+
+// Hook: React state that changes only when the trigger's state changes.
+const squats = useBodyTrigger({ move: "squat" });
+// squats.count = reps so far; squats.active = at the bottom of a rep.
+```
+
+Pass exactly one of `pose` or `move`. Left / right are the person's own sides on every camera.
+
+## Poses
+
+`pose` takes BodyInstruction's vocabulary (`BODY_POSES` from `@vincentt-xr/sdk/instructions`) and
+its classifier (`analyzeBodyShape` + `matchBodyPose`), so a pose that completes a
+[`BodyInstruction`](/api/body-instructions) also fires a `BodyTrigger`.
+
+| You want                         | `pose`                                                     | Needs      |
+| -------------------------------- | ---------------------------------------------------------- | ---------- |
+| T-pose                           | `"tPose"`                                                  | upper body |
+| Hands up                         | `"handsUp"` (alias of `"hooray"`)                          | upper body |
+| Hands on hips                    | `"handsOnHips"`                                            | upper body |
+| Arms crossed                     | `"armsCrossed"`                                            | upper body |
+| Squat position                   | `"squat"`                                                  | whole body |
+| Raise one arm                    | `"raiseLeft"` / `"raiseRight"`                             | upper body |
+| Hands on head, thumbs up, salute | `"handsOnHead"`, `"thumbUp"`, `"bothThumbsUp"`, `"salute"` | upper body |
+| Lean, one leg, star              | `"leanLeft"` / `"leanRight"`, `"oneLeg"`, `"star"`         | whole body |
+
+`pose` also takes a rule, a match function or a `defineBodyPose` definition:
+
+```tsx
+<BodyTrigger pose={{ arms: { left: "raised", right: "out" } }} onBegin={point} />
+<BodyTrigger pose={(shape) => shape.arms.left.up > 1.2} onBegin={reach} />
+```
+
+A pose begins once it has been held (the match is smoothed over about 80 ms, then `holdMs`, 150 by
+default) and ends when it is released. A one-frame flicker of the classifier neither ends nor
+restarts it. Pose defaults: `BODY_POSE_TRIGGER_DEFAULTS` (enter 0.6, exit 0.25, hold 150 ms, loss
+grace 250 ms).
+
+Poses read the same on portrait, landscape and square feeds. Tracker x and y each span the frame,
+so on a 9:16 portrait feed a horizontal distance reads 1.78 times too long next to a vertical one;
+the trigger multiplies x by the frame's aspect ratio (`useXRFeedAspect()`) before classifying.
+`handsUp` and `hooray` are the same pose: each answers the other.
+
+## Moves
+
+`move` takes a `BODY_MOVES` name (`BODY_MOVE_NAMES`) or your own `defineBodyMove` definition.
+
+| `move`                       | Begins                                                                                          | Ends                     | Needs      | `side` | Cooldown |
+| ---------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------ | ---------- | ------ | -------- |
+| `"jump"`                     | shoulders, hips and both feet rise together above the standing level (take-off)                 | on landing               | whole body | –      | 150 ms   |
+| `"squat"`                    | from standing, both thighs drop toward level (bottom of a rep)                                  | back up standing         | whole body | –      | 200 ms   |
+| `"wave"`                     | a raised forearm swings side to side (three turning points within 1.5 s)                        | when the swinging stops  | the arm    | yes    | 600 ms   |
+| `"clap"`                     | the hands meet fast (apart to touching within 0.25 s)                                           | when they part           | both arms  | –      | 100 ms   |
+| `"punch"`                    | an arm snaps from bent to straight (within 0.2 s) at shoulder height, to the camera or the side | when it bends back       | the arm    | yes    | 150 ms   |
+| `"armFlap"`                  | both arms, out like wings, beat down together                                                   | when the arms rise again | both arms  | –      | 100 ms   |
+| `"kick"`                     | one foot swings up fast with the knee lifted and the lower leg raised                           | when the foot is down    | whole body | yes    | 300 ms   |
+| `"stepLeft"` / `"stepRight"` | hips and both feet move to the person's own left / right by half a shoulder width within 0.7 s  | when the body settles    | whole body | –      | 300 ms   |
+| `"march"`                    | each alternating knee (or foot) lift; the first lift only starts the count                      | when that foot is down   | whole body | –      | 0        |
+
+Every move counts: `state.count` goes up on each begin (jumps, reps, claps, punches, flaps, kicks,
+steps, march steps). `onEnd` with `endReason: "released"` means the move completed. `state.side` is the
+limb behind the begin (`wave`, `punch`, `kick`, `march`) or the step direction.
+
+`side` (`"left"`, `"right"`, default `"either"`) picks the limb a `wave`, `punch` or `kick` watches.
+
+### Telling moves apart
+
+The moves are built not to answer each other: a squat is not a jump (the feet stay down), a march
+step is not a kick (the foot hangs under the knee), a heel kick is not a kick (the knee stays down),
+lowering the arms from the front is not a flap, dropping or raising a bent arm is not a punch, a slow
+reach is not a punch, hands brought together slowly are not a clap, walking toward the camera is not
+a step, and a single knee raise is not marching.
+
+Moves read body motion in the frame, so a camera that moves fast reads as motion too: a jerked
+handheld phone can look like a jump or a step. Slow drift never does. For leg moves, prop the phone up.
+
+## Whole body in view
+
+Leg moves need shoulders, hips, knees and ankles in view. A joint counts when MediaPipe's visibility
+is at least 0.2 and it sits no more than 0.12 past the frame edge (`BODY_MOVE_VIEW_LIMITS`; the
+frame spans −1..1). Both bars are deliberate: MediaPipe often reports 0.2 to 0.5 for ankles that are
+in frame, and an ankle can sit just past the edge when the feet are partly cropped. Legs cut off by a
+front camera held close sit far past the edge with visibility near 0, so leg moves never fire.
+Tune per trigger with `view`:
+
+```tsx
+<BodyTrigger move="jump" view={{ visible: 0.3, edge: 0.05 }} onBegin={hop} />
+```
+
+The leg poses (`squat`, `leanLeft`, `leanRight`, `oneLeg`, `star`) use BodyInstruction's own check
+(visibility at least `BODY_SHAPE_LIMITS.visible`, 0.5). The trigger reports what it needs:
+
+- `state.view`: `"full"` (needs the whole body) or `"upper"`.
+- `state.inView`: whether what it needs is in view now.
+
+```tsx
+const jump = useBodyTrigger({ move: "jump", onBegin: hop });
+// Show "Step back so your whole body is in view" while this is true.
+const needsStepBack = jump.view === "full" && !jump.inView;
+```
+
+Leaving view while a move is active is tracking loss: after `lostGraceMs` it ends with
+`endReason: "lost"`, and the move's memory starts fresh when the body is back.
+
+## Thresholds
+
+Arm moves measure in shoulder widths (like BodyInstruction's classifier). Leg moves measure vertical
+against vertical (torso length, the standing shin), so distance from the camera and the frame's aspect
+ratio cancel. Defaults are in `BODY_MOVE_LIMITS`; each move lists the keys it reads in
+`BODY_MOVES[name].limitKeys`. Override per trigger:
+
+```tsx
+<BodyTrigger move="jump" limits={{ jumpRise: 0.2 }} onBegin={hop} />
+<BodyTrigger move="squat" limits={{ squatDown: 0.75 }} onBegin={rep} /> {/* shallower squats count */}
+```
+
+| Key                                                               | Default              | Meaning                                                                                                   |
+| ----------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------------------------------------------- |
+| `jumpRise` / `jumpLand`                                           | 0.12 / 0.06          | take-off rise and landing tolerance, torso lengths                                                        |
+| `jumpMaxAirMs` / `jumpLevelMs`                                    | 900 / 300            | longest air time; how fast the standing level follows slow drift                                          |
+| `squatDown` / `squatUp`                                           | 0.6 / 0.85           | thigh height ÷ shin height at the bottom / back up (standing ≈ 1)                                         |
+| `waveForearmUp`, `waveAmplitude`, `waveReversals`, `waveWindowMs` | 0.25, 0.3, 3, 1500   | forearm raised; swing between turning points; turning points within the window                            |
+| `clapApart`, `clapTogether`, `clapWithinMs`, `clapRelease`        | 0.6, 0.3, 250, 0.45  | hands apart, then together within the time; parted again                                                  |
+| `punchBent`, `punchStraight`, `punchWithinMs`, `punchBand`        | 0.75, 0.93, 200, 0.9 | arm straightness (shoulder → wrist ÷ arm length, in 3D) from bent to straight; wrist near shoulder height |
+| `flapStroke`, `flapWithinMs`, `flapTop`, `flapOut`                | 0.7, 500, −0.4, 0.3  | both wrists drop this far, from a top near shoulder level, out to the sides                               |
+| `kickLift`, `kickKnee`, `kickShin`, `kickWithinMs`                | 0.35, 0.15, 0.6, 600 | foot and knee above the standing ones; the foot not hanging under the knee                                |
+| `stepShift`, `stepWithinMs`                                       | 0.5, 700             | sideways travel of hips and feet, shoulder widths                                                         |
+| `marchLift`, `marchDrop`, `marchWithinMs`                         | 0.2, 0.08, 1200      | knee or foot above the other one; alternation window                                                      |
+
+For poses, `limits` also takes BodyInstruction's classifier thresholds (`BODY_SHAPE_LIMITS` keys).
+The lifecycle options are the gesture engine's (`enterThreshold`, `exitThreshold`, `holdMs`,
+`cooldownMs`, `minConfidence`, `lostGraceMs`), per trigger.
+
+## Options
+
+`useBodyTrigger({ pose | move, … })` and `<BodyTrigger … />` take:
+
+- `pose` or `move` (exactly one).
+- `side`: `"either"` (default), `"left"`, `"right"`.
+- `limits`: `Partial<BodyMoveLimits & BodyShapeLimits>`.
+- `view`: `Partial<BodyMoveViewLimits>` (`{ visible, edge }`, the leg moves' view gate).
+- The lifecycle options above.
+- `bodyIndex` (default 0), `active` (default `true`; `false` pauses and releases the body model),
+  `targetFps`.
+- `onBegin`, `onEnd`, `onChange`: called from the frame loop with that frame's `BodyTriggerState`.
+
+`BodyTriggerState` is `GestureState` (`active`, `value`, `confidence`, `tracked`, `phase`,
+`justBegan`, `justEnded`, `endReason`, `startedAtMs`, `durationMs`, `count`) plus `kind`
+(`"pose"` | `"move"`), `view`, `inView` and `side`.
+
+## Custom moves
+
+```ts
+import { defineBodyMove } from "@vincentt-xr/sdk/tracking";
+
+// Both wrists above the head, then down below the hips within a second: a "touch the floor".
+const touchFloor = defineBodyMove({
+  name: "touchFloor",
+  label: "Touch the floor",
+  description: "Hands from overhead to below the knees within 1 s.",
+  view: "full",
+  createMemory: () => ({ upAt: -Infinity }),
+  measure: (frame, memory, nowMs) => {
+    const { joints: j } = frame;
+    if (j.leftWrist.y > j.nose.y && j.rightWrist.y > j.nose.y) memory.upAt = nowMs;
+    const down = j.leftWrist.y < j.leftKnee.y && j.rightWrist.y < j.rightKnee.y;
+    return { value: down && nowMs - memory.upAt < 1000 ? 1 : 0 };
+  },
+});
+
+<BodyTrigger move={touchFloor} onBegin={celebrate} />;
+```
+
+`measure(frame, memory, nowMs, { side, limits })` runs once per new tracker sample of a body in
+view and returns `{ value, side? }` (`value` ≥ 0.5 while the move is happening, for the default
+thresholds). `frame` (`BodyMoveFrame`) carries BodyInstruction's `shape`, the person's own `joints`
+(`joints.leftWrist` is their left wrist) in square units (x multiplied by the frame's aspect, so x and
+y compare), `visibility` and `seen` per joint (the view gate above), `shoulderWidth`, `torso`
+(vertical shoulder-to-hip length), `leftward` (the tracker-x direction of the person's left),
+`upperInView`, `fullInView` and `aspect`. Memory is recreated whenever the body is lost or leaves
+view. Define moves at module scope: a definition is keyed by its `name`.
+
+The pure layer is on `@vincentt-xr/sdk/low-level` for tests and custom runners:
+`createBodyMoveFrame(landmarks, { mirrored, visibility, aspect, view })`, `createBodyTrigger(target, options)`,
+`stepBodyTrigger(machine, frame | null, nowMs, fresh?)`, `snapshotBodyTriggerState`,
+`bodyTriggerStatesEqual`, `setBodyTriggerOptions`, `resolveBodyTriggerOptions`,
+`resolveBodyMoveLimits`, `BODY_POSE_TRIGGER_DEFAULTS`, `sideJoint` and `TimedSeries` (windowed
+min / max over a fixed ring, for custom moves).
+
+## Performance
+
+All body triggers read one shared body model; each tracker sample is analysed once and shared by
+every trigger reading it. Moves keep a short fixed-size history and measure only new tracker samples,
+not every rendered frame.
+
+## Manual validation
+
+Run `/gestures` (webcam by default; the fallback clip shows a full body). The body triggers section
+lists every pose and move with a light and, for moves, a count.
+
+| Case             | Steps                                                       | Expect                                                         |
+| ---------------- | ----------------------------------------------------------- | -------------------------------------------------------------- |
+| Poses            | hold a T-pose, hands up, hands on hips, arms crossed, squat | That row lights while held, off when released                  |
+| Jump             | stand back (whole body in view), jump three times           | `jump` ×3; no `squat`, `march` or `step`                       |
+| Squat reps       | three squats                                                | `squat` ×3, counted at the bottom                              |
+| Wave             | wave with your left hand, `side` = right                    | No count; set `side` = left, it counts with side `left`        |
+| Clap             | clap four times                                             | `clap` ×4; slowly pressing the palms together: no count        |
+| Punch            | jab at the camera with each hand                            | `punch` counts with the right side                             |
+| Arm flap         | flap both arms like wings                                   | `armFlap` counts each downstroke                               |
+| Kick, march      | kick once; then march in place                              | `kick` ×1; `march` counts steps, `kick` stays still            |
+| Steps            | step to your left, then to your right                       | `stepLeft` then `stepRight`, on the webcam and rear camera     |
+| Legs out of view | come close so only the upper body shows                     | Leg rows turn amber ("needs full body") and never count        |
+| Fallback clip    | "Person Fullbody" (portrait, standing still)                | No pose lit (not `squat`); leg rows in view, counts at 0       |
+| Feed shape       | switch the stage between portrait and landscape             | The same poses light; the "leg view gate" readout says in view |

package/docs/api/camera-permission.mdx

@@ -128,7 +128,7 @@ next to `XRErrorReason`, for callers that classify their own
 
 ## Manual validation
 
-Run `/camera-permission`:
+Run `/camera`:
 
 | Case         | Steps                                     | Expect                                                            |
 | ------------ | ----------------------------------------- | ----------------------------------------------------------------- |

package/docs/api/coordinate-spaces.mdx

@@ -80,10 +80,13 @@ less than the frame (zoom > 1 never does; a contain-shaped source does);
 
 - Positions never need a flip: landmarks already live in the (possibly
   mirrored) feed, and the feed is what you see.
-- **Direction signs** do. Holistic hand labels and the iris `offset.x` are
-  corrected with `isMirroredSourceKind(sourceKind)` so "the person's left"
-  means the same thing on every source. Use the same helper, never
-  `sourceKind === "webcam"` inline.
+- **Direction signs and side names** do. Every SDK left / right is the
+  person's own side on every source (GROUNDING "Left and right"); MediaPipe's
+  own sides (landmark indices, labels, blendshape names) are swapped on the
+  mirrored webcam. The SDK corrects them with `isMirroredSourceKind(sourceKind)`
+  / `useXRMirrored()`; for your own reads use `personSide(side, mirrored)`,
+  `feedSidedName(name, mirrored)` and `personLeftSign(mirrored)` from
+  `@vincentt-xr/sdk`, never `sourceKind === "webcam"` inline.
 - `mirrorFeedX` flips a feed point's x (for a direction vector, or to compare
   a selfie feed with a straight recording).
 

package/docs/api/cross-camera.mdx

@@ -50,7 +50,8 @@ unprojected to the scene lands where the preview shows it on every camera.
 Consumers never flip positions; `useXRMirrored()` (low-level, the same
 answer as `isMirroredSourceKind(sourceKind)`) is for _direction_ semantics
 only, such as which hand is the person's left. Every tracker and the
-coordinate toolkit already key off the kind.
+coordinate toolkit already key off the kind, so every SDK left / right is the
+person's own side on every camera (GROUNDING "Left and right").
 
 ## Orientation
 
@@ -76,7 +77,7 @@ on the face through a rotation and a camera switch. Use
 
 ## Manual validation
 
-Run `/camera-switching` on a phone:
+Run `/camera` on a phone:
 
 | Case                | Steps                               | Expect                                                                 |
 | ------------------- | ----------------------------------- | ---------------------------------------------------------------------- |

package/docs/api/debug-ui-scene-editor.mdx

@@ -32,11 +32,10 @@ from final preview/runtime surfaces.
 
 ## In-repo examples
 
-- `/viewport-editor`: general 3D scene-object editor.
 - `/unified-architecture`: unified face-tracking scene using
   `XRSceneEditorProvider`, `XRSceneViewport`, hierarchy, inspector, toolbar, and
   media switching.
-- `/testing-hook`: multi-face face mesh setup backed by the same reusable
+- `/face-mesh?faces=two`: multi-face face mesh setup backed by the same reusable
   editor primitives.
 - `/debugger`: debugger overlay mounted beside a minimal tracked scene.
 
@@ -129,6 +128,28 @@ It renders sections for components present on the selected object:
 
 All edits route through `onComponentChange` or `onObjectChange`.
 
+## Docked windows
+
+The hierarchy and inspector float by default: draggable glass cards over the
+page. To put them in your own layout — a side panel or a settings drawer — pass
+`docked`. A docked window renders in place: no portal, no fixed position, no
+dragging. It takes the container's width, grows with its content, and drops the
+glass card so it blends into the host.
+
+```tsx
+<XRSceneEditorProvider defaultStore={store}>
+  <aside className="my-side-panel">
+    <SceneHierarchyPanel docked />
+    <SceneInspectorPanel docked />
+  </aside>
+  <XRSceneViewport />
+</XRSceneEditorProvider>
+```
+
+The provider must wrap the layout that hosts the docked panels. `docked` also
+works on `HierarchyPanel`, `ObjectInspectorPanel`, `ScreenImageSettingsPanel`,
+`ScreenTextSettingsPanel`, `ProductHotspotPanel` and `XRDiagnosticsPanel`.
+
 ## `ViewportToolbar`
 
 Small optional toolbar for transient viewport controls.

package/docs/api/external-face-mesh.mdx

@@ -0,0 +1,231 @@
+---
+title: "External Face Mesh"
+description: "Wear your own head or face model on the tracked face: it follows the head pose and the face's expression, with any topology and UV layout."
+sidebarTitle: "External Face Mesh"
+icon: "masks-theater"
+---
+
+`ExternalFaceMesh` puts your own face model on the tracked face. It follows the head pose and the
+face's expression in two ways at once:
+
+- **Landmarks:** the tracked landmarks bend the mesh, so the jaw opens and the lips and cheeks move.
+- **Morphs:** the model's own ARKit-named morph targets play the face's blendshapes. That gives an
+  authored blink and raised brows, and the eyeballs turn with the gaze.
+
+It is the foundation of the face-character pipeline
+([#194](https://github.com/vincentt-xr/sdk/issues/194)). Expression controllers, pinned accessories
+and face reprojection build on the same `FaceRig`.
+
+Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { useLoader } from "@react-three/fiber";
+import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js";
+import { FaceTracker } from "@vincentt-xr/sdk/tracking";
+import { ExternalFaceMesh } from "@vincentt-xr/sdk/face-effects";
+
+const Character = () => {
+  const head = useLoader(GLTFLoader, headUrl); // authored in canonical centimetres
+  return (
+    <FaceTracker>
+      <ExternalFaceMesh model={head.scene} />
+    </FaceTracker>
+  );
+};
+```
+
+It reads the parent `FaceTracker`'s face (or a `HeadBinding`'s) and never acquires the face model
+itself. The model is cloned with its geometry, so one loaded model can back several faces.
+
+## Authoring the model
+
+Author the model in the **canonical head frame**, the same frame as `CanonicalHeadFrame` and the v3
+head occluder `.blend` in `assets-src/head-occluder/`:
+
+- **Units and axes:** centimetres, +Y up, +Z out of the face toward the camera.
+- **Origin:** landmark 4, just above the nose tip, sits at `(0, -0.46, 7.59)`.
+
+The easiest way is to open the occluder `.blend` and fit your head over it. Its first 468 vertices
+are the tracked face landmarks, so the eyes, nose and mouth line up when your model sits on them.
+
+Any topology and any UV layout works. The face area should lie close to the canonical face surface:
+the closer it is, the more of the face follows the expression. The rest (skull, ears, neck,
+accessories) can go anywhere and stays rigid.
+
+## How it follows the face
+
+**What drives it** is set by `drive`:
+
+| `drive`            | What moves it                                                                                                    |
+| ------------------ | ---------------------------------------------------------------------------------------------------------------- |
+| `"auto"` (default) | `"morphs"` for a full ARKit set (mouth morphs too), `"hybrid"` with only eye or brow morphs, else `"landmarks"`. |
+| `"hybrid"`         | Landmarks for the jaw, lips and cheeks; the model's morphs for the eyes and brows.                               |
+| `"morphs"`         | Only the model's morph targets, driven by the blendshapes; otherwise just the head pose.                         |
+| `"landmarks"`      | Only the landmarks bend the mesh.                                                                                |
+| `"none"`           | The head pose only.                                                                                              |
+
+For a model with a full ARKit set, such as Face Cap's, the authored morphs look best, so `"auto"`
+picks `"morphs"`. Binding then plays no part, because nothing is bent.
+
+Why hybrid for eye and brow morphs: MediaPipe only tracks the edge of each eyelid, 16 points around each eye. Bending a
+sculpted lid with those points drags its edge down without rolling the lid over the eyeball. That
+leaves slits at the corners and creases the lid. An authored `eyeBlink` morph closes it cleanly, so
+hybrid masks the eye and brow landmarks (`FACE_REGION_MASKS`) and lets the morphs move those parts.
+
+**Morph names** map to MediaPipe's 51 blendshape channels (`FACE_MORPH_NAMES`, ARKit names). The
+match accepts:
+
+- ARKit names, e.g. `eyeBlinkLeft`;
+- Face Cap and Blender styles, e.g. `eyeBlink_L`, `eyeBlink.L`, `eyeBlink_left`;
+- any case;
+- exporter prefixes such as `blendShape1.`.
+
+`Left` is the model's own left (landmark 263's side, MediaPipe's left), as in ARKit. It is driven
+mesh to mesh, so the model's eye in front of the person's blinking eye blinks on every feed (on the
+mirrored webcam that is MediaPipe's right channel for the person's real left eye; GROUNDING "Left
+and right"). MediaPipe has no `tongueOut`.
+Morphs that no channel drives are left alone, so your own animation keeps them. The model's rest pose
+must have every morph at 0.
+
+**Binding** happens once, at load. Each vertex is bound to a point on the canonical face surface,
+which is MediaPipe's 853 face triangles. The mouth and eye openings are holes, so a lip never binds
+across the mouth. `binding` picks how:
+
+| `binding`          | Binds by                                                                             |
+| ------------------ | ------------------------------------------------------------------------------------ |
+| `"auto"` (default) | UV when the mesh's UVs are a canonical layout (most UV hits agree with 3D), else 3D. |
+| `"uv"`             | UV only, for a mesh whose face UVs follow `CANONICAL_FACE_UVS` (either v direction). |
+| `"surface"`        | The closest point in 3D. Any topology and layout.                                    |
+
+**Influence** depends on how far a vertex sits from the face surface. Within `falloff[0]` (0.8 cm)
+it follows fully; it fades to rigid at `falloff[1]` (2.5 cm). The back of the head, the ears and the
+neck therefore stay rigid.
+
+**Parts:** each mesh of the model follows in one of three ways (`parts`, by object name):
+
+- `"surface"`: every vertex follows its own point. Use it for skin.
+- `"follow"`: the whole mesh moves as one with the point under its centre, keeping its shape. Use
+  it for eyeballs, teeth or a nose ring. A follow part named like an eye (`eye…`) also turns with
+  the gaze (`eyeLookIn/Out/Up/Down`, up to `gazeDegrees` = `[25, 20]`).
+- `"rigid"`: head pose only. Use it for a helmet or hair.
+
+By default, a mesh whose median vertex sits more than 0.3 cm behind the face (eyeballs, teeth) is
+`"follow"`, and everything else is `"surface"`.
+
+**Neutral.** Only the expression moves the model, not the person's own face shape or resting
+expression. With `neutral: "auto"` (default), the rig averages the landmarks and the blendshape
+scores over the first `neutralMs` (600 ms) after the face is found, and subtracts that rest face. The
+blendshapes are rescaled, so a resting 0.1 reads 0 and a full blink still reads 1. It recaptures every time the face is found again, and when
+`neutralKey` changes: bind it to a "Recalibrate" button and ask the person to relax and look at the
+camera. With `neutral: "none"` the model also takes on the person's face shape (wider eyes, a longer
+jaw).
+
+## `ExternalFaceMesh`
+
+| Prop                  | Default       | What it does                                                                                |
+| --------------------- | ------------- | ------------------------------------------------------------------------------------------- |
+| `model`               | —             | The face model (e.g. a GLTF scene) in the canonical head frame.                             |
+| `drive`               | `"auto"`      | What moves it: `"auto"`, `"hybrid"`, `"morphs"`, `"landmarks"` or `"none"`.                 |
+| `binding`             | `"auto"`      | `"auto"`, `"uv"` or `"surface"`.                                                            |
+| `falloff`             | `[0.8, 2.5]`  | Full influence up to the first distance (cm), rigid beyond the second.                      |
+| `parts`               | auto          | Per mesh name: `"surface"`, `"follow"` or `"rigid"`.                                        |
+| `expression`          | `1`           | 0 = rigid, 1 = the tracked face, > 1 exaggerates.                                           |
+| `morphGain`           | —             | A gain per channel on its morph, e.g. `{ eyeBlinkLeft: 1.2 }`.                              |
+| `gazeDegrees`         | `[25, 20]`    | How far a full look turns an eyeball: sideways, up/down (degrees).                          |
+| `onBind`              | —             | Called once per model with every mesh's binding (readouts, debug views).                    |
+| `faceIndex`           | the tracker's | Which tracked face (when not inside a `FaceRig`; likewise the rows below).                  |
+| `smoothing`           | `0.5`         | Head pose EMA while still (`1` = none). Fast turns follow at once, so a shake never trails. |
+| `expressionSmoothing` | `0.6`         | Expression EMA, `1` = none. MediaPipe's depth is noisy; a little keeps the mesh calm.       |
+| `holdMs`              | `250`         | Keep the last pose and expression through shorter dropouts.                                 |
+| `neutral`             | `"auto"`      | `"auto"` subtracts the captured rest face; `"none"` keeps the person's face shape.          |
+| `neutralMs`           | `600`         | How long the neutral capture averages.                                                      |
+| `neutralKey`          | —             | A new value captures the neutral again.                                                     |
+| `leadMs`              | `0`           | Landmark lead between inferences (One Euro + velocity, at most this long). 0 = raw.         |
+| `onTrackingChange`    | —             | Called once on found (`true`) and once on lost (`false`).                                   |
+| `children`            | —             | More content in the canonical head frame.                                                   |
+
+## `FaceRig`
+
+`FaceRig` is the engine underneath, and every face-character component shares one. It is a group in
+the canonical head frame, like `CanonicalHeadFrame`, that also publishes the face's expression. Put
+several components inside one `FaceRig` to share it:
+
+```tsx
+<FaceTracker>
+  <FaceRig neutralKey={recalibrations}>
+    <ExternalFaceMesh model={head.scene} />
+    <ExternalFaceMesh model={beard.scene} />
+  </FaceRig>
+</FaceTracker>
+```
+
+`useFaceRig()` returns a ref to the live `FaceRigState`. Read it inside `useFrame`:
+
+- `tracking`: `"tracked"`, `"held"` or `"lost"`.
+- `pose`: `{ position, quaternion, scale }`, canonical cm → world.
+- `local`: the 468 landmarks in the head frame (cm).
+- `residual`: the expression, as each landmark's offset from the rest face (cm, 468 × 3).
+- `morphs`: the 51 blendshape channels (0..1, neutral removed).
+- `neutral`, `morphNeutral` and `neutralState`.
+
+The rig holds its last state through dropouts shorter than `holdMs`, hides when the face is lost,
+and snaps on reacquire.
+
+The pure pieces are exported for custom renderers:
+
+- `stepFaceRig(state, worldLandmarks, nowMs, options, scores)`, `createFaceRigState()` and
+  `recalibrateFaceRig(state)`.
+- `bindFaceMesh(positionsCm, uvs, { mode, falloff, uvTolerance })` and
+  `deformFaceMesh(binding, rest, residual, out, amount, toLocal)`.
+- `bindFaceModel(root, options)`, `planFaceDrive(meshes, mode)` and
+  `driveFaceModel(meshes, plan, rig, options)`.
+- `faceMorphChannel(name)`, `faceMorphRegion`, `FACE_GAZE_CHANNELS` and `FACE_REGION_MASKS`.
+- The canonical data: `CANONICAL_FACE_POSITIONS`, `CANONICAL_FACE_TRIANGLES` and
+  `CANONICAL_FACE_UVS`.
+
+## Example
+
+`/external-face-mesh` wears the three.js "facecap" head when it is present. The file goes in the
+git-ignored `example/src/assets/models/local/facecap-canonical.glb`; it has no redistribution
+licence, so it is never committed. That copy is prepared as follows:
+
+- aligned to the canonical frame;
+- ears raised to the anatomical height (brow to nose base);
+- every morph zeroed at rest (the original kept a frame of its animation).
+
+It carries all 52 ARKit morphs, of which 51 are tracked. Without it, the page uses the canonical
+head.
+
+The page has controls for:
+
+- drive, binding, expression and falloff;
+- a binding heat map (red follows, blue rigid);
+- keeping the face shape;
+- recalibrating the neutral;
+- the head occluder and two faces.
+
+## Manual visual test plan
+
+Open `/external-face-mesh` on the webcam.
+
+1. **Placement**: the model sits on your head, with its eyes, nose and mouth on yours, and stays
+   there as you turn and tilt.
+2. **Neutral**: look at the camera relaxed when it appears. At rest it keeps its own shape: eyes
+   not stretched, jaw not lengthened. Tick "keep the person's face shape" and it conforms to yours;
+   untick it.
+3. **Expression**: open your mouth (the jaw drops and the lips part, never stretching across), smile
+   (the mouth corners and cheeks move), raise your brows. With drive "none", nothing moves but the
+   pose.
+4. **Blink and gaze** (facecap, drive "auto" = morphs): blink one eye, then both. The lid rolls down over the
+   eye and closes, with no slit at the corners and no crease. Look left, right, up and down: the
+   eyeballs turn with you. Open your mouth and smile: facecap's own jawOpen and smile shapes play.
+   Switch drive to "landmarks" to see the difference.
+5. **Parts**: the eyes and teeth move as whole pieces, never squashed. The back of the head and the
+   ears stay rigid; the heat map shows red on the face fading to blue.
+6. **Recalibrate**: start with your mouth open, then relax and press Recalibrate. The rest pose is
+   corrected.
+7. **Dropouts**: a brief cover of the camera holds the head. Longer, and it hides, then snaps back
+   on reacquire.
+8. **Two faces**: on the two-people image, each face wears its own copy.

package/docs/api/face-actions.mdx

@@ -37,21 +37,23 @@ const isPouting = hasFaceAction(face, "mouthPout");
 
 ## Actions
 
-| Name                            | Meaning                      |
-| ------------------------------- | ---------------------------- |
-| `browRaise`                     | Either brow is raised        |
-| `eyeBlink`                      | Both eyes blink              |
-| `eyeBlinkEither`                | Either eye blinks            |
-| `eyeBlinkLeft`, `eyeBlinkRight` | Viewer-left/right eye blinks |
-| `mouthOpen`                     | Jaw opens                    |
-| `mouthPout`                     | Lips pucker                  |
-| `headUp`, `headDown`            | Pitch from neutral           |
-| `headTurnLeft`, `headTurnRight` | Yaw from neutral             |
-| `headTiltLeft`, `headTiltRight` | Roll from neutral            |
+| Name                            | Meaning                                                          |
+| ------------------------------- | ---------------------------------------------------------------- |
+| `browRaise`                     | Either brow is raised                                            |
+| `eyeBlink`                      | Both eyes blink                                                  |
+| `eyeBlinkEither`                | Either eye blinks                                                |
+| `eyeBlinkLeft`, `eyeBlinkRight` | The person's own left / right eye blinks                         |
+| `mouthOpen`                     | Jaw opens                                                        |
+| `mouthPout`                     | Lips pucker                                                      |
+| `headUp`, `headDown`            | Pitch from neutral                                               |
+| `headTurnLeft`, `headTurnRight` | Yaw from neutral, toward the person's own left / right           |
+| `headTiltLeft`, `headTiltRight` | Roll from neutral, toward the person's own left / right shoulder |
 
 Blendshape values are in `[0, 1]`. Head values are non-negative radians. Up and
-left follow the SDK convention and are positive before they are converted to an
-action magnitude.
+the person's own left follow the SDK convention and are positive before they
+are converted to an action magnitude. Left and right are the person's own side
+on every feed: the mirrored webcam, the rear camera, images and video
+(GROUNDING "Left and right").
 
 ## Neutral head pose
 
@@ -89,15 +91,17 @@ left, right, both, and either-eye movement cases. Use `mouthOpen`, `mouthPout`,
 and `browRaise` for the matching physical movements. These actions report
 observable pose/blendshape movement; they do not infer emotion.
 
-Left/right blink names use the mirrored viewer-facing convention used by effect
-editors. Raw `FaceInfo.blendshapes` retain MediaPipe's anatomical-side names.
+Left/right blink names are the person's own eyes on every feed. Raw
+`FaceInfo.blendshapes` keep MediaPipe's names (`eyeBlinkLeft` = MediaPipe's left
+eye: the person's own left on an unmirrored feed, their right on the mirrored
+webcam); the actions read the right one for `face.mirrored`.
 The lifecycle guarantees one `stay` frame between `begin` and `end`, including
 for a blink that drops below its release threshold on the next input frame.
 
 For image-based happy, sad, angry, fear, disgust, surprise, neutral, and unknown
 classification, use the separate `useFaceAttributeExpression` hook. The SDK
 does not fabricate these labels from movement thresholds. The verified
-`/facial-movement-expression` page exposes both independent surfaces.
+`/face-actions` page exposes both independent surfaces.
 
 Use `/face-actions` to view every value, calibrate neutral pose, adjust the
 selected action, and repeat the live verification checklist. All actions and

package/docs/api/face-attribute-expression.mdx

@@ -82,6 +82,6 @@ Without those headers, ONNX Runtime falls back to single-threaded WASM.
 Treat classifications as model estimates of visible facial expression—not
 ground-truth emotion, intent, identity, or mental state.
 
-Use the verified `/facial-movement-expression` page to inspect the live winning
+Use the verified `/face-actions` page to inspect the live winning
 class and all seven probabilities. Its real-camera model flow, including every
 expression class, was verified on 26 July 2026.

package/docs/api/face-binding-anchors.mdx

@@ -7,7 +7,9 @@ icon: "locate-fixed"
 
 Use `TrackingAnchor` to attach content to semantic face locations. Build editor
 dropdowns from `FACE_TARGETS`; do not copy landmark indices or invent target
-strings. Left and right use the tracked person's perspective.
+strings. Left and right are the tracked person's own side on every feed (GROUNDING "Left and
+right"): `face.leftEarlobe` is their real left ear lobe on the mirrored webcam (the screen's left)
+and on a video (the screen's right).
 
 ## Presets
 
@@ -33,7 +35,7 @@ does not provide anatomical earlobe points.
 ## Custom points
 
 For `face.custom`, pass `customAnchorPoint`. X and Y are clamped to `[-1, 1]`;
-the origin is face center, X runs from the person's left to right, Y runs from
+the origin is face center, X runs from the person's own left to right on every feed, Y runs from
 chin to forehead, and positive Z points toward the camera.
 
 ```tsx