@vincentt-xr/sdk 2.0.0-alpha.7 → 2.0.0-alpha.9

package/docs/api/face-swap.mdx

@@ -0,0 +1,350 @@
+---
+title: "Face Swap"
+description: "Two people swap faces, a photo or a painted face goes on every face, or your own face is mirrored, live on the camera, in the wearer's skin tone and light."
+sidebarTitle: "Face Swap"
+icon: "people-arrows"
+---
+
+`FaceSwap` and `FaceMirror` draw a face on a tracked face, live on the camera:
+
+- **Two people swap faces.** It starts by itself when two faces are in view.
+- **A photo or a painted face** goes on every face (or only one).
+- **Your own face, mirrored:** one half doubled, the halves swapped, or the eyes and mouth upside
+  down.
+
+The drawn face moves with the wearer's expression, takes the wearer's skin tone and light, and
+fades in centimetres inside the face edge. Both components start the face model themselves.
+Import from `@vincentt-xr/sdk/face-effects` ([#275](https://github.com/vincentt-xr/sdk/issues/275)).
+
+## Quick start
+
+```tsx
+import { VideoBackground, XRProvider, XRScene } from "@vincentt-xr/sdk";
+import { FaceMirror, FaceSwap } from "@vincentt-xr/sdk/face-effects";
+
+<XRProvider>
+  <XRScene>
+    <VideoBackground />
+    {/* Two people swap faces automatically; with one face the camera shows as is */}
+    <FaceSwap onStatus={(s) => setSwapping(s.active)} />
+  </XRScene>
+</XRProvider>;
+
+// Any photo: File, Blob, URL, <img>, canvas or ImageBitmap
+<FaceSwap source={{ kind: "photo", src: file }} onStatus={(s) => setWarnings(s.warnings)} />;
+// A face painted in the face atlas: keeps its paint, takes the room's light
+<FaceSwap source={{ kind: "canonicalTexture", src: paintedUrl }} />;
+// Your own face, the left half twice
+<FaceMirror variant="leftLeft" />;
+```
+
+## Which component
+
+| Goal                                                           | Use                                                                      |
+| -------------------------------------------------------------- | ------------------------------------------------------------------------ |
+| Two people swap faces, or a photo / painted face on every face | `FaceSwap`                                                               |
+| Your own face mirrored                                         | `FaceMirror variant`                                                     |
+| Your own face texture on a live face (a custom effect)         | `FaceTextureLayer face`, fed by `useFaceTexture` or `useLiveFaceTexture` |
+| A face drawn into a picture or a render target, or no React    | `FaceTransfer` with `createFaceTexture`                                  |
+
+**Mounting.** Mount them anywhere in `<XRScene>`, also inside a `<FaceTracker>`. They draw in
+screen space and need no `FaceRig`. With no `FaceTracker` above, they start the face model
+themselves: `FaceSwap` with two faces for a two-person swap, otherwise one. The first component to
+start the model sets its face count, so anything else that may start it first needs the same
+`numFaces`. A console warning names the fix when a face the component needs can never be tracked.
+Keep hands in front with `<HandTracker><HandCutout /></HandTracker>`.
+
+## FaceSwap
+
+### Sources
+
+| `source`                                             | What is drawn                                                                                                      |
+| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
+| `{ kind: "eachOther" }` (default)                    | Faces 0 and 1 swap.                                                                                                |
+| `{ kind: "photo", src, mirrored?, pick?, maxSize? }` | The face found in a photo, on every tracked face. `pick`: `"largest"` (default) or the n-th face from the left.    |
+| `{ kind: "canonicalTexture", src, mirrored? }`       | A face painted in the face atlas (a URL or a `THREE.Texture`), on every tracked face.                              |
+| `{ kind: "faceTexture", face }`                      | A `FaceTexture` you made (`useFaceTexture`, `createFaceTexture`, `useLiveFaceTexture`); it stays yours to dispose. |
+
+`mirrored` says the image itself is mirrored (some phones save selfies mirrored). A mirrored selfie
+cannot be detected from the pixels, so it is your app's toggle.
+
+### Props
+
+Every setting below is also a prop, flat (`<FaceSwap matchLighting={0.5} />`), plus:
+
+| Prop        | Default                  | What it does                                                                        |
+| ----------- | ------------------------ | ----------------------------------------------------------------------------------- |
+| `source`    | `{ kind: "eachOther" }`  | The face drawn (above).                                                             |
+| `amount`    | `1`                      | 0..1, or a ref read every frame, so it can fade without re-rendering.               |
+| `faceIndex` | every tracked face       | Photo, painted and `faceTexture` sources: only this face. Ignored for two people.   |
+| `numFaces`  | 2 for two people, else 1 | Faces to start the face model with.                                                 |
+| `quality`   | `"auto"`                 | `"auto"` follows the session's quality tier; or `"high"`, `"medium"`, `"low"`.      |
+| `active`    | `true`                   | `false` hides it and stops its passes and its face model, but keeps a loaded photo. |
+| `onStatus`  |                          | Called whenever the status changes (not every frame).                               |
+
+The status is `{ active, faces, source, warnings, error }`:
+
+- `active`: drawn now (fading in, shown or fading out);
+- `faces`: faces in the newest face result;
+- `source`: `"waiting"` (no source yet, or fewer than two ready faces for a two-person swap),
+  `"loading"`, `"ready"` or `"error"`;
+- `warnings`: for photos, any of `turned` (head turned more than 25°), `eyesClosed`, `mouthOpen`,
+  `small` (eyes less than 60 px apart) and `severalFaces`;
+- `error`: `{ code, message }` with code `noFace`, `decode`, `cors`, `model` or `aborted`.
+
+### Behaviour
+
+- **Two people.** The swap starts once both faces have been tracked for 5 face results and fades
+  in over 200 ms. It holds through a dropout of up to 250 ms, then fades back to the plain camera
+  (also after 1 s with no face results at all). MediaPipe reordering the faces changes nothing.
+  Where the faces overlap, the larger face draws on top. With three or more faces, only faces 0 and
+  1 swap.
+- **Photos and painted faces.** Each tracked face fades in with its first result and out 250 ms
+  after it is lost.
+- **Own eyes and mouth.** By default the wearer keeps their own live eyes and mouth inside the drawn
+  face (`ownEyes`, `ownMouth`), so blinks, gaze, teeth and tongue stay real. Turn them off for the
+  source's own eyes and mouth: a photo's or painting's stay still.
+- **Expression.** The wearer's expression moves the drawn face. A still face has a neutral, closed
+  mouth, so its lips stretch when the wearer talks.
+
+## Settings and presets
+
+| Field             | Range    | Default   | What it does                                                                              |
+| ----------------- | -------- | --------- | ----------------------------------------------------------------------------------------- |
+| `preset`          |          | `natural` | `natural`, `keepColors`, `raw`, or `"custom"`.                                            |
+| `matchSkinTone`   | 0..1     | 1         | The drawn face takes the wearer's skin tone, exposure and white balance.                  |
+| `matchLighting`   | 0..1     | 1         | The wearer's light and shadow fall on the drawn face.                                     |
+| `edgeMatch`       | 0..1     | 1         | The band inside the face edge always takes the wearer's tone and light, so no seam shows. |
+| `edgeSoftness`    | 0.3..3   | 1.2       | Width of the fade inside the face edge, cm.                                               |
+| `edgeInset`       | 0..1.5   | 0.3       | Pulls the edge inside the tracked outline, cm.                                            |
+| `ownEyes`         | boolean  | true      | The wearer keeps their live eyes.                                                         |
+| `ownMouth`        | boolean  | true      | The wearer keeps their live mouth interior.                                               |
+| `featureSoftness` | 0.1..0.6 | 0.25      | Soft rim around the kept eyes and mouth, cm (0.1 for painted faces).                      |
+| `detail`          | 0..1     | 1         | Sharpness of the drawn face; lower softens it to match a soft camera.                     |
+| `smoothing`       | 0..1     | 0.35      | Landmark smoothing. Shared per face: the most any component asks for wins.                |
+| `amount`          | 0..1     | 1         | Overall strength.                                                                         |
+
+| Preset       | For                       | Sets                                                                         |
+| ------------ | ------------------------- | ---------------------------------------------------------------------------- |
+| `natural`    | photos and two people     | Full skin-tone and light match, 1.2 cm edge, own eyes and mouth.             |
+| `keepColors` | painted or stylised faces | No skin-tone match, 0.7 light match, 1 cm edge: the paint keeps its colours. |
+| `raw`        | a mask look, or debugging | No matching, a short 0.4 cm edge: the face as captured.                      |
+
+- Without `preset`, a painted face starts from `keepColors` with a 0.1 cm feature rim; photos and
+  two people start from `natural`.
+- A named preset always means its values: an explicit value that moves a field the preset sets
+  makes it `"custom"`. Smoothing, inset, feature softness and amount are never set by a preset.
+- `resolveFaceSwapSettings(input, { source })` resolves props (defaults, then the preset, then your
+  values, clamped). `patchFaceSwapSettings(prev, patch)` applies a panel edit: choosing a preset
+  applies it, editing a field it sets makes it `"custom"`. `FACE_SWAP_SETTINGS_FIELDS` drives a
+  settings panel; `FaceSwapSettingsPanel` from `@vincentt-xr/sdk/debug-ui` is a ready one.
+
+## Colour and edges
+
+**Colour.** The drawn face is multiplied by a gain per colour channel, in linear light, computed per
+vertex:
+
+- **Skin tone** (`matchSkinTone`): the ratio of the wearer's mean skin colour to the source's. It
+  carries skin tone, exposure and white balance.
+- **Light** (`matchLighting`): the ratio of the wearer's local, low-frequency skin colour to the
+  source's, relative to their means. It is applied at full strength for brightness and half
+  strength for colour, so coloured light does not tint the skin. A painted face only takes the
+  wearer's light: its own painted shading keeps its contrast.
+- The gain is clamped to 0.25–4.
+
+The statistics are skin only: brows, eyes, lips and nostrils are masked out, and outliers
+(speculars, beard, glasses frames) are rejected. They are blurred to 64² and smoothed over 250 ms,
+so a flicker in one frame never shows.
+
+**Edges.** Every distance is in cm on the face, measured from a smoothed face outline, so the fade
+lines are smooth curves. The alpha is the product of:
+
+- the edge fade: from `edgeInset` inside the outline, over `edgeSoftness`;
+- how squarely the wearer's face points at the camera there;
+- how much of the source was seen there (a photo's turned-away cheek is filled from the other side
+  when it is baked);
+- soft rims of `featureSoftness` around the kept eyes and mouth.
+
+Inside the fade, a band twice as wide always takes the wearer's tone and light (`edgeMatch`), so
+the drawn face meets the real skin with no seam even when `matchSkinTone` is 0.
+
+## FaceMirror
+
+```tsx
+<FaceMirror variant="leftLeft" />   // both halves are your own LEFT half
+<FaceMirror variant="rightRight" /> // both halves are your own RIGHT half
+<FaceMirror variant="flip" />       // left and right swapped: a left wink shows on the right
+<FaceMirror variant="upsideDown" /> // each eye and the mouth flipped top to bottom in place
+```
+
+| Prop            | Default            | What it does                                                                 |
+| --------------- | ------------------ | ---------------------------------------------------------------------------- |
+| `variant`       | required           | `leftLeft`, `rightRight`, `flip` or `upsideDown`.                            |
+| `matchLighting` | `1`                | The real side light stays where it falls; `0` shows the raw mirrored pixels. |
+| `edgeSoftness`  | `1.2`              | Width of the fade inside the face edge, cm.                                  |
+| `smoothing`     | `0.35`             | Landmark smoothing.                                                          |
+| `amount`        | `1`                | 0..1, or a ref.                                                              |
+| `faceIndex`     | every tracked face | Only this face.                                                              |
+| `numFaces`      | `1`                | Faces to start the face model with.                                          |
+| `quality`       | `"auto"`           | As for `FaceSwap`.                                                           |
+| `active`        | `true`             | `false` hides it and stops its passes.                                       |
+
+- Left and right are the person's own sides on the mirrored webcam, the rear camera and videos
+  alike.
+- The eyes and mouth are part of every variant: they always come from the mirrored face, so a left
+  wink in `leftLeft` winks both eyes.
+- Where the far side of a turned face was not seen, the real face shows.
+
+## Sides
+
+Left and right always mean the person's own side. A face texture's `mirrored` flag says which side
+of the canonical face is the person's left: `true` for a live face on the mirrored webcam or a
+selfie saved mirrored, `false` for a photo as stored, a painted face, and a live face on the rear
+camera or a video. When a face is drawn on another, the sides are matched through that flag and the
+exact landmark mirror map (`FACE_UNWRAP_MIRROR`, `resolveFaceSourceMirror`), never by flipping the
+texture. A photo's left cheek lands on the wearer's left cheek on every feed.
+
+## Face textures and FaceTextureLayer
+
+A `FaceTexture` is one face laid flat in the face atlas (`FACE_ATLAS_UVS`): its `map`, its skin
+statistics `low` and `mean`, `mirrored`, and for photos the detected `landmarks`.
+
+```tsx
+import {
+  FaceTextureLayer,
+  useFaceTexture,
+  useLiveFaceTexture,
+} from "@vincentt-xr/sdk/face-effects";
+
+// Inside <XRScene>: these need the renderer.
+const photo = useFaceTexture({ kind: "photo", src: file }); // { face, status, warnings, error, faceCount }
+const second = useLiveFaceTexture(1); // stable, updated in place; null until made
+
+<FaceTextureLayer face={photo.face} />;
+<FaceTextureLayer face={second} faceIndex={0} />;
+```
+
+- `useFaceTexture(source)` bakes a photo or painted face once per URL or object, aborts a stale load
+  and disposes the face on change or unmount.
+- `useLiveFaceTexture(faceIndex)` is shared and refcounted per face: every component asking for the
+  same face shares one unwrap and one set of statistics. It updates only when a new face result
+  arrives, from smoothed landmarks, so it never swims.
+- `FaceTextureLayer` draws any face texture on a tracked face with the same settings as `FaceSwap`,
+  plus `layout` (`asIs`, `flip`, `leftLeft`, `rightRight`, `upsideDown`) and `targetStats`. It starts
+  the face model one commit after it mounts, so `useLiveFaceTexture(1)` above it starts the model
+  with two faces.
+- Without React: `createFaceTexture(renderer, source)` bakes a face (you dispose it), and
+  `FaceTransfer` draws it: `setTarget(landmarks, aspect)` or a live map, `setSource`,
+  `setTargetStats`, `setLayout`, `setSettings`, `setAmount`, then render its `mesh` into the scene or
+  into a picture-sized render target.
+- `detectFaceImage(image)` finds the faces in a still image (the same face model, run in image mode
+  on the CPU).
+
+## Painting a face
+
+A painted face is one image in the face atlas layout:
+
+- **Layout.** Upright, chin at the bottom, image left = the character's own RIGHT, as on an
+  unmirrored photo. The template labels both halves.
+- **Coverage.** Opaque about 5 mm past the face outline; the fade sits inside it. Straight alpha.
+- **Look.** A neutral face, eyes open, lips closed, in flat even light: `matchLighting` adds the
+  room's light.
+- **Eyes and mouth.** Paint the eye and mouth fills too: they show when `ownEyes` or `ownMouth` is
+  off. Keep liner and lip lines outside the dashed feature rims in the template, or the wearer's own
+  lid and lips show through them.
+- **Size.** 1024², WebP or PNG.
+
+Template: [`docs/guides/face-texture-template.svg`](/guides/face-texture-template.svg), or in your
+app `createFaceTextureGuideSvg()` / `createFaceTextureGuidePngBlob()`, which draw the outline, the
+edge fade lines of a preset, the feature rims, the skin-statistics area and the side labels.
+`createFaceTextureSideTestSvg()` draws a test face with an "L" on the character's own left cheek:
+worn as a source, it reads like a sticker on the wearer's left cheek. The Blender pipeline used for
+the two example faces is in `assets-src/face-swap/README.md`.
+
+## Layering
+
+The swap, the mirror and `FaceTextureLayer` _are_ the face (render order −2):
+
+| Layer                                        | Order         | With the swap                                                                                    |
+| -------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------ |
+| Camera background                            | −1000         | Under it.                                                                                        |
+| `HandCutout`                                 | −900          | Hands stay in front.                                                                             |
+| `FaceRetouch`                                | −2.5          | Covered where the swap draws; do not combine.                                                    |
+| `FaceSwap`, `FaceMirror`, `FaceTextureLayer` | −2            | The face. One face effect per face: not with `FaceWarp3D`, `FaceMorph` or an `ExternalFaceMesh`. |
+| Makeup, `LipEffect`                          | −1.99 … −1.55 | Paint on top, shaded from the raw camera.                                                        |
+| `FaceInset`                                  | −1.5          | Shows the raw camera part.                                                                       |
+| Head occluder                                | −1            | Unchanged.                                                                                       |
+| Accessories                                  | 0+            | Draw on top.                                                                                     |
+
+## Performance
+
+Targets: about 30 fps on a mid-range phone with one feature on. The numbers below are estimates
+until they are measured on devices.
+
+| Mode                  | GPU per frame            | Per face result                              | One-time                                                                                             |
+| --------------------- | ------------------------ | -------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
+| Photo or painted face | 1 draw, about 0.3–0.5 ms | skin statistics at 6–15 Hz, under 0.1 ms     | Photo: decode 20–60 ms; detector start 0.3–1.5 s (first photo); detection 30–100 ms; bake under 5 ms |
+| Mirror                | 1 draw                   | one unwrap, 512–1024², about 0.2–0.5 ms      | none                                                                                                 |
+| Two people            | 2 draws, about 0.6–1 ms  | two unwraps with mirror fill, about 0.6–1 ms | none                                                                                                 |
+
+- **Two people cost most in inference:** two-face tracking runs on a larger analysis image and
+  re-runs face detection while fewer than two faces are tracked. Expect 1.6–2.5× the one-face
+  inference time; `FaceSwap` cannot reduce it.
+- **Quality tiers** (`quality: "auto"` follows the session tier):
+
+  | Tier   | Mesh                     | Live map max | Mirror fill (two people) | Statistics |
+  | ------ | ------------------------ | ------------ | ------------------------ | ---------- |
+  | high   | level 2, 14.5k triangles | 1024         | on                       | 15 Hz      |
+  | medium | level 2                  | 768          | on                       | 10 Hz      |
+  | low    | level 1, 3.6k triangles  | 512          | off                      | 6 Hz       |
+
+  The live map is also capped by the face's size on screen.
+
+- **Memory:** a photo or painted face is about 5.6 MB of GPU memory; a live map 1.4–5.6 MB per face.
+  The photo detector is a second MediaPipe instance (20–40 MB, with its own small WebGL context); it
+  closes after 30 s idle or with `disposeFaceImageLandmarker()`.
+
+Device results (device, browser, tier, fps, date) are recorded here once measured.
+
+## Privacy
+
+The SDK accepts any photo, so a real person's face can be swapped. Photos are decoded in memory
+only: nothing is uploaded, stored or put in the URL, and a baked face is disposed when it is
+replaced. No real person's face ships with the SDK. Ask for consent before using someone's photo;
+whether captures that contain a swap carry a disclosure is your product's decision.
+
+## Example
+
+`/face-reprojection` has `?mode=swap` and `?mode=mirror` (with `?mode=character` for the face
+character):
+
+- `?mode=swap&with=eachOther&source=sdk-video-5`: two people swap on the two-person clip;
+- `?mode=swap&with=painted-female&preset=keepColors`: a painted face on yours;
+- `?mode=swap&with=sideTest`: the side test;
+- `?mode=mirror&variant=upsideDown&matchLighting=0`: the Thatcher effect, raw.
+
+## Manual visual test plan
+
+Record each run with the device and the date.
+
+1. **Sides.** `?mode=swap&with=sideTest` on the webcam, the rear camera and a clip: the "L" lands on
+   the person's own left cheek every time. With a sticker on your real left cheek: `leftLeft` shows
+   it on both cheeks, `rightRight` on neither, `flip` on the right. A left wink in `leftLeft` winks
+   both eyes.
+2. **Registration.** `<FaceTextureLayer face={useLiveFaceTexture(0)} layout="asIs" />` looks like the
+   plain camera while talking and turning: no shift, no swimming.
+3. **Turned head.** At about 30° of yaw in a two-person swap, the nose covers the far cheek and no
+   cheek is painted over the nose.
+4. **Two people.** On `sdk-video-5` and with two real people: when one leaves, the camera is back to
+   normal within about 0.5 s; when they return, the swap comes back after a few frames and the fade;
+   when the faces reorder, there is no colour flash.
+5. **Photos.** Local photos only, never committed: an EXIF-rotated portrait, a 12 MP image, a face
+   turned 30° (the far cheek is filled; `turned` warning), closed eyes (warning), a group photo (the
+   largest face is used).
+6. **Colour.** Several skin tones, side light, warm and cool white balance, and a dark-skin /
+   light-skin pair: no seam with `natural` at `edgeSoftness` 0.5, 1.2 and 3. `raw` shows the raw
+   difference. A painted face with `keepColors` keeps its paint and blends at the edge.
+7. **Occluders.** A hand over the face (`HandCutout`), glasses and earrings drawing on top, bangs.
+8. **Mirror variants.** Each one, including `upsideDown` with the mouth open.
+9. **Performance.** Every tier on a Pixel 6a, a Galaxy A54 and an iPhone 12, with
+   `useXRDiagnostics`. Memory stays flat after 10 photo re-picks.

package/docs/api/face-warp-3d.mdx

@@ -0,0 +1,166 @@
+---
+title: "Face Warp 3D"
+description: "Reshape the live face with blend shapes — bigger eyes, a longer nose, chubby cheeks — using the camera image itself as the texture."
+sidebarTitle: "Face Warp 3D"
+icon: "face-grin-tongue-wink"
+---
+
+`FaceWarp3D` reshapes the person's real face with blend shapes (shape keys), as Spark AR's face mesh
+deformation did: bigger eyes, a longer nose, chubby cheeks, a big forehead.
+
+- **Real pixels.** Every pixel is the camera image itself, moved, so the result is photoreal and
+  the person's own expression carries on underneath.
+- **Weight 0** leaves the camera image untouched.
+
+It is part of the face-character pipeline ([#194](https://github.com/vincentt-xr/sdk/issues/194)).
+Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { FaceWarp3D } from "@vincentt-xr/sdk/face-effects";
+import { FaceTracker } from "@vincentt-xr/sdk/tracking";
+
+<FaceTracker>
+  <FaceWarp3D deformation={gltf.scene} weights={{ bigEyes: 0.8, longNose: 0.3 }} />
+</FaceTracker>;
+```
+
+## FaceWarp3D or FaceMorph?
+
+They are different effects.
+
+|              | `FaceWarp3D`                                                                                              | [`FaceMorph`](/api/face-morph)                                            |
+| ------------ | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
+| What you see | The person, photoreal: their own skin, hair and expression, reshaped                                      | Another object (a ball, a box, a character shape) wearing the camera face |
+| How          | Shape keys move the camera's own pixels; weight 0 = the image untouched                                   | A mesh is rendered, textured with the unwrapped camera face               |
+| Best for     | Big eyes, a big nose, chubby or slim faces, chibi or alien proportions, your face shaped like a character | Turning into a shape, transformation sweeps, stylised heads               |
+| Avoid for    | Changing the skull or hair (face only)                                                                    | Realistic heads (their skull shows the model's material, not hair)        |
+
+Use one face effect per face; they all draw over the face.
+
+## Authoring blend shapes
+
+Shape keys live on a model in the canonical head frame (centimetres, Y up, Z out of the face), exported
+as glTF morph targets.
+
+- **On the canonical face (exact).** Sculpt on the reference mesh
+  `assets-src/face-character/canonical-face-reference.glb` in the SDK repo, or export
+  `createCanonicalFaceGeometry()` with three's `GLTFExporter`. Its 468 vertices are the face
+  landmarks, in order. Every key applies exactly, and key names become the `weights` names.
+- **On any other mesh (transferred).** A head in the same frame, such as Snap's face + skull mesh
+  with a sculpted key, has its keys transferred: each point of the face takes the offset at its
+  closest point on the mesh's base surface.
+- **A model's own shape.** A model without shape keys gives one key, `"shape"`, measured along the
+  face's normal: the face takes the model's proportions. The facecap example reads best this way.
+
+### In Blender
+
+1. **Import.** File → Import → glTF 2.0, with default settings. Leave its location, rotation and
+   scale as they are.
+2. **Add a key.** Object Data properties → Shape Keys: click `+` once for the Basis, then `+` again
+   for your key. Rename it (`bigEyes`).
+3. **Sculpt.** With the key selected, sculpt or edit the mesh, with X symmetry on for a balanced face.
+   Slide the key's value to preview.
+4. **Keep the vertices.** Never add, delete, merge or reorder them: no remesh, no "merge by
+   distance", no applied subdivision. Blender's glTF import and export keep the vertices and their
+   order (tested).
+5. **Export.** File → Export → glTF 2.0 (`.glb`), with Data → Shape Keys on and Apply Modifiers off.
+   +Y up is the default.
+6. **More keys.** One file can carry many keys (up to `FACE_WARP_MAX_KEYS`, 16, apply). Give each its
+   own slider through `weights`.
+
+The warp mesh is the face with the eyes and mouth covered, so keys can move the eyes and lips too.
+Two **skirt** rings past the face's edge stretch the surrounding hair, ears and neck pixels. That lets
+the edge of the face move (puffed cheeks, a smaller chin) without uncovering the original or leaving
+a seam.
+
+## Props
+
+| Prop           | Default | What it does                                                                      |
+| -------------- | ------- | --------------------------------------------------------------------------------- |
+| `deformation`  | `null`  | The model with shape keys (see above). `null` shows the face mesh only.           |
+| `weights`      | —       | A weight per shape key, by name. A key left out is at 1. Also a ref.              |
+| `weight`       | `1`     | Overall strength, multiplying every key. Also a ref.                              |
+| `expression`   | `1`     | The face's own expression: 0 freezes it, more than 1 exaggerates it. Also a ref.  |
+| `subdivisions` | `2`     | Face mesh density (2 ≈ 7,500 vertices; smoother sculpts).                         |
+| `opacity`      | `1`     | Opacity of the warped face.                                                       |
+| `onKeys`       | —       | Receives the shape-key names found on `deformation`, for sliders.                 |
+| rig props      | —       | `faceIndex`, `smoothing`, `holdMs`, `neutral` … (when it runs its own `FaceRig`). |
+
+## Animating without re-rendering
+
+`weights`, `weight` and `expression` also take a ref (`LiveValue<T>`), read every frame. Drive them from
+`useFrame`, or from a `FaceExpressionController` `set` output, without re-rendering React:
+
+```tsx
+const keys = useRef({ bigEyes: 0 });
+useFrame(({ clock }) => {
+  keys.current.bigEyes = (Math.sin(clock.elapsedTime) + 1) / 2;
+});
+<FaceWarp3D deformation={gltf.scene} weights={keys} />;
+```
+
+## How it works
+
+1. **Live face.** Every frame the CPU writes the live face (`FaceRigState.local`, 471 points with the
+   eye and mouth centres) into a small float texture. That is all the per-frame CPU work.
+2. **Deformation, on the GPU.** The vertex shader interpolates each vertex's live point, then adds
+   the shape keys' offsets (a float texture) times their weights:
+   `position = live + weight · Σ key · offset`. Skirt vertices follow their edge vertex with a
+   fading share of its move.
+3. **Projective texture.** The shader projects each vertex's undeformed live position to the
+   screen. The camera image is drawn full-screen in the same space, so that point is the camera
+   texel under it. Every fragment samples there, which gives an exact identity at weight 0 even
+   while the head moves.
+
+**Limitation (shared with Spark AR):** shrinking a feature far inside the face (a tiny mouth) can
+show the original pixels that were there. Enlarging is always clean.
+
+**Pure pieces:**
+
+- `createFaceWarpMesh`, `bindFaceWarpDeformation`;
+- `stepFaceWarp3D`, the CPU reference of the shader;
+- `createFaceWarpGpuData` and `writeFaceWarpLiveTexels`, what the shader reads;
+- `createCanonicalFaceGeometry`, `subdivideFaceUnwrap` and `createClosestPointSearch`.
+
+## Migrating from the old FaceWarp3D
+
+The 2.0.0-alpha.9 FaceWarp3D replaces the proportion sliders. These are removed:
+
+- `FaceWarp3DSettings`, `FACE_WARP_3D_PRESET` and `DEFAULT_FACE_WARP_3D_SETTINGS`;
+- `CanonicalFaceMeshRenderer`'s `faceWarp3D` prop;
+- `FaceWarp3DHead`;
+- the `FaceWarp3DSettingsPanel` debug panel.
+
+To get the same looks, sculpt them as shape keys on the canonical face and give each a weight.
+
+## Example
+
+`/face-warp` in its 3D mode (`?mode=3d`; the old `/face-warp-3d` redirects there) has three sources:
+
+- **facecap:** its own shape (local; the default when present);
+- **demo:** shape keys sculpted in code on the canonical face (big eyes, long nose, chubby cheeks,
+  tiny mouth, big forehead), with one slider each;
+- **Lens Studio head:** the chibi shape key, sculpted on Snap's face + skull + ears mesh
+  (local, git-ignored).
+
+It also has controls for weight (or a looping animation, driven through a ref), expression and
+opacity.
+
+## Manual visual test plan
+
+Open `/face-warp?mode=3d` on the webcam.
+
+1. **Weight 0**: the image is untouched. No edge, band or shimmer, even while you move.
+2. **Big eyes**: your eyes grow, your lids still blink, and your gaze still moves.
+3. **Long nose / chubby cheeks / big forehead**: each reads clearly from the front and stays
+   photoreal (your own skin and hair). The edge of the face blends into the hair, ears and neck.
+4. **Expressions**: talk, smile and raise your brows with keys on. Your expression carries through
+   the shape. `expression` 0 freezes it; 2 exaggerates it.
+5. **Lens Studio head**: the cheeks puff and the forehead swells into the hair, with no double chin
+   or seam.
+6. **Head turn**: turn and tilt. The warp stays on the face and the skirt follows.
+7. **Animate**: the warp breathes in and out smoothly.
+8. **Dropout**: cover the camera. The warp holds briefly, hides when the face is lost, and comes
+   back.

package/docs/api/faceless.mdx

@@ -0,0 +1,143 @@
+---
+title: "Faceless (experimental)"
+description: "Erase the eyes, brows, nose and mouth with the person's own skin, or even the whole face into a mannequin. Every tracked face, with makeup, glasses and liquify on top."
+sidebarTitle: "Faceless"
+icon: "face-meh-blank"
+---
+
+`Faceless` erases a face's features with the person's own skin. The face keeps its real light, colour
+and outline; only the eyes, brows, nose and mouth are gone.
+
+- **Two looks.** `"real"` erases the features and keeps the rest of the face as the camera sees it,
+  with grain matched to the skin's own texture. `"smooth"` evens the whole face into a mannequin.
+- **Blinks and an open mouth never show.** The erase covers the eye and mouth openings too.
+- **It is the face.** It draws in the face layer, so makeup, designs, glasses and hats draw over it,
+  and `FaceLiquify` warps it.
+
+Experimental: its props and look may change in a later alpha. Import from
+`@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { VideoBackground, XRScene } from "@vincentt-xr/sdk";
+import { Faceless } from "@vincentt-xr/sdk/face-effects";
+
+<XRScene>
+  <VideoBackground />
+  <Faceless />
+</XRScene>;
+```
+
+It starts the face model itself; no `FaceTracker` is needed.
+
+## Looks, amount and grain
+
+```tsx
+<Faceless style="smooth" />;
+<Faceless amount={0.5} />;
+<Faceless grain={0} />;
+```
+
+- **`style`**: `"real"` (default) or `"smooth"`.
+- **`amount`**: 0 is the real face, 1 is faceless. Pass a ref (`{ current: n }`) to fade it every
+  frame without re-rendering.
+- **`grain`**: 0..1, `"real"` only. 1 (default) matches the fine detail measured on the person's own
+  cheeks, forehead and chin; 0 leaves the erased skin smooth. There is no grain in the eye and mouth
+  openings or on the lips, which stretch on screen when they open.
+
+## Faces
+
+- **Every tracked face** goes faceless, up to four. `faceIndex` picks one; `numFaces` sets how many
+  faces the face model tracks.
+- Inside a `FaceTracker`, it follows that tracker's face.
+- A face fades in with its first result and out shortly after it is lost.
+
+## Hands in front (experimental)
+
+With `HandCutout` in the scene, a hand over the face stays in front of the faceless face.
+
+```tsx
+import { HandCutout, HandTracker } from "@vincentt-xr/sdk/tracking";
+
+<>
+  <HandTracker>
+    <HandCutout />
+  </HandTracker>
+  <Faceless />
+</>;
+```
+
+## Props
+
+| Prop          | Default            | Meaning                                                                        |
+| ------------- | ------------------ | ------------------------------------------------------------------------------ |
+| `style`       | `"real"`           | `"real"`: features erased with the own skin; `"smooth"`: the whole face evened |
+| `amount`      | 1                  | 0 the real face … 1 faceless, or a ref read every frame                        |
+| `grain`       | 1                  | Grain on the erased skin, 0..1 (`"real"` only)                                 |
+| `faceIndex`   | every tracked face | Only this face (default: the enclosing `FaceTracker`'s face, else every face)  |
+| `numFaces`    | 1                  | Faces the face model tracks when this starts it                                |
+| `smoothing`   | the live face's    | Landmark smoothing 0..1                                                        |
+| `quality`     | `"auto"`           | Mesh level and statistics rate, as face swap's                                 |
+| `active`      | true               | False: nothing drawn                                                           |
+| `renderOrder` | −2                 | The face layer                                                                 |
+
+## Your own layer
+
+`useFacelessTexture(faceIndex, options)` returns the faceless face texture for your own
+`FaceTextureLayer`, for example to fade it with your own logic or draw it on another face:
+
+```tsx
+import { FaceTextureLayer, useFacelessTexture } from "@vincentt-xr/sdk/face-effects";
+
+const Blank = () => {
+  const face = useFacelessTexture(0, { style: "smooth" });
+  return (
+    <FaceTextureLayer
+      face={face}
+      faceIndex={0}
+      preset="raw"
+      ownEyes={false}
+      ownMouth={false}
+      unseen="cover"
+    />
+  );
+};
+```
+
+- `ownEyes` and `ownMouth` off draw the erased eye and mouth openings.
+- `unseen="cover"` also covers the parts the camera sees edge-on (nostril undersides, the lips'
+  seam); with the default `"fade"` the camera shows through there.
+
+Lower level: `FacelessTexture` is the composite pass (`update(renderer, liveFace, { style, grain })`,
+`dispose()`), with its masks from `createFacelessMask` and `createFacelessStretchMask` and its shader
+`FACELESS_FRAGMENT`.
+
+## Limits
+
+- **Beards and stubble** under the erased mouth are replaced by smooth skin; the stubble around it
+  stays.
+- **Real glasses** are part of the camera image: the lenses are erased with the eyes, the frames
+  outside the eye area stay. Draw 3D glasses over the faceless face instead.
+- **Cost.** One composite per face per new face result, at the live face texture's size. Check phone
+  performance before shipping a multi-face effect.
+
+## Example
+
+`/face-reprojection` in the SDK playground, mode "faceless", has both looks, the amount and grain
+sliders, hands in front and the face count. Settings are in the URL: `?mode=faceless`,
+`?fstyle=smooth`, `?famount=0.5`, `?grain=0`. The two-people clip shows two faces.
+
+## Manual visual test plan
+
+Open `/face-reprojection?mode=faceless` on the webcam.
+
+1. **Real**: eyes, brows, nose and mouth are gone. No brow line, lash line, lip line or nostril
+   remains; the erased skin has the face's own colour, light and fine texture.
+2. **Smooth**: the whole face is even skin, with no feature or line showing through.
+3. **Blink and open the mouth wide**: nothing shows, and no streaks appear.
+4. **Turn the head** far left and right, and tilt: the erase stays on the features with no edge
+   sliding off the face.
+5. **Amount**: 0 is the real face, 0.5 a half fade, 1 faceless.
+6. **Hands**: with hands in front, a hand over the mouth stays in front of the face.
+7. **Two faces**: on the two-people clip, both faces are faceless.