@vincentt-xr/sdk 2.0.0-alpha.8 → 2.0.0-alpha.9

package/docs/api/face-reprojection.mdx

@@ -0,0 +1,154 @@
+---
+title: "Face Reprojection"
+description: "The live camera face, worn by a face model in the model's own UV layout: the whole face, or only the eyes and mouth on a character."
+sidebarTitle: "Face Reprojection"
+icon: "masks-theater"
+---
+
+`FaceReprojection` puts the person's real face from the camera onto a face model
+([`ExternalFaceMesh`](/api/external-face-mesh)). The model can use any UV atlas.
+
+- **The whole face:** the person's face on the character's head.
+- **Only the eyes and / or mouth:** the classic talking-character effect.
+- **The eyes on the eyeballs** as well, if you choose.
+
+It is part of the face-character pipeline ([#194](https://github.com/vincentt-xr/sdk/issues/194),
+#199). Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { ExternalFaceMesh, FaceReprojection } from "@vincentt-xr/sdk/face-effects";
+import { FaceTracker } from "@vincentt-xr/sdk/tracking";
+
+<FaceTracker>
+  <ExternalFaceMesh model={gltf.scene}>
+    <FaceReprojection region="eyesAndMouth" />
+  </ExternalFaceMesh>
+</FaceTracker>;
+```
+
+## How it works
+
+Two offscreen passes run per camera frame (at most `fps` a second):
+
+1. **Unwrap.** The camera face is laid out in the canonical face UV layout.
+   - Every landmark samples the camera image where it was tracked.
+   - The eye and mouth openings, which are holes in the canonical surface, are filled. So the eyes
+     and the inside of the mouth come along.
+   - Skin seen edge-on (a cheek in a profile view) fades out instead of smearing, and so does the
+     outer edge of the face.
+2. **Remap.** Every vertex of the model is bound once to its closest point on that filled canonical
+   face. The model is drawn in its own UV layout and reads the unwrap there. A few pixels of edge
+   padding stop seams at the edges of UV islands.
+
+The result is applied to the model's meshes through a patched clone of each material:
+
+- **unlit** (default): as the camera saw it;
+- **lit**: as the base colour, shaded by the scene lights.
+
+**Mirroring, aspect and resize** need nothing. The camera image and the landmarks share one space (a
+mirrored webcam is mirrored before tracking), and the unwrap follows the anatomy rather than the
+screen.
+
+## Props
+
+| Prop               | Default       | What it does                                                                                             |
+| ------------------ | ------------- | -------------------------------------------------------------------------------------------------------- |
+| `meshes`           | skin parts    | Which meshes wear it, by object name. They must share one atlas. Add the eyeballs for real eyes.         |
+| `region`           | `"face"`      | `"face"`, `"eyes"`, `"mouth"` or `"eyesAndMouth"` (soft areas around the openings, lids and lips in).    |
+| `amount`           | `1`           | How much of the camera face shows, 0..1.                                                                 |
+| `shading`          | `"unlit"`     | `"unlit"`: the camera's colours. `"lit"`: shaded by the scene lights.                                    |
+| `mask`             | —             | Red channel, in the model's UV layout (load it like the model's map; glTF textures use `flipY = false`). |
+| `canonicalMask`    | —             | Red channel, in the canonical face UV layout.                                                            |
+| `resolution`       | `1024`        | Output texture size, px (mipmapped).                                                                     |
+| `unwrapResolution` | `512`         | Canonical-layout texture size, px.                                                                       |
+| `padding`          | `2`           | Edge padding around UV islands, px (0–4).                                                                |
+| `fps`              | `30`          | The most updates a second (`0` = every new camera frame).                                                |
+| `facing`           | `[0.05, 0.3]` | Cosines: skin turned further than the first is hidden, fully kept from the second.                       |
+| `falloff`          | `[1, 3]`      | Distance from the canonical face (cm) over which a vertex stops taking the face.                         |
+| `apply`            | `true`        | Patch the meshes' materials. With `false`, only render; use `onTexture` with your own material.          |
+| `onTexture`        | —             | Receives `{ texture, unwrap }` once they exist.                                                          |
+
+**Eyes and lids.** Eyeballs (parts named `eye…`) read the eye openings. Skin and teeth read the face
+and the mouth opening, never the eyes. A lid's edge sits in the eye opening, and wearing the eye there
+would show when a blink morph closes the lid.
+
+**Face index and tracking.**
+
+- It uses the enclosing rig's face (`FaceRigState.faceIndex`, from `FaceRig` / `ExternalFaceMesh`
+  `faceIndex`).
+- It updates only on frames where that face is tracked, and keeps the last texture through a
+  dropout.
+- Both textures are freed on unmount.
+
+## Your own material or pipeline
+
+- **Your own material.** Pass `apply={false}` and use `onTexture`, or patch any built-in material
+  with `patchFaceReprojectionMaterial(material, createFaceReprojectionUniforms())`. The textures
+  hold premultiplied alpha.
+- **Without React.** `FaceReprojector` is the GPU engine:
+
+  - `setTargets([{ geometry, canonical, openings }])`;
+  - `render(renderer, cameraTexture, landmarks)` once per frame;
+  - read `texture` / `unwrapTexture`;
+  - `dispose()` when done.
+
+  It takes five more options for the unwrap, all off by default:
+
+  | Option          | Default              | Meaning                                                                                                                                                                               |
+  | --------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+  | `uvs`           | `CANONICAL_FACE_UVS` | The unwrap's layout, one `[u, v]` per landmark. `FACE_ATLAS_UVS` unwraps into the face atlas; a `canonicalMask` must then be in that layout.                                          |
+  | `unwrapMipmaps` | `false`              | Mipmap the unwrap, for reading it smaller than it is.                                                                                                                                 |
+  | `mirrorFill`    | `false`              | Fill the far side of a turned face from the other side, once per unwrap; only where the mirror twin was seen clearly better. Points outside the image are filled from their twin too. |
+  | `inFrame`       | on with `mirrorFill` | Drop the parts of the face outside the image instead of stretching the edge pixels over them.                                                                                         |
+  | `unwrapPadding` | `0`                  | Padding of the unwrap, px (0–16). It pads alpha too: coverage grows at the face's edge and into unseen holes, so padded texels count as seen.                                         |
+
+  With `uvs`, `texture` (the remap into `setTargets`' own atlas) follows the layout, but the
+  canonical paths do not: `bindFaceReprojection(...).canonicalUv`, a `faceUv` attribute and
+  `patchFaceReprojectionMaterial(..., { attributes: true })` read the unwrap in the canonical
+  layout and would wear a scrambled face. Convert the binding first:
+  `writeFaceReprojectionLayoutUv(binding, FACE_ATLAS_UVS)` is the `faceUv` for that unwrap.
+
+- **Pure helpers.** `getFaceUnwrapMesh`, `getFaceUnwrapUvs` (the unwrap's UVs in a layout),
+  `writeFaceUnwrapFrame`, `writeFaceFacing` (how squarely each of the 471 unwrap vertices faces
+  the camera), `bindFaceReprojection`, `writeFaceReprojectionLayoutUv` and the region helpers.
+
+## Face swap and mirror
+
+To put a face on another live face instead of on a model, use [Face Swap](/api/face-swap):
+`FaceSwap` swaps two people's faces or puts a photo or painted face on every face, and
+`FaceMirror` mirrors your own face. They unwrap the face with the same `FaceReprojector`, into the
+face atlas (`FACE_ATLAS_UVS`) rather than the canonical layout, and draw it back on the tracked face
+in screen space. `/face-reprojection?mode=swap` and `?mode=mirror` show them.
+
+## Example
+
+`/face-reprojection` runs on the local facecap character (see
+[External Face Mesh](/api/external-face-mesh#example)). It has controls for region, shading,
+amount, facing, fps, resolution and face index, and an option to put your eyes on the eyeballs. It
+previews both textures in the top-right corner: the remap in the model's atlas, and the unwrap. Without the
+local model, only the unwrap shows.
+
+## Manual visual test plan
+
+Open `/face-reprojection` on the webcam with the local facecap model.
+
+1. **Whole face**: your face is on the character's skin, sharp and aligned, with no seam at the
+   edge. The ears, the scalp and the back of the head keep the character's own skin, and the edge
+   fades softly.
+2. **Head turn**: turn your head. The cheek turning away fades to the character's skin instead of
+   smearing. The near side stays sharp.
+3. **Expressions**: open your mouth (your teeth and tongue show in the character's mouth), smile and
+   raise your brows. Blink: the character blinks (its own morph) and the lids show your lids.
+4. **Eyes + mouth**: switch region to "eyes + mouth". Only your eyes and lips show, with soft edges,
+   on the character's grey skin. Tick "your eyes on the eyeballs" and your eyes move with your gaze.
+5. **Shading**: switch to "lit". The face takes the scene lighting. The amount slider fades it.
+6. **Mirror and source**: switch between the webcam and the sample videos and image. Each real eye
+   lands on the character's eye in front of it, on every source (mesh to mesh; GROUNDING "Left and
+   right").
+7. **Resolution / fps**: change the resolution (the previews resize) and the fps (lower updates
+   less often). Nothing leaks: switching back and forth keeps the frame rate.
+8. **Dropout**: cover the camera briefly. The face holds, then the character hides and comes back
+   with a fresh face.
+9. **Second face**: with two people, "second face" puts the second person's face on the character.

package/docs/api/face-retouching.mdx

@@ -52,22 +52,30 @@ space. They default to the full frame. The legacy compare demo uses
 
 For multiple people, render one `FaceRetouch` per person and pass `faceIndex`.
 `numFaces` defaults to `faceIndex + 1`, so `<FaceRetouch faceIndex={1} />`
-requests tracking for at least two faces. Multiple passes are transparent
-outside their own face mask, so they can be stacked over the same camera
-background.
+requests tracking for at least two faces. Each pass is see-through outside
+its own face mask, so they stack over the same camera background at the same
+draw order.
 
 ```tsx
 const faceCount = 2;
 
 {
   Array.from({ length: faceCount }, (_, faceIndex) => (
-    <FaceRetouch
-      key={faceIndex}
-      {...controls}
-      faceIndex={faceIndex}
-      numFaces={faceCount}
-      renderOrder={-1000 + faceIndex}
-    />
+    <FaceRetouch key={faceIndex} {...controls} faceIndex={faceIndex} numFaces={faceCount} />
   ));
 }
 ```
+
+## Draw order
+
+`FaceRetouch` draws in the opaque face layer at `FACE_RETOUCH_RENDER_ORDER`
+(-2.5, from `@vincentt-xr/sdk/face-effects`), with depth testing:
+
+- after the camera background (-1000) and `HandCutout` (-900), so a hand in
+  front of the face stays sharp instead of being smoothed;
+- before the face effects (`FaceWarp3D`, `FaceMorph`, characters at -2) and
+  makeup (-1.99 to -1.55), which draw over the retouched skin.
+
+Mount it at the top of the scene, not inside a tracker group (a group's
+render order would move it). Override `renderOrder` only to place a custom
+effect between the retouch and the face.

package/docs/api/face-swap.mdx

@@ -0,0 +1,350 @@
+---
+title: "Face Swap"
+description: "Two people swap faces, a photo or a painted face goes on every face, or your own face is mirrored, live on the camera, in the wearer's skin tone and light."
+sidebarTitle: "Face Swap"
+icon: "people-arrows"
+---
+
+`FaceSwap` and `FaceMirror` draw a face on a tracked face, live on the camera:
+
+- **Two people swap faces.** It starts by itself when two faces are in view.
+- **A photo or a painted face** goes on every face (or only one).
+- **Your own face, mirrored:** one half doubled, the halves swapped, or the eyes and mouth upside
+  down.
+
+The drawn face moves with the wearer's expression, takes the wearer's skin tone and light, and
+fades in centimetres inside the face edge. Both components start the face model themselves.
+Import from `@vincentt-xr/sdk/face-effects` ([#275](https://github.com/vincentt-xr/sdk/issues/275)).
+
+## Quick start
+
+```tsx
+import { VideoBackground, XRProvider, XRScene } from "@vincentt-xr/sdk";
+import { FaceMirror, FaceSwap } from "@vincentt-xr/sdk/face-effects";
+
+<XRProvider>
+  <XRScene>
+    <VideoBackground />
+    {/* Two people swap faces automatically; with one face the camera shows as is */}
+    <FaceSwap onStatus={(s) => setSwapping(s.active)} />
+  </XRScene>
+</XRProvider>;
+
+// Any photo: File, Blob, URL, <img>, canvas or ImageBitmap
+<FaceSwap source={{ kind: "photo", src: file }} onStatus={(s) => setWarnings(s.warnings)} />;
+// A face painted in the face atlas: keeps its paint, takes the room's light
+<FaceSwap source={{ kind: "canonicalTexture", src: paintedUrl }} />;
+// Your own face, the left half twice
+<FaceMirror variant="leftLeft" />;
+```
+
+## Which component
+
+| Goal                                                           | Use                                                                      |
+| -------------------------------------------------------------- | ------------------------------------------------------------------------ |
+| Two people swap faces, or a photo / painted face on every face | `FaceSwap`                                                               |
+| Your own face mirrored                                         | `FaceMirror variant`                                                     |
+| Your own face texture on a live face (a custom effect)         | `FaceTextureLayer face`, fed by `useFaceTexture` or `useLiveFaceTexture` |
+| A face drawn into a picture or a render target, or no React    | `FaceTransfer` with `createFaceTexture`                                  |
+
+**Mounting.** Mount them anywhere in `<XRScene>`, also inside a `<FaceTracker>`. They draw in
+screen space and need no `FaceRig`. With no `FaceTracker` above, they start the face model
+themselves: `FaceSwap` with two faces for a two-person swap, otherwise one. The first component to
+start the model sets its face count, so anything else that may start it first needs the same
+`numFaces`. A console warning names the fix when a face the component needs can never be tracked.
+Keep hands in front with `<HandTracker><HandCutout /></HandTracker>`.
+
+## FaceSwap
+
+### Sources
+
+| `source`                                             | What is drawn                                                                                                      |
+| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
+| `{ kind: "eachOther" }` (default)                    | Faces 0 and 1 swap.                                                                                                |
+| `{ kind: "photo", src, mirrored?, pick?, maxSize? }` | The face found in a photo, on every tracked face. `pick`: `"largest"` (default) or the n-th face from the left.    |
+| `{ kind: "canonicalTexture", src, mirrored? }`       | A face painted in the face atlas (a URL or a `THREE.Texture`), on every tracked face.                              |
+| `{ kind: "faceTexture", face }`                      | A `FaceTexture` you made (`useFaceTexture`, `createFaceTexture`, `useLiveFaceTexture`); it stays yours to dispose. |
+
+`mirrored` says the image itself is mirrored (some phones save selfies mirrored). A mirrored selfie
+cannot be detected from the pixels, so it is your app's toggle.
+
+### Props
+
+Every setting below is also a prop, flat (`<FaceSwap matchLighting={0.5} />`), plus:
+
+| Prop        | Default                  | What it does                                                                        |
+| ----------- | ------------------------ | ----------------------------------------------------------------------------------- |
+| `source`    | `{ kind: "eachOther" }`  | The face drawn (above).                                                             |
+| `amount`    | `1`                      | 0..1, or a ref read every frame, so it can fade without re-rendering.               |
+| `faceIndex` | every tracked face       | Photo, painted and `faceTexture` sources: only this face. Ignored for two people.   |
+| `numFaces`  | 2 for two people, else 1 | Faces to start the face model with.                                                 |
+| `quality`   | `"auto"`                 | `"auto"` follows the session's quality tier; or `"high"`, `"medium"`, `"low"`.      |
+| `active`    | `true`                   | `false` hides it and stops its passes and its face model, but keeps a loaded photo. |
+| `onStatus`  |                          | Called whenever the status changes (not every frame).                               |
+
+The status is `{ active, faces, source, warnings, error }`:
+
+- `active`: drawn now (fading in, shown or fading out);
+- `faces`: faces in the newest face result;
+- `source`: `"waiting"` (no source yet, or fewer than two ready faces for a two-person swap),
+  `"loading"`, `"ready"` or `"error"`;
+- `warnings`: for photos, any of `turned` (head turned more than 25°), `eyesClosed`, `mouthOpen`,
+  `small` (eyes less than 60 px apart) and `severalFaces`;
+- `error`: `{ code, message }` with code `noFace`, `decode`, `cors`, `model` or `aborted`.
+
+### Behaviour
+
+- **Two people.** The swap starts once both faces have been tracked for 5 face results and fades
+  in over 200 ms. It holds through a dropout of up to 250 ms, then fades back to the plain camera
+  (also after 1 s with no face results at all). MediaPipe reordering the faces changes nothing.
+  Where the faces overlap, the larger face draws on top. With three or more faces, only faces 0 and
+  1 swap.
+- **Photos and painted faces.** Each tracked face fades in with its first result and out 250 ms
+  after it is lost.
+- **Own eyes and mouth.** By default the wearer keeps their own live eyes and mouth inside the drawn
+  face (`ownEyes`, `ownMouth`), so blinks, gaze, teeth and tongue stay real. Turn them off for the
+  source's own eyes and mouth: a photo's or painting's stay still.
+- **Expression.** The wearer's expression moves the drawn face. A still face has a neutral, closed
+  mouth, so its lips stretch when the wearer talks.
+
+## Settings and presets
+
+| Field             | Range    | Default   | What it does                                                                              |
+| ----------------- | -------- | --------- | ----------------------------------------------------------------------------------------- |
+| `preset`          |          | `natural` | `natural`, `keepColors`, `raw`, or `"custom"`.                                            |
+| `matchSkinTone`   | 0..1     | 1         | The drawn face takes the wearer's skin tone, exposure and white balance.                  |
+| `matchLighting`   | 0..1     | 1         | The wearer's light and shadow fall on the drawn face.                                     |
+| `edgeMatch`       | 0..1     | 1         | The band inside the face edge always takes the wearer's tone and light, so no seam shows. |
+| `edgeSoftness`    | 0.3..3   | 1.2       | Width of the fade inside the face edge, cm.                                               |
+| `edgeInset`       | 0..1.5   | 0.3       | Pulls the edge inside the tracked outline, cm.                                            |
+| `ownEyes`         | boolean  | true      | The wearer keeps their live eyes.                                                         |
+| `ownMouth`        | boolean  | true      | The wearer keeps their live mouth interior.                                               |
+| `featureSoftness` | 0.1..0.6 | 0.25      | Soft rim around the kept eyes and mouth, cm (0.1 for painted faces).                      |
+| `detail`          | 0..1     | 1         | Sharpness of the drawn face; lower softens it to match a soft camera.                     |
+| `smoothing`       | 0..1     | 0.35      | Landmark smoothing. Shared per face: the most any component asks for wins.                |
+| `amount`          | 0..1     | 1         | Overall strength.                                                                         |
+
+| Preset       | For                       | Sets                                                                         |
+| ------------ | ------------------------- | ---------------------------------------------------------------------------- |
+| `natural`    | photos and two people     | Full skin-tone and light match, 1.2 cm edge, own eyes and mouth.             |
+| `keepColors` | painted or stylised faces | No skin-tone match, 0.7 light match, 1 cm edge: the paint keeps its colours. |
+| `raw`        | a mask look, or debugging | No matching, a short 0.4 cm edge: the face as captured.                      |
+
+- Without `preset`, a painted face starts from `keepColors` with a 0.1 cm feature rim; photos and
+  two people start from `natural`.
+- A named preset always means its values: an explicit value that moves a field the preset sets
+  makes it `"custom"`. Smoothing, inset, feature softness and amount are never set by a preset.
+- `resolveFaceSwapSettings(input, { source })` resolves props (defaults, then the preset, then your
+  values, clamped). `patchFaceSwapSettings(prev, patch)` applies a panel edit: choosing a preset
+  applies it, editing a field it sets makes it `"custom"`. `FACE_SWAP_SETTINGS_FIELDS` drives a
+  settings panel; `FaceSwapSettingsPanel` from `@vincentt-xr/sdk/debug-ui` is a ready one.
+
+## Colour and edges
+
+**Colour.** The drawn face is multiplied by a gain per colour channel, in linear light, computed per
+vertex:
+
+- **Skin tone** (`matchSkinTone`): the ratio of the wearer's mean skin colour to the source's. It
+  carries skin tone, exposure and white balance.
+- **Light** (`matchLighting`): the ratio of the wearer's local, low-frequency skin colour to the
+  source's, relative to their means. It is applied at full strength for brightness and half
+  strength for colour, so coloured light does not tint the skin. A painted face only takes the
+  wearer's light: its own painted shading keeps its contrast.
+- The gain is clamped to 0.25–4.
+
+The statistics are skin only: brows, eyes, lips and nostrils are masked out, and outliers
+(speculars, beard, glasses frames) are rejected. They are blurred to 64² and smoothed over 250 ms,
+so a flicker in one frame never shows.
+
+**Edges.** Every distance is in cm on the face, measured from a smoothed face outline, so the fade
+lines are smooth curves. The alpha is the product of:
+
+- the edge fade: from `edgeInset` inside the outline, over `edgeSoftness`;
+- how squarely the wearer's face points at the camera there;
+- how much of the source was seen there (a photo's turned-away cheek is filled from the other side
+  when it is baked);
+- soft rims of `featureSoftness` around the kept eyes and mouth.
+
+Inside the fade, a band twice as wide always takes the wearer's tone and light (`edgeMatch`), so
+the drawn face meets the real skin with no seam even when `matchSkinTone` is 0.
+
+## FaceMirror
+
+```tsx
+<FaceMirror variant="leftLeft" />   // both halves are your own LEFT half
+<FaceMirror variant="rightRight" /> // both halves are your own RIGHT half
+<FaceMirror variant="flip" />       // left and right swapped: a left wink shows on the right
+<FaceMirror variant="upsideDown" /> // each eye and the mouth flipped top to bottom in place
+```
+
+| Prop            | Default            | What it does                                                                 |
+| --------------- | ------------------ | ---------------------------------------------------------------------------- |
+| `variant`       | required           | `leftLeft`, `rightRight`, `flip` or `upsideDown`.                            |
+| `matchLighting` | `1`                | The real side light stays where it falls; `0` shows the raw mirrored pixels. |
+| `edgeSoftness`  | `1.2`              | Width of the fade inside the face edge, cm.                                  |
+| `smoothing`     | `0.35`             | Landmark smoothing.                                                          |
+| `amount`        | `1`                | 0..1, or a ref.                                                              |
+| `faceIndex`     | every tracked face | Only this face.                                                              |
+| `numFaces`      | `1`                | Faces to start the face model with.                                          |
+| `quality`       | `"auto"`           | As for `FaceSwap`.                                                           |
+| `active`        | `true`             | `false` hides it and stops its passes.                                       |
+
+- Left and right are the person's own sides on the mirrored webcam, the rear camera and videos
+  alike.
+- The eyes and mouth are part of every variant: they always come from the mirrored face, so a left
+  wink in `leftLeft` winks both eyes.
+- Where the far side of a turned face was not seen, the real face shows.
+
+## Sides
+
+Left and right always mean the person's own side. A face texture's `mirrored` flag says which side
+of the canonical face is the person's left: `true` for a live face on the mirrored webcam or a
+selfie saved mirrored, `false` for a photo as stored, a painted face, and a live face on the rear
+camera or a video. When a face is drawn on another, the sides are matched through that flag and the
+exact landmark mirror map (`FACE_UNWRAP_MIRROR`, `resolveFaceSourceMirror`), never by flipping the
+texture. A photo's left cheek lands on the wearer's left cheek on every feed.
+
+## Face textures and FaceTextureLayer
+
+A `FaceTexture` is one face laid flat in the face atlas (`FACE_ATLAS_UVS`): its `map`, its skin
+statistics `low` and `mean`, `mirrored`, and for photos the detected `landmarks`.
+
+```tsx
+import {
+  FaceTextureLayer,
+  useFaceTexture,
+  useLiveFaceTexture,
+} from "@vincentt-xr/sdk/face-effects";
+
+// Inside <XRScene>: these need the renderer.
+const photo = useFaceTexture({ kind: "photo", src: file }); // { face, status, warnings, error, faceCount }
+const second = useLiveFaceTexture(1); // stable, updated in place; null until made
+
+<FaceTextureLayer face={photo.face} />;
+<FaceTextureLayer face={second} faceIndex={0} />;
+```
+
+- `useFaceTexture(source)` bakes a photo or painted face once per URL or object, aborts a stale load
+  and disposes the face on change or unmount.
+- `useLiveFaceTexture(faceIndex)` is shared and refcounted per face: every component asking for the
+  same face shares one unwrap and one set of statistics. It updates only when a new face result
+  arrives, from smoothed landmarks, so it never swims.
+- `FaceTextureLayer` draws any face texture on a tracked face with the same settings as `FaceSwap`,
+  plus `layout` (`asIs`, `flip`, `leftLeft`, `rightRight`, `upsideDown`) and `targetStats`. It starts
+  the face model one commit after it mounts, so `useLiveFaceTexture(1)` above it starts the model
+  with two faces.
+- Without React: `createFaceTexture(renderer, source)` bakes a face (you dispose it), and
+  `FaceTransfer` draws it: `setTarget(landmarks, aspect)` or a live map, `setSource`,
+  `setTargetStats`, `setLayout`, `setSettings`, `setAmount`, then render its `mesh` into the scene or
+  into a picture-sized render target.
+- `detectFaceImage(image)` finds the faces in a still image (the same face model, run in image mode
+  on the CPU).
+
+## Painting a face
+
+A painted face is one image in the face atlas layout:
+
+- **Layout.** Upright, chin at the bottom, image left = the character's own RIGHT, as on an
+  unmirrored photo. The template labels both halves.
+- **Coverage.** Opaque about 5 mm past the face outline; the fade sits inside it. Straight alpha.
+- **Look.** A neutral face, eyes open, lips closed, in flat even light: `matchLighting` adds the
+  room's light.
+- **Eyes and mouth.** Paint the eye and mouth fills too: they show when `ownEyes` or `ownMouth` is
+  off. Keep liner and lip lines outside the dashed feature rims in the template, or the wearer's own
+  lid and lips show through them.
+- **Size.** 1024², WebP or PNG.
+
+Template: [`docs/guides/face-texture-template.svg`](/guides/face-texture-template.svg), or in your
+app `createFaceTextureGuideSvg()` / `createFaceTextureGuidePngBlob()`, which draw the outline, the
+edge fade lines of a preset, the feature rims, the skin-statistics area and the side labels.
+`createFaceTextureSideTestSvg()` draws a test face with an "L" on the character's own left cheek:
+worn as a source, it reads like a sticker on the wearer's left cheek. The Blender pipeline used for
+the two example faces is in `assets-src/face-swap/README.md`.
+
+## Layering
+
+The swap, the mirror and `FaceTextureLayer` _are_ the face (render order −2):
+
+| Layer                                        | Order         | With the swap                                                                                    |
+| -------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------ |
+| Camera background                            | −1000         | Under it.                                                                                        |
+| `HandCutout`                                 | −900          | Hands stay in front.                                                                             |
+| `FaceRetouch`                                | −2.5          | Covered where the swap draws; do not combine.                                                    |
+| `FaceSwap`, `FaceMirror`, `FaceTextureLayer` | −2            | The face. One face effect per face: not with `FaceWarp3D`, `FaceMorph` or an `ExternalFaceMesh`. |
+| Makeup, `LipEffect`                          | −1.99 … −1.55 | Paint on top, shaded from the raw camera.                                                        |
+| `FaceInset`                                  | −1.5          | Shows the raw camera part.                                                                       |
+| Head occluder                                | −1            | Unchanged.                                                                                       |
+| Accessories                                  | 0+            | Draw on top.                                                                                     |
+
+## Performance
+
+Targets: about 30 fps on a mid-range phone with one feature on. The numbers below are estimates
+until they are measured on devices.
+
+| Mode                  | GPU per frame            | Per face result                              | One-time                                                                                             |
+| --------------------- | ------------------------ | -------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
+| Photo or painted face | 1 draw, about 0.3–0.5 ms | skin statistics at 6–15 Hz, under 0.1 ms     | Photo: decode 20–60 ms; detector start 0.3–1.5 s (first photo); detection 30–100 ms; bake under 5 ms |
+| Mirror                | 1 draw                   | one unwrap, 512–1024², about 0.2–0.5 ms      | none                                                                                                 |
+| Two people            | 2 draws, about 0.6–1 ms  | two unwraps with mirror fill, about 0.6–1 ms | none                                                                                                 |
+
+- **Two people cost most in inference:** two-face tracking runs on a larger analysis image and
+  re-runs face detection while fewer than two faces are tracked. Expect 1.6–2.5× the one-face
+  inference time; `FaceSwap` cannot reduce it.
+- **Quality tiers** (`quality: "auto"` follows the session tier):
+
+  | Tier   | Mesh                     | Live map max | Mirror fill (two people) | Statistics |
+  | ------ | ------------------------ | ------------ | ------------------------ | ---------- |
+  | high   | level 2, 14.5k triangles | 1024         | on                       | 15 Hz      |
+  | medium | level 2                  | 768          | on                       | 10 Hz      |
+  | low    | level 1, 3.6k triangles  | 512          | off                      | 6 Hz       |
+
+  The live map is also capped by the face's size on screen.
+
+- **Memory:** a photo or painted face is about 5.6 MB of GPU memory; a live map 1.4–5.6 MB per face.
+  The photo detector is a second MediaPipe instance (20–40 MB, with its own small WebGL context); it
+  closes after 30 s idle or with `disposeFaceImageLandmarker()`.
+
+Device results (device, browser, tier, fps, date) are recorded here once measured.
+
+## Privacy
+
+The SDK accepts any photo, so a real person's face can be swapped. Photos are decoded in memory
+only: nothing is uploaded, stored or put in the URL, and a baked face is disposed when it is
+replaced. No real person's face ships with the SDK. Ask for consent before using someone's photo;
+whether captures that contain a swap carry a disclosure is your product's decision.
+
+## Example
+
+`/face-reprojection` has `?mode=swap` and `?mode=mirror` (with `?mode=character` for the face
+character):
+
+- `?mode=swap&with=eachOther&source=sdk-video-5`: two people swap on the two-person clip;
+- `?mode=swap&with=painted-female&preset=keepColors`: a painted face on yours;
+- `?mode=swap&with=sideTest`: the side test;
+- `?mode=mirror&variant=upsideDown&matchLighting=0`: the Thatcher effect, raw.
+
+## Manual visual test plan
+
+Record each run with the device and the date.
+
+1. **Sides.** `?mode=swap&with=sideTest` on the webcam, the rear camera and a clip: the "L" lands on
+   the person's own left cheek every time. With a sticker on your real left cheek: `leftLeft` shows
+   it on both cheeks, `rightRight` on neither, `flip` on the right. A left wink in `leftLeft` winks
+   both eyes.
+2. **Registration.** `<FaceTextureLayer face={useLiveFaceTexture(0)} layout="asIs" />` looks like the
+   plain camera while talking and turning: no shift, no swimming.
+3. **Turned head.** At about 30° of yaw in a two-person swap, the nose covers the far cheek and no
+   cheek is painted over the nose.
+4. **Two people.** On `sdk-video-5` and with two real people: when one leaves, the camera is back to
+   normal within about 0.5 s; when they return, the swap comes back after a few frames and the fade;
+   when the faces reorder, there is no colour flash.
+5. **Photos.** Local photos only, never committed: an EXIF-rotated portrait, a 12 MP image, a face
+   turned 30° (the far cheek is filled; `turned` warning), closed eyes (warning), a group photo (the
+   largest face is used).
+6. **Colour.** Several skin tones, side light, warm and cool white balance, and a dark-skin /
+   light-skin pair: no seam with `natural` at `edgeSoftness` 0.5, 1.2 and 3. `raw` shows the raw
+   difference. A painted face with `keepColors` keeps its paint and blends at the edge.
+7. **Occluders.** A hand over the face (`HandCutout`), glasses and earrings drawing on top, bangs.
+8. **Mirror variants.** Each one, including `upsideDown` with the mouth open.
+9. **Performance.** Every tier on a Pixel 6a, a Galaxy A54 and an iPhone 12, with
+   `useXRDiagnostics`. Memory stays flat after 10 photo re-picks.