@vincentt-xr/sdk 2.0.0-alpha.8 → 2.0.0-alpha.9

package/docs/api/instructions.mdx

@@ -0,0 +1,323 @@
+---
+title: "Instructions"
+description: "Prompts that ask for a gesture, an expression or a pose with a 3D hint and your copy — ready-made presets, and every layer underneath open for your own vocabulary, detection, look and layout."
+sidebarTitle: "Instructions"
+icon: "person-chalkboard"
+---
+
+An instruction asks the person to do something and waits until they do. It shows a 3D hint, puts
+your copy under it, and completes once the pose has been held for `holdMs`. Everything lives in one door:
+
+```tsx
+import { … } from "@vincentt-xr/sdk/instructions";
+```
+
+There are three ready-made presets:
+
+- [Hand Instructions](/api/hand-instructions)
+- [Face Instructions](/api/face-instructions)
+- [Body Instructions](/api/body-instructions)
+
+Each preset is built from four open layers, and you can use any layer on its own from outside the SDK.
+
+| Layer          | What you get                                                                                                                |
+| -------------- | --------------------------------------------------------------------------------------------------------------------------- |
+| **Vocabulary** | The built-in gestures, expressions and poses; `defineHandGesture` / `defineFaceExpression` / `defineBodyPose` for your own. |
+| **Detection**  | `useHandGesture`, `useFaceExpression`, `useBodyPose`, and `useInstructionHold` for any condition.                           |
+| **Hint**       | `HandGestureHint`, `FaceExpressionHint`, `BodyPoseHint` with `look` and `loop`, and the kit to build a hint on any model.   |
+| **Layout**     | `<Instruction>`: any detector, any symbol, your text style, motion and proportions.                                         |
+
+The rest of this page goes from small changes to a fully custom prompt.
+
+## 1. Restyle a preset
+
+Every preset takes the same layout and text options, plus `look` and `loop` for its hint. Each is
+merged over that preset's default, so you only write what you change.
+
+```tsx
+<HandInstruction
+  gesture="peace"
+  model={hand.scene}
+  poses={hand.animations}
+  size={0.2} // a fraction of the screen height, or "sm" | "md" | "lg"
+  look={{
+    ghost: { color: "#7dd3fc", core: 0.35 }, // tint, opacity facing you, at the rim (`rim`), rim whitening
+    halo: false, // turn a layer off
+    glow: { color: "#38bdf8", full: 1 },
+    highlight: { opacity: 1 },
+  }}
+  loop={{ seconds: 1.6, stagger: 0 }} // timing, hold share, start, easing, finger stagger (hand)
+  textStyle={{ fontFamily: "Georgia, serif", strokeColor: "#0c4a6e", strokeWidth: 6 }}
+  motion={{ ring: { color: "#38bdf8" }, pop: 0.2, bob: 0 }}
+  layout={{ gap: 0.2, textHeight: 0.35 }}
+/>
+```
+
+**`look`** (`HintLook`) controls how the hint is drawn:
+
+- `ghost` (`GhostOptions`): `color`, `core` (opacity facing you), `rim` (opacity at the silhouette),
+  `whiten`.
+- `parts`: overrides per model material name, e.g. `{ iris: { color: "#2a2732" } }`; `null` removes
+  a default tint.
+- `halo`: `width` in model units, `color`, `opacity`.
+- `glow`: `color`, `size`, `idle` and `full` opacity.
+- `highlight`: `opacity`, `whiten`, and `idle` (the share shown while relaxed).
+- `matcap`: your own shading texture.
+- `material`: any three.js material, replacing the ghost.
+
+Any layer can be `false` to turn it off. The defaults are exported:
+
+- `HAND_HINT_LOOK` and `HAND_HINT_LOOP`
+- `FACE_HINT_LOOK` and `FACE_HINT_LOOP`
+- `BODY_HINT_LOOK` and `BODY_HINT_LOOP`
+
+`mergeHintLook(base, over)` merges two looks the same way the presets do.
+
+**`loop`** (`HintLoop`) controls how the hint animates:
+
+- `seconds` per cycle; `0` holds the pose.
+- `hold`: the share of the cycle held at the full pose.
+- `from`: where each cycle starts, 0 = the model's rest.
+- `easing`.
+- The hand also takes `stagger`, how far apart the fingers move; `0` moves them together.
+
+**Text and layout** apply to every preset and to `<Instruction>`:
+
+- `textStyle` / `nudgeTextStyle`: any `TextLabel` style (font, weight, stroke, shadow, background),
+  merged over `INSTRUCTION_TEXT_STYLE`.
+- `color`: a shortcut for the text colour.
+- `motion` (`InstructionMotion`):
+  - `fadeIn` and `fadeOut`, in seconds;
+  - `rise`, `bob`, `pop`, `popSeconds` and `nudgePulse`;
+  - `ring`: `{ color, seconds, opacity, from, to }`, or `false`.
+- `layout` (`InstructionLayout`): `textHeight`, `gap`, `nudgeHeight`, `textWidth` and `edgeMargin`.
+  Each is relative to the size, except `edgeMargin`, which is a fraction of the screen height.
+- `renderText(slot)`: draw each line yourself. You get `{ text, role, width, height, renderOrder,
+style }`. Use transparent materials, because the layout fades every mesh under it.
+
+## 2. Your own gesture, expression or pose
+
+A definition holds three things:
+
+- its default copy (`label`);
+- how it is detected: a `rule`, a `match` function, or both, in which case both must pass;
+- how the hint shows it (`hint`).
+
+Pass a definition anywhere a built-in name goes: the presets, the hooks, the hints and the pure
+`match*` functions.
+
+```tsx
+const hangLoose = defineHandGesture({
+  label: "Hang loose",
+  rule: { digits: { thumb: true, index: false, middle: false, ring: false, pinky: true } },
+  match: (shape) => shape.curl.pinky < 0.35, // and the pinky really straight
+  hint: { clip: "callMe", roll: 103, highlight: ["thumb", "pinky"] },
+});
+
+const yawn = defineFaceExpression({
+  label: "Big yawn",
+  rule: { atLeast: { jawOpen: 0.7 } },
+  hint: { targets: { surprised: 1, smile: 0.3 } }, // a weighted mix of morph targets
+});
+
+const flex = defineBodyPose({
+  label: "Show your muscles",
+  match: (shape) =>
+    shape.arms.left.bend > 70 && shape.arms.right.bend > 70 && shape.arms.left.up > 0,
+  hint: { clip: "flex", arms: ["left", "right"] }, // a clip in your own figure
+});
+
+<HandInstruction gesture={hangLoose} model={hand.scene} poses={hand.animations} />;
+```
+
+The hint can name a clip in your own model:
+
+- **Hand:** a `HandHintPose`: an authored `clip`, or a procedural pose (`curl`, `spread`,
+  `thumbTuck`, `thumbTo`), plus the view (`yaw`, `roll`, `aimThumb`) and `highlight`.
+- **Face:** `{ target }` or `{ targets }`.
+- **Body:** `{ clip, arms }`.
+
+`resolveHandGesture`, `resolveFaceExpression` and `resolveBodyPose` turn a name or definition into
+the full definition.
+
+## 3. Tune detection
+
+Every hook takes the same kinds of input:
+
+- a built-in name;
+- a rule;
+- a match function:
+  - hand: `(shape, side) => boolean`;
+  - face: `(blendshapes, face) => boolean`;
+  - body: `(shape) => boolean`;
+- a definition.
+
+They also take `holdMs`, `graceMs`, `onComplete` and `active`.
+
+The hand and body hooks take `limits`: thresholds overridden for that detector only. The defaults,
+`HAND_SHAPE_LIMITS` and `BODY_SHAPE_LIMITS`, are frozen. The pure classifiers take them too:
+
+```tsx
+const thumb = useHandGesture("thumbUp", { limits: { thumbVertical: 0.5 }, holdMs: 800 });
+const shape = analyzeBodyShape(landmarks, { mirrored, limits: { raisedAboveShoulder: 1 } });
+```
+
+Every hook returns an **`InstructionDetector`** plus its own readout (`shapesRef`, `faceRef`,
+`shapeRef`):
+
+- `status`: re-renders only when it changes;
+- `holdRef`: `progress` 0..1;
+- `matchedRef`: matched on this frame;
+- `isInView()`;
+- `reset()`.
+
+**Any condition.** `useInstructionHold(match, options)` holds any test you write; `match` runs every
+frame. To combine detectors, read their `matchedRef`s, and call those hooks first in the same
+component:
+
+```tsx
+const hand = useHandGesture("palm");
+const face = useFaceExpression("smile");
+const both = useInstructionHold(() => hand.matchedRef.current && face.matchedRef.current, {
+  isInView: () => hand.isInView() && face.isInView(),
+  holdMs: 600,
+  onComplete: start,
+});
+```
+
+A new name, rule or definition value starts a new prompt. A match function is read live and doesn't
+restart the hold. For a new prompt with the same function, call `reset()` or change the component's
+React `key`.
+
+## 4. A fully custom prompt: `<Instruction>`
+
+The layout on its own: any detector, any symbol.
+
+```tsx
+<Instruction
+  detector={both} // any hook result, or your own { status, isInView }
+  text="Wave and smile"
+  nudgeText="Show your hand and your face" // default null (no nudge) here
+  symbol={(height, renderOrder) => (
+    <TextLabel
+      text="🖐️😊"
+      bgColor="transparent"
+      aspect={2}
+      scale={[height * 2, height, 1]}
+      renderOrder={renderOrder}
+    />
+  )}
+  renderText={({ text, width, height, renderOrder }) => (
+    <TextLabel
+      text={text}
+      bgColor="#fde68a"
+      color="#0f172a"
+      borderRadius={60}
+      bgFit="text"
+      aspect={width / height}
+      scale={[width, height, 1]}
+      renderOrder={renderOrder}
+    />
+  )}
+/>
+```
+
+It takes every option from section 1, plus `symbolAspect` (a taller slot, e.g. 2.1 for a standing
+figure) and `active`. Mount it inside `XRScene`; it draws in its own `ScreenSpaceUI`, above the AR
+content (`INSTRUCTION_RENDER_ORDER`, 2000 and up). The symbol can be:
+
+- a preset hint, e.g. `HandGestureHint` with your definition and `look`;
+- an image (`ScreenImage`), a `SpriteAnimation` or an emoji;
+- your own 3D model.
+
+## 5. A hint on your own model
+
+The presets' hints are built from a small kit you can use on any model:
+
+- **`useGhostModel(model, look, { base, renderOrder })`** makes a ghost copy of the model.
+  - It applies a matcap per part, a depth pre-pass, the halo and the glow, all from one `look`.
+  - It returns `object` to render, and `meshes` (each with its twins, which need the same morph
+    weights).
+  - `setHighlight(bone => 0..1)` lights up bones; call `setStrength(amount)` every frame.
+- **`HintStage`** handles the staging. It fits `span` to `height`, mirrors the model, draws the glow
+  disc behind it and runs the auto orbit.
+- **`sampleHintLoop(time, loop)`** samples the loop: rest → pose (held) → rest. It returns `{ amount,
+drive, phase, closing }`.
+- **`createClipBlend(root, fromClip, toClip)`** and **`createMorphBlend(meshes, targets)`** pose the
+  model; `apply(amount)` each frame.
+- **`orbitSpan(points, middle, amplitude)`** fits the model over the whole orbit.
+
+```tsx
+const MyHint = ({ height, renderOrder }: { height: number; renderOrder: number }) => {
+  const ghost = useGhostModel(
+    face.scene,
+    { ghost: { color: "#fde68a" } },
+    { base: FACE_HINT_LOOK, renderOrder },
+  );
+  const blend = useMemo(
+    () =>
+      createMorphBlend(
+        ghost.meshes.flatMap(({ mesh, twins }) => [mesh, ...twins]),
+        { surprised: 1, wink: 1 },
+      ),
+    [ghost.meshes],
+  );
+  const frame = useMemo(() => {
+    const box = new THREE.Box3().setFromObject(ghost.object);
+    const size = box.getSize(new THREE.Vector3());
+    return {
+      offset: box.getCenter(new THREE.Vector3()).negate(),
+      span: Math.max(size.x, size.y) * 1.2,
+    };
+  }, [ghost.object]);
+  useFrame(({ clock }) => {
+    const { amount } = sampleHintLoop(clock.elapsedTime, { seconds: 2 });
+    blend.apply(amount);
+    ghost.setStrength(amount);
+  });
+  return (
+    <HintStage
+      height={height}
+      span={frame.span}
+      orbit={20}
+      glow={ghost.glowMaterial}
+      renderOrder={renderOrder}
+    >
+      <group position={frame.offset}>
+        <primitive object={ghost.object} />
+      </group>
+    </HintStage>
+  );
+};
+```
+
+The material factories are exported for full control: `createGhostMaterial`, `createHaloMaterial`,
+`createGlowMaterial`, `createDepthPrepassMaterial`, `createHintMatcap`, `addHintTwins` and
+`assignHintMaterials`. Their opacities, colours and widths are uniforms (`userData.ghost`,
+`userData.halo`). The hold timing is pure too: `createGestureHoldState`, `stepGestureHold`,
+`GESTURE_HOLD_DEFAULT_MS` (400) and `GESTURE_HOLD_GRACE_MS` (150).
+
+## Example
+
+`/custom-instruction` has three demos, built only from this door:
+
+1. **Hang loose:** a custom gesture on the hand preset, with a restyled look, loop, text style and
+   ring.
+2. **Palm and smile:** an open palm and a smile held together. It uses an emoji symbol and a pill
+   text component.
+3. **Surprised wink:** your own hint, mixing two morph targets, for an expression with a match
+   function.
+
+## Manual visual test plan
+
+Open `/custom-instruction`.
+
+1. **Restyled preset**: the hang-loose hand is cyan with no halo and a blue glow. The thumb and
+   pinky are brighter, the fingers move together, and the text is a serif with a dark stroke. On the
+   webcam, do the gesture: a large cyan ring plays, and "Nice!" replaces the copy.
+2. **Any condition**: the emoji pair stands over a yellow pill. Show an open palm only: it stays
+   `detected`. Then smile with the palm up: it holds, then completes.
+3. **Own hint**: the gold face loops into a surprised wink with a dark-gold halo. A plain wink or a
+   plain open mouth doesn't complete it; both together do.
+4. **Presets unchanged**: `/hand-instructions`, `/face-instructions` and `/body-instructions` look
+   and behave as before. The exception is peace, point and the counts, which now show the palm.

package/docs/api/iris-tracking.mdx

@@ -28,15 +28,19 @@ useFrame(() => {
 
 ## Landmarks and sides
 
-| Side (person's) | Center | Ring (outer, top, inner, bottom) | Inner / outer corner | Upper / lower lid |
-| --------------- | ------ | -------------------------------- | -------------------- | ----------------- |
-| `left`          | 473    | 474, 475, 476, 477               | 362 / 263            | 386 / 374         |
-| `right`         | 468    | 469, 470, 471, 472               | 133 / 33             | 159 / 145         |
-
-Sides follow the face-mesh topology, so they are the person's anatomical
-sides on every source: mirrored webcam, straight video or image. The data
-needs the refined mesh (478 points); with fewer landmarks each eye reports
-`detected: false`. These indices are exported as `IRIS_LANDMARKS`. The
+| MediaPipe side | Center | Ring (outer, top, inner, bottom) | Inner / outer corner | Upper / lower lid |
+| -------------- | ------ | -------------------------------- | -------------------- | ----------------- |
+| `left`         | 473    | 474, 475, 476, 477               | 362 / 263            | 386 / 374         |
+| `right`        | 468    | 469, 470, 471, 472               | 133 / 33             | 159 / 145         |
+
+`result.left` / `result.right` are the person's own eyes on every source
+(GROUNDING "Left and right"). The table lists MediaPipe's sides
+(`IRIS_LANDMARKS`): its left eye sits on the image's right of the analysed
+frame, which is the person's left eye on a straight video, image or rear
+camera, and their RIGHT eye on the mirrored webcam. The SDK reads the mesh
+eye for the feed (`IRIS_LANDMARKS[personSide(side, mirrored)]`), so app code
+never swaps. The data needs the refined mesh (478 points); with fewer
+landmarks each eye reports `detected: false`. The
 allocation-free helpers behind the hook (`createIrisResult`,
 `updateIrisResult`, `computeIrisEye`, `IrisLead`) live on
 `@vincentt-xr/sdk/low-level` for pipelines that already hold their own
@@ -230,13 +234,14 @@ the example does.
 
 Run `/iris-tracking`:
 
-| Case                 | Steps                                   | Expect                                                             |
-| -------------------- | --------------------------------------- | ------------------------------------------------------------------ |
-| Shader material      | default, change color / scale / opacity | Both irises recoloured live; pupil follows the iris offset         |
-| Image material       | mode `image`, presets or a URL          | Image covers each iris, tinted by color; mirror flips it           |
-| Custom material      | "custom shader (ring)"                  | Pulsing ring from the shared uniforms                              |
-| Tracking loss        | cover the face / "Person Shaking Head"  | Effect holds ~250 ms then disappears; no stale eyes                |
-| Multi-face           | "Two People Smiling", face index 0 / 1  | Effect moves to the selected face                                  |
-| Mirrored camera      | webcam                                  | Left/right stay anatomical; the eye on the viewer's left is `left` |
-| Portrait / landscape | resize the pane                         | Quads stay centred on the irises at the right size                 |
-| Closed eyes          | blink                                   | Effect fades with confidence                                       |
+| Case                 | Steps                                   | Expect                                                        |
+| -------------------- | --------------------------------------- | ------------------------------------------------------------- |
+| Shader material      | default, change color / scale / opacity | Both irises recoloured live; pupil follows the iris offset    |
+| Image material       | mode `image`, presets or a URL          | Image covers each iris, tinted by color; mirror flips it      |
+| Custom material      | "custom shader (ring)"                  | Pulsing ring from the shared uniforms                         |
+| Tracking loss        | cover the face / "Person Shaking Head"  | Effect holds ~250 ms then disappears; no stale eyes           |
+| Multi-face           | "Two People Smiling", face index 0 / 1  | Effect moves to the selected face                             |
+| Mirrored camera      | webcam, `eye` left                      | Only your real left eye; on screen it is the eye on the left  |
+| Straight video       | a clip, `eye` left                      | Only the person's real left eye; on screen it is on the right |
+| Portrait / landscape | resize the pane                         | Quads stay centred on the irises at the right size            |
+| Closed eyes          | blink                                   | Effect fades with confidence                                  |

package/docs/api/made-up-camera.mdx

@@ -0,0 +1,131 @@
+---
+title: "Made-up Camera"
+description: "Give face insets, face textures and face swap the camera image with the makeup on it: one shared, opt-in, feed-size render of camera plus makeup."
+sidebarTitle: "Made-up Camera"
+icon: "wand-magic-sparkles"
+---
+
+Makeup is drawn over the live face in the main render. A component that samples the camera image
+itself (a face inset, a face texture, a face swap's live source) therefore shows the bare face.
+`<MadeUpCamera>` gives the components inside it the camera image with the makeup on it.
+
+It is off unless mounted. While mounted it renders one feed-size texture of camera plus makeup per
+frame, shared by every `<MadeUpCamera>`. Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { Blush, FaceInset, LipEffect, MadeUpCamera } from "@vincentt-xr/sdk/face-effects";
+import { FaceTracker } from "@vincentt-xr/sdk/tracking";
+
+<FaceTracker>
+  <Blush />
+  <LipEffect preset="lipstick" shade="classicRed" />
+  <MadeUpCamera>
+    {/* The giant mouth wears the lipstick */}
+    <FaceInset part="mouth" magnify={1.8} />
+  </MadeUpCamera>
+</FaceTracker>;
+```
+
+## What follows it
+
+Inside a `<MadeUpCamera>`:
+
+| Component                          | What shows the makeup                                                                                                 |
+| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
+| `FaceInset`, `useFaceInsetTexture` | The cut-out part                                                                                                      |
+| `FaceMorph`, `FaceReprojection`    | The camera-face texture they wear                                                                                     |
+| Face swap live faces               | The face map. A face's map is shared: one consumer inside a `<MadeUpCamera>` decides for all of that face's consumers |
+
+Components outside it keep the plain camera.
+
+`FaceWarp3D` always keeps the plain camera. Nest the makeup in the warp instead: it already rides it.
+
+The skin probe, the face swap's colour statistics and the makeup's own camera sampling always read the
+plain camera, so makeup never feeds back into itself.
+
+## What it holds
+
+- Every face-atlas layer drawn between `MAKEUP_RENDER_ORDER.prepass` and `.restore`, on every face:
+  makeup regions, `LipEffect`, `MakeupDesign`, custom `FaceAtlasLayer`s at makeup orders, and the
+  self-occlusion passes.
+- `HandCutout`, so hands stay bare.
+- Anything you add with `object.layers.enable(MADE_UP_CAMERA_LAYER)` (layer 30; 31 is the external background layer).
+
+Faces riding a `FaceWarp3D` or `FaceMorph` are drawn unwarped, where the camera sees them, with the
+unwarped face's normals.
+
+The pass is lit by the lights the main camera sees and writes and blends sRGB colour exactly as the
+canvas does, with the same shader programs, so the makeup's pixels match the screen. It does not hold:
+
+- `FaceRetouch`, face swaps or other face effects;
+- face-atlas layers outside the makeup orders;
+- the canvas's multisampling (antialiasing) or any post-processing.
+
+## Double makeup
+
+Makeup drawn on the same face again on top shows twice:
+
+- makeup nested in a `FaceMorph` that is wrapped in `<MadeUpCamera>`;
+- a wrapped face swap that lands on a face wearing makeup of its own (that makeup draws over the
+  swapped face).
+
+Wrap only the components that should show the makeup.
+
+## Your own materials
+
+`useMadeUpCamera()` holds the shared target and returns a handle, or `null` before mount:
+
+```tsx
+import { useMadeUpCamera } from "@vincentt-xr/sdk/face-effects";
+
+const madeUp = useMadeUpCamera();
+<mesh>
+  <planeGeometry />
+  <meshBasicMaterial map={madeUp?.ready ? madeUp.texture : null} toneMapped={false} />
+</mesh>;
+```
+
+| Handle            | Meaning                                                                                              |
+| ----------------- | ---------------------------------------------------------------------------------------------------- |
+| `texture`         | Camera plus makeup. Same layout as the camera texture (uv, aspect, image top at v = 1), sRGB         |
+| `ready`           | The texture holds the current camera image: false before the first pass and while the camera is gone |
+| `frame`           | Passes rendered so far                                                                               |
+| `width`, `height` | The texture's size: the camera image's                                                               |
+
+- `useMadeUpCameraTexture()` returns the nearest `<MadeUpCamera>`'s texture while it is ready, else
+  the session camera texture (null without a camera, never a stale made-up frame). The SDK
+  components use it.
+- `feedTextureVersion(texture)` returns a feed texture's content version: the pass count for the
+  made-up texture (its `version` never changes), `texture.version` for the camera. Compare it between
+  frames to skip work when the image did not change.
+
+## Cost
+
+Every frame while any holder is mounted:
+
+- the makeup is drawn a second time;
+- two feed-size passes: the camera copied under the makeup, then a copy into the sRGB texture;
+- a second walk of the scene graph.
+
+GPU memory: three 4-byte-per-pixel buffers at the camera image's size (colour, depth-stencil, output).
+That is about 4 MB for a 390 × 844 feed and 25 MB at 1080 × 1920.
+
+The pass runs at frame priority -0.3, after the face surfaces update. Frame callbacks at the default
+priority read this frame's image. The face swap's live map (priority -0.75) reads the previous
+frame's. Shadow maps are not re-rendered for the pass.
+
+## Manual visual test plan
+
+On the webcam, with blush and a red lipstick:
+
+1. **Inset**: a mouth inset inside `<MadeUpCamera>` shows the lipstick, with the same colour and edge
+   as on the face. Outside it, the inset shows bare lips.
+2. **Match**: put a `FaceInset part="face" magnify={1}` inside it. The face looks the same as without
+   the inset.
+3. **Hands**: a hand over the cheek (with `HandCutout`) stays bare in the inset.
+4. **Turned head**: the far cheek's blush stops at the nose in the inset, as on screen.
+5. **Face swap**: with makeup on face 0 only, a two-person swap inside it shows that makeup where
+   face 1 is.
+6. **Off**: unmount the `<MadeUpCamera>`. The frame time returns to its earlier value.

package/docs/api/makeup-skin-tone.mdx

@@ -0,0 +1,135 @@
+---
+title: "Makeup Skin Tone"
+description: "Read the person's skin depth, undertone, lip and brow colours from the skin probe, and get the curated palette shades that suit them, for shade-finder UIs."
+sidebarTitle: "Makeup Skin Tone"
+icon: "swatchbook"
+---
+
+`useMakeupSkinTone` reads the face's skin probe back from the GPU a few times a second and returns
+what a shade finder needs:
+
+- **Skin depth**: `fair`, `light`, `medium`, `tan`, `deep` or `rich`.
+- **Undertone**: `cool`, `neutral` or `warm`.
+- **The person's own colours**: settled skin, lips and brow hair, as sRGB hex.
+- **Recommendations**: for every makeup region, the curated palette shades (`MAKEUP_SHADES`) nearest
+  what that region would paint on this person, best first.
+
+The SDK draws no labels. Makeup does not need this readout: `shade: "auto"` already adapts on the
+GPU. Import from `@vincentt-xr/sdk/face-effects`.
+
+## Quick start
+
+```tsx
+import { Blush, Contour, useMakeupSkinTone } from "@vincentt-xr/sdk/face-effects";
+
+const tone = useMakeupSkinTone(); // null until the skin probe has settled
+<Contour shade={tone?.recommendations.contour[0] ?? "medium"} />;
+<Blush shade={tone?.recommendations.blush[0] ?? "rose"} />;
+```
+
+Call it inside the canvas. It works alone: it starts the face model when no `FaceTracker` is above
+it, and starts the face's skin probe when no makeup runs one.
+
+## Options
+
+| Option       | Default         | Meaning                                                        |
+| ------------ | --------------- | -------------------------------------------------------------- |
+| `faceIndex`  | rig / tracker   | Which face: the enclosing `FaceRig` or `FaceTracker`'s, else 0 |
+| `numFaces`   | `faceIndex + 1` | Faces to start the face model with, when this hook starts it   |
+| `intervalMs` | 500             | Milliseconds between reads, at least 200                       |
+| `count`      | 3               | Shades recommended per region                                  |
+| `quality`    | `"auto"`        | The skin probe's quality (`useFaceSkin`)                       |
+| `enabled`    | `true`          | `false` stops reading and releases the probe                   |
+
+## The readout
+
+| Field             | Meaning                                                                                                     |
+| ----------------- | ----------------------------------------------------------------------------------------------------------- |
+| `skin`            | The settled skin colour as the camera shows it, sRGB hex                                                    |
+| `lab`             | Its CIELAB (D65) `[L*, a*, b*]`                                                                             |
+| `depth`, `ita`    | Depth from the Individual Typology Angle, `atan((L* − 50) / b*)` in degrees                                 |
+| `undertone`       | From the CIELAB hue angle                                                                                   |
+| `hue`, `chroma`   | The skin's CIELAB hue (degrees) and chroma, for your own thresholds                                         |
+| `lips`, `brows`   | The person's lip and brow-hair colours, or `null` while under a quarter of them was seen                    |
+| `recommendations` | Palette shade names per region: `lips`, `contour`, `blush`, `highlighter`, `eyeshadow`, `eyeliner`, `brows` |
+
+| Depth    | ITA         |
+| -------- | ----------- |
+| `fair`   | over 55°    |
+| `light`  | 41° to 55°  |
+| `medium` | 28° to 41°  |
+| `tan`    | 10° to 28°  |
+| `deep`   | −30° to 10° |
+| `rich`   | under −30°  |
+
+Undertone: hue under 50° is `cool`, over 60° is `warm`, between is `neutral`. Deeper skin has a lower
+hue whatever its undertone, so below `L*` 50 both cuts drop 0.25° per unit: at `L*` 20 they are 42.5°
+and 52.5°. A skin with chroma under 6 (a grey or washed-out image) is always `neutral`. The thresholds
+are `MAKEUP_SKIN_TONE`.
+
+## How shades are chosen
+
+Each region gets a target colour: what `shade: "auto"` paints on this person.
+
+- **Lips**: their own lips, deepened.
+- **Blush**: their skin moved toward rose.
+- **Contour**: their skin, deepened.
+- **Highlighter**: their skin, lifted.
+- **Brows**: their brow hair.
+- **Eyeshadow**: a transition shade, the skin deepened.
+- **Eyeliner**: the brow hair, darkened.
+
+Warm skin leans blush, highlighter and eyeshadow toward peach, gold and bronze; cool skin toward rose,
+pearl and taupe. The recommendations are the palette shades nearest each target (CIELAB ΔE). Unseen
+lips or brows fall back to colours derived from the skin.
+
+## Camera light
+
+The readout measures the image, not the person:
+
+- **White balance** moves the undertone. A warm room reads warm; a cool screen glow reads cool.
+- **Exposure** moves the depth. A backlit face reads deeper.
+
+Present the result as a suggestion the user can change, never as a label. Ask for even, neutral
+light when it matters.
+
+## Cost
+
+No per-frame allocation. Each read is one 24 × 1 packing draw and an asynchronous copy to the CPU
+(`FaceSkinProbe.readStatsAsync`), with one read in flight at a time. The hook re-renders only when the
+readout changes.
+
+## Photos
+
+`computeMakeupSkinTone(stats)` takes any `FaceSkinStats`:
+
+```ts
+import {
+  FaceSkinProbe,
+  computeMakeupSkinTone,
+  readFaceSkinStats,
+} from "@vincentt-xr/sdk/face-effects";
+
+const probe = new FaceSkinProbe({ renderer: gl });
+probe.renderStill(gl, photoTexture, photoLandmarks);
+const data = await probe.readStatsAsync(gl);
+const tone = data ? computeMakeupSkinTone(readFaceSkinStats(data)) : null;
+probe.dispose();
+```
+
+Also exported: `classifyMakeupSkinDepth`, `classifyMakeupUndertone`, `nearestMakeupShades`,
+`MAKEUP_SKIN_DEPTHS`, `MAKEUP_UNDERTONES`.
+
+## Manual visual test plan
+
+Show `tone.depth`, `tone.undertone`, `tone.skin` and the first blush, contour and lip shade in a
+debug readout on the webcam.
+
+1. **Settles**: the readout appears within a second of the face showing, and stays steady while you
+   hold still.
+2. **Light**: switch from daylight to a warm lamp. The undertone moves toward warm (the documented
+   caveat); the shades follow.
+3. **Turn away**: past about 35° the readout holds its last value.
+4. **Lips**: wear a strong lipstick. The lip recommendations move toward it within a few seconds.
+5. **Two faces**: `faceIndex: 1` reads the second person.
+6. **Cost**: the frame time does not change with the hook mounted.