@vincentt-xr/sdk 2.0.0-alpha.1 → 2.0.0-alpha.11

package/docs/api/makeup.mdx

@@ -0,0 +1,353 @@
+---
+title: "Makeup"
+description: "Lipstick, contour, blush, highlighter, eyeshadow, eyeliner, brows and painted designs on the live face, shaded against the person's own skin and light, with looks, shades and six finishes."
+sidebarTitle: "Makeup"
+icon: "brush"
+---
+
+Makeup paints the live face. Each region is its own component, imported from
+`@vincentt-xr/sdk/face-effects`:
+
+| Region        | Component      | Paints                                             |
+| ------------- | -------------- | -------------------------------------------------- |
+| `lips`        | `LipEffect`    | Lipstick, gloss, tint, lip designs                 |
+| `contour`     | `Contour`      | Cheek hollows, jawline, temples, sides of the nose |
+| `blush`       | `Blush`        | The cheeks                                         |
+| `highlighter` | `Highlighter`  | Cheekbones, brow bone, nose bridge, cupid's bow    |
+| `eyeshadow`   | `Eyeshadow`    | The lids, crease, lower lash line and inner corner |
+| `eyeliner`    | `Eyeliner`     | Upper and lower lash lines, wings                  |
+| `brows`       | `Brows`        | The person's own brows, tinted, filled or reshaped |
+| —             | `MakeupDesign` | Your own art: freckles, face gems, graphic liner   |
+
+Paint is shaded relative to the person's own skin, lip colour and light, so a shade reads as its named
+colour on every skin tone and follows the room light. Every edit (look, shade, shape, finish, side) is
+a uniform change, never a shader recompile.
+
+## Quick start
+
+```tsx
+import { VideoBackground, XRScene } from "@vincentt-xr/sdk";
+import {
+  Blush,
+  Brows,
+  Contour,
+  Eyeliner,
+  Eyeshadow,
+  Highlighter,
+  LipEffect,
+  MAKEUP_LOOKS,
+} from "@vincentt-xr/sdk/face-effects";
+
+const look = MAKEUP_LOOKS.softGlam;
+
+<XRScene>
+  <VideoBackground />
+  <Contour {...look.contour} />
+  <Blush {...look.blush} shade="coral" />
+  <Highlighter {...look.highlighter} />
+  <Eyeshadow {...look.eyeshadow} />
+  <Eyeliner {...look.eyeliner} />
+  <Brows {...look.brows} />
+  <LipEffect {...look.lips} />
+</XRScene>;
+```
+
+Spread a look region into its component and put overrides after the spread. No prop is required:
+`<Blush />` paints the default blush. `LipEffect` suspends once on mount, so keep it under a
+`<Suspense>`.
+
+## Where it binds
+
+No `FaceTracker` is needed.
+
+- **Alone**: the component starts the face model itself, with `numFaces` faces (at least
+  `faceIndex + 1`). Pass the same `numFaces` to every face component.
+- **Inside a `FaceTracker`**: it paints that tracker's face.
+- **Nested in a `FaceRig`, `FaceWarp3D` or `FaceMorph`**: it rides that face, warp included. Makeup
+  outside a warp does not follow it.
+
+Every region on one face shares one live surface and one skin probe. A face lost for under 250 ms
+holds; then the makeup fades out.
+
+## Looks
+
+`MAKEUP_LOOKS[name]` holds a `label`, a `description` and partial settings for all seven regions.
+A region a look leaves bare is `{ enabled: false }`.
+
+| Look       | `MAKEUP_LOOKS` key |
+| ---------- | ------------------ |
+| Natural    | `natural`          |
+| Soft glam  | `softGlam`         |
+| Sun-kissed | `sunKissed`        |
+| Glam       | `glam`             |
+| Smoky eye  | `smokyEye`         |
+| Bold lip   | `boldLip`          |
+| Editorial  | `editorial`        |
+| Glitter    | `glitter`          |
+
+`MAKEUP_LOOK_NAMES` lists them in this order. Only `natural` uses `"auto"` shades.
+
+## Settings
+
+Every region has flat, serializable settings (`BlushSettings`, `ContourSettings`, …;
+`MakeupSettings[region]` maps each region to its type). They are the component's props.
+
+| Field                                           | Meaning                                                                                   |
+| ----------------------------------------------- | ----------------------------------------------------------------------------------------- |
+| `enabled`                                       | `false` draws nothing and holds nothing (default `true`)                                  |
+| `shape`                                         | A placement preset (`MAKEUP_SHAPE_NAMES[region]`) or `"custom"`. Lips: `preset`           |
+| `shade`                                         | A `MAKEUP_SHADES[region]` name, `"auto"` where supported, or `"custom"`                   |
+| `color`                                         | CSS colour. A colour that differs from the shade wins, and `shade` becomes `"custom"`     |
+| `opacity`                                       | Linear amount 0..1. `0` hides the region but keeps it ready                               |
+| `coverage`                                      | 0 sheer (filters the person's own colour, never lightens) … 1 opaque                      |
+| `finish`, `shine`                               | One of six finishes (below) and its highlight strength 0..1                               |
+| `sparkleColor`, `sparkleDensity`, `sparkleSize` | Flakes for `shimmer` and `glitter`: colour (`""` = from the shade), density 0..1, size mm |
+| `softness`                                      | Edge feather 0..1                                                                         |
+| `side`                                          | `"both"`, `"left"` or `"right"`: the person's own side on every feed                      |
+
+Lips have the same fields except `softness` and `side` (`edgeSoftness` sets their edge), with
+`preset` in place of `shape`.
+
+Region fields:
+
+| Region        | `shape` values                                                                               | Extra fields                                                                   |
+| ------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
+| `contour`     | `soft`, `sculpted`, `bronzed`, `nose`                                                        | `size` 0.5..1.5                                                                |
+| `blush`       | `apples`, `lifted`, `draped`, `sunKissed`                                                    | `size` 0.5..1.5; `lift` −1..1 (positive toward the temple)                     |
+| `highlighter` | `cheekbones`, `strobe`, `glow`, `nose`                                                       | `size` 0.5..1.5                                                                |
+| `eyeshadow`   | `lid`, `crease`, `smoky`, `cutCrease`, `halo`, `winged`                                      | `height`, `crease` + `creaseShade`, `lower`, `inner` + `innerShade` (all 0..1) |
+| `eyeliner`    | `tightline`, `classic`, `winged`, `catEye`, `smudged`, `kohl`                                | `thickness` mm 0.2..4; `wing` mm 0..15; `wingAngle` 0..45°; `lower`; `smudge`  |
+| `brows`       | `natural`, `softArch`, `straight`, `highArch`, `bold`, `thin`                                | `boldness` (0.5 = their own brow); `fill` (0 tints the hairs only); `strokes`  |
+| `lips`        | `preset`: `lipstick`, `gloss`, `matte`, `tint`, `metallic`, `gradient`, `shimmer`, `glitter` | The `LipEffect` fields                                                         |
+
+`shape: "custom"` applies no preset: the region keeps its default placement, with the fields as
+given.
+
+### Resolving and editing
+
+Resolution order: defaults, then the shape, then the finish's `shine` and sparkle values, then the
+shade's colour, then explicit values. Numbers are clamped; invalid values fall back.
+
+```tsx
+import {
+  patchMakeupSettings,
+  resolveMakeupLook,
+  resolveMakeupSettings,
+} from "@vincentt-xr/sdk/face-effects";
+
+const blush = resolveMakeupSettings("blush", { shape: "lifted", shade: "coral" });
+const glitter = patchMakeupSettings("blush", blush, { finish: "glitter" });
+const look = resolveMakeupLook("softGlam", { eyeliner: { shape: "winged" } });
+```
+
+- Spread-then-override works for partials (a look's regions, your own objects).
+- A resolved object carries the values its shape and finish produced, so `{ ...resolved, finish:
+"glitter" }` changes nothing else. Change resolved settings with `patchMakeupSettings`.
+- `MAKEUP_SETTINGS_FIELDS[region]` (lips: `LIP_EFFECT_SETTINGS_FIELDS`) are typed `SettingsPanel`
+  fields for an editing panel. Pass what the panel emits through `patchMakeupSettings`.
+
+## Shades and `"auto"`
+
+`MAKEUP_SHADES[region]` maps generic names (`coral`, `champagne`, `softBlack`, …) to sRGB hex;
+`MAKEUP_SHADE_NAMES[region]` lists them in palette order.
+
+`shade: "auto"` takes the colour from the person, on the GPU, with no readback:
+
+| Region        | `"auto"` colour                   |
+| ------------- | --------------------------------- |
+| `lips`        | their own lip colour, deepened    |
+| `blush`       | their skin hue, moved toward rose |
+| `contour`     | their skin, darkened              |
+| `highlighter` | their skin, lifted                |
+| `brows`       | their brow-hair colour            |
+
+Eyeshadow and eyeliner have no `"auto"`. With `"auto"`, `color` is shown for the first ~0.3 s, until
+the skin probe has measured the colour. To show a shade finder, read the person's depth, undertone
+and best palette shades with [`useMakeupSkinTone`](/api/makeup-skin-tone).
+
+## Finishes
+
+| `finish`   | Look                                                        |
+| ---------- | ----------------------------------------------------------- |
+| `matte`    | The skin's own shine removed                                |
+| `satin`    | A soft sheen                                                |
+| `gloss`    | A wet highlight that keeps its strength at low opacity      |
+| `shimmer`  | Satin plus dense fine flakes                                |
+| `metallic` | A highlight tinted by the shade, from the room light        |
+| `glitter`  | Sparse large flakes that sparkle with head and light motion |
+
+Flakes stick to the skin and fade to their average below 1.5 px, so they never crawl. The low
+quality tier turns shimmer into a smooth sheen and halves glitter density.
+
+## Sides
+
+`side` is the person's own side on every feed: the mirrored webcam, the rear camera, images and
+video. `<Blush side="left" />` paints the person's left cheek, which is on the screen's left on the
+mirrored webcam and on the screen's right on the rear camera.
+
+## Hands in front
+
+Add `HandCutout` so a hand over the face stays bare and in front of the makeup:
+
+```tsx
+import { HandCutout, HandTracker } from "@vincentt-xr/sdk/tracking";
+
+<XRScene>
+  <VideoBackground />
+  <HandTracker>
+    <HandCutout />
+  </HandTracker>
+  <Blush shade="rose" />
+  <LipEffect preset="lipstick" shade="classicRed" />
+</XRScene>;
+```
+
+`HandCutout` draws each hand's silhouette as depth at the near plane before the face layer, so
+makeup is hidden where a hand is.
+
+## Two people
+
+Give each component `faceIndex` and the same `numFaces`:
+
+```tsx
+<>
+  <Blush shade="rose" faceIndex={0} numFaces={2} />
+  <Blush shade="peach" faceIndex={1} numFaces={2} />
+  <LipEffects
+    maxFaces={2}
+    faces={(faceIndex) => (faceIndex === 0 ? { preset: "matte" } : { preset: "gloss" })}
+  />
+</>
+```
+
+Inside `<FaceTracker numFaces={2} faceIndex={n}>`, the component paints that tracker's face.
+`LipEffects` draws one lip effect per face; with a single config the faces share one material.
+
+## Draw order
+
+Makeup draws in the face layer (`MAKEUP_RENDER_ORDER`):
+
+| Order | What                                                       |
+| ----- | ---------------------------------------------------------- |
+| −900  | `HandCutout`                                               |
+| −2.5  | `FaceRetouch`                                              |
+| −2    | The face: `FaceWarp3D`, `FaceMorph`, characters, face swap |
+| −1.9  | `contour`                                                  |
+| −1.86 | `blush`                                                    |
+| −1.83 | `highlighter`                                              |
+| −1.8  | `eyeshadow`                                                |
+| −1.76 | `eyeliner`                                                 |
+| −1.72 | `brows`                                                    |
+| −1.68 | `design` (`MakeupDesign`)                                  |
+| −1.64 | `lips`                                                     |
+| −1.5  | `FaceInset`                                                |
+| −1    | Head occluder                                              |
+| 0+    | Accessories                                                |
+
+Hands and glasses draw over makeup. On a turned head the far cheek stops at the nose (high and
+medium quality tiers). Effects that show the camera image (`FaceInset`, `FaceMorph`, `FaceReprojection`, face swap) show
+it without makeup; wrap them in [`MadeUpCamera`](/api/made-up-camera) to include it. `FaceWarp3D`
+always shows the plain camera: nest the makeup in the warp instead.
+
+## Designs
+
+`MakeupDesign` paints art you made in the face atlas, shaded like makeup:
+
+```tsx
+import { MakeupDesign } from "@vincentt-xr/sdk/face-effects";
+
+<MakeupDesign
+  texture="/makeup/freckles.svg"
+  region={["cheeks", "nose"]}
+  coverage={0.45}
+  opacity={0.8}
+/>;
+```
+
+The art's alpha is where it paints; its colours are the paint. See
+[Makeup Design Textures](/guides/makeup-design-textures) for painting it and
+[Face Atlas Textures](/guides/face-atlas-textures) for the layout.
+
+## Custom materials
+
+Every region component and `MakeupDesign` take `material`: a material node, an instance you own, or
+a factory of `MakeupMaterialContext`. Lips have their own material contract (see
+[Lip Shaders](/examples/lip-shaders)). Pass `ctx.uniforms` (kept current from the settings and bound to the face)
+and build the shader from the `MAKEUP_GLSL` chunks:
+
+```tsx
+import * as THREE from "three";
+import {
+  Blush,
+  MAKEUP_BLENDING,
+  MAKEUP_GLSL,
+  type MakeupMaterialContext,
+} from "@vincentt-xr/sdk/face-effects";
+
+<Blush
+  shape="draped"
+  material={(ctx: MakeupMaterialContext) => (
+    <shaderMaterial
+      glslVersion={THREE.GLSL3}
+      {...MAKEUP_BLENDING}
+      depthWrite={false}
+      uniforms={ctx.uniforms}
+      vertexShader={MAKEUP_GLSL.atlasVertex}
+      fragmentShader={`${MAKEUP_GLSL.common}${MAKEUP_GLSL.shading}${MAKEUP_GLSL.sparkle}${MAKEUP_GLSL.finish}${MAKEUP_GLSL.shapes}
+        void main() {
+          MakeupSample s = makeupSampleGeometry();
+          float k = blushShape(s.rest) * s.fade * uOpacity;
+          if (k < 1.0 / 255.0) discard;
+          makeupSampleColor(s);
+          makeupOutput(s, makeupDiffuse(s, uColor, uCoverage), k, vec3(0.0));
+        }`}
+    />
+  )}
+/>;
+```
+
+- The factory runs once per region and factory, not on edits: read live values from `ctx.uniforms`
+  and `ctx.skin`. Keep a factory that returns a `THREE.Material` stable (module scope or
+  `useCallback`).
+- To drive one material yourself on every face, make it with `createMakeupMaterial(region)`, edit it
+  with `updateMakeupMaterial(material, { settings })` and pass it as `sharedMaterial`. You dispose
+  it.
+- The eye masks are in `MAKEUP_GLSL.eyes`; the design sampler is in `MAKEUP_GLSL.design`.
+
+## Cost and limits
+
+- One draw per region per face, over only the triangles its shape can reach. Pixels outside the
+  shape are discarded before any texture read.
+- One GPU program serves contour, blush and highlighter; each eye region and designs have their own.
+  Each is compiled once on first mount; the region shows when it is ready, within 2 s.
+- Eye shapes are placed from MediaPipe's lash and brow landmarks. Very hooded lids, brows under
+  bangs, or brow landmarks off the real brow shift them.
+- A long eyeliner wing reads up to ~15 % shorter on the far side of a turned eye.
+- The skin probe needs the face in view. Until it settles, `"auto"` shows `color`.
+
+## Example
+
+`/makeup` in the SDK playground has the eight looks and a custom mix, a panel per region, the lip
+effect with designs and custom materials, the skin-tone readout with "apply", one or two faces, a
+compare toggle and hands in front. Settings are kept in the URL: `?look=glam&faces=2&hands=0`.
+
+## Manual visual test plan
+
+Open `/makeup` on the webcam, in even light.
+
+1. **Looks**: step through all eight. Each matches its description; nothing flickers or swims while
+   you move your head.
+2. **Compare**: toggle "makeup on". Only the made-up regions change.
+3. **Skin tones**: on the two-people clip, the same opaque shade reads as the same colour on both
+   faces.
+4. **Auto**: `natural` adapts to your own lips, skin and brows within about a second.
+5. **Sides**: blush with `side` left paints your real left cheek (the screen's left on the webcam).
+   On a clip, it paints the person's left cheek (the screen's right).
+6. **Hands**: a hand over the mouth and cheek stays bare. Turn "hands in front" off: the makeup
+   draws over the hand.
+7. **Turned head**: the far cheek's blush stops at the nose.
+8. **Blinks**: close one eye. Its lower liner and lower shadow fade; the other eye's stay.
+9. **Two faces**: faces "two" paints both people; "second face" gives the second person its own look.
+10. **Finishes**: glitter flakes stick to the skin and sparkle with head motion; gloss shows a moving
+    highlight.
+11. **Dropout**: cover the camera. The makeup holds briefly, then fades out.

package/docs/api/material-render-state.mdx

@@ -0,0 +1,132 @@
+---
+title: "Material Render State API"
+description: "MeshRenderer material draw-state controls: depth, color mask, culling, blending, and polygon offset."
+sidebarTitle: "Material Render State"
+icon: "sliders"
+---
+
+Material render state lives on `MeshRendererSettings.material.renderState`.
+
+```ts
+import {
+  DEFAULT_MESH_RENDERER_RENDER_STATE,
+  normalizeMeshRendererRenderState,
+  type MeshRendererRenderState,
+} from "@vincentt-xr/sdk";
+```
+
+## Type definition
+
+```ts
+type MeshRendererRenderState = {
+  depthTest: boolean;
+  depthWrite: boolean;
+  depthFunc:
+    | "never"
+    | "less"
+    | "equal"
+    | "lessEqual"
+    | "greater"
+    | "notEqual"
+    | "greaterEqual"
+    | "always";
+  colorWrite: boolean;
+  colorMask: {
+    r: boolean;
+    g: boolean;
+    b: boolean;
+    a: boolean;
+  };
+  cullMode: "back" | "front" | "off";
+  blendMode: "normal" | "additive" | "subtractive" | "multiply" | "none";
+  polygonOffset: {
+    enabled: boolean;
+    factor: number;
+    units: number;
+  };
+};
+```
+
+## Defaults
+
+```ts
+const DEFAULT_MESH_RENDERER_RENDER_STATE = {
+  depthTest: true,
+  depthWrite: true,
+  depthFunc: "lessEqual",
+  colorWrite: true,
+  colorMask: { r: true, g: true, b: true, a: true },
+  cullMode: "back",
+  blendMode: "normal",
+  polygonOffset: { enabled: false, factor: 0, units: 0 },
+};
+```
+
+Use `normalizeMeshRendererRenderState(partial)` when reading serialized scenes
+that may omit nested defaults.
+
+## Usage
+
+```ts
+const meshRenderer = createDefaultComponent("MeshRenderer", object.id);
+
+meshRenderer.settings.material.renderState = {
+  depthTest: true,
+  depthWrite: false,
+  depthFunc: "lessEqual",
+  colorWrite: true,
+  colorMask: { r: true, g: false, b: false, a: true },
+  cullMode: "off",
+  blendMode: "additive",
+  polygonOffset: {
+    enabled: true,
+    factor: -1,
+    units: -1,
+  },
+};
+```
+
+## Field behavior
+
+| Field           | Behavior                                                                                |
+| --------------- | --------------------------------------------------------------------------------------- |
+| `depthTest`     | Enables/disables depth testing before drawing the material.                             |
+| `depthWrite`    | Enables/disables writing this material into the depth buffer.                           |
+| `depthFunc`     | Selects the depth comparison function.                                                  |
+| `colorWrite`    | All-channel color write gate.                                                           |
+| `colorMask`     | Per-channel RGBA color write gates.                                                     |
+| `cullMode`      | `"back"` renders front faces, `"front"` renders back faces, `"off"` renders both sides. |
+| `blendMode`     | Maps to Three.js built-in blending modes.                                               |
+| `polygonOffset` | Offsets depth to reduce z-fighting or tune layering.                                    |
+
+`colorMask` is not a plain Three.js material field. The SDK applies it as scoped
+WebGL draw state around each mesh render and restores the previous color mask
+after the object renders.
+
+## Asset-backed meshes
+
+When `MeshRendererSettings.assetUrl` is set, the GLTF scene is cloned and mesh
+materials are cloned before SDK material/render-state overrides are applied.
+This prevents two objects using the same asset URL from mutating the shared Drei
+GLTF cache.
+
+## Debug UI
+
+`MeshRendererSettingsPanel` exposes render-state controls:
+
+- Depth Test
+- Depth Write
+- Depth Function
+- Color Write
+- Mask R/G/B/A
+- Cull Mode
+- Blend Mode
+- Polygon Offset enabled/factor/units
+
+All edits should route back to the selected object's `MeshRenderer` component
+through `onComponentChange`.
+
+## Explicit non-goal
+
+`instanceCount` is not part of material render state. It belongs to mesh
+instancing and should be added as a separate mesh/rendering API when needed.

package/docs/api/multi-subject.mdx

@@ -0,0 +1,185 @@
+---
+title: "Multi-Subject Tracking Policy"
+description: "The identity and association contract for faces, hands, bodies and holistic results: what an index means, how many subjects each tracker reports, how to select one, what happens on loss and reappearance, and how subjects are matched across trackers."
+sidebarTitle: "Multi-Subject Policy"
+icon: "people-group"
+---
+
+## Identity contract
+
+- **An index is a result-array position, not an identity.** `faceIndex`,
+  `handIndex`, `getFace(i)`, `getLandmarks2D(i)` read position `i` of the
+  current frame's result. The order is whatever MediaPipe emits; it can
+  change between frames when subjects move, overlap, or one is lost. Nothing
+  in the SDK promises that `faceIndex: 1` is the same person two frames
+  later.
+- **No learned identity.** The SDK never recognises people. The only
+  continuity it offers is spatial: the `sticky` selection below follows the
+  subject nearest to the previously selected one.
+- **Bodies carry a stable id.** The body model gives each tracked body an
+  `id` (`node.getBodies()`, see [Bodies](#bodies)) that stays with the person
+  across result-order changes, two people crossing and dropouts under 250 ms.
+  It is spatial too: someone who leaves for longer comes back with a new id.
+- **Single-subject APIs are unchanged.** `useFaceInfo()`, `<FaceTracker>`,
+  `<HandTracker>`, `<TrackingAnchor>` keep reading index 0 by default; the
+  policy is an opt-in option on top.
+
+## Maximum subjects
+
+| Tracker  | Max | Default | Option                                                                                                 |
+| -------- | --- | ------- | ------------------------------------------------------------------------------------------------------ |
+| face     | 10  | 1       | `numFaces` on `<FaceTracker>` / `useFaceInfo(s)`; the first acquirer of the shared model wins          |
+| hand     | 2   | 2       | none (two hands, `handedness` labels them)                                                             |
+| gesture  | 1   | 1       | none                                                                                                   |
+| body     | 2   | 1       | `numBodies` on the body features and `useBodyIds`; the model looks for the most any live consumer asks |
+| holistic | 1   | 1       | none (single person by design)                                                                         |
+
+`XR_MAX_SUBJECTS` carries this table. Every consumer that needs more than one
+face must ask for it; a consumer that asks for fewer than the first acquirer
+gets the larger count. The count the face model started with is the read-only
+`numFaces` on its node (`useXRModelNode<XRModelNodeFaceTracking>(XRModel.FACE_TRACKER)`
+from `@vincentt-xr/sdk/low-level`), so a consumer that asked for more can tell
+that it got fewer.
+
+## Selecting one subject
+
+```tsx
+import { useFaceInfo, selectSubject, faceCandidates } from "@vincentt-xr/sdk/tracking";
+
+// Hook form (faces): pick by policy instead of a fixed index.
+const face = useFaceInfo({ numFaces: 4, subject: { selection: "largest" } });
+const follow = useFaceInfo({ numFaces: 4, subject: { selection: "sticky", stickyRadius: 0.2 } });
+
+// Pure form (any tracker): build candidates, select, carry the memory.
+const result = selectSubject(faceCandidates(results.faceLandmarks), policy, previousMemory);
+```
+
+| `selection`              | Picks                                                                                                                                                                                                                     |
+| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `index`                  | position `policy.index` (default 0) — the historical behaviour                                                                                                                                                            |
+| `largest`                | the biggest subject (closest to the camera)                                                                                                                                                                               |
+| `nearest-center`         | closest to the feed centre                                                                                                                                                                                                |
+| `leftmost` / `rightmost` | by feed x, as shown in the preview                                                                                                                                                                                        |
+| `sticky`                 | the same `id` as the previous pick when the tracker keeps ids (bodies); else nearest to it within `stickyRadius` (feed units, default 0.2), same `label` (handedness) preferred; otherwise `fallback` (default `largest`) |
+
+`selectSubject` returns `{ index, reason, memory }`; `index` is `null` when
+there is no candidate. `faceCandidates(landmarkArrays)` and
+`jointCandidates(handsOrBodies)` build candidates (feed-space centre and
+size) from face landmarks or tracker-space joints; `bodyCandidates(bodies)`
+does it for tracked bodies and carries their ids.
+
+## Loss and reappearance
+
+- A subject that disappears for a few frames is bridged by the tracker's
+  hold (`holdMs`, 250 ms by default, through `FaceTrackMemory` / the readers'
+  hold): the last result is replayed, then dropped.
+- `sticky` keeps its memory through a miss, so a subject that reappears
+  near where it was is re-selected; one that reappears elsewhere is a new
+  subject and the policy's `fallback` decides. When the selection switches
+  to the fallback, the memory moves with it (a deliberate switch, not a jump
+  back later).
+- Reappearance never restores a previous index: read the policy's pick, not
+  a remembered position.
+
+## Bodies
+
+The body model looks for up to two people. Ask for a second with `numBodies`
+(default 1): looking for two, MediaPipe detects every frame, which costs about
+twice as much per detection even with one person in view (desktop: 9 ms for
+one body, 14 ms asking for two with one person in view, 20 ms with two
+people). Ask for two only when the scene needs them; on slow devices cap the
+body model's inference rate (`trackers.body.targetFps`).
+
+Every body feature follows a chosen person. `bodyId` follows a stable id,
+`subject` picks by policy, `bodyIndex` reads a result position (the default,
+0); `bodyId` wins over `subject`, which wins over `bodyIndex`. `numBodies`
+defaults to 2 with `bodyId` or `subject`, else 1.
+
+| feature                                   | options                                       |
+| ----------------------------------------- | --------------------------------------------- |
+| `BodyTracker` (and its `TrackingAnchor`s) | `bodyId`, `subject`, `numBodies`, `bodyIndex` |
+| `BodyRig`                                 | the same                                      |
+| `BodyTrigger` / `useBodyTrigger`          | the same                                      |
+| `BodyInstruction` / `useBodyPose`         | the same                                      |
+| `useGesture` (body gestures)              | the same                                      |
+
+Per-person masks (`PersonCutout bodyId`, `useBodyMaskTexture`) split the person mask by
+these ids; see [Person Cutout](/api/person-cutout#one-person-bodyid-subject-usebodymasktexture).
+Fitted children of a `BodyTracker` (`ShoulderOccluder`, `Necklace`,
+`NeckOccluder`) follow the person it follows. `HandRig`'s forearm steering
+reads the arms of every tracked body.
+
+```tsx
+import {
+  BodyRig,
+  BodyTracker,
+  TrackingAnchor,
+  useBodyIds,
+  useBodySubject,
+} from "@vincentt-xr/sdk/tracking";
+
+// One avatar per person.
+const Avatars = () => {
+  const ids = useBodyIds(); // re-renders only when someone arrives or leaves
+  return ids.map((id) => <BodyRig key={id} bodyId={id} model={avatar.scene} />);
+};
+
+// A badge that stays on one person as they move around another.
+<BodyTracker subject={{ selection: "sticky", fallback: "leftmost" }}>
+  <TrackingAnchor target="body.leftShoulder">{badge}</TrackingAnchor>
+</BodyTracker>;
+
+// Custom per-frame logic.
+const { read } = useBodySubject({ subject: { selection: "largest" } });
+useFrame(() => {
+  const body = read(); // { pose, id, held, index } or null
+});
+```
+
+**Identity rules.** A detection is matched to the bodies already tracked by
+torso position (predicted from its motion) and size. A newcomer gets an id
+once seen 3 times over 100 ms (the tracker reports a part of a person, such
+as a crouching dancer's legs, as another body for a frame or two); the first
+person into an empty scene gets one at once. A missing body is held 250 ms
+(`held: true`, its last pose), then dropped. The matcher is `BodyIdentity`
+(`/low-level`, `BODY_IDENTITY_DEFAULTS`).
+
+**Limits.** Two people touching (holding hands) can have a leg read toward
+the other person. Two people cut off at the shoulders (a selfie) are found
+intermittently, so the second is renumbered on each return. Results and
+measurements: `example/BODY_EVALUATION.md` (Two people).
+
+## Association across trackers
+
+`associateSubjects(primary, secondary, maxDistance = 0.35)` greedily matches
+the nearest centres between two candidate lists (face ↔ body, hand ↔ body),
+each subject at most once, ignoring pairs farther than `maxDistance` (feed
+units). That proximity-in-the-same-frame rule is the whole association
+contract; with one body (the default) or the single-person holistic model it
+reduces to "the body belongs to the nearest face".
+
+## Manual validation
+
+Run `/multi-subject` (faces: starts on the two-people image, `numFaces` 4):
+
+| Case       | Steps                                           | Expect                                              |
+| ---------- | ----------------------------------------------- | --------------------------------------------------- |
+| Candidates | two / three people images                       | one grey dot per face, count in the status line     |
+| Rules      | largest / nearest-center / leftmost / rightmost | the cyan ring moves to the matching face            |
+| Index      | `index` with 0..3                               | ring on that array position, none when out of range |
+| Sticky     | `sticky` on "Two People Smiling" video          | the ring stays on the same person as they move      |
+| Loss       | cover a face (webcam)                           | ring holds ~250 ms, then follows the fallback       |
+
+Bodies (`/multi-subject?subjects=bodies`, up to 2; webcam, else a local two-person clip, else
+"Two People Smiling"):
+
+| Case     | Steps                                     | Expect                                                                    |
+| -------- | ----------------------------------------- | ------------------------------------------------------------------------- |
+| Ids      | two people in view                        | a skeleton each in its own colour, numbers 1 and 2 on the chests          |
+| Crossing | walk past each other                      | each keeps their number and colour                                        |
+| Dropout  | step out briefly (under a quarter second) | the skeleton fades and comes back with the same number                    |
+| Return   | leave for a few seconds and come back     | a new number                                                              |
+| Rules    | leftmost / rightmost / sticky             | the cyan ring on that person; with `sticky` it stays on them as they move |
+| Avatars  | "avatar on each person"                   | a Y Bot on each person, each following its own body                       |
+| Bench    | "identity bench" on the two-person clip   | ids 1, 2; no swaps; no dropouts                                           |
+| Cost     | "performance", up to 1 then 2, one person | inference about 1.6× with 2 (desktop 9 → 14 ms)                           |