@vincentt-xr/sdk 2.0.0-alpha.6 → 2.0.0-alpha.8

package/GROUNDING.md

@@ -10,11 +10,12 @@ or spelunk `node_modules` for the API.
 The API is split into task-domain entry points. Import each symbol from the door
 this reference shows for it; the imports here are literal.
 
-- `@vincentt-xr/sdk` — **core**: providers, scene, screen-space layout, runtime renderers (`Transform3D`, `MeshRenderer`, `SceneObjectRenderer`), `TextLabel`, `Panel`, `SpriteAnimation`, `VideoBackground`, `AspectRatioContainer`, capture/share (`useFrameCapture`, `useMediaRecorder`, `dataURLtoFile`), audio.
+- `@vincentt-xr/sdk` — **core**: providers, scene, screen-space layout, runtime renderers (`Transform3D`, `MeshRenderer`, `SceneObjectRenderer`), `TextLabel`, `Panel`, `SpriteAnimation`, `VideoBackground`, `AspectRatioContainer`, capture/share (`useFrameCapture`, `useMediaRecorder`, `dataURLtoFile`), audio, capability detection (`useXRCapabilities`, `detectXRCapabilities`, `resolveTrackerSupport`), camera permission (`CameraPermissionGate`, `useCameraPermission`), adaptive quality (`useXRQuality`, `XRQualityController`), persistent tracking config (`useXRTrackingConfig`), asset lifecycle (`useXRAsset`, `XRAssetCache`, `defineTextureAsset` / `defineGltfAsset` / `defineVideoAsset`).
 - `@vincentt-xr/sdk/scene-object` — **public authoring state**: scene-object/component types, factories, selectors, and immutable store operations.
-- `@vincentt-xr/sdk/tracking` — **trackers**: face/hand/gesture/body/segmentation, `TrackingAnchor`, `GestureTrigger`, the bare `FaceMesh`, `useFaceResults`.
+- `@vincentt-xr/sdk/tracking` — **trackers**: face/hand/gesture/body/holistic/segmentation, `TrackingAnchor`, `GestureTrigger`, the bare `FaceMesh`, `useFaceResults`, gestures, iris, device calibration (`useXRCalibration`), multi-subject policy (`selectSubject`, `useFaceInfo({ subject })`).
 - `@vincentt-xr/sdk/face-effects` — **face deep**: FaceMesh material config, retouch, canonical mesh, head-binding.
-- `@vincentt-xr/sdk/low-level` — **escape hatch**: raw model-node reads, selector hooks, custom-tracker plumbing. Reach here only when core + tracking can't express it.
+- `@vincentt-xr/sdk/product` — **placement AR** for product pages: `ProductViewer` (Google `<model-viewer>`, optional peer, lazy-loaded), `useProductAr`. A separate mode from the session: placement AR is model-viewer, face/body AR is the session, never both at once.
+- `@vincentt-xr/sdk/low-level` — **escape hatch**: raw model-node reads, selector hooks (`useXRLifecycle`, `useXRTrackerStatus`, …), tracking diagnostics (`useXRDiagnostics`), custom-tracker plumbing. Reach here only when core + tracking can't express it.
 
 Trackers (`<FaceTracker>`, `<GestureTracker>`, `<HandTracker>`, `<BodyTracker>`)
 **self-register** when mounted — no explicit `registerXRPipeline` call. Per-frame
@@ -94,6 +95,19 @@ import { ScreenSpaceUI, ScreenTransform } from "@vincentt-xr/sdk";
 
 Coordinates are normalized: -1 to +1, with +y up. The center of the screen is `(0, 0)`. The anchors rect controls BOTH size and placement of the child — it stretches the child (plane, TextLabel, sprite) to fill the rect. Don't add `scale`/`position`/`planeGeometry args` to size or move a screen-anchored child; set the anchors.
 
+Canvas screen pixels map across the live viewport on both axes, including
+landscape and square displays. They are not automatically letterboxed. Preserve
+an image or video's own aspect with its fit mode inside a screen rect, not with
+manual point conversion. `ScreenTransform2DSettings`, used by serialized editor
+layouts, is intentionally contain-fitted so text and its transform guide retain
+their authored aspect ratio.
+
+For a custom R3F mesh that needs a translated design point, use
+`screenPixelsToScreenSpace({ point, parentSize })`; it applies the same separate
+X/Y mapping as the screen primitives. `screenPixelsToUnits` is scalar conversion
+for uniform distances and is not a substitute for point placement on a
+non-design-aspect viewport.
+
 ```tsx
 // Full-screen frame
 <ScreenTransform anchors={{ left: -1, right: 1, top: 1, bottom: -1 }}>
@@ -528,10 +542,47 @@ projection correction. Author a Transform with `position.y = 0` to opt out.
 Use it only when bridging custom runtime code; ordinary scene-object rendering
 should use `SceneObjectRenderer`.
 
+SDK material types and shader presets are convenience defaults, not a closed
+material system. For an advanced effect, pass `materialOverride` to
+`FaceMeshRenderer`. It accepts a material node, a three.js material instance
+(set on the mesh directly, so one instance can serve many meshes), or a factory
+returning either, invoked per visible face geometry with `{ cameraTexture, geometryKey, faceUv, lipEdge }`. `faceUv` is
+the shared live camera-UV `BufferAttribute` and `lipEdge` the static lip band
+position (both `null` until the canonical mesh has loaded), bound to every
+group's geometry as `attribute vec2 faceUv` and `attribute float lipEdge`. The override takes
+precedence over presets and stays runtime-only: do not place `THREE.Material`,
+uniforms, or factories in `FaceMeshRendererSettings` or a persisted
+`SceneObjectStore`.
+
+```tsx
+<FaceMeshRenderer
+  value={faceMeshSettings}
+  materialOverride={({ cameraTexture, geometryKey }) => (
+    <shaderMaterial
+      key={geometryKey}
+      uniforms={{ cameraTexture: { value: cameraTexture } }}
+      vertexShader={vertexShader}
+      fragmentShader={fragmentShader}
+      transparent
+    />
+  )}
+/>
+```
+
+For serialized scene objects, pass runtime overrides separately to
+`SceneObjectRenderer`, keyed by the FaceMeshRenderer component ID. R3F owns a
+material node's lifecycle; factories must return stable, ordinary React material
+nodes and must not mutate private renderer internals.
+
 `CanonicalFaceMeshRenderer` remains exported from `@vincentt-xr/sdk/face-effects`
 for backwards compatibility and low-level experiments, but new editor and
 scene-object work should use `FaceMeshRenderer`.
 
+Every canonical face mesh (including `FaceMeshRenderer` and `LipEffect`) bridges
+tracker dropouts with `holdMs` (default 250) and `predictMs` (default 0),
+both `CanonicalFaceMeshRendererProps`. The contract, defaults, and limits are
+in "Tracker dropouts — the one bridge to use"; do not add a second hold on top.
+
 ### `FaceRetouch` — legacy landmark skin retouch
 
 Use `FaceRetouch` from `@vincentt-xr/sdk/face-effects` for the legacy
@@ -572,6 +623,350 @@ For multiple people, render one `FaceRetouch` per face:
 optional horizontal UV clipping props in `0..1` space, useful for split-screen
 comparison demos.
 
+### Iris tracking — `useIrisTracking()` and `<IrisEffect>`
+
+Iris data comes from the ten refined iris landmarks the shared face tracker
+already emits: `useIrisTracking` (`@vincentt-xr/sdk/tracking`) acquires the
+same `XRModel.FACE_TRACKER` as `FaceTracker` / `useFaceInfo`, so there is no
+second detector. `<IrisEffect>` (`@vincentt-xr/sdk/face-effects`) draws a
+material over each iris.
+
+```tsx
+import { useIrisTracking, IRIS_LANDMARKS } from "@vincentt-xr/sdk/tracking";
+import { IrisEffect } from "@vincentt-xr/sdk/face-effects";
+
+<IrisEffect mode="shader" color="#4fa3ff" opacity={0.85} smoothing={15} />;
+<IrisEffect mode="image" textureUrl="/iris.png" scale={1.1} mirrored />;
+
+const { result, status } = useIrisTracking({ faceIndex: 0, smoothing: 15 });
+useFrame(() => {
+  if (!result.detected) return;
+  const { center, radius, offset, openness, confidence } = result.left;
+});
+```
+
+- **Result** (`IrisResult`, stable object mutated per frame): `version`,
+  `timestampMs`, `faceIndex`, `detected`, `held`, `confidence`, and `left` /
+  `right` (`IrisEye`): `center` + `ring` + `eyeCenter` in MediaPipe image
+  space (0..1, top-left, Y down, like `FaceInfo.landmarks`), `trackerCenter`
+  in tracker space, `radius` / `eyeWidth` in image-width units, `openness`
+  (lid gap / eye width), `offset` ([-1, 1] per axis, +x the person's left,
+  +y up, on every source: the horizontal sign is corrected on the mirrored
+  webcam feed) and `confidence`. Indices: `IRIS_LANDMARKS` (left center 473, ring
+  474–477; right center 468, ring 469–472; corners and lids per side). Sides
+  follow the mesh topology and are anatomical on every source, mirrored
+  webcam included; positions need no flip.
+- **`offset` is NOT gaze.** It is where the iris sits inside the eyelid
+  opening and ignores head pose and eyeball geometry. The SDK provides no
+  gaze estimation; do not present `offset` as one. For blinks and
+  eye-look expressions use blendshapes (`useFaceInfo` / `useFaceAction`).
+- **Confidence**: MediaPipe scores no iris point and emits them for closed
+  eyes, so `eye.confidence` = `openness / 0.2` clamped to 0..1, scaled by
+  the hold decay; `result.confidence` is 1 live, decaying to 0 over
+  `holdMs` while `held`. Gate effects on it; the built-in materials fade with
+  `irisConfidence`. `status` is `loading` → `ready` / `error` for the model.
+- **Shared contracts**: dropouts use `FaceTrackMemory` (`holdMs` 250,
+  `predictMs` 0); `smoothing` is `TrackingAnchor`'s One Euro on centers and
+  radius (default 0 = raw; ~10 tames jitter, but it is a 1 Hz-class low-pass
+  that lags a 2–3 Hz head shake, so keep it low for fast motion), reset on
+  re-acquire; `leadMs` (default 80, 0 off)
+  advances the centers along their last sample-to-sample velocity between
+  inference samples, so a 24 fps face model reads continuous at 60+ fps
+  instead of hold-and-jump (raise the face `targetFps` first when the lag
+  itself is the problem); `faceIndex` / `numFaces`
+  select faces like `useFaceInfo` (array index, first consumer sets
+  `numFaces`). When the face stays lost beyond `holdMs` the effect hides at
+  once: no stale eye effects.
+- **`<IrisEffect>`**: flat `IrisEffectSettings` props (`eye` both | left |
+  right, `mode` shader | image, `color`, `textureUrl`, `tintImage` (multiply
+  the image by `color`; off by default, images show as authored), `scale` × iris
+  radius, `opacity`, `targetFps` (face inference cap; 0 = session, first
+  face consumer wins, so request the same value everywhere; a later change
+  takes effect only once every holder has released the model — a lone
+  `<IrisEffect>` re-acquires and re-initializes, a `registerXRPipeline`
+  entry pins the first value for the session).
+  **_`targetFps: 60` gives the best iris following_** (fast head motion
+  reads continuous) **_but use it with caution_**: it runs face inference
+  at 2.5× the template's 24 fps on every frame, which heats and throttles
+  phones. Ship 60 only when the effect is the point of the experience and
+  the target devices have been checked; otherwise keep the session default
+  and rely on `leadMs`, or pick 30 as the middle ground. Then `smoothing`,
+  `leadMs`, `mirrored` flips the material only, `faceIndex`, `holdMs`) plus `numFaces`, `active`, `predictMs`, `texture`
+  (pre-loaded, beats `textureUrl`), `material` (a three material or a
+  factory `(IrisMaterialContext) => Material`, called once per eye for each
+  `material` identity; the effect disposes what a factory created when the
+  factory changes or on unmount, an instance stays yours; read the live
+  texture from `uniforms.map`, `context.texture` is only the texture known
+  when the factory ran), `renderOrder`, `onTracking(handle)`. Each eye is a quad spanning
+  the iris diameter on the z=0 plane; the built-in shader is a procedural
+  iris whose pupil follows `offset`; the image material covers the quad
+  tinted by `color`. Every material gets the same stable uniforms
+  (`IrisUniforms`: `irisCenter`, `irisRadius`, `irisOffset`,
+  `irisConfidence`, `irisOpenness`, `time`, `color`, `opacity`, `mirror`,
+  `tintImage`, `map`), refreshed in place by `updateIrisUniforms`; the fragment chunks
+  `IRIS_SHADER_FRAGMENT` / `IRIS_IMAGE_FRAGMENT` and `IRIS_VERTEX_SHADER`
+  are exported as starting points. Debug panel: `IrisEffectSettingsPanel`
+  (`@vincentt-xr/sdk/debug-ui`) with `resolveIrisEffectSettings` /
+  `patchIrisEffectSettings`.
+- Use iris tracking for eye decorations, pupil overlays and shader inputs;
+  blendshapes for blink/wink/eye-look expressions; `PinToMesh` / FaceMesh
+  materials for content that must deform with the skin. To hide the iris
+  behind the eyelids, draw a `CanonicalFaceMeshRenderer` with
+  `showEyeGeometry={false}` at a higher `renderOrder` than the effect and a
+  `materialOverride` that samples the camera texture through the mesh's
+  live `faceUv` attribute with `transparent: true` and `depthTest: false`
+  (the example's "eyelid mask"); it repaints the feed over the face except
+  the eye openings. It must be transparent: three draws every opaque object
+  before any transparent one, and the iris quads are transparent. Full contract and the
+  manual test plan: `docs/api/iris-tracking.mdx`; example:
+  `/iris-tracking`.
+- **Low-level iris seam** (`@vincentt-xr/sdk/low-level`): `createIrisResult`,
+  `updateIrisResult(result, landmarks, nowMs, { faceIndex, held,
+faceConfidence, mirrored })`, `computeIrisEye`, `createIrisEye`,
+  `IRIS_FULL_OPENNESS` and `IrisLead` are the allocation-free helpers behind
+  `useIrisTracking`, for bring-your-own-landmarks pipelines (a recorded
+  clip, a custom tracker). App code reads `useIrisTracking`; only reach for
+  these when you already hold 478-point landmarks.
+
+### `<LipEffect>` — lip tint / lipstick overlay
+
+Use `LipEffect` from `@vincentt-xr/sdk/face-effects` to paint the lips of a
+tracked face. It renders only the canonical lip-surface triangles (between the
+MediaPipe outer and inner lip rings) and blends a design texture, or a solid
+colour, over the live camera pixels. Mount a `<FaceTracker>` first.
+
+```tsx
+import { LipEffect } from "@vincentt-xr/sdk/face-effects";
+
+<LipEffect preset="gloss" designTexture="/lips/gloss.png" color="#e85d75" />;
+```
+
+Props are `Partial<LipEffectSettings>` plus `faceIndex`, `designTexture` (URL or
+`THREE.Texture`, canonical face UV space; omit it for a solid `color` fill),
+`lowerDesignTexture` (lower lip, used with `splitLips`), `material`,
+`sharedMaterial` (a material instance to use instead of the built-in one),
+`maxUpdateFps` (cap how often the lip mesh follows the landmarks), `holdMs`
+(default 250: keep the lips on the last landmarks across tracker dropouts) and
+`predictMs` (default 0: extrapolate the held landmarks along their last
+velocity), and `renderOrder` (default `30`). `LipEffectSettings` is flat and
+serializable:
+
+| Field                                                  | Meaning                                                                                              |
+| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- |
+| `preset`                                               | `"custom"` or one of `lipstick`, `gloss`, `matte`, `tint`, `metallic`, `gradient`                    |
+| `maskMode`                                             | `"outer-and-inner-lips"` (default, mouth interior cut out) or `"outer-lips"`                         |
+| `color`, `opacity`, `intensity`                        | Tint, coverage, and how much the design is multiplied by the tint (0 = as painted)                   |
+| `shading`                                              | How much the camera's own lip light/shadow modulates the paint (0 = flat)                            |
+| `gloss`, `roughness`, `highlightColor`                 | Specular highlight amount, spread, and colour                                                        |
+| `colorMode`                                            | `"texture"` (sample the design when given) or `"solid"` (procedural: colour + gradient only)         |
+| `gradientColor`, `gradientAmount`, `gradientDirection` | Second colour blended `outward` from the mouth line, or `vertical` / `horizontal` across the lip box |
+| `edgeSoftness`                                         | Feather at the outer lip boundary as a fraction of the lip band (0 = hard edge, default 0.12)        |
+| `innerLipColor`, `innerLipAmount`, `innerLipWidth`     | Separate tint for the inner lip near the mouth line (and the mouth interior in `outer-lips` mode)    |
+| `splitLips`, `lowerLipColor`, `lowerLipOpacity`        | Give the lower lip its own colour and opacity; `lowerDesignTexture` prop gives it its own texture    |
+| `textureScaleX/Y`, `textureOffsetX/Y`                  | Design transform about the lip centre; offset is in lip widths/heights                               |
+| `textureRotation`, `textureFlipX/Y`                    | Degrees counter-clockwise as seen on the face; mirror across the lip centre                          |
+
+Resolution order is defaults, then the named preset, then every explicit prop.
+Helpers: `resolveLipEffectSettings(partial)`, `patchLipEffectSettings(current, patch)`
+(switching `preset` re-applies that preset), `LIP_EFFECT_PRESETS`,
+`LIP_EFFECT_PRESET_KEYS`, `DEFAULT_LIP_EFFECT_SETTINGS`,
+`LIP_EFFECT_SETTINGS_FIELDS` (typed `SettingsPanel` fields), and
+`createLipTextureTransform(settings, bounds)` (the UV sampling `Matrix3`).
+
+**Live controls.** `LipEffectSettingsPanel` from `@vincentt-xr/sdk/debug-ui` edits
+the same settings object that drives rendering; spread what it emits onto the
+component:
+
+```tsx
+import { LipEffectSettingsPanel } from "@vincentt-xr/sdk/debug-ui";
+
+const [settings, setSettings] = useState(() => resolveLipEffectSettings({ preset: "lipstick" }));
+<LipEffect {...settings} designTexture={url} />;
+<LipEffectSettingsPanel value={settings} onChange={setSettings} />;
+```
+
+**Design texture contract.** A square image in the canonical face UV atlas (same
+layout as `Canonical_Face_Color.png`). Only pixels under the lip triangles are
+sampled: roughly x 38–62 %, y 65–74 % of the image. Transparent pixels leave the
+skin untouched (alpha drives the blend); RGB is multiplied by `color` at
+`intensity`, so paint white/light for a pure tint. Image-left is the subject's
+right lip corner: screen-right on the mirrored webcam view, screen-left for
+image/video sources. Image-top is `v = 1`. Never remap UVs or assume a lips-only
+canvas; use the texture transform settings to fit a design instead. Generate
+the template instead of drawing it: `createLipUvGuideSvg(mesh)`,
+`createLipUvGuideDataUrl(mesh, { region: "lips" })`, and
+`createLipUvGuidePngBlob(mesh)` (browser) from the `useCanonicalFaceMesh()`
+result; `getLipUvBounds(mesh)` for the numbers (`CANONICAL_LIP_UV_BOUNDS` is the
+shipped mesh's fallback); `lipUvToPixel(u, v, size)` for the `flipY` pixel
+mapping. The shipped `docs/guides/lip-uv-template.svg` / `.png` are regenerated
+with `pnpm dlx tsx scripts/generate-lip-uv-guide.ts --png`; the `/lip-effect`
+example page has a `Texture preview / UV guide` toggle and `Download SVG / PNG`
+buttons. Full guide: `docs/guides/lip-effect-textures.mdx`.
+
+**Several faces.** `LipEffects` renders one lip effect per face slot:
+
+```tsx
+import { LipEffects } from "@vincentt-xr/sdk/face-effects";
+
+<FaceTracker numFaces={2} />;
+<LipEffects maxFaces={2} faces={{ preset: "gloss", designTexture: url }} />;
+<LipEffects
+  maxFaces={2}
+  faces={(faceIndex) =>
+    faceIndex === 0 ? { preset: "matte" } : { preset: "gloss", designTexture: url }
+  }
+/>;
+```
+
+`faces` is one config for all faces or a picker per face (`null` skips a
+face); each config takes everything `LipEffect` takes except `faceIndex`. With a
+single config the faces share one built-in material (`shareMaterial`, default
+on for a single config); `maxUpdateFps` and `renderOrder` apply to all faces.
+Match `maxFaces` to the tracker's `numFaces`.
+
+**Performance.** The built-in material is a plain three.js object:
+`createLipEffectMaterial()`, `updateLipEffectMaterial(material, inputs)`, and the
+owning hook `useLipEffectMaterial(inputs)` (inputs: `cameraTexture`,
+`designTexture`, `lowerDesignTexture`, `settings`, `lipUvBounds`,
+`subtractMouth`). Pass one instance as `sharedMaterial` to several `LipEffect`s,
+or return a material instance from a `material` factory, and all of them draw
+with the same program and uniforms. `maxUpdateFps` throttles the per-frame
+landmark-to-vertex update (visibility still follows face presence every frame).
+Tracking cost itself is set on the session: `<XRProvider perf={{ targetFps,
+camRes }}>` lowers inference rate and the analysed resolution, and
+`<FaceTracker targetFps>` overrides it per tracker. `useLipDesignTextures(upper,
+lower)` resolves URL-or-texture props through R3F's loader cache, so faces that
+share a URL share a texture.
+
+**Custom material.** `material` accepts a React material node, a three.js
+material instance, or a factory that receives a stable
+`LipEffectMaterialContext` and returns either:
+
+```ts
+type LipEffectMaterialContext = {
+  cameraTexture: THREE.Texture | null;
+  designTexture: THREE.Texture | null;
+  faceUv: THREE.BufferAttribute; // live camera UVs, bound as `attribute vec2 faceUv`
+  lipEdge: THREE.BufferAttribute; // static band position, `attribute float lipEdge` (0 outer ring, 1 mouth line)
+  lowerDesignTexture: THREE.Texture | null;
+  faceIndex: number;
+  maskMode: LipMaskMode;
+  lipUvBounds: LipUvBounds; // lip region of the UV atlas
+  settings: LipEffectSettings; // the resolved settings
+};
+```
+
+The factory runs once per lip geometry group (plus the mouth interior in
+`outer-lips` mode) and only after the canonical mesh has loaded. Honour
+`maskMode` with `stencilWrite stencilRef={1}
+stencilFunc={THREE.NotEqualStencilFunc}` plus `KeepStencilOp` for
+`stencilFail` / `stencilZFail` / `stencilZPass` (three.js only runs the
+comparison while `stencilWrite` is true; the keep-ops stop the material from
+altering the mouth stencil it tests). That is what the exported
+`LipEffectMaterial`'s `subtractMouth` prop does; reuse that
+component (`cameraTexture`, `designTexture`, `settings`, `lipUvBounds`,
+`subtractMouth`, `lowerDesignTexture`) when only the settings need to differ.
+`LIP_EFFECT_VERTEX_SHADER` / `LIP_EFFECT_FRAGMENT_SHADER` are the built-in GLSL
+sources, and `docs/examples/lip-shaders.mdx` documents four factories (animated
+gloss, noise texture, iridescence, time-based shimmer) built on the vertex
+shader. The parsed mesh exposes the same band position as `lipEdgeDistance`, and
+`getLipUvBounds(mesh).mouthV` is the mouth line in UV space.
+
+### `<SegmentationOverlay>` — 2D semantic region layers
+
+Use `SegmentationOverlay` from `@vincentt-xr/sdk/face-effects` to draw regions
+of a semantic segmentation mask (`lips`, `eyes`, `brows`, `face_skin`, `mouth`,
+`bg`, `person`; or any label an external model emits, such as `beard` / `ears`
+from a face-parsing model, which the SDK itself does not provide)
+as a tinted, screen-aligned 2D layer over the camera. It needs no canonical 3D
+geometry.
+
+```tsx
+import {
+  SegmentationOverlay,
+  useFaceRegionMask,
+  usePersonMask,
+} from "@vincentt-xr/sdk/face-effects";
+
+const faces = useFaceRegionMask(); // lips, mouth, eyes, brows, face_skin — from face landmarks
+const person = usePersonMask(); // bg, person — from portrait segmentation
+<SegmentationOverlay mask={faces} label="lips" color="#c0304f" blendMode="multiply" />;
+<SegmentationOverlay mask={person} label="bg" color="#000" opacity={0.6} blendMode="normal" />;
+```
+
+**2D versus 3D.** Pick `SegmentationOverlay` for screen-space tints, cut-outs
+and stamps, and for regions without landmarks (background, person, hair,
+beard, ears, the latter three only with an external mask). Pick `LipEffect` /
+`FaceMeshRenderer` when a design must follow
+the lip or face surface under rotation with real shading. The 2D layer samples
+`texture` in screen space and takes its lighting from the camera pixels via
+`blendMode`; it is one full-screen quad and one small texture.
+
+**Mask data.** `SemanticMask = { width, height, indices: Uint8Array (row 0 =
+top), labels: string[], space?: "feed" | "source", version }`. Feed any
+model's per-pixel class ids into it with `createSemanticMask` /
+`clearSemanticMask` (face-effects, also tracking), `SEMANTIC_MASK_LABELS` and
+`FACE_REGION_LABELS` name the built-in labels. Mutate in place and bump
+`version`; the overlay uploads lazily per version, and draws nothing while
+`version` is 0. The rasterizer primitives (`fillPolygon`,
+`fillLandmarkPolygon`, `semanticMaskLabelIndex`, `rasterizeFaceRegions`,
+`FACE_REGION_POLYGONS`) are `@vincentt-xr/sdk/low-level`.
+
+**Coordinates.** `"feed"` (default) is the session camera texture — what
+trackers see, cover-cropped and webcam-mirrored — and needs no setup; the
+overlay quad matches `VideoBackground`. `"source"` is the raw media frame:
+the overlay cover-fits it from the mask's aspect and the camera texture size
+(`computeMaskUvTransform`); set `mirrored` when the feed is mirrored relative to
+that source (webcam). Masks are nearest-filtered class indices at their own
+resolution; `feather` (mask texels) softens edges.
+
+**Props** are `Partial<SegmentationOverlaySettings>` (`label` — up to four,
+comma-separated; `invert`; `color`; `opacity`; `blendMode` `normal` |
+`multiply` | `screen` | `overlay`; `feather`; `mirrored`) plus `mask`,
+`texture` (screen-space image), `material` (node, `THREE.Material`, or factory
+receiving `{ cameraTexture, maskTexture, maskTransform, labelIndices, mask,
+settings }`), `renderOrder`, `visible`. `SegmentationOverlaySettingsPanel`
+(`@vincentt-xr/sdk/debug-ui`) edits the settings live. The internals are
+`@vincentt-xr/sdk/low-level`: the material object API
+(`createSegmentationOverlayMaterial` / `updateSegmentationOverlayMaterial` /
+`useSegmentationOverlayMaterial`), the shaders
+(`SEGMENTATION_OVERLAY_VERTEX_SHADER` / `_FRAGMENT_SHADER`), mask textures
+(`createSemanticMaskTexture` / `uploadSemanticMask` / `useSemanticMaskTexture`)
+and `computeMaskUvTransform`. `usePersonMask` and `SegmentationOverlay` are
+also exported from `@vincentt-xr/sdk/tracking` for person / background work.
+
+**Hair.** `useHairMask({ modelUrl?, quality?, active? })` runs MediaPipe's Hair
+Segmenter through `XRModel.SEMANTIC_SEGMENTATION` and returns `{ mask, status,
+error, labels }` with labels `bg` / `hair`; `status` is `loading` → `ready` |
+`error`, and the mask stays at `version` 0 until the first inference. By
+default the model loads from MediaPipe's published file, integrity-pinned in
+the SDK manifest (verified; a republish fails loudly); the same hash is pinned
+under the SDK CDN path `/assets/xr/models/hair_segmenter.tflite` for when the
+file is uploaded there. `useHairMask()` with no options works. An inference
+error is transient and clears on the next good frame.
+`useSemanticSegmentationMask({ preset: "hair" | "multiclass", … })` is the
+generic form (`multiclass` adds `body_skin`, `face_skin`, `clothes`,
+`others`); one preset per session, first acquirer wins. Hair is neither
+landmark geometry (no hair points) nor the portrait segmenter (person as one
+class): use it for hair tints, recolours, highlights and cut-outs as 2D
+effects. Full contract: `docs/api/segmentation-overlay.mdx` → "Hair
+segmentation".
+
+**Faces and performance.** `useFaceRegionMask({ faceIndex: "all" | n,
+regions, resolution = 256, maxUpdateFps, holdMs = 250, predictMs = 0 })`
+merges all faces or isolates one; `holdMs` keeps a dropped face's mask for
+that long (small or distant faces make the tracker blink) and `predictMs`
+extrapolates the held landmarks along their last velocity (`FaceTrackMemory`
+is the reusable bridge);
+make one provider and overlay per face for per-face looks. `usePersonMask({
+maxUpdateFps = 15 })` acquires the segmentation model and reads the mask back
+from the GPU each update, so keep its rate low and share it; it covers every
+person in frame, and its mask stays at `version` 0 (overlay draws nothing)
+until the model has loaded. Use `blendMode="normal"` for `bg` / `person`;
+`multiply` is for tinting skin and is nearly invisible on dark backgrounds. Combine labels in
+one overlay instead of stacking quads. Full reference, including the manual
+visual test plan: `docs/api/segmentation-overlay.mdx`.
+
 ### `<FaceTracker>` — Face tracking (468 landmarks)
 
 Detects faces and provides face transform to children via context. Supports
@@ -871,6 +1266,13 @@ import { FaceTracker, FaceMesh } from "@vincentt-xr/sdk/tracking";
 </FaceTracker>;
 ```
 
+The composite face shader (camera + canonical texture with blend modes) and the
+`FaceWarp3DHead` shader end with `#include <colorspace_fragment>`, so the face
+renders at the same brightness as the camera feed and built-in materials;
+`FaceWarp3DHead` tags its UV reference texture sRGB for the same reason. Any
+custom `ShaderMaterial` layered over the feed must include that chunk too (see
+"Custom shaders" under lip effects).
+
 ### `<HandTracker>` — Hand skeleton (21 joints)
 
 The hand-skeleton tracker. Use it for 21-joint hand landmarks — anchoring to `hand.indexTip`, `hand.wrist`, etc. It also exposes a crude open-vs-closed grip on `node.gesture` (only `closed_fist` / `open_palm`), but that is a side effect of the skeleton, not a gesture recognizer. **For any named gesture, use `<GestureTracker>`** (next section) — it is the gesture door; `<HandTracker>` is the skeleton door.
@@ -885,6 +1287,13 @@ Supports multiple detected hands: `hand` selects `"left"` or `"right"` when that
 hand is available; `handIndex` falls back to positional selection. Existing
 one-hand scenes keep working via the default first-hand selection.
 
+Low-level `node.coordinates` are `TrackerPoint` values: center-origin, X right,
+Y up, and positive Z toward the viewer. X/Y are in `[-1, 1]`; Z is
+tracker-relative depth, not a normalized screen axis. For a bespoke cursor or
+overlay, call `trackerPointToScreenPixels({ point, viewportSize })` from
+`@vincentt-xr/sdk`. The SDK already owns the mirror and origin conversion; do
+not repeat it in app code.
+
 ### `<GestureTracker>` — Named gestures
 
 The gesture door — the component that recognizes named gestures. Its `node.gesture` is one of these exact strings (copy verbatim — these are the only valid values):
@@ -920,12 +1329,183 @@ import { BodyTracker, TrackingAnchor } from "@vincentt-xr/sdk/tracking";
 </BodyTracker>;
 ```
 
+Low-level body landmarks use the same `TrackerPoint` contract as hand and
+gesture points. `TrackingAnchor` is preferred for attachment; use
+`trackerPointToScreenPixels` only when rendering a bespoke 2D projection.
+
+### `<HolisticTracker>` — Face + hands + pose from one inference
+
+Opt-in, additive: one MediaPipe Holistic inference per frame for a **single
+person**, shared by every consumer. Use it when a scene needs face, hands and
+body together; keep the individual trackers when it needs one stream (the
+holistic model is heavier than any single one, 13.7 MB). Mounting
+`<FaceTracker>` beside `<HolisticTracker>` runs two models — pick one per scene.
+
+```tsx
+import { HolisticTracker, TrackingAnchor } from "@vincentt-xr/sdk/tracking";
+
+<HolisticTracker holdMs={250} hands="both">
+  <HolisticTracker.Face>
+    <TrackingAnchor target="face.noseTip" smoothing={15}>
+      {/* … */}
+    </TrackingAnchor>
+  </HolisticTracker.Face>
+  <HolisticTracker.Hand hand="right">
+    <TrackingAnchor target="hand.indexTip">{/* … */}</TrackingAnchor>
+  </HolisticTracker.Hand>
+  <HolisticTracker.Body>
+    <TrackingAnchor target="body.leftShoulder">{/* … */}</TrackingAnchor>
+  </HolisticTracker.Body>
+</HolisticTracker>;
+```
+
+- **Sub-trackers** `HolisticTracker.Face` / `.Hand hand="left"|"right"` /
+  `.Body` provide the same tracker context as `FaceTracker` / `HandTracker` /
+  `BodyTracker`: every `face.*`, `hand.*`, `body.*` anchor target (including
+  `body.neck` and `face.custom`) and `smoothing` work unchanged. The group is
+  hidden while its stream is absent; origins are the nose tip, the wrist, the
+  head. The holistic graph has no face transformation matrix, so there is
+  **no head rotation**; use `<FaceTracker>` when children must roll with the
+  head. `FaceMesh`, the face readers, face effects and head binding read
+  `XRModel.FACE_TRACKER`, not holistic.
+- **Unified result**: `useHolisticTracking()` → `{ node, ready, status,
+error }`; read `node.getResult()` inside `useFrame` (stable object, mutated
+  in place, `version` bumps every frame). `HolisticResult` = `face`
+  (478 `NormalizedLandmark`s in MediaPipe image space like `FaceTracker`'s
+  `detectionResults`, plus `blendshapes` name → score), `leftHand` /
+  `rightHand` (21 `TrackerPoint`s like `HandTracker`, plus `handedness` and
+  metric `worldLandmarks`), `pose` (33 `TrackerPoint`s like `BodyTracker`,
+  plus `visibility[]` and `worldLandmarks`). Each stream is `null` when
+  absent. Coordinates get no source-dependent mirroring, matching Hand/Body.
+  Hand **labels** are anatomical on every source: holistic assigns hands from
+  the pose, whose left/right follow the image, so the SDK swaps them on the
+  mirrored webcam feed (`leftHand` = the person's left hand, which appears on
+  the viewer's left in the selfie view). `hands="left"` therefore keeps the
+  person's left hand, the same hand `<HandTracker hand="left">` selects. The
+  pose's left/right landmarks (shoulders 11/12, wrists 15/16, …) are swapped
+  the same way, so `pose` and the hand streams always describe the same
+  physical hand and `getHolisticSignals` can pair them; this differs from
+  `BodyTracker`, whose indices follow the image.
+- **Pre-release asset host**: the default model is MediaPipe's published
+  `holistic_landmarker.task` on `storage.googleapis.com` (SHA-384 pinned), not
+  the SDK CDN, until the CDN copy is published. Pass `modelUrl` to self-host
+  it; the hash for the CDN path is already pinned.
+- **Confidence**: face/hands `1` while detected, pose = mean visibility.
+  Each stream runs through `FaceTrackMemory` (the one dropout bridge):
+  `holdMs` 250 / `predictMs` 0 by default; while held, `held` is true and
+  `confidence` decays linearly to 0 at `holdMs`. `holdMs={0}` drops at once.
+- **Delegate**: `delegate` is `"auto"` (default) | `"GPU"` | `"CPU"`.
+  MediaPipe's holistic face-blendshape stage has no WebGL kernel for one of
+  its ops, so with `outputFaceBlendshapes` the GPU graph fails on its first
+  frame; `"auto"` rebuilds the graph on the CPU once (one `console.warn`,
+  status dips to `loading` for the rebuild, no refetch) and
+  `node.getDelegate()` then reports `"CPU"`. CPU inference is several times
+  slower (tens to hundreds of ms per frame on a laptop); when blendshapes
+  are not needed, `outputFaceBlendshapes={false}` keeps the whole graph on
+  the GPU. A fixed `"GPU"` policy reports the failure as `error` instead.
+- **Options**: model-creation (`modelUrl`, the five `min*Confidence`
+  thresholds, `outputFaceBlendshapes`, `delegate`) come from the first
+  mounted consumer and re-acquire when changed; `targetFps` overrides `perf.targetFps`;
+  `hands` (`both` | `left` | `right` | `none`), `analysisMaxDimension` (768),
+  `holdMs`, `predictMs` apply live (last writer wins); `active` pauses;
+  `onStatusChange(status, error)` reports `loading` → `ready` / `error` (a
+  transient inference error recovers on the next good frame). The default
+  model is MediaPipe's published `holistic_landmarker.task`, SHA-384 pinned.
+- **Combined signals**: `getHolisticSignals(result)` →
+  `HolisticSignals` with presence per stream, `smile` / `mouthOpen`
+  blendshape scores, `leftHandRaised` / `rightHandRaised` (pose wrist above
+  its shoulder), `leftPalmOpen` / `rightPalmOpen` (three of four fingers
+  extended) and `greeting` (raised open hand while smiling; right wins).
+  `isHandRaised` / `isPalmOpen` are exported alone. Pass an `out` object to
+  avoid per-frame allocation.
+- **Low-level**: `XRModelNodeHolisticTracking` (`@vincentt-xr/sdk/low-level`)
+  = `{ getResult, getRawResult, getStatus, getError, getDelegate, setOptions }`. Debug
+  panel: `HolisticTrackerSettingsPanel` (`@vincentt-xr/sdk/debug-ui`) with
+  `resolveHolisticTrackerSettings` / `toHolisticTrackerProps`. Full contract
+  and manual test plan: `docs/api/holistic-tracker.mdx`; example:
+  `/holistic-tracking`.
+
 ### `<Segmentation type="portrait">` — Background segmentation (declarative)
 
 Registers the portrait segmentation pipeline. The resulting mask feeds into `<VideoBackground segmentationMask={...} />` for background replacement. Mount it alongside a ref or state that holds the mask texture.
 
+For a **tinted or textured person / background layer** rather than a background
+swap, use `usePersonMask()` with `<SegmentationOverlay mask={…} label="bg" />`
+(or `label="person"`). For **hair**, use `useHairMask()` with `label="hair"`
+(MediaPipe Hair Segmenter via `XRModel.SEMANTIC_SEGMENTATION`). For **background
+replacement** (person kept, background swapped for any media), use
+`<PersonCutout>` over a full-screen `ScreenImage`; see "Camera background". Both are exported from this door as well as from
+`@vincentt-xr/sdk/face-effects`; `usePersonMask` acquires the segmentation model
+itself, so no `<Segmentation>` is needed for it. See `<SegmentationOverlay>` in
+the face-effects section for the full contract.
+
 ---
 
+## Multi-subject policy — indices, counts, selection, association
+
+**An index is a result-array position, not an identity**: `faceIndex` /
+`handIndex` / `getFace(i)` read position `i` of the current frame; the order
+can change between frames and nothing promises the same person at the same
+index later. The SDK never recognises people; the only continuity is spatial
+(`sticky`). Single-subject defaults (index 0) are unchanged.
+
+Max subjects (`XR_MAX_SUBJECTS`): face 10 (`numFaces`, default 1, first
+acquirer of the shared model wins), hand 2 (fixed, `handedness` labels them),
+gesture 1, body 1, holistic 1 (single person).
+
+```tsx
+import {
+  useFaceInfo,
+  selectSubject,
+  faceCandidates,
+  jointCandidates,
+  associateSubjects,
+} from "@vincentt-xr/sdk/tracking";
+
+const face = useFaceInfo({ numFaces: 4, subject: { selection: "largest" } }); // or "nearest-center" | "leftmost" | "rightmost" | "index"
+const same = useFaceInfo({ numFaces: 4, subject: { selection: "sticky", stickyRadius: 0.2 } }); // follows the previous pick by proximity; fallback "largest"
+const result = selectSubject(faceCandidates(results.faceLandmarks), policy, previousMemory); // pure: { index | null, reason, memory }; carry result.memory to the next frame
+const pairs = associateSubjects(faceCandidates(faces), jointCandidates(bodies), 0.35); // nearest-centre pairs, each subject once
+```
+
+Loss: the tracker hold (`holdMs` 250) replays the last result, then drops it;
+`sticky` keeps its memory through a miss and re-selects a subject that
+reappears nearby, otherwise the fallback decides and the memory moves with it.
+Reappearance never restores an index. Full contract: `docs/api/multi-subject.mdx`;
+example: `/multi-subject`.
+
+## Tracker dropouts — the one bridge to use
+
+The face landmarker loses and re-finds a face for a frame or two at a time
+when the face is small, distant, fast, or turned hard. Raw detection results
+show those gaps as `faceLandmarks` momentarily empty. **Do not write your own
+hold, smoothing, or interpolation for this in app code.** The SDK has one
+mechanism, used by every face surface, and it is the source of truth:
+
+- `FaceTrackMemory` (`@vincentt-xr/sdk/face-effects`) remembers each face's
+  last landmarks and a smoothed velocity. A missing face is replayed for
+  `holdMs`, extrapolated along that velocity for the first `predictMs`, then
+  held still, then dropped. Depth (`z`) rides along when present.
+- Rendered surfaces use it with `holdMs: 250` and `predictMs: 0` by default:
+  `CanonicalFaceMeshRenderer` (so `FaceMeshRenderer` and `LipEffect` /
+  `LipEffects`) and `useFaceRegionMask` for 2D overlays. The face readers
+  (`useFaceInfo` / `useFaceInfos` / `useFaceDetection`) apply the same
+  250 ms hold to presence and lifecycle events through their own `holdMs`;
+  they report, they do not extrapolate. Set `holdMs: 0` only when immediate
+  disappearance is the point. Keep `predictMs` at or below 300: it
+  extrapolates linearly and will overshoot a sharp reversal for up to that
+  window.
+- Measured on the SDK's "Person Fullbody" clip, hold 250 with predict 150 cut
+  empty frames from 23 % to 2 %; the remaining gaps are longer than the hold
+  and are correctly dropped rather than shown stale.
+- The dropouts themselves are a tracking-input problem. When they are
+  frequent, raise the analysed resolution with `perf.camRes` before reaching
+  for a longer hold.
+
+Custom consumers of raw landmarks (a bespoke mesh, a custom material factory
+that reads `detectionResults`) should run them through `FaceTrackMemory` with
+the same defaults so they stay in step with the SDK's own surfaces.
+
 ## Tracker performance & lifecycle
 
 ML tracker models are the heaviest thing in an XR app — each one runs a MediaPipe pipeline on the camera feed. Two rules keep apps from overheating the device and from a scene-transition gesture bug.
@@ -948,7 +1528,7 @@ Scope a tracker to a single scene only when just that scene uses it (e.g. a `<Fa
 
 ### First load: each tracker you mount is a download
 
-Before a tracker reports anything, the viewer's phone downloads its model and the shared MediaPipe engine (`vision_wasm_internal.wasm`, served gzip-compressed, ~2.9 MB). The SDK fetches the two in parallel, and trackers mounted together share one engine download. Model sizes: face 3.8 MB, hand 7.8 MB, gesture 8.4 MB, body 5.8 MB. All are cached by the browser for a year, so a returning viewer downloads nothing.
+Before a tracker reports anything, the viewer's phone downloads its model and the shared MediaPipe engine (`vision_wasm_internal.wasm`, served gzip-compressed, ~2.9 MB). The SDK fetches the two in parallel, and trackers mounted together share one engine download. Model sizes: face 3.8 MB, hand 7.8 MB, gesture 8.4 MB, body 5.8 MB, holistic 13.7 MB (face + hands + pose in one). All are cached by the browser for a year, so a returning viewer downloads nothing.
 
 So every extra tracker costs a first-time viewer its full model on mobile data. Mount only the trackers the experience actually reads; a face-only effect should not also mount `<HandTracker>`.
 
@@ -973,7 +1553,45 @@ Session lifecycle preserves that ownership state: `session.stop()` pauses every
 tracker without releasing its MediaPipe resources, and the next `session.start()`
 resumes only models that still have at least one active owner. Static-image
 detection pulses use the same suspension path, so source switches do not
-accidentally reactivate an `active={false}` tracker.
+accidentally reactivate an `active={false}` tracker. A tracker whose last
+owner unmounts while the session runs is disposed, and the instance a later
+mount acquires starts its own inference loop (the run guard is per instance),
+so remounting a sole-owner tracker mid-session is safe.
+
+A failed `session.setMediaSource()` (webcam denied, no device, a stream that
+never produces frames) rejects and leaves the session where it was: an idle
+session stays idle, a running session clears its swap gate and keeps rendering,
+and the next `setMediaSource()` binds normally with no stop/start in between.
+Each bind owns a generation; a webcam denial that lands after a later bind
+already succeeded is dropped instead of overwriting the healthy session's
+`error`. Callers therefore recover from a blocked camera by selecting another
+source once.
+
+### Adaptive quality — budget + tiers (on by default)
+
+`<XRProvider>` mounts an `XRQualityController` unless `quality={false}`. It
+samples frame time, tracker inference, memory and device signals every
+second and steps between declared tiers: `targetFps` and `analysisScale`
+through the session perf config, `renderScale` through `<XRScene>`'s pixel
+ratio, `camRes` only when a tier sets `rebindCamera: true`.
+
+```tsx
+import { XRProvider, useXRQuality, DEFAULT_QUALITY_TIERS, DEFAULT_PERFORMANCE_BUDGET } from "@vincentt-xr/sdk";
+
+<XRProvider quality={{ tiers: DEFAULT_QUALITY_TIERS, budget: { maxFrameMs: 33 }, minTier: "medium" }}>…</XRProvider>
+<XRProvider quality={{ adaptive: false, initialTier: "medium" }}>…</XRProvider> // pinned, still observable
+<XRProvider quality={false}>…</XRProvider>                                       // no controller
+
+const { tier, index, tiers, decision, history, signals, locked, setTier, lock, unlock, reset } = useXRQuality();
+```
+
+- Defaults: tiers high / medium / low (`targetFps` 30 / 24 / 15, `renderScale` 1 / 0.8 / 0.6, `analysisScale` 1 / 0.85 / 0.7); budget `maxFrameMs` 1000/24, `maxInferenceMs` 24, `sampleMs` 1000, `downgradeAfter` 2, `upgradeAfter` 6, `upgradeHeadroom` 0.75, `cooldownMs` 4000. All plain JSON; `resolveQualityConfig` validates partials.
+- Bounded (`minTier` / `maxTier`), hysteretic (streaks), rate-limited (cooldown), reversible (steps back up with headroom). Judges thermal → memory → inference → frame time. At the floor and still over budget it sets `signals.thermal = "suspected"` and changes nothing. Hidden tabs are never judged. A per-tracker `targetFps` prop still beats the tier; the app's `perf` is the base and is restored when the controller unmounts.
+- Every change is a decision `{ index, tier, reason, at }` (`init`, `frame-time`, `inference`, `memory`, `thermal`, `recovered`, `manual`, `locked`, `reset`) kept in `history`; the state is in the session store (`useXRQualityState()`, low-level) and in `useXRDiagnostics().quality` / the diagnostics panel.
+- `perf.analysisScale` (0.25..1) is the new knob the tiers drive; set it by hand only with `quality={false}`.
+- Pure seam (low-level): `stepQualityController`, `judgeSample`, `createQualityState`, `setQualityTier`, `lockQuality`, `resetQuality`, `resolveInitialTierIndex`, `readDeviceSignals`.
+
+Full contract: `docs/api/adaptive-quality.mdx`; example: `/adaptive-quality`.
 
 ### `perf` — cap inference rate + camera resolution
 
@@ -1005,6 +1623,320 @@ against hashes pinned in the package. A missing hash for
 `https://cdn.vincentt.studio` fails closed; custom consumer-owned asset URLs
 remain best-effort and warn when no pinned hash exists.
 
+### Capability detection — decide before the session starts
+
+Run the check on the landing screen (no `<XRProvider>` needed). It never opens
+the camera and never downloads a model body: camera = secure context +
+`getUserMedia` + video-input count + Permissions API state; rendering = WebGL
+version, OffscreenCanvas, WebAssembly (WebGPU is reported, not required);
+models = one `HEAD` request per tracker you list.
+
+```tsx
+import { useXRCapabilities, resolveTrackerSupport, XRModel } from "@vincentt-xr/sdk";
+import type { XRCapabilities, XRFallbackReason } from "@vincentt-xr/sdk";
+
+const { capabilities, status, refresh } = useXRCapabilities({
+  models: [XRModel.FACE_TRACKER], // optional asset probes
+});
+if (status === "ready" && capabilities) {
+  const face = resolveTrackerSupport(capabilities, XRModel.FACE_TRACKER);
+  if (!face.supported) renderFallback(face.reason); // { code, message, recoverable }
+  if (face.needsPermission) renderCameraPrimer(); // first getUserMedia will prompt
+}
+```
+
+- `capabilities.camera.status`: `"supported" | "permission-prompt" | "permission-denied" | "unsupported" | "unknown"`. Denied and unsupported are distinct; `reason.recoverable` says whether the viewer can fix it (grant permission, close the other tab).
+- `capabilities.rendering.status` + `webglVersion` (0/1/2), `offscreenCanvas`, `wasm`, `webgpu`.
+- `capabilities.models[model]`: `status`, `reason`, `url`, `bytes` (`unknown` for models not listed). The `SEMANTIC_SEGMENTATION` probe covers the default `selfie_multiclass_256x256` preset only; the hair preset loads a different file.
+- `capabilities.canRunTracking`: rendering supported and camera not unsupported.
+- `detectXRCapabilities(options)` is the same check as a promise. `refresh()` re-runs it (after the viewer granted permission).
+
+**Fallback vocabulary.** Every degraded state is an `XRFallbackReason =
+{ code, message, recoverable }`. Branch on `code`:
+`camera-permission-denied`, `camera-in-use`, `no-camera`,
+`camera-insecure-context`, `camera-unsupported`, `camera-lost`,
+`webgl-unavailable`, `offscreen-canvas-unavailable`, `wasm-unavailable`,
+`model-unreachable`, `model-integrity`, `model-init-failed`, `unknown`.
+`message` is display prose and may be reworded. `resolveTrackerSupport`
+returns the most fundamental blocker first (rendering → camera → model).
+
+Inside a session, a component that does not own a tracker reads an actionable
+status without touching the model:
+
+```tsx
+import { useXRTrackerStatus } from "@vincentt-xr/sdk/low-level";
+
+const { status, fallback } = useXRTrackerStatus(XRModel.FACE_TRACKER);
+// status: "unavailable" (nobody acquired it) | "loading" | "ready" |
+//         "blocked" (camera failed; fallback = the session error) |
+//         "error" (the model's initialize failed; fallback = its cause)
+```
+
+`useXRModel(...)` (the owner) also returns `fallback` next to `error`;
+`useXRModelErrors()` lists every model whose initialize failed.
+
+### Camera permission & privacy — gate, request, retry, recover
+
+Six creator-facing states: `unknown` (never asked; the next request prompts),
+`requesting`, `granted`, `denied` (declined or dismissed; asking again
+prompts), `blocked` (persistently denied in the site settings; asking again
+fails silently), `unavailable` (no camera / insecure context / unsupported /
+held by another app, see `reason`). `denied` vs `blocked` is decided by
+reading the Permissions API after the failed request.
+
+```tsx
+import { CameraPermissionGate, useCameraPermission } from "@vincentt-xr/sdk";
+
+// Gate: children mount only while access is granted; no provider needed.
+<CameraPermissionGate
+  fallback={({ state, reason, canPrompt, request }) => (
+    <Primer state={state} reason={reason} onAllow={canPrompt ? () => request() : undefined} />
+  )}
+>
+  <XRProvider>…</XRProvider>
+</CameraPermissionGate>;
+
+// Hook: query on mount (no prompt), prompt from a button.
+const { state, reason, canPrompt, usable, request, refresh } = useCameraPermission();
+```
+
+- `request()` must run from a user gesture; the probe stream is stopped at once (indicator off, nothing captured). `auto` prompts on mount. Both follow site-setting changes live.
+- The built-in fallback is accessible (`aria-live` status, a real button, disabled + explained when `blocked`); keep those in a custom one.
+- Inside a session, `useXRSessionCameraPermission()` (`@vincentt-xr/sdk/low-level`) reports the session's own state (`requesting` during its webcam bind, `granted`, the classified failure, `blocked` / `unavailable` after a revoke). Stream/canvas sources leave it `unknown`.
+- Retry / recover without remounting: `session.setMediaSource({ source: XRMediaSource.WEBCAM })` then `session.start()`; a failed bind leaves the session where it was.
+- Privacy: `session.stop()` stops the tracks (indicator off); a revoke or lost camera does so automatically; no model runs before `start()`, so no frame is processed before explicit access; frames never leave the device.
+- Plain helpers (core): `queryCameraPermission`, `requestCameraAccess({ keepStream, constraints })`, `subscribeCameraPermission`, `canPromptForCamera`, `CAMERA_PERMISSION_MESSAGES`. `classifyCameraFailure(reason, permissionState)` + `mapDomExceptionToReason(err)` are `@vincentt-xr/sdk/low-level` (they take an `XRErrorReason`).
+- Camera choice: see "Cross-camera & orientation" below.
+
+### Cross-camera & orientation — front / rear, devices, mirror policy
+
+```tsx
+import { XRMediaSource, listCameras, findCameraByFacing } from "@vincentt-xr/sdk";
+import { useXRCameraFacing, useXRMirrored, useXROrientation } from "@vincentt-xr/sdk/low-level";
+
+await session.setMediaSource({ source: XRMediaSource.WEBCAM, facingMode: "environment" }); // or { deviceId }
+const facing = await session.switchCamera({ facingMode: "environment" }); // live switch; starts a bound/stopped session
+const cameras = await listCameras(); // [{ deviceId, label, facing }] — labels need permission
+```
+
+- `facingMode` (`"user"` default, `"environment"`) is a soft hint; `deviceId` is exact and wins. `cameraFacing` / `useXRCameraFacing()` report what the track says (a one-camera laptop answers `environment` with `user`). `useXRCameraDeviceId()` reports the device.
+- Same request = no-op; a different facing / device / mirror policy releases the old tracks once and rebinds under the same source canvas (models keep their input). `running` / `paused` are preserved.
+- **Mirror policy is keyed on the source kind**, and the kind follows the _effective_ facing the track reports: front camera `"webcam"` mirrored (even when `environment` was requested on a one-camera device); rear camera `"rear-camera"`, images, videos straight; `mirrored: boolean` on the bind pins it. Preview and tracking share the feed, so positions never need flipping; `useXRMirrored()` (= `isMirroredSourceKind(sourceKind)`) is for direction semantics only.
+- Orientation: `useXROrientation()` → `{ type: "portrait" | "landscape", angle }`; on change the session re-reads the video size so `useXRFeedCrop()` and landmark-anchored content re-align at once. `calibration.orientation` is only for raw sideways captures the browser does not rotate.
+- Camera-switch lifecycle tests: `lib/hooks/useXRSession/lifecycle.test.ts`. Full contract and phone checklist: `docs/api/cross-camera.mdx`; example: `/camera-switching`.
+
+Full contract: `docs/api/camera-permission.mdx`; example: `/camera-permission`.
+
+### Persistent tracking configuration — `useXRTrackingConfig`
+
+One serializable, validated, versioned document (`XRTrackingConfig`,
+`version: 1`) for tracker selection and tuning, projected onto the provider,
+the tracker components, the camera request and model options. Runtime
+resources (streams, textures, models, session / permission / quality state,
+calibration measurements) are excluded by construction.
+
+```tsx
+import {
+  XRProvider,
+  useXRTrackingConfig,
+  resolveTrackingConfig,
+  parseTrackingConfig,
+} from "@vincentt-xr/sdk";
+import { FaceTracker } from "@vincentt-xr/sdk/tracking";
+
+const tracking = useXRTrackingConfig({
+  storageKey: "myapp.tracking",
+  initial: { trackers: { hand: { enabled: true } } },
+}); // ABOVE the provider
+if (!tracking.hydrated) return null; // wait for the persisted document: a provider re-init would restart the camera
+<XRProvider {...tracking.providerProps}>…</XRProvider>; // perf, lifecycle, quality
+const face = tracking.trackerProps("face"); // { numFaces, targetFps?, holdMs, predictMs, active }
+{
+  face.active && <FaceTracker numFaces={face.numFaces} targetFps={face.targetFps} />;
+}
+await session.switchCamera(tracking.cameraRequest); // { facingMode, deviceId?, mirrored? }
+tracking.update({ perf: { targetFps: 20 } });
+tracking.reset();
+tracking.importJson(text);
+tracking.json;
+tracking.warnings;
+```
+
+- Sections: `trackers.{face,hand,gesture,body,holistic,segmentation,semanticSegmentation}` (`enabled`, `targetFps | null` = inherit, face `numFaces 1..10 / holdMs / predictMs / smoothing`, holistic `hands / holdMs / predictMs / analysisMaxDimension / outputFaceBlendshapes / confidence / delegate`, segmentation `quality`, semantic `preset / quality`), `perf { targetFps, camRes, cameraZoom, analysisScale }`, `lifecycle { pauseWhenHidden }`, `quality` (adaptive partial config or `false`), `camera { facingMode, deviceId, mirrored | "auto" }`, `fallbacks { onUnsupported: "block" | "degrade", delegate }`, `assets { holisticModelUrl, semanticSegmentationModelUrl }`, `updatedAt`.
+- `resolveTrackingConfig(input, base?)` fills from `DEFAULT_TRACKING_CONFIG` (face only, perf 24 fps / 720 / 1.2, adaptive quality on, front camera, degrade), clamps and type-checks every leaf, returns `warnings` instead of throwing; `mergeTrackingConfig` patches; `serializeTrackingConfig` / `parseTrackingConfig` round-trip, drop unknown fields and migrate older `version`s onto the defaults (`migrated: true` + warning; the hook re-saves). `loadTrackingConfig` / `saveTrackingConfig` / `clearTrackingConfig` (`localStorage`, key `vincentt.xr.tracking-config`, injectable storage).
+- Projections: `toProviderProps`, `toTrackerProps(config, kind)`, `toCameraRequest`, `toModelOptions(config, model)`, `enabledModels(config)` (probe these with `useXRCapabilities`).
+
+Full contract: `docs/api/tracking-config.mdx`; example: `/tracking-config`.
+
+### Product placement AR — `<ProductViewer>` + `useProductAr`
+
+"View in your space" goes through the device's native path (WebXR on Android
+Chrome, Scene Viewer on Android, Quick Look on iOS from a USDZ); the SDK's
+camera session has no world tracking. The SDK wraps Google's
+`<model-viewer>` (`@google/model-viewer`, optional peer dependency, lazy-loaded
+on first render, bundles its own three.js ≈ 1 MB — keep it on product routes).
+
+```tsx
+import { ProductViewer, useProductAr } from "@vincentt-xr/sdk/product";
+import { useXRCapabilities } from "@vincentt-xr/sdk";
+
+const ar = useProductAr(); // { status, supportedModes, canActivate, reason, loading, error, bind, activate }
+<ProductViewer
+  src="/hat.glb"
+  iosSrc="/hat.usdz"
+  poster="/hat.webp"
+  alt="Hat"
+  placement="floor"
+  scale="auto"
+  onReady={ar.bind}
+/>;
+{
+  ar.supportedModes.length > 0 && (
+    <button onClick={() => ar.activate()} disabled={!ar.canActivate}>
+      View in your space
+    </button>
+  );
+}
+useXRCapabilities().capabilities?.placement; // { webxr, sceneViewer, quickLook, any, reason } — offer AR only when `any`
+```
+
+- Props map onto model-viewer attributes: `src`, `iosSrc` (USDZ, iOS), `poster`, `alt`, `ar`, `arModes` (preference order, default webxr → scene-viewer → quick-look), `placement` (`floor` | `wall`), `scale` (`auto` | `fixed` = true size), `cameraControls`, `autoRotate`, `loading`, `environmentImage`, `shadowIntensity`, `exposure`, `attributes` (pass-through); events `onLoad`, `onError`, `onArStatus`, `onReady(element)`; children are slotted (`slot="ar-button"`, hotspots).
+- `activate()` must run from a user gesture. `status`: `not-presenting` → `session-started` → `object-placed` (model-viewer's `ar-status`), `failed`, or `unavailable` (no supported mode; `reason.code` `placement-unsupported`).
+- **Camera handoff (WebXR only)**: WebXR runs in the page and needs the camera the tracking session holds; Scene Viewer and Quick Look open a native activity and never touch it. Under an `<XRProvider>` with a running / paused webcam session, `activate()` stops the session first when the mode about to be used is `webxr` and restores the same facing and lifecycle (a paused session comes back paused) when AR ends (`handoffCamera: false` opts out; `releaseSessionForAr(session, mode)` is the pure form). Stream / canvas sources are untouched. Keep `arModes` on the viewer and `modes` on the hook identical (both default to `DEFAULT_PRODUCT_AR_MODES`).
+- Assets: GLB for all platforms (small, Draco / KTX2), USDZ for Quick Look produced at publishing time (never at runtime), both on the CDN.
+
+Full contract: `docs/api/product-ar.mdx`; example: `/product-ar`.
+
+**Variants, hotspots, dimensions** (same door):
+
+```tsx
+import { ProductHotspot, ProductDimensions, useProductVariants, useProductHotspots, useProductDimensions } from "@vincentt-xr/sdk/product";
+import type { ProductHotspotData, ProductViewerElement } from "@vincentt-xr/sdk/product";
+
+const variants = useProductVariants(el);     // KHR_materials_variants: { variants, variant, setVariant } — materials only, carry into WebXR / Scene Viewer; Quick Look shows the USDZ as exported
+const dims = useProductDimensions(el);       // { size: [w, h, d], center } metres, null until load
+const hotspots = useProductHotspots(el, { initial, onChange }); // { hotspots, placeFromPointer(clientX, clientY, label), add, update, remove, clear, json, importJson }
+<ProductViewer scale="fixed" onReady={(e) => setEl(e)}>
+  {hotspots.hotspots.map((h) => <ProductHotspot key={h.id} hotspot={h} />)}   // { id, position: [x,y,z], normal: [x,y,z], label?, description? } in model metres
+  <ProductDimensions dimensions={dims} unit="metric" />                       // W / H / D labels on the bounding box; pair with scale="fixed"
+</ProductViewer>
+```
+
+Pure helpers: `parseHotspots` / `serializeHotspots` (validated authoring data), `hotspotAttributes`, `nextHotspotId`, `dimensionLines`, `formatLength`. Debug UI: `ProductHotspotPanel` (`@vincentt-xr/sdk/debug-ui`) for click-to-place authoring and JSON export. Full contract: `docs/api/product-details.mdx`; example: `/product-details`.
+
+### Device compatibility — matrix, smoke, evidence
+
+Supported: iOS WebKit 16.4+, Android Chrome 110+, desktop Chrome / Edge /
+Safari; Firefox best effort; insecure contexts and no-WebGL unsupported.
+Platform differences the SDK already absorbs: no `video.captureStream` on
+iOS (feed the camera, not a hosted clip), Permissions API camera descriptor
+missing on Firefox (`unknown` until the first prompt), `deviceMemory` only on
+Chromium. Full table: `docs/compatibility.mdx`.
+
+```tsx
+import { XRModel } from "@vincentt-xr/sdk";
+import {
+  runCompatChecks,
+  useXRContext,
+  COMPAT_SCENARIOS,
+  MANUAL_COMPAT_SCENARIOS,
+} from "@vincentt-xr/sdk/low-level";
+
+const { session } = useXRContext();
+const report = await runCompatChecks(session, {
+  models: [XRModel.FACE_TRACKER],
+  label: "iPhone 13",
+}); // stops the session afterwards unless keepSessionRunning
+report.results; // [{ id, status: "pass" | "fail" | "skip", cause: "capability" | "model" | "lifecycle" | "performance" | null, code, detail, durationMs }]
+report.summary; // { pass, fail, skip, causes }
+```
+
+- Automated scenarios: `capabilities`, `session-start`, `model-ready`, `model-detects`, `pause-resume`, `stop-start`, `performance`. It opens the webcam (or `stream`), acquires and releases the models itself, never throws.
+- Manual scenarios (`MANUAL_COMPAT_SCENARIOS`): permission prompt / blocked recovery, rear camera, rotation, background-return, camera interrupted, sustained use, low-end device. `/compat` runs both and exports the JSON evidence (`XRCompatReport`, schema `version: 1`).
+- Failure attribution: `classifyCompatCause(code, layer)` maps SDK codes (`camera-*`, `no-camera`, `webgl-*` → capability; `model-*` → model; `timeout:lifecycle`, `feed-stalled`, `lifecycle:*` → lifecycle; `fps-low`, `inference-high` → performance).
+- CI: `pnpm smoke` (Playwright, headless Chromium, fake camera) runs `/compat?auto=1` and uploads the evidence; `.github/workflows/compat-smoke.yml`, advisory.
+
+### Session lifecycle — start / pause / resume / stop / reset
+
+```
+idle ──setMediaSource──> media-bound ──start──> running ──stop──> stopped
+                              ↑                 │    ↑               │
+                              │           pause │    │ resume         │
+                              │                 ▼    │               │
+                              │               paused ─┘               │
+                              └────────────reset──────────────────────┘
+```
+
+- `session.pause()` — `running → paused`: feed loop stops, every model pauses, the video element pauses; the camera stream stays open. No-op in any other state. `session.resume()` (or `start()`) continues with no rebind. `setMediaSource()` while paused binds the new media and stays paused.
+- `session.stop()` releases the stream/tracks and keeps the models initialized; `start()` afterwards needs `setMediaSource()` first. Binding the _same_ source again (the webcam after a privacy-mode stop) re-opens it; the "unchanged source" short-circuit only applies while the media is live. `session.reset()` also releases every model and clears errors.
+- **Provider unmount disposes the session** (route change away from the AR page): tracks and models are released exactly once and later calls throw `"… called after the provider unmounted"`. Keep one `<XRProvider>` above the pages that share a session.
+- **Camera loss.** A live video track that ends (unplugged, revoked, taken by another app) or a camera permission that flips to denied releases the media once, suspends the models, sets `useXRError()` to `reason: "camera-lost"` (or `"permission-denied"`) and parks the session in `stopped`. Recover with `setMediaSource()` + `start()`. Tracks the session replaced itself during a source swap never trigger this.
+- **Camera constraints.** The webcam bind asks for 24–60 fps and `camRes` width; a camera that cannot promise the minimum (cheap, virtual or fake devices) is retried once with a soft frame-rate request, so a slow camera binds instead of failing with `OverconstrainedError` (which now reads as `unsupported` when even the relaxed request fails).
+- **Visibility.** The session always retries `video.play()` when a hidden tab returns (iOS Safari pauses the camera track). `<XRProvider lifecycle={{ pauseWhenHidden: true }}>` additionally pauses the whole session while hidden and resumes on return; a session the app paused itself stays paused.
+- `useXRLifecycle()` (`@vincentt-xr/sdk/low-level`) subscribes to the state: `"idle" | "media-bound" | "running" | "paused" | "stopped"`.
+
+Full contract and manual steps: `docs/api/session-lifecycle.mdx`; example: `/session-health`.
+
+### Tracking diagnostics — `useXRDiagnostics()` and `<XRDiagnosticsPanel>`
+
+Opt-in: sampled on an interval (default 250 ms, never per frame) only while a
+consumer is mounted, so it costs nothing in production until rendered. Gate it
+behind a debug flag.
+
+```tsx
+import { useXRDiagnostics } from "@vincentt-xr/sdk/low-level";
+import { XRDiagnosticsPanel } from "@vincentt-xr/sdk/debug-ui";
+
+const d = useXRDiagnostics({ intervalMs: 250, models: [XRModel.FACE_TRACKER] });
+d.session; // lifecycle, ready, error, source/sourceKind, sourceWidth/Height, renderWidth/Height, feedFps, feedFrames
+d.models[0]; // model, label, state, inferenceMs, inferenceFps, inferences, skippedFrames,
+// detections, confidence, status, error, paused, targetFps, owners
+d.inferenceMsTotal; // ms of ML per frame across running models
+
+<XRDiagnosticsPanel models={[XRModel.FACE_TRACKER, XRModel.HAND_TRACKER]} />; // DOM panel, outside the canvas
+```
+
+`state` per model: `"unavailable"` (nobody acquired it), `"loading"`,
+`"ready"`, `"paused"` (no active owner, or the session is not running),
+`"error"`. `models` defaults to every `XRModel`, so unacquired models show as
+unavailable. `detections`/`confidence`: face = count / none; hand, gesture =
+count / score; body, holistic = count / mean landmark visibility; segmenters
+= 1 when a mask was produced / none. `skippedFrames` counts RAF ticks the
+model did not analyse (its `targetFps` cap, or busy). The legacy
+`DebuggerProvider` / `DebuggerOverlay` (camera fps, render time, memory)
+still exists; prefer this hook for tracker health.
+
+### Device calibration — `useXRCalibration()`
+
+Orientation, mirror mode, scale/offset, face distance and body proportions,
+measured once, persisted (`localStorage`, key `vincentt.xr.calibration`),
+reset, and applied **only where you call it**. Raw tracker results are never
+altered.
+
+```tsx
+import { useXRCalibration } from "@vincentt-xr/sdk/tracking";
+
+const cal = useXRCalibration({ storageKey: "myapp.calibration" }); // persist: false keeps it in memory
+cal.calibrateFace(face.landmarks); // reference face (feed-space landmarks); false without a face
+cal.calibrateBody(poseFeedLandmarks); // reference body (33 pose landmarks in feed space)
+cal.update({ orientation: 90, mirrored: "auto", scale: 1.1, offset: { x: 0, y: 0 } });
+cal.applyToFeedPoint(face.landmarks[4]); // new point; input untouched
+cal.faceDistance(face.landmarks); // 1 at the reference distance, > 1 closer, null without a face/reference
+cal.bodyScale(poseFeedLandmarks); // 1 at the reference, null without a body/reference
+cal.reset(); // defaults + clears storage
+cal.calibration; // { version, orientation, mirrored, scale, offset, face, body, updatedAt }
+```
+
+`mirrored: "auto"` follows the session (only the webcam is mirrored), so it
+never flips a point; `true`/`false` flip only when they disagree with the
+session. Pure helpers for non-React code: `applyCalibrationToFeedPoint`,
+`measureFaceCalibration`, `measureBodyCalibration`, `estimateFaceDistance`,
+`estimateBodyScale`, `updateCalibration`, `resetCalibration`,
+`loadCalibration`, `saveCalibration`, `clearCalibration` (all
+`@vincentt-xr/sdk/tracking`). Body landmarks from `getLandmarks2D()` are
+tracker space; map to feed with `trackerToFeed` before measuring. Full
+contract: `docs/api/diagnostics-calibration.mdx`; example: `/session-health`.
+
 ---
 
 ## Reading tracker state
@@ -1092,6 +2024,75 @@ useFrame(() => {
 > above; reach for the raw read only when you need the session handle for something the
 > selector can't express.
 
+### `useGesture` / `<GestureListener>` — reusable gesture detection (face, hand, body)
+
+One engine for pinch, grab, open palm, wave, hand raise, nod, head shake,
+mouth open, smile, blink, brow raise, arm raise and T-pose, plus custom
+gestures. Import from `@vincentt-xr/sdk/tracking`. Each gesture acquires the
+shared face, hand or body model it needs (no second detector); mount hooks
+once above scene switches like any tracker.
+
+```tsx
+import { GestureListener, useGesture } from "@vincentt-xr/sdk/tracking";
+
+const pinch = useGesture("pinch", { hand: "right", onBegin: () => grab(), onEnd: () => drop() });
+<GestureListener gesture="headNod" onBegin={() => confirm()} />;
+```
+
+- **Built-ins** (`GESTURE_NAMES`, `GESTURES[name]` with `kind`, `description`,
+  `defaults`): face `mouthOpen`, `smile`, `blink`, `browRaise`, `headNod`,
+  `headShake`; hand `pinch`, `grab`, `openPalm`, `handRaised`, `wave`, and
+  the finger-count signs `oneFinger`, `peaceSign`, `threeFingers`,
+  `fourFingers`, `fist`, `palm`, `thumbUp`, `thumbDown` (the weakest required
+  finger scores, so 1 only when every finger agrees; `grab` / `openPalm` are
+  the thumb-agnostic mean-curl versions of `fist` / `palm`); body
+  `armRaisedLeft`, `armRaisedRight`, `armRaised`, `bothArmsRaised`, `tPose`.
+  Hand measures are scale-free (÷ palm size), body measures are in shoulder
+  widths, body sides are the person's. Nod / shake / wave are oscillations
+  (three turning points above an amplitude within a window) and ship with a
+  600 ms cooldown so one nod counts once.
+- **Engine contract** (`GestureDetectorOptions`): per frame the definition
+  yields `{ value, confidence }` or `null` (no subject). Confidence below
+  `minConfidence` (0.5) or no sample = loss; an active gesture survives loss
+  for `lostGraceMs` (250) with its value frozen, then ends with
+  `endReason: "lost"`. Inactive → `begin` at `value >= enterThreshold`
+  (0.5) after `holdMs` (0) of staying above it, never during `cooldownMs`
+  (0) after the last end; active → `end` ("released") at
+  `value <= exitThreshold` (enter − 0.15). A `begin` always gets one `stay`
+  frame. `GestureState`: `active`, `value`, `confidence`, `tracked`, `phase`
+  begin | stay | end | none, `justBegan`, `justEnded`, `endReason`,
+  `startedAtMs`, `durationMs` (as of the last published change, not a
+  per-frame clock; derive a live duration from `startedAtMs`), `count`.
+  Face/hand/body confidence is
+  presence-based (1 while tracked); custom definitions may supply a real
+  score.
+- **Options** (`UseGestureOptions`): the detector options above, `active`,
+  `faceIndex` / `numFaces`, `hand` / `handIndex`, `bodyIndex`, `targetFps`,
+  `onBegin` / `onEnd` / `onChange` (frame-loop callbacks with that frame's
+  state). The hook returns React state that changes only when the gesture's
+  state changes. `resolveGestureDetectorOptions(overrides, defaults)` is the
+  merge the hook applies.
+- **Custom**: `defineGesture({ name, kind, description, defaults?,
+createMemory?, measure })`; `measure(input, memory, nowMs)` gets a
+  `FaceInfo | null` (face) or `TrackerPoint[] | undefined` (hand: the 21
+  joints; body: the 33 pose points). Low-level seam
+  (`@vincentt-xr/sdk/low-level`): `stepGestureDetector`,
+  `createGestureDetectorState`, `snapshotGestureState`,
+  `gestureStatesEqual`, the measures (`measurePinch`, `measureGrab`,
+  `measureOpenPalm`, `measureHandHeight`, `measureArmRaise`,
+  `measureTPose`, `measureFingerPattern({ fingers, thumb })` with `null`
+  for "either", `measureThumb`, `fingerExtensions`, `thumbExtension`,
+  `thumbDirection`), `HAND_JOINTS`, `POSE_JOINTS`, `OscillationTracker`.
+- **Versus the others**: `<GestureTrigger>` fires once on a named static
+  hand pose from the gesture-recognizer model; `useGesture` measures
+  continuous and temporal gestures from the landmark trackers with tunable
+  thresholds. `useFaceAction` is the face-only predecessor; the face gestures
+  reuse its measurements and add cooldown, confidence, loss grace and the
+  temporal nod / shake. Debug panel: `GestureSettingsPanel`
+  (`@vincentt-xr/sdk/debug-ui`) with `resolveGestureSettings` /
+  `patchGestureSettings` / `GESTURE_SETTINGS_FIELDS`. Full contract and the
+  manual test plan: `docs/api/gestures.mdx`; example: `/gestures`.
+
 ### `<GestureTrigger gestures={["victory"]} onTrigger={(g) => ...}>` — Declarative one-shot gesture handler
 
 Fires once per gesture recognition event. The simplest way to act on a gesture — prefer it over a manual `useFrame` poll. Two modes:
@@ -1131,6 +2132,62 @@ In a v2-template project this is already mounted in `src/App.tsx` — do NOT add
 treat the camera feed as a real scene plane should render a separate editor
 video plane instead of changing `VideoBackground` semantics.
 
+### `<PersonCutout>` — background replacement as two screen layers
+
+The easy path for virtual backgrounds and person cut-outs: any full-screen
+background surface below, the person-masked live camera above.
+
+```tsx
+import { FULL_SCREEN_ANCHORS, PersonCutout, ScreenImage, ScreenSpaceUI } from "@vincentt-xr/sdk";
+
+<ScreenSpaceUI>
+  <ScreenImage
+    src="/backgrounds/beach.jpg"
+    fit="cover"
+    anchors={FULL_SCREEN_ANCHORS}
+    renderOrder={0}
+  />
+  <PersonCutout renderOrder={100} />
+</ScreenSpaceUI>;
+```
+
+- **Background**: a `ScreenImage` with an image, `{ kind: "gif" }`, or
+  `{ kind: "texture", texture: videoTexture }`, or a custom shader plane inside
+  `<ScreenTransform anchors={FULL_SCREEN_ANCHORS}>`; give it a lower
+  `renderOrder` than the cut-out (default `100`). Do not mirror it unless the
+  content must match a mirrored webcam (`flipX`).
+- **Foreground**: `PersonCutout` is a `ScreenImage` whose source is the session
+  camera texture and whose `maskTexture` is the portrait segmentation mask. The
+  camera texture is already cover-cropped and webcam-mirrored, and the mask is
+  in that same feed space; the mask is sampled through the image's own fit
+  transform, so alignment holds under any aspect ratio, crop or flip. Props:
+  the `ScreenImage` props except `source` / mask props, plus `invert` (show the
+  background only), `fallback` (`"camera"` default: unmasked feed while the
+  model loads or fails; `"hidden"`: nothing), `active`, `onStatusChange(status,
+error)` with `loading` | `ready` | `error`. It acquires `<Segmentation
+type="portrait">`'s model itself and shares it with `VideoBackground` and
+  `usePersonMask`.
+- `usePersonMaskTexture()` returns `{ texture, status, error }` for custom
+  compositions; the mask is white (RGB 255) on the person and black elsewhere
+  with alpha 255 everywhere, so read it as `maskChannel="green"` (what three's
+  `alphaMap` reads too). `ScreenImage` gained `maskTexture`, `maskChannel`
+  (`alpha` | `red` | `green` | `luminance`), `maskInvert`, `maskFeather` and
+  `maskErode` for any image; the mask always samples through the image's fit
+  transform and takes precedence over advanced blend modes.
+- **Edge quality**: `maskFeather` is a 3×3 blur radius in mask texels (`0` =
+  hard edge); `maskErode` (0..1, needs feather) shrinks the kept region into
+  that soft edge so the source's background fringe (green-screen spill, halo)
+  is trimmed. `PersonCutout` defaults to `maskFeather={4}` / `maskErode={0.7}`,
+  measured on the 720-px segmenter mask; raise erode for spill, lower it when
+  thin features (fingers, hair tips) vanish. Neither fixes mask lag on fast
+  motion, which is the segmenter's temporal error, not an edge problem.
+- **2D versus 3D**: use this for background replacement and person-in-scene
+  effects (one quad, one mask, covers body, hair and clothes). Use the face
+  mesh when content must stick to the face surface with perspective and
+  shading; use the head occluder when 3D objects must hide behind the head in
+  depth, which a 2D mask cannot do. Full contract and the manual test plan:
+  `docs/api/person-cutout.mdx`; example: `/person-cutout`.
+
 ---
 
 ## Mesh primitives and textures
@@ -1379,6 +2436,56 @@ authoring model into center, size, rotation, pivot offset, and aspect. Do not co
 the default `720x1280` root or `32` reference-pixel constant into another renderer
 or guide.
 
+### Coordinate spaces — one toolkit
+
+Six spaces, named once; every conversion lives in `@vincentt-xr/sdk` and does
+its origin, axis, crop and mirror change exactly once. Never chain a manual
+flip, crop or design-canvas conversion after a helper.
+
+- **source**: raw media pixels (top-left, Y down), never mirrored.
+- **feed**: the camera texture, 0..1 top-left Y down — the source
+  cover-cropped to the render size, enlarged by `perf.cameraZoom` (1.2)
+  about the centre, mirrored for the **webcam only**. Every landmark and SDK
+  mask lives here. The feed fills the viewport, so feed and viewport agree
+  per axis on every aspect ratio.
+- **tracker**: feed remapped to center origin, X right, Y up, Z toward the
+  viewer (`TrackerPoint`: hand, body, holistic nodes).
+- **viewport**: the rendered canvas in CSS pixels; **screen**: 720×1280
+  design pixels, center origin, Y up; **world**: three.js, tracked content on
+  the z = 0 plane.
+
+`FeedCrop` (`computeFeedCrop({ source, feed, zoom, mirrored })`) is the one
+description of source → feed: the session's blit loop draws with it,
+`computeMaskUvTransform` (now with `zoom`) fits source-space masks with it,
+and the conversions use it. `useXRFeedCrop()` (`/low-level`) returns the live
+one (`null` until media is bound).
+
+Conversions: `sourceToFeed` / `feedToSource` (+ `isFeedPointInSource`),
+`feedToTracker` / `trackerToFeed`, `feedToViewport` / `viewportToFeed`,
+`feedToScreenPixels` / `screenPixelsToFeed`, `feedToWorld` / `trackerToWorld`
+(camera, optional `planeZ`, depth-independent), `worldToFeed` /
+`worldToTracker` (`z` = NDC depth), `computeFitLayout(source, destination,
+"cover" | "contain", zoom)`. Mirroring: positions never need a flip; direction
+signs do — use `isMirroredSourceKind(sourceKind)` (webcam or unknown → true),
+never `sourceKind === "webcam"` inline; `mirrorFeedX` flips a feed x.
+Orientation: `orientSourcePoint` / `orientedSize` rotate a sideways capture
+(0 / 90 / 180 / 270, clockwise) before `sourceToFeed`; the browser already
+orients webcam frames. The trackers (`HandTracker`, `GestureTracker`,
+`BodyTracker`, `HolisticTracker`, `FaceTracker`, `IrisEffect`), the face
+model's mirror handling, the viewport layout and the mask fit all run on
+these helpers; `unprojectToZZero` remains as a thin alias of `feedToWorld`.
+Diagrams and worked examples: `docs/api/coordinate-spaces.mdx`.
+
+```ts
+import { feedToSource, feedToWorld, trackerToFeed, feedToViewport } from "@vincentt-xr/sdk";
+import { useXRFeedCrop } from "@vincentt-xr/sdk/low-level";
+
+const crop = useXRFeedCrop(); // source → feed, live
+const sourcePx = crop ? feedToSource(face.landmarks[1], crop) : null; // back onto the raw frame
+const css = feedToViewport(trackerToFeed(hand.coordinates[8]), viewportSize); // DOM overlay
+feedToWorld(face.landmarks[1], camera, mesh.position); // onto the z = 0 plane
+```
+
 ### Screen/world coordinate utilities
 
 Import the `*ScreenPixels` coordinate helpers, `screenPointToRay`, and
@@ -1388,6 +2495,11 @@ design pixels are Three.js world units.
 SDK screen pixels use the Effect House-style `720x1280` design canvas, center
 origin, and +Y up.
 
+`docs/api/screen-space-coordinates.mdx` is the API source of truth for which
+helper owns each conversion. Keep this grounding summary and that page aligned;
+raw MediaPipe face landmarks are normalized image points, while SDK low-level
+hand, gesture, and body reads are center-origin tracker points.
+
 ```ts
 const units = screenPixelsToUnits(120, { parentSize: screenDimensions });
 const pixels = screenUnitsToPixels(units, { parentSize: screenDimensions });
@@ -1416,8 +2528,8 @@ const placementRay = screenPointToRay({
 - Use `clientToScreenPixels` / `screenPixelsToClient` for pointer events and DOM
   overlays; pass the viewport element's `getBoundingClientRect()` result.
 - Use `normalizedToScreenPixels` / `screenPixelsToNormalized` for normalized
-  viewport coordinates with top-left origin. These values are not automatically
-  camera-crop adjusted.
+  viewport coordinates with top-left origin — a feed point is one. They do not
+  undo the camera crop; `feedToSource` with the session's `FeedCrop` does.
 - Use `mediaToScreenPixels` / `screenPixelsToMedia` with the shared
   `XRMediaLayout` for camera/image pixels. They apply cover/contain offsets,
   scaling, and optional display mirroring; do not manually repeat that math.
@@ -1677,8 +2789,12 @@ For debug pages that need a camera/image/video picker, import
 applies the selected debug media preset to the current XR session and starts
 the face-tracking media pipeline used by the debug face examples. The older
 `useMediaSourceSession(activeMediaSource, { pipeline })` name remains available
-for custom debug pipelines. Use these only in debug/example UI; production
-runtime should set the session media source directly.
+for custom debug pipelines. `defaultMediaSources` lists the hosted clips first
+(so `defaultMediaSources[0]` is a video and a page opens without a camera
+prompt or a camera at all), then `webcam`, then the stills; pick the webcam
+entry explicitly when a page must start on the live camera. Use these only in
+debug/example UI; production runtime should set the session media source
+directly.
 
 ### Debug UI usage in Harness client apps
 
@@ -2105,8 +3221,10 @@ if (error?.reason === "camera-in-use") {
 The shape is `{ error, message, reason }`:
 
 - **`reason`** — the stable, machine-readable identity. Branch on this. One of
-  `"permission-denied"`, `"camera-in-use"`, `"no-camera"`, `"unsupported"`, or
-  `"unknown"`.
+  `"permission-denied"`, `"camera-in-use"`, `"no-camera"`, `"unsupported"`,
+  `"camera-lost"` (the camera stopped mid-session; see "Session lifecycle"), or
+  `"unknown"`. `sessionReasonToFallback(reason)` (`@vincentt-xr/sdk/low-level`)
+  maps it onto the `XRFallbackReason` vocabulary.
 - **`message`** — human prose for display. It may be reworded in any release, so
   never match on its text.
 - **`error`** — a legacy constant kept for backwards compatibility. Carries no
@@ -2120,10 +3238,61 @@ request; `"unknown"` is anything unrecognized.
 
 ---
 
+## Asset lifecycle & GPU memory — `useXRAsset`, `XRAssetCache`
+
+One identity, one load, one owner list, one disposal per asset. Own three.js
+loaders over the SDK's verifying fetch; no drei.
+
+```tsx
+import {
+  XRAssetCacheProvider,
+  createAssetCache,
+  defineGltfAsset,
+  defineTextureAsset,
+  defineVideoAsset,
+  useXRAsset,
+  useXRAssetStats,
+  useXRAssetCache,
+} from "@vincentt-xr/sdk";
+
+const HAT = defineGltfAsset(url); // { key, load(signal), dispose, bytes }
+const { value, status, error } = useXRAsset(HAT); // shared value; reference released on unmount
+<XRAssetCacheProvider cache={createAssetCache({ budgetBytes: 64 * 1048576 })}>
+  …
+</XRAssetCacheProvider>;
+const stats = useXRAssetStats(); // assets, referencedAssets, usageBytes, reclaimableBytes, budgetBytes, overBudget, evictions, hits, misses
+```
+
+- `cache.acquire(asset, ownerId)` / `release(key, ownerId)`: in-flight loads are shared; owners are ids (re-renders and StrictMode never double-count); releasing during a load aborts it.
+- Unreferenced values stay cached until the budget (default 256 MiB) needs room — LRU, unreferenced only, `dispose` exactly once — or `disposeUnreferenced()` / `disposeAll()`. `setBudget`, `evict(bytes)`, `preload(asset)` (warm without a reference), `get(key)`, `list()`, `subscribe()`.
+- Bytes are estimates: `estimateTextureBytes`, `estimateGeometryBytes`, `estimateObjectBytes`; `disposeObject3D` releases everything under a scene node once.
+- Route change: owners release on unmount; `<XRAssetCacheProvider>` disposes unreferenced values when it unmounts (`disposeUnreferencedOnUnmount`, default true). Tracker-owned resources are the session registry's, released once on provider unmount.
+- Custom assets: write your own `XRAssetDefinition` (`key` + `load` honouring `signal` + `dispose` + `bytes`).
+
+Full contract: `docs/api/asset-lifecycle.mdx`; example: `/asset-lifecycle`.
+
+---
+
 ## Capture & share
 
 Grab a photo of the live render or record the canvas, then save or share it.
 
+### Camera, layout, mirror, and capture parity
+
+One owner composes media pixels before they are displayed, tracked, or captured.
+For direct session media, the session owns the webcam source and mirrors it once
+before its cover-cropped camera texture reaches trackers and R3F content. For
+`XRViewport mediaStrategy="internal"`, the viewport owns source selection,
+fit, and mirror state; it sends that already composed canvas to the session, so
+the tracker, visible media surface, and capture target share the same pixels.
+
+Resize and source changes recompute this composition before the next tracker
+frame. Do not add a second CSS, canvas, or shader mirror to a session camera
+texture or viewport media canvas. `captureFrame()` and `captureStream()` capture
+the active XRScene canvas when present, otherwise the DOM viewport media canvas.
+They capture canvas composition only; arbitrary DOM HUD elements are not part of
+the image or recording.
+
 ### `useFrameCapture()` + `dataURLtoFile()` — Single-shot photo
 
 Use `useFrameCapture` for UI/effect logic. It exposes `status`, `capturePhoto`,
@@ -2144,7 +3313,13 @@ await photo.capturePhoto();
 ```
 
 For synchronous low-level code, `session.captureFrame()` remains available via
-`useXRContext` from `@vincentt-xr/sdk/low-level`.
+`useXRContext` from `@vincentt-xr/sdk/low-level`. It also exposes
+`session.captureStream(fps?)`, which records the same active canvas:
+
+```tsx
+const rec = useMediaRecorder();
+await rec.start(session.captureStream(30));
+```
 
 ### `useMediaRecorder()` — Record the canvas to a video Blob
 
@@ -2156,8 +3331,7 @@ the first supported codec (mp4 → webm). Pass `audio: true` to mix in the mic,
 import { useMediaRecorder } from "@vincentt-xr/sdk";
 
 const rec = useMediaRecorder();
-const canvas = document.querySelector("#vincentt-xr-canvas canvas");
-await rec.start(canvas.captureStream(30));
+await rec.start(session.captureStream(30));
 // ...later
 rec.stop(); // -> rec.recordedBlob is the video Blob
 ```