@vincentt-xr/sdk 2.0.0-alpha.6 → 2.0.0-alpha.8

package/docs/api/camera-permission.mdx

@@ -0,0 +1,142 @@
+---
+title: "Camera Permission & Privacy"
+description: "Distinguish unknown, requesting, granted, denied, blocked and unavailable camera states; request, retry and recover without remounting; release the camera cleanly; keep every frame on the device until the viewer allows access."
+sidebarTitle: "Camera Permission"
+icon: "camera"
+---
+
+Camera access is the first thing that can go wrong in an AR experience, and
+the browser's `NotAllowedError` hides four different situations. The SDK
+splits them into states a creator can act on, offers the request flow as a
+hook and a gate component, and guarantees no camera frame is processed
+before explicit, usable access.
+
+## States
+
+| State         | Meaning                                                                              | Next step                                                 |
+| ------------- | ------------------------------------------------------------------------------------ | --------------------------------------------------------- |
+| `unknown`     | Never asked (the next request prompts), or the browser cannot say.                   | Show a primer and a request button.                       |
+| `requesting`  | A prompt is open.                                                                    | Wait.                                                     |
+| `granted`     | Usable.                                                                              | Mount the session.                                        |
+| `denied`      | This prompt was declined or dismissed; the next request prompts again.               | Offer "try again".                                        |
+| `blocked`     | Persistently denied in the site settings; a request fails without a prompt.          | Explain how to change the setting.                        |
+| `unavailable` | No camera, insecure context, unsupported browser, or held by another app (`reason`). | Fall back; `reason.recoverable` says if a retry can help. |
+
+`denied` and `blocked` are told apart by reading the Permissions API _after_
+the failed request: `denied` there means blocked. `reason` is an
+`XRFallbackReason` (`camera-permission-denied`, `camera-in-use`,
+`no-camera`, `camera-insecure-context`, `camera-unsupported`, `camera-lost`).
+
+## Before the session: the gate
+
+```tsx
+import { CameraPermissionGate, useCameraPermission } from "@vincentt-xr/sdk";
+
+// Component form: children render only while access is granted.
+<CameraPermissionGate
+  fallback={({ state, reason, canPrompt, request }) => (
+    <Primer state={state} reason={reason} onAllow={canPrompt ? () => request() : undefined} />
+  )}
+>
+  <XRProvider>
+    <Experience />
+  </XRProvider>
+</CameraPermissionGate>;
+
+// Hook form, for your own layout.
+const { state, reason, canPrompt, usable, request, refresh } = useCameraPermission();
+```
+
+Neither needs an `<XRProvider>`. On mount they only _query_ the permission
+(no prompt); `request()` prompts, and must run from a user gesture (a
+button) as browsers require. The probe stream is stopped at once, so the
+camera indicator goes off again and nothing is captured: the call only
+moves the permission. Pass `auto` to prompt on mount. Both follow the
+site-setting changes through the Permissions API, so a viewer who allows
+the camera in the address bar flips the gate without a reload.
+
+The built-in fallback of `<CameraPermissionGate>` announces state changes
+with `aria-live="polite"`, keeps the request behind a real `<button>`,
+disables it when a prompt cannot help, and labels the region "Camera access".
+Keep those in a custom fallback: a visible button, `aria-live` on the
+status text, `aria-disabled` when blocked, and the explanation for
+`blocked` next to the button rather than in a toast.
+
+## Inside the session
+
+The session reports its own live state through `useXRSessionCameraPermission()`
+(`@vincentt-xr/sdk/low-level`): `requesting` while its webcam
+`getUserMedia` is pending, `granted` once bound, the classified failure
+otherwise, and `blocked` / `unavailable` after a mid-session revoke or a
+lost camera. Stream and canvas sources leave it `unknown`.
+
+```tsx
+const { state, reason } = useXRSessionCameraPermission();
+```
+
+### Request, retry, recover without remounting
+
+```tsx
+const { session } = useXRContext();
+const retry = async () => {
+  await session.setMediaSource({ source: XRMediaSource.WEBCAM }); // prompts again if allowed
+  await session.start();
+};
+```
+
+A failed bind leaves the session where it was (`idle` stays idle, a running
+session keeps its old source), so the same provider, models and scene are
+reused. A revoke mid-session stops the session with a `camera-lost` /
+`permission-denied` error; the same two calls recover it.
+
+### Privacy: releasing the camera
+
+- `session.stop()` stops every media track: the browser's camera indicator
+  goes off and no frame is read. Models stay initialized, so a later
+  `setMediaSource` + `start` is fast. Use this for an in-app "privacy mode".
+- A revoke (`permission-denied`) or a lost device (`camera-lost`) releases
+  the tracks once and suspends every model automatically.
+- No model runs before `session.start()`, and `start()` needs a bound
+  source, so no camera data is processed before explicit access. The gate
+  adds the second guarantee: the session is not even mounted until
+  `granted`.
+- Frames never leave the device: inference runs in the page, and the only
+  network traffic is the one-time model and engine download.
+
+## Choosing the camera
+
+`setMediaSource({ source: XRMediaSource.WEBCAM, facingMode: "environment" })`
+opens the rear camera; `"user"` (default) the front one; `deviceId` picks a
+specific camera from `enumerateDevices()`. Binding again with a different
+facing swaps the camera mid-session. The front camera is mirrored
+(`sourceKind` `"webcam"`), the rear camera is drawn straight (`sourceKind`
+`"rear-camera"`), and `mirrored: true | false` overrides that for a camera
+whose facing is unknown. Trackers and the coordinate toolkit follow the
+source kind, so content stays aligned either way. `useXRCameraFacing()`
+(low-level) reports the bound facing. The full cross-camera and orientation
+contract lands with the next sprint item.
+
+## Low-level helpers
+
+`queryCameraPermission()`, `requestCameraAccess({ keepStream, constraints })`,
+`subscribeCameraPermission(listener)` and `canPromptForCamera(state)` are
+plain functions (core door) for non-React code; `CAMERA_PERMISSION_MESSAGES`
+holds the default copy per state. `classifyCameraFailure(reason,
+permissionState)` and `mapDomExceptionToReason(err)` live in `/low-level`
+next to `XRErrorReason`, for callers that classify their own
+`getUserMedia` failure.
+
+## Manual validation
+
+Run `/camera-permission`:
+
+| Case         | Steps                                     | Expect                                                            |
+| ------------ | ----------------------------------------- | ----------------------------------------------------------------- |
+| First visit  | load                                      | gate `unknown`, no prompt until "Allow camera"                    |
+| Allow        | click Allow, accept                       | `requesting` → `granted`; the experience mounts and binds         |
+| Dismiss      | click Allow, close the prompt             | `denied`, "Try again" enabled                                     |
+| Block        | deny with "never" / site setting          | `blocked`, button disabled, settings hint shown                   |
+| Unblock      | allow in the address bar                  | gate flips to `granted` without reload                            |
+| Revoke       | "simulate revoke" (or revoke in settings) | session `blocked` / `unavailable`, lifecycle `stopped`            |
+| Recover      | "retry camera"                            | prompt if needed, lifecycle `running`                             |
+| Privacy mode | "privacy mode (stop camera)"              | camera indicator off, lifecycle `stopped`; "retry camera" resumes |

package/docs/api/capture-recording.mdx

@@ -33,6 +33,12 @@ after exposing a failure through `status`, `error`, and `onError`. For
 synchronous low-level code, `session.captureFrame()` remains available through
 `useXRContext` from `@vincentt-xr/sdk/low-level`.
 
+The SDK captures the active composition owner: an `XRScene` canvas when one is
+mounted, otherwise the `XRViewport` DOM media canvas. For viewport-managed
+media, source selection, cover/contain layout, and mirror state are composed
+once before display, tracking, and capture. Arbitrary DOM HUD elements are not
+part of canvas capture.
+
 | Returned field   | Meaning                                                      |
 | ---------------- | ------------------------------------------------------------ |
 | `status`         | `idle`, `capturing`, `completed`, or `error`                 |
@@ -53,7 +59,9 @@ an existing audio stream.
 
 ```tsx
 import { useMediaRecorder } from "@vincentt-xr/sdk";
+import { useXRContext } from "@vincentt-xr/sdk/low-level";
 
+const { session } = useXRContext();
 const recorder = useMediaRecorder({
   audio: true,
   onRecordingStart: () => console.log("recording"),
@@ -63,7 +71,7 @@ const recorder = useMediaRecorder({
   onRecordingComplete: uploadVideo,
   onError: reportError,
 });
-await recorder.start(canvas.captureStream(30));
+await recorder.start(session.captureStream(30));
 recorder.pause();
 recorder.resume();
 recorder.stop();

package/docs/api/coordinate-spaces.mdx

@@ -0,0 +1,165 @@
+---
+title: "Coordinate Spaces"
+description: "One toolkit for source, feed, tracker, viewport, screen and world coordinates: cover crop, zoom, mirroring, orientation, and documented conversions."
+sidebarTitle: "Coordinate Spaces"
+icon: "arrows-to-dot"
+---
+
+Every tracking consumer meets the same six spaces. Import the conversions from
+`@vincentt-xr/sdk`; each one performs its origin, axis, crop and mirror change
+exactly once, so never chain a manual flip or crop after it.
+
+## The spaces
+
+```
+ source (media px)         feed (0..1)                 viewport (CSS px)
+ ┌──────────────────┐      ┌────────┐                  ┌────────┐
+ │ raw video/image  │ cover│        │  identity        │        │
+ │ 1280 × 720       │ crop │ 720 ×  │  per axis        │ 390 ×  │
+ │ never mirrored   │──────▶ 1280   │◀────────────────▶│ 844    │
+ └──────────────────┘ zoom │ webcam │                  │        │
+                     mirror│ mirror │                  └────────┘
+                           └───┬────┘                       │
+                               │ x·2−1, 1−y·2                │ design canvas
+                               ▼                             ▼
+                           tracker (−1..1, Y up)        screen (720×1280 design px, Y up)
+                               │
+                               │ camera ray → z = 0 plane
+                               ▼
+                           world (three.js)
+```
+
+| Space        | Origin / axes                                  | Who produces it                                                |
+| ------------ | ---------------------------------------------- | -------------------------------------------------------------- |
+| **source**   | top-left, Y down, media pixels, never mirrored | the `<video>` / `<img>` the session was given                  |
+| **feed**     | top-left, Y down, 0..1                         | the camera texture; every landmark and SDK mask                |
+| **tracker**  | center, X right, Y up, Z toward the viewer     | hand, body, holistic nodes (`TrackerPoint`)                    |
+| **viewport** | top-left, Y down, CSS pixels                   | the rendered canvas, pointer events via `clientToScreenPixels` |
+| **screen**   | center, Y up, 720×1280 design pixels           | `ScreenSpaceUI`, `ScreenImage`, labels                         |
+| **world**    | three.js world; tracked content on z = 0       | `TrackingAnchor`, face mesh, iris quads                        |
+
+The feed is the source **cover-cropped to the render size**, enlarged by
+`perf.cameraZoom` (default 1.2) about the centre, and **mirrored for the
+webcam only**. Image and video sources are drawn straight. Because the feed
+fills the viewport, feed and viewport agree per axis; a landmark at feed
+(0.5, 0.5) is the middle of the canvas on every aspect ratio.
+
+## The feed crop, once
+
+`FeedCrop` is the single description of source → feed. The session's blit
+loop draws with it, `computeMaskUvTransform` fits source-space masks with it,
+and the point conversions use it, so what the trackers saw and what you
+compute agree.
+
+```ts
+import { computeFeedCrop, feedToSource, sourceToFeed } from "@vincentt-xr/sdk";
+import { useXRFeedCrop } from "@vincentt-xr/sdk/low-level";
+
+// Inside the provider: the live crop (null until media is bound).
+const crop = useXRFeedCrop();
+
+// Or build one yourself for an offline pipeline.
+const offline = computeFeedCrop({
+  source: { width: 1280, height: 720 },
+  feed: { width: 720, height: 1280 },
+  zoom: 1.2,
+  mirrored: true, // webcam
+});
+
+// A landmark back onto the raw frame (for a crop, a server call, an overlay on the <video>).
+const px = feedToSource({ x: 0.62, y: 0.4 }, crop); // { x, y } in source pixels
+// A detection made on the raw frame into landmark space.
+const fp = sourceToFeed({ x: 800, y: 300 }, crop); // { x, y } in 0..1
+```
+
+`feedToSource` can return points outside the source when the feed shows
+less than the frame (zoom > 1 never does; a contain-shaped source does);
+`isFeedPointInSource` tells you.
+
+## Mirroring
+
+- Positions never need a flip: landmarks already live in the (possibly
+  mirrored) feed, and the feed is what you see.
+- **Direction signs** do. Holistic hand labels and the iris `offset.x` are
+  corrected with `isMirroredSourceKind(sourceKind)` so "the person's left"
+  means the same thing on every source. Use the same helper, never
+  `sourceKind === "webcam"` inline.
+- `mirrorFeedX` flips a feed point's x (for a direction vector, or to compare
+  a selfie feed with a straight recording).
+
+## Camera orientation
+
+A frame delivered sideways (a native capture, a rotated file) must be
+rotated before it becomes the source. `orientSourcePoint(point, size,
+orientation)` and `orientedSize(size, orientation)` move a point and a frame
+size through a clockwise 0 / 90 / 180 / 270 rotation; apply them before
+`sourceToFeed`. The browser already orients webcam frames, so the session
+does not rotate.
+
+## Conversions
+
+| From → to              | Helper                                                              |
+| ---------------------- | ------------------------------------------------------------------- |
+| source ↔ feed         | `sourceToFeed`, `feedToSource` (with a `FeedCrop`)                  |
+| feed ↔ tracker        | `feedToTracker`, `trackerToFeed`                                    |
+| feed ↔ viewport       | `feedToViewport`, `viewportToFeed` (viewport size)                  |
+| feed ↔ screen         | `feedToScreenPixels`, `screenPixelsToFeed`                          |
+| feed / tracker → world | `feedToWorld`, `trackerToWorld` (camera, optional `planeZ`)         |
+| world → feed / tracker | `worldToFeed`, `worldToTracker` (`z` is NDC depth)                  |
+| media layout           | `computeFitLayout(source, destination, "cover" \| "contain", zoom)` |
+
+The screen-space helpers (`normalizedToScreenPixels`, `worldToScreenPixels`,
+`screenPointToRay`, `mediaToScreenPixels` …) are unchanged and documented in
+[Screen-Space Coordinates](/api/screen-space-coordinates); a feed point is a
+"normalized viewport point" there.
+
+### Worked examples
+
+**A hand joint to a DOM overlay**
+
+```ts
+const joint = handNode.hands?.[0].coordinates?.[8]; // tracker space
+const feed = trackerToFeed(joint); // 0..1
+const css = feedToViewport(feed, { width: el.clientWidth, height: el.clientHeight });
+badge.style.transform = `translate(${css.x}px, ${css.y}px)`;
+```
+
+**A face landmark into the 3D scene** (what `TrackingAnchor` does)
+
+```ts
+useFrame(() => {
+  const tip = face.landmarks[1]; // feed space, from useFaceInfo
+  feedToWorld(tip, camera, mesh.position); // onto the z = 0 tracking plane
+});
+```
+
+**A tap to the raw frame** (send the touched pixel to a server)
+
+```ts
+const screen = clientToScreenPixels({ point: { x: e.clientX, y: e.clientY }, rect });
+const feed = screenPixelsToFeed(screen, viewportSize);
+const sourcePx = feedToSource(feed, useXRFeedCrop()!);
+```
+
+**A mask made on the raw video**
+
+```ts
+computeMaskUvTransform({
+  space: "source",
+  maskWidth: 1280,
+  maskHeight: 720,
+  feedWidth,
+  feedHeight,
+  zoom: 1.2, // the session's perf.cameraZoom
+  mirrored: true, // webcam
+});
+```
+
+## Regression coverage
+
+`lib/coordinates/coordinates.test.ts` pins: the cover crop with zoom against
+the blit loop's rectangle for landscape→portrait, portrait→landscape and 4:3
+pairs; webcam mirroring (source left edge → feed right edge, round trip);
+feed/tracker/viewport/screen round trips and agreement with
+`trackerPointToScreenPixels`; feed→world depth independence and plane
+selection; and the mask fit matching the feed crop, zoom included.

package/docs/api/cross-camera.mdx

@@ -0,0 +1,88 @@
+---
+title: "Cross-Camera & Orientation"
+description: "Front / rear camera selection and switching, device enumeration, explicit mirror policy for preview and tracking, and orientation-safe alignment."
+sidebarTitle: "Cross-Camera"
+icon: "camera-rotate"
+---
+
+## Choosing and switching cameras
+
+```tsx
+import { XRMediaSource, listCameras, findCameraByFacing } from "@vincentt-xr/sdk";
+
+// Bind: front (default) or rear, or a specific device.
+await session.setMediaSource({ source: XRMediaSource.WEBCAM, facingMode: "environment" });
+await session.setMediaSource({ source: XRMediaSource.WEBCAM, deviceId });
+
+// Switch on a live session (models, scene and lifecycle stay; a bound or
+// stopped session is started). Resolves to the facing the track reports.
+const facing = await session.switchCamera({ facingMode: "environment" });
+
+// Enumerate (labels appear once permission is granted).
+const cameras = await listCameras(); // [{ deviceId, label, facing: "user" | "environment" | null }]
+const rear = findCameraByFacing(cameras, "environment");
+```
+
+`facingMode` is a soft hint: a laptop with one camera answers `environment`
+with that camera, and `cameraFacing` then reports what the track says.
+`deviceId` is exact and beats the hint. The same request again is a no-op;
+a different facing, device or mirror policy releases the old stream and
+tracks once and binds the new camera under the same source canvas, so every
+model keeps its input binding. `useXRCameraFacing()` and
+`useXRCameraDeviceId()` (low-level) read the result.
+
+## Mirror policy
+
+There is exactly one mirror decision per session and it is keyed on the
+source kind:
+
+| Source                       | `sourceKind`      | Mirrored |
+| ---------------------------- | ----------------- | -------- |
+| front camera (`user`)        | `webcam`          | yes      |
+| rear camera (`environment`)  | `rear-camera`     | no       |
+| image / video stream, canvas | `image` / `video` | no       |
+
+The kind follows the _effective_ facing (what the track reports), so a
+one-camera laptop that answers an `environment` request with its front
+camera is still mirrored; `mirrored: true | false` on the bind pins it
+either way. Preview and tracking read the same feed, so a landmark
+unprojected to the scene lands where the preview shows it on every camera.
+Consumers never flip positions; `useXRMirrored()` (low-level, the same
+answer as `isMirroredSourceKind(sourceKind)`) is for _direction_ semantics
+only, such as which hand is the person's left. Every tracker and the
+coordinate toolkit already key off the kind.
+
+## Orientation
+
+`useXROrientation()` (low-level) reports `{ type: "portrait" | "landscape",
+angle }` from `screen.orientation` (viewport aspect as the fallback). On a
+change the session re-reads the live video size, so `useXRFeedCrop()` and
+everything built on the coordinate toolkit re-align without waiting for the
+next frame; the blit loop also resyncs the source size every frame. Content
+anchored to landmarks (`TrackingAnchor`, `feedToWorld`, …) therefore stays
+on the face through a rotation and a camera switch. Use
+`calibration.orientation` only for sources the browser does not rotate
+(a raw sideways capture).
+
+## Lifecycle guarantees
+
+- Switching releases the previous stream exactly once (`ended` from the
+  replaced tracks is ignored), keeps models initialized, and keeps
+  `running` / `paused` as it was; `switchCamera` additionally starts a
+  `media-bound` / `stopped` session.
+- A switch that fails (device unplugged, in use) leaves the session where it
+  was with the error published; call again or pick another device.
+- `session.dispose()` (provider unmount) removes the orientation listeners.
+
+## Manual validation
+
+Run `/camera-switching` on a phone:
+
+| Case                | Steps                               | Expect                                                                 |
+| ------------------- | ----------------------------------- | ---------------------------------------------------------------------- |
+| Front camera        | load                                | `front · mirrored`; the cyan dot sits on the nose                      |
+| Rear camera         | "front → rear"                      | `rear · straight`; text in view reads correctly; dot still on the nose |
+| Device list         | after permission                    | labelled cameras; tapping one opens it (device id shown)               |
+| Mirror override     | mirror = straight, "apply mirror"   | front feed un-mirrored; dot still aligned                              |
+| Rotate              | turn the phone                      | orientation line flips, source size updates, dot stays aligned         |
+| Switch while paused | pause (session-health), switch here | stays paused; resume runs on the new camera                            |

package/docs/api/diagnostics-calibration.mdx

@@ -0,0 +1,97 @@
+---
+title: "Diagnostics & Calibration"
+description: "Opt-in tracking diagnostics (readiness, inference time and rate, dropped frames, confidence, errors) and a device calibration workflow for orientation, mirror, scale, face distance and body proportions."
+sidebarTitle: "Diagnostics & Calibration"
+icon: "gauge-high"
+---
+
+## Tracking diagnostics
+
+```tsx
+import { useXRDiagnostics } from "@vincentt-xr/sdk/low-level";
+import { XRDiagnosticsPanel } from "@vincentt-xr/sdk/debug-ui";
+
+// Data (any component under <XRProvider>, outside the R3F canvas or inside):
+const diagnostics = useXRDiagnostics({ intervalMs: 250, models: [XRModel.FACE_TRACKER] });
+diagnostics.session.feedFps; // frames the session blits per second
+diagnostics.models[0].state; // "unavailable" | "loading" | "ready" | "paused" | "error"
+diagnostics.models[0].inferenceMs; // last inference, ms
+diagnostics.models[0].inferenceFps; // inferences per second (~1 s window)
+diagnostics.models[0].skippedFrames; // RAF ticks not analysed (throttle / busy)
+diagnostics.models[0].detections; // faces / hands / bodies in the last inference
+diagnostics.models[0].confidence; // primary-subject score, when the model has one
+
+// Panel (debug UI, outside the canvas):
+<XRDiagnosticsPanel models={[XRModel.FACE_TRACKER, XRModel.HAND_TRACKER]} />;
+```
+
+The hook samples on an interval, never per frame, and only while a consumer
+is mounted. Nothing is collected in production unless you render it, so it is
+safe to ship behind a debug flag. The `models` list defaults to every
+`XRModel`, so models nobody acquired show as `unavailable`.
+
+Per-model fields come from the model instance (`getDiagnostics()` on every
+`BaseXRModel`): inference timing is recorded around each MediaPipe call;
+`detections` and `confidence` are what each model can report (face: count,
+no score; hand / gesture: count and the handedness or gesture score; body /
+holistic: count and mean landmark visibility; segmenters: 1 when a mask was
+produced, no score). `targetFps` is the per-tracker cap when one was
+requested. `owners` lists who holds the model and who wants it running.
+
+Session fields mirror the store: lifecycle, ready, source and kind, camera
+and feed sizes, feed fps and the session error. `inferenceMsTotal` sums the
+last inference of every running model, a quick "how much ML per frame".
+
+## Calibration
+
+Cameras differ in orientation, mirroring and field of view, and people stand
+at different distances. The calibration workflow measures a device once,
+persists it, and applies it only where you ask. **Raw tracker results are
+never altered.**
+
+```tsx
+import { useXRCalibration } from "@vincentt-xr/sdk/tracking";
+
+const cal = useXRCalibration({ storageKey: "myapp.calibration" });
+
+// Capture references while the person stands at their normal distance:
+cal.calibrateFace(faceInfo.landmarks); // feed-space face landmarks
+cal.calibrateBody(poseLandmarksInFeedSpace); // 33 pose landmarks, feed space
+
+// Device transform:
+cal.update({ orientation: 90, mirrored: "auto", scale: 1.1, offset: { x: 0, y: 0.02 } });
+
+// Apply where it matters:
+const nose = cal.applyToFeedPoint(faceInfo.landmarks[4]); // new point, input untouched
+const distance = cal.faceDistance(faceInfo.landmarks); // 1 at reference, >1 closer
+const bodyScale = cal.bodyScale(poseLandmarksInFeedSpace); // 1 at reference
+
+cal.reset(); // defaults; clears the persisted value
+```
+
+| Field             | Meaning                                                                                                                   |
+| ----------------- | ------------------------------------------------------------------------------------------------------------------------- |
+| `orientation`     | `0 / 90 / 180 / 270`: clockwise rotation applied to feed points.                                                          |
+| `mirrored`        | `"auto"` follows the session (only the webcam is mirrored); `true`/`false` flip only when they disagree with the session. |
+| `scale`, `offset` | Uniform scale about the feed centre, then a feed-unit nudge.                                                              |
+| `face`            | `faceWidth` (temples 234↔454) and `eyeDistance` (33↔263) in feed-width units, plus `capturedAt`.                        |
+| `body`            | `shoulderWidth` (11↔12), `hipWidth` (23↔24), `torsoLength` (mid-shoulder to mid-hip).                                   |
+
+The pure helpers (`applyCalibrationToFeedPoint`, `measureFaceCalibration`,
+`measureBodyCalibration`, `estimateFaceDistance`, `estimateBodyScale`,
+`updateCalibration`, `resetCalibration`, `loadCalibration`,
+`saveCalibration`, `clearCalibration`) work without React. Persistence uses
+`localStorage` under `vincentt.xr.calibration` by default, validates every
+field on load, and drops blobs from another `version`.
+
+## Manual validation
+
+Run `/session-health`:
+
+| Case              | Steps                         | Expect                                                                    |
+| ----------------- | ----------------------------- | ------------------------------------------------------------------------- |
+| Diagnostics panel | load, pause / resume          | face + body `ready`, inference ms / fps live; `paused` and 0 fps on pause |
+| Unavailable model | untick "mount hand tracker"   | hand row reads `unavailable`; tick again → `loading` → `ready`            |
+| Face calibration  | "calibrate face", move closer | distance rises above 1; reload keeps the reference                        |
+| Orientation       | set 90°                       | calibrated nose point rotates; raw point unchanged                        |
+| Reset             | "reset"                       | defaults, distance `–`, storage cleared                                   |

package/docs/api/face-effects.mdx

@@ -13,6 +13,12 @@ This section is split into focused pages:
   paint onto the live face mesh.
 - [Face Retouching](/api/face-retouching) for the legacy skin-retouch shader
   and its controls.
+- [Segmentation Overlay](/api/segmentation-overlay) for 2D semantic region
+  layers (lips, eyes, brows, face skin, person, background) without 3D
+  geometry, and when to prefer them over the canonical mesh.
+- [Lip Effect Textures](/guides/lip-effect-textures) for `LipEffect`: the
+  design-texture canvas contract, the generated lip UV template, mask modes,
+  and the custom material context.
 
 Use `@vincentt-xr/sdk/tracking` for trackers, `FaceInfo`, Face Actions, and the
 bare deforming `FaceMesh`. Use `@vincentt-xr/sdk/face-effects` for the modular

package/docs/api/face-mesh.mdx

@@ -25,6 +25,12 @@ Prefer `FaceMeshRenderer` for editor/runtime scene-object integrations. It owns
 face tracking internally through `FaceTracker`, reads live landmarks, deforms
 the canonical face mesh, and uses the shared FaceMesh material settings.
 
+Face mesh landmarks arrive as raw MediaPipe image coordinates: X/Y use a
+top-left normalized image plane and Z is tracker-relative depth. The renderer
+projects those points into its camera scene itself. They are not SDK
+center-origin tracker points, so do not feed them to
+`trackerPointToScreenPixels` or apply an additional mirror/origin conversion.
+
 ```tsx
 import {
   DEFAULT_FACE_MESH_RENDERER,
@@ -58,6 +64,35 @@ faceMesh.settings = {
 patch)` is the edit-time source of truth: use it whenever material, shader,
 texture, blend, or mask state changes.
 
+### Runtime custom materials
+
+Material types and shader presets are quick SDK defaults. Use
+`materialOverride` when the face needs a custom Three.js material or shader.
+It accepts a material node or a factory that receives the current camera texture,
+the geometry key, and `faceUv`: the shared live camera-UV `BufferAttribute`
+(`null` until the canonical mesh loads), also bound to every group's geometry as
+`attribute vec2 faceUv`. This prop wins over preset settings and is intentionally not
+part of `FaceMeshRendererSettings`, because materials, uniforms, and factories
+are not serializable scene data.
+
+```tsx
+<FaceMeshRenderer
+  value={faceMeshSettings}
+  materialOverride={({ cameraTexture, geometryKey }) => (
+    <shaderMaterial
+      key={geometryKey}
+      uniforms={{ cameraTexture: { value: cameraTexture } }}
+      vertexShader={vertexShader}
+      fragmentShader={fragmentShader}
+    />
+  )}
+/>
+```
+
+When rendering a serialized scene store, supply the same runtime-only override
+through `SceneObjectRenderer`'s `faceMeshMaterialOverrides`, keyed by the
+FaceMeshRenderer component ID. R3F owns material node disposal.
+
 FaceMesh `Transform` is intentionally not normal world-space `Transform3D`.
 `SceneObjectRenderer` passes an authored `Transform` into `FaceMeshRenderer`,
 and FaceMesh applies supported values in MediaPipe landmark space before camera