@vincentt-xr/sdk 2.0.0-alpha.8 → 2.0.0-alpha.9

package/docs/api/hand-rig.mdx

@@ -0,0 +1,535 @@
+---
+title: "Hand Rig"
+description: "Both tracked hands as world-space poses, a driver that animates a skinned 3D hand model (or a glove) from them, hand occlusion for rings, and hand tattoos."
+sidebarTitle: "Hand Rig"
+icon: "hand"
+---
+
+The hand tracker detects up to two hands. `HandRig` drives a skinned 3D hand model from either of
+them: the model's palm is fitted to the tracked palm (position, rotation, size) and every finger bone
+swings toward its tracked joint. Underneath, the hand model publishes both hands as **world-space
+poses** with a stable left / right identity.
+
+Import from `@vincentt-xr/sdk/tracking`.
+
+## Quick start
+
+```tsx
+import { useLoader } from "@react-three/fiber";
+import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js";
+import { HandRig } from "@vincentt-xr/sdk/tracking";
+
+const Hands = () => {
+  const gltf = useLoader(GLTFLoader, handUrl); // a model following the rig contract
+  return (
+    <>
+      <HandRig hand="left" model={gltf.scene} />
+      <HandRig hand="right" model={gltf.scene} />
+    </>
+  );
+};
+```
+
+Mount `HandRig` anywhere in the scene, not inside a tracker. It acquires the shared hand model
+itself. One loaded model can drive both hands: each rig clones it, skeleton included. Example:
+`/hand-rig`.
+
+## Hand poses (`HandPose`)
+
+`node.poses` / `node.getHandPose(hand)` on the hand model node
+(`useXRModelNode(XRModel.HAND_TRACKER)`, `/low-level`) give each hand by its **anatomical** side:
+
+| field         | meaning                                                                                                            |
+| ------------- | ------------------------------------------------------------------------------------------------------------------ |
+| `handedness`  | the person's own hand, on every source (mirrored webcam or not)                                                    |
+| `score`       | MediaPipe's handedness confidence, 0..1                                                                            |
+| `world`       | 21 joints, **centimetres**, origin at the hand centre, three.js axes: +X screen right, +Y up, +Z toward the viewer |
+| `image`       | the same 21 joints in tracker space (centre origin, ±1, Y up) — what `HandTracker` publishes as `coordinates`      |
+| `timestampMs` | `performance.now()` of the detection                                                                               |
+| `held`        | `true` while a lost hand is being held (see below)                                                                 |
+
+Joint order and names: `HAND_JOINT_NAMES` (`wrist`, `thumbCmc`, `thumbMcp`, `thumbIp`, `thumbTip`,
+`indexMcp` … `pinkyTip`) = MediaPipe landmark order; `HAND_JOINT_INDEX[name]` is the index.
+`world` is the true 3D shape of the hand (finger bend, which way the palm faces), independent of its
+distance. `image` is where it is on screen. `handPoseToScene(pose, camera)` (`/tracking`) combines the two into scene
+positions: the world shape, scaled and turned in the image plane onto the tracked image points (weak
+perspective; the hand's centre sits on the `z = 0` tracking plane).
+
+`node.hands[i]` also carries `worldCoordinates` and `handednessScore` per raw detection.
+
+### Handedness is anatomical
+
+MediaPipe labels a hand as it appears in the frame it is given. The SDK analyzes the feed as
+displayed, which is mirrored for the front webcam, so its raw labels were swapped there. The SDK now
+corrects them (`anatomicalHandedness`). `HandTracker hand="left"`, `node.handedness` and the poses all
+mean the person's left hand on the webcam, the rear camera, images and video.
+
+### Stable identity (`assignHandPoses`, `/low-level`)
+
+MediaPipe labels each hand independently, so a hand can be called the wrong side for a frame (edge-on)
+or both hands the same side. The rules, in order:
+
+1. Two hands with different, confident labels take their labels.
+2. Otherwise, while both sides were seen in the last 500 ms, the pairing that keeps each wrist nearest
+   to where that side's wrist was wins.
+3. Two hands with the same label and no history: the higher score keeps it.
+4. One hand with a low-confidence label sitting where the OTHER hand just was is that hand, mislabeled.
+
+A lost hand is held for `HAND_POSE_DEFAULT_HOLD_MS` (200 ms, marked `held`), then becomes `null`. When
+it comes back it lands on the same side, never the other rig.
+
+## `HandRig`
+
+| prop               | default     | meaning                                                                                 |
+| ------------------ | ----------- | --------------------------------------------------------------------------------------- |
+| `hand`             | —           | `"left"` / `"right"`, the person's hand to follow                                       |
+| `model`            | —           | the loaded model (e.g. `gltf.scene`); cloned per rig                                    |
+| `modelHand`        | `"right"`   | the chirality the model is authored as                                                  |
+| `smoothing`        | `0.5`       | EMA factor for the hand and finger poses, `1` = none                                    |
+| `fingerLengths`    | `"tracked"` | `"tracked"`: segments take the tracked lengths; `"model"`: keep the model's proportions |
+| `showHeld`         | `true`      | keep drawing a hand during the short hold                                               |
+| `material`         | —           | replaces every mesh material of this rig (caller-owned)                                 |
+| `onModel`          | —           | called once with the rig's clone (shadows, render order …)                              |
+| `onTrackingChange` | —           | fires once on found (`true`) and once on lost (`false`)                                 |
+| `active`           | `true`      | `false` pauses the hand model's inference                                               |
+
+**Mirroring.** A right hand seen in a mirror looks like a left hand. On the mirrored webcam, the
+person's right hand therefore has a LEFT hand's shape on screen, and a right-hand model is mirrored
+onto it (`screenChirality`). One model serves both hands on every source.
+
+**Tracking.** While a hand is lost the rig hides. On reacquire it snaps to the new pose rather than
+sliding in. A jump of more than `HAND_RIG_SNAP_PALMS` (2.5) palm lengths in one frame also snaps
+(about 22 cm on an average hand). It's measured in the model's own units (`handRigSnapDistance`), so
+models authored in metres, centimetres or millimetres behave the same. Same contract as the head fits.
+
+**Finger lengths.** The model is scaled as a whole by the palm fit. With `fingerLengths="tracked"`
+each segment then takes the tracked hand's length, ratio clamped to `HAND_RIG_LENGTH_RANGE`
+(0.75–1.35). Segments are lengthened **along the bone only**, so fingers keep the model's thickness:
+the next joint moves out, and the last segment stretches along its own axis.
+
+## Rings and other finger jewellery: `HandOccluder` + `HandAttachment`
+
+`HandOccluder` is a `HandRig` whose hand writes depth but no colour and draws first
+(`HAND_OCCLUDER_RENDER_ORDER` = −1). The camera shows the real hand, and anything behind it (the
+back half of a ring, the far side of a watch strap) fails the depth test and disappears.
+`HandAttachment` puts content on a finger bone of the enclosing rig, so it rides on the same fit:
+
+```tsx
+import { HandAttachment, HandOccluder } from "@vincentt-xr/sdk/tracking";
+
+<HandOccluder hand="right" model={handGltf.scene} inflate={-0.1}>
+  <HandAttachment joint="ringMcp" along={0.6}>
+    <primitive object={gemRing} />
+  </HandAttachment>
+  <HandAttachment joint="middlePip" along={0.5}>
+    <primitive object={midiRing} />
+  </HandAttachment>
+</HandOccluder>;
+```
+
+**`HandAttachment` props**
+
+| prop     | default | meaning                                                                                                                                                    |
+| -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `joint`  | —       | the finger bone, named by the joint it starts at: `indexMcp` = base segment, `indexPip` middle, `indexDip` last; thumb `thumbCmc` / `thumbMcp` / `thumbIp` |
+| `along`  | `0.5`   | where along that segment: 0 = at the joint, 1 = at the next                                                                                                |
+| `offset` | —       | `{ position, rotation, scale }` in the attachment frame (model units, radians, multiplier)                                                                 |
+
+**The attachment frame.** +Y runs along the finger toward the tip, +Z points out of the palm, so −Z
+is the back of the hand. Units are the model's (cm for the SDK hand). Author a ring around +Y with
+its stone at −Z and it sits on the back of the finger, whatever the rig's own bone axes are
+(`handAttachmentLocal`, `/low-level`). Attachments follow the bone, its tracked length and the
+knuckle offsets, the hand's visibility and its mirroring. On the other chirality the content is
+mirrored too; symmetric pieces like rings are unaffected. Use it inside a `HandRig` as well (e.g.
+nails on a visible skinned hand).
+
+**`HandOccluder` props**: everything `HandRig` takes except `material`, plus:
+
+- `inflate` (model units, default `0`) pushes the occluder along its normals after skinning. Use
+  it negative to shrink.
+- `showDebugMaterial`: draw the occluder as translucent grey.
+- `opacity` (0.5) for that debug view.
+
+**Sizing rule.** A ring must stay **outside** the occluder, or the occluder hides its front as
+well. Real fingers vary and the SDK hand's low-poly fingers are wider than they are tall. The
+example places base rings past the webbing (`along={0.6}`), uses a 1.0 cm inner radius (0.85 cm for
+a midi ring), and shrinks the occluder by 1 mm (`inflate={-0.1}`). If a ring's front disappears,
+lower `inflate`. If its back shows through the finger, raise it.
+
+**Knuckles.** The palm fit is rigid, so a hand whose knuckles are spread wider or narrower than the
+model's would start its fingers a few millimetres off, enough to put a ring beside the finger. With
+`fingerLengths="tracked"` (the default) each digit's first bone also moves to its tracked knuckle
+(`HandRigSolution.knuckles`, clamped to `HAND_RIG_KNUCKLE_SHIFT`, 0.2 palm lengths), eased like the
+lengths.
+
+**Render order.**
+
+- The occluder draws at −1, only on the hand model's own meshes (flagged
+  `userData[HAND_RIG_MODEL_MESH]`).
+- Attachments keep their own order (0) and depth-test against the occluder.
+- The occluder ends 1.2 cm past the wrist, like the hand model, so a watch on the wrist bone's
+  neighbour isn't occluded below that point.
+
+Example: `/hand-rig?rings=1`, with three rings from `assets-src/accessories/rings-v1.blend` (gold band,
+gem ring, rose-gold midi). It has:
+
+- a ring picker per finger
+- occluder on/off
+- the debug view
+- an inflate slider
+- a found/lost log
+- a hide-able panel
+
+## Watches and bracelets (experimental): `WristAttachment`, the forearm bone and the arm mask
+
+Experimental: the props and the behaviour may change in a later alpha. Known gaps are tracked in
+vincentt-xr/sdk#319.
+
+`WristAttachment` puts content on the wrist of the enclosing `HandRig` / `HandOccluder`. A function
+child gets the wrist's size, so a strap or a bangle fits the tracked hand. `forearm` on the occluder
+adds an arm behind it, so the back of the strap disappears behind the real arm. `armMask` makes the
+wristwear follow the real arm:
+
+```tsx
+import { HandOccluder, WristAttachment } from "@vincentt-xr/sdk/tracking";
+
+<HandOccluder hand="right" model={handGltf.scene} inflate={-0.1} forearm armMask="skin">
+  <WristAttachment along={1.6}>
+    {(wrist) => <Watch rx={wrist.width / 2 + 0.25} rz={wrist.depth / 2 + 0.25} />}
+  </WristAttachment>
+</HandOccluder>;
+```
+
+**`WristAttachment` props**
+
+| prop       | default | meaning                                                                                    |
+| ---------- | ------- | ------------------------------------------------------------------------------------------ |
+| `along`    | `1`     | how far up the forearm from the wrist joint, in model units (cm for the SDK hand)          |
+| `offset`   | —       | `{ position, rotation, scale }` in the wrist frame (model units, radians, multiplier)      |
+| `children` | —       | content in the wrist frame, or `(wrist: WristSize) => ReactNode` to size it from the wrist |
+
+**The forearm bone.** Wristwear rides on the rig's `forearm` bone: a child of `wrist` that starts at
+the tracked wrist point and runs up the arm. The SDK hand has it (`assets-src/hands/build_forearm.py`
+adds it in Blender). For other models, `HandRig` makes one in code (`ensureForearmBone`). Author
+wristwear in Blender by parenting it to `forearm`.
+
+**The wrist frame.** +Y runs toward the fingers (so −Y goes up the arm), +Z points out of the palm
+(−Z is the back of the wrist, where a watch face goes), +X runs across the wrist. Its origin is the
+wrist joint. It is built from the rig's rest pose, so it is the same whatever the model's own bone
+axes are (`wristFrameLocal`, `/low-level`).
+
+**The wrist's size** (`WristSize`, from `wristSizeOf`, measured on the rig at rest, model units):
+
+- `palmWidth`: index knuckle to pinky knuckle.
+- `palmLength`: wrist to middle knuckle.
+- `width`: `WRIST_WIDTH_OF_PALM` (0.9) × `palmWidth`.
+- `depth`, back to palm: `WRIST_DEPTH_OF_WIDTH` (0.7) × `width`.
+
+Make a strap's inner ellipse a little larger than `width / 2` by `depth / 2`. The example adds
+0.25 cm, so the strap stays outside the arm occluder and its front never hides.
+
+**How the forearm is aimed.** Before every render the rig aims the forearm bone once, for everything
+on that wrist (`HandRigWrist`):
+
+1. **Steadied.** When a hand turns edge-on, one camera can't tell which side of the palm faces it,
+   and the tracked hand can flip for a few frames. A turn of more than 35° is only taken once it
+   lasts 200 ms and 4 hand detections (`WristRotationFilter`), so flips don't spin the watch.
+2. **Aligned with the arm** (`armMask` on `HandRig` / `HandOccluder`). From the wrist it scans
+   across the arm in a segmentation mask: first an arc around the wrist, then cross-sections 2–8 cm
+   up the arm, then a line through their middles. That gives the arm's direction (bends up to
+   95°), its middle and its width. The forearm then:
+   - lines up with the arm in full once the mask is sure;
+   - moves onto the arm's middle (at most 2 cm);
+   - keeps its way while the hand points at the camera;
+   - eases back to the hand's own axis when the arm is lost.
+3. **Fitted to the width,** gently: a scale across the arm of 0.85 to 1.2 on what rides on the bone.
+
+| `armMask`    | the arm is                                                                                                                                                                         |
+| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `"skin"`     | the multiclass segmenter's bare skin (`body_skin`), apart from the face and the clothes                                                                                            |
+| `"portrait"` | the person mask: cheaper, but has no edges where the arm is in front of the body                                                                                                   |
+| `{ … }`      | options: `kind`, `turn`, `fitWidth`, the easing (`smoothing`, `holdMs`, `fadeMs`, `maxSkewMs`, `maxShift`) and the measure (`along`, `widthAt`, `maxWidth`, `minWidth`, `maxTurn`) |
+
+Masks arrive later than hand poses, so each mask is measured against the hand pose from the same
+moment (`ArmTracker`; both segmenters stamp their masks with `timestampMs`). `"skin"` runs the
+multiclass segmenter, which shares `XRModel.SEMANTIC_SEGMENTATION` with the hair mask: the first
+acquirer in a session picks the preset.
+
+**The forearm occluder.** `forearm` on `HandOccluder` (`true`, or `{ length, scale }`) adds a
+depth-only `ForearmOccluder` on the forearm bone: a tapered elliptic tube from the wrist up the arm.
+
+- `length` is in palm lengths (`FOREARM_OCCLUDER_LENGTH`, 2.5).
+- It widens by `FOREARM_OCCLUDER_FLARE` (1.25) toward the elbow; `scale` makes it wider (> 1) or
+  narrower.
+- It reaches `FOREARM_OCCLUDER_OVERLAP` (0.15 palm lengths) into the hand, so there is no gap at the
+  wrist.
+- It shares the occluder's debug view.
+
+Mount it on its own inside a `HandRig` (`<ForearmOccluder length scale showDebugMaterial opacity />`),
+or build the mesh yourself with `createForearmGeometry(wrist, { length, scale })`.
+
+**Known gaps** (vincentt-xr/sdk#319):
+
+- The hand tracker misses some poses: a fist pressed on the face, a hand bent toward the camera.
+- With the hand pointing at the camera, the bone can't yet lie along an arm seen side-on.
+- Width readings differ between the two masks.
+
+Example: `/hand-rig?rings=1&wrist=watch&armMask=skin` (also `wrist=bracelet` / `both`,
+`armMask=portrait`). The watch and the bead bracelet are built in code, and the watch hands show the
+real time. `debug=1` draws the mask, the hand's axis, the cross-sections across the arm and the arm's
+axis; `forearm=0` turns the arm occluder off to compare.
+
+## Gloves: a model that wears `HandRig`
+
+A glove is a hand model. Build it on the same rig and `HandRig` wears it, fitted to the person's
+palm and finger lengths. There is no glove component: the rig contract below is the whole
+interface.
+
+```tsx
+import { useLoader } from "@react-three/fiber";
+import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js";
+import { HandRig } from "@vincentt-xr/sdk/tracking";
+
+const Gloves = () => {
+  const glove = useLoader(GLTFLoader, gloveUrl); // e.g. the example's glove-v1.glb
+  return (
+    <>
+      <HandRig hand="left" model={glove.scene} />
+      <HandRig hand="right" model={glove.scene} />
+    </>
+  );
+};
+```
+
+The glove's materials are lit (`MeshPhysicalMaterial` from the GLB), so the scene needs lights or an
+environment.
+
+**Recipe for your own glove** (or a gauntlet, a mitten, a robot hand):
+
+1. Start from a rigged hand's skin (`assets-src/hands/hand-v1.blend`). Keep its armature, weights,
+   joint names and rest positions.
+2. Make it slightly thicker than the hand: push the surface out along its normals (the example uses
+   2.8 mm). The real fingers then stay hidden under it in every pose. The thickness scales with the
+   hand, because the whole rig does.
+3. Weight anything past the wrist (a cuff) to `wrist` only. The tracker has no forearm landmark, so
+   a cuff follows the palm's direction: keep it short.
+4. Add UVs and textures as for any model, then export a GLB with skinning, deform bones only and
+   +Y up. No animation is needed.
+
+**The example glove** (`glove-v1.glb`, ~195 KB) is a winter knit glove: a deep red `knit` body and
+a cream ribbed `rib` cuff, with knit textures generated procedurally in Blender. It sits 2.8 mm
+outside `hand-v1` everywhere, is quantised (`KHR_mesh_quantization`, which a plain `GLTFLoader`
+reads) and uses `KHR_materials_sheen` for the soft fuzz. To recolour it, set `material.color` on the
+material named `knit` or `rib`; clone the material first to change one hand only. It's built by
+`assets-src/hands/build_glove.py` from `hand-v1.blend`, so the hand's CC BY 4.0 credit applies.
+Example: `/hand-rig?glove=1` (`&gloveHands=left` gloves one hand only).
+
+## Tattoos: `HandTattoo`
+
+`HandTattoo` puts art on the skin of the enclosing `HandRig` / `HandOccluder`, on the back of the hand
+or the palm. The art is projected onto the model's surface in its **rest pose**, so it bends,
+stretches and turns with the fingers and the palm. Inside a `HandOccluder` it inks the real hand in
+the camera image:
+
+```tsx
+import { HandOccluder, HandTattoo } from "@vincentt-xr/sdk/tracking";
+
+<HandOccluder hand="right" model={handGltf.scene} inflate={-0.1}>
+  <HandTattoo src={roseUrl} placement="back" size={5} opacity={0.85} />
+</HandOccluder>;
+```
+
+- **Blending.** `blend="multiply"` (the default) multiplies the camera image by the art, so skin
+  shading and texture show through the ink, and white or transparent parts leave the skin as it
+  is. `blend="normal"` lays the art's own colours over the skin, for stickers or light ink on dark
+  skin.
+- **Occlusion.** The `HandOccluder` draws the hand's depth first, so the art disappears where the
+  hand covers itself (curled fingers over a palm tattoo). Keep the occluder's `inflate` at or below
+  0: the art is drawn on the model's own surface, and a grown occluder covers it.
+- **On a visible model** (inside a `HandRig` with a skin or a glove) the art prints on that model.
+- **No UVs needed.** The placement is measured from the rig's joints, so any model on the rig
+  contract works, `hand-v1` included (it ships without UVs).
+
+**`HandTattoo` props**
+
+| prop          | default                    | meaning                                                                                                              |
+| ------------- | -------------------------- | -------------------------------------------------------------------------------------------------------------------- |
+| `src`         | —                          | an image URL (loaded as sRGB) or a `THREE.Texture` you own (set `colorSpace = SRGBColorSpace`)                       |
+| `placement`   | `"back"`                   | `"back"` of the hand or `"palm"`                                                                                     |
+| `size`        | 0.55 palm lengths (4.8 cm) | the art's width in model units (cm for the SDK hand). Its height follows the image's aspect                          |
+| `offset`      | `[0, 0]`                   | `[towardThumb, towardFingers]` from the centre of that side, model units                                             |
+| `rotation`    | `0`                        | radians, counter-clockwise as seen looking at that side with the fingers up                                          |
+| `opacity`     | `0.9`                      | 0..1                                                                                                                 |
+| `blend`       | `"multiply"`               | `"multiply"` (ink) or `"normal"` (the art's own colours)                                                             |
+| `renderOrder` | `1`                        | `HAND_TATTOO_RENDER_ORDER`: after the hand model (transparent pass), so it blends over the camera image or the model |
+
+**Where it lands.** The centre of each side is half-way from the wrist to the knuckles. The art is
+projected straight onto that side and fades out where the surface turns more than ~65° away (the
+edges of the hand), so it never wraps onto the other side. For a piece that wraps round the hand,
+mount two `HandTattoo`s. Several tattoos per hand are fine: each one is one more draw of the hand
+mesh.
+
+**Mirroring.** On the other chirality (a right-hand model drawn as a left hand) the placement mirrors
+with the hand: `offset[0]` stays toward the thumb. The art itself is flipped back, so it never reads
+mirrored, and `rotation` still turns it counter-clockwise on screen.
+
+**Art.** Dark line art on a transparent PNG works best with `multiply` (a white background works
+too), at 512–1024 px. The example draws placeholder art in code (`tattooArt.ts`: a nautical star and
+a swallow).
+
+**Why a component, not a UV texture.** The SDK hand ships without UVs, and every model would unwrap
+differently. Projecting from the rig's rest joints gives the same placement on any compatible model,
+in physical units, and keeps the art where an artist expects it: on the back or the palm, upright
+with the fingers up.
+
+**Pure pieces** (`/low-level`): `handTattooFrame(binding, placement)` (the projector's frame in rig
+space), `handGeometryToRoot(binding, mesh)` (geometry to rest space, quantised meshes included),
+`handTattooMatrix(frame, geometryToRoot, { size, offset })` (the shader's matrix), `handTattooUv`
+(the shader's math in JS), `createHandTattooMaterial`, `setHandTattooBlend`,
+`HAND_TATTOO_FACING_FADE`. `HAND_TATTOO_DEFAULT_SIZE` and `HAND_TATTOO_RENDER_ORDER` are in
+`/tracking`.
+
+Example: `/hand-rig?tattoo=1`. It has back / palm placement, two placeholder arts, opacity, size,
+rotation, offsets and blend. With `glove=1` the art prints on the glove.
+
+## The rig contract (for your own hand model)
+
+A compatible model has a node named after each joint it rotates, placed **at** that joint:
+
+```
+wrist ─┬ thumbCmc → thumbMcp → thumbIp → (thumbTip)
+       ├ indexMcp → indexPip → indexDip → (indexTip)
+       ├ middleMcp → middlePip → middleDip → (middleTip)
+       ├ ringMcp → ringPip → ringDip → (ringTip)
+       ├ pinkyMcp → pinkyPip → pinkyDip → (pinkyTip)
+       └ (forearm)
+```
+
+- `HAND_RIG_REQUIRED_JOINTS` (the 16 non-tip nodes) must exist. Missing ones throw with their names.
+  The `…Tip` nodes are optional; without them the tip is extrapolated from the last bone.
+- **Any rest pose, units, bone roll and authoring axes work.** The driver measures every rotation
+  from the model's own rest pose, so a relaxed, curled rest hand is fine.
+- One chirality per model (`modelHand`). Draw the other by mirroring; don't ship two meshes.
+- `forearm` is optional (experimental): a bone from the wrist joint up the arm. The rig aims it
+  (steadied, and along the arm in the mask with `armMask`), and wristwear rides on it. Without it,
+  `HandRig` makes one in code. Weight forearm geometry (a sleeve, a glove's cuff) to it so it follows
+  the arm rather than the palm. The example model ends at the wrist with an alpha fade and carries
+  the bone only.
+- Keep the deforming bones' children on their local axis (Blender's +Y does this) so the last
+  segment can stretch along it.
+
+**How it solves**, per frame (`solveHandRig`, pure):
+
+1. `handPoseToScene` maps the tracked hand into the scene.
+2. If the hand's on-screen chirality differs from `modelHand`, the live points are mirrored in X and
+   the mirror goes on the root.
+3. Root: the Umeyama similarity from the rig's rest palm (`HAND_RIG_PALM_JOINTS`: wrist, thumb CMC,
+   four MCPs) onto the live palm.
+4. Fingers: each bone (`HAND_RIG_BONES`, parents first) follows its parent rigidly, then swings by the
+   shortest arc so it points at its live child joint.
+5. Lengths: tracked segment length over rest length per bone (`solution.lengths`).
+
+`bindHandRig`, `solveHandRig`, `applyHandRigSolution`, `applyHandRigLengths`, `handRigRootMatrix`,
+`handRigSnapDistance`, `screenChirality`, the `HAND_RIG_*` tables and the identity pieces
+(`assignHandPoses`, `anatomicalHandedness`, `handWorldLandmarkToJoint`) are exported from
+**`@vincentt-xr/sdk/low-level`**, for apps that animate a rig outside React or record poses.
+`/tracking` keeps `HandRig`, the pose contract (`HAND_JOINT_NAMES`, `HAND_JOINT_INDEX`, `HandPose`,
+`HandPoses`, `HandJointName`, `HAND_POSE_DEFAULT_HOLD_MS`), `handPoseToScene` (the blessed way to
+place a pose in the scene, e.g. for a joint overlay) and `HAND_RIG_REQUIRED_JOINTS`.
+
+## The example model
+
+`assets-src/hands/hand-v1.blend` (README beside it): "Hand (low poly)" by scribbletoad, CC BY 4.0. It
+was rigged for the SDK by running MediaPipe on renders of the mesh to place the joints, with knuckle
+edge rings, 21 joint nodes, a 17 cm right hand and a wrist alpha fade. The 1 k-triangle cage is
+subdivided once on export (5.6 k triangles, ~230 KB). The GLB also carries 11 gesture poses as
+animations, used by [Hand Instructions](/api/hand-instructions).
+
+GLTFLoader turns depth writes off for alpha-blended materials. The example turns them back on for the
+hand so curled fingers don't show through the palm, because only the wrist edge is see-through. A
+replacement material keeps the wrist fade with `vertexColors: true, transparent: true`.
+
+## Manual visual test plan
+
+Open `/hand-rig` with the tracked joints overlay on.
+
+1. **Webcam (mirrored)**: raise your right hand. The panel says `right` and the rig is a mirrored
+   right-hand model. Raise the left: `left`. Both together: two rigs, thumbs on the correct sides.
+2. **Unmirrored source** (a video preset or a local clip): the person's right hand reads `right`
+   (on screen-left when they face the camera).
+3. **Rotation**: palm to camera, back to camera, palm sideways, wrist bent. The rig turns with the
+   hand, the fingertips sit on the tip dots, and the hand ends at the wrist without a stub.
+4. **Fingers**: fist, point, peace sign, OK sign, spread. Each finger bends at its knuckles, nothing
+   tears.
+5. **Lengths**: switch fingers to "model proportions" and back. Tracked lengths reach the tip dots;
+   finger thickness does not change.
+6. **Loss and reacquisition**: hide one hand. Its rig vanishes after a moment, the panel logs `lost`.
+   Bring it back: the same rig returns (`found`), snapped, without sliding in from elsewhere. Cross
+   your hands and uncross: the rigs stay with their hands.
+7. **Partial occlusion**: cover some fingers with the other hand or an object. MediaPipe still
+   estimates all 21 joints, so the covered fingers follow its guess (expect some wobble). The rig
+   hides only when the hand itself is lost.
+8. **Materials**: switch the look to gloves and ghost. Each rig gets its own material and the wrist
+   still fades.
+
+### Hand occluder (`/hand-rig?rings=1`)
+
+1. **Palm to camera**: each ring shows only its front arc. The back arc and the gem (on the back of
+   the hand) are hidden behind the finger.
+2. **Back of the hand to camera**: the gem faces you, and the band's palm side is hidden.
+3. **Turn the hand edge-on**: the near half of each ring shows, the far half hides, with no
+   flicker at the finger's silhouette.
+4. **Occluder off**: the whole ring draws over the finger, which is the failure case.
+5. **Debug**: the grey occluder hand sits on the real one and each ring wraps its occluder finger.
+6. **Inflate**: lower it until a ring's front stays whole at every angle. Raise it until the back
+   no longer shows through.
+7. **Placement**: rings sit on the right fingers past the webbing. Bend the fingers and the rings
+   ride along. Spread and close the fingers and each ring stays centred on its finger.
+8. **Both hands, both cameras**: the left and right hands each wear their rings. On the mirrored
+   webcam the gem still sits on the back of the hand.
+
+### Wristwear, experimental (`/hand-rig?rings=1&wrist=both&armMask=skin`)
+
+1. **Back of the hand to camera**: the watch case sits on the strap on the back of the wrist, 12
+   toward the thumb and 3 toward the fingers, showing the real time. The strap wraps the wrist to
+   its edges. The bracelet sits above the watch, up the arm.
+2. **Palm to camera**: the watch case is hidden behind the arm and the strap's palm side shows. The
+   bracelet's far beads hide and the near ones show.
+3. **Turn the wrist edge-on and back** a few times: the watch does not flip or spin round the arm.
+   A real turn is taken after a moment.
+4. **Bend the wrist** (a fist hanging from a level arm), with `debug=1`: the yellow arm axis and the
+   grey forearm run along the real forearm, not along the hand, and the watch stays across the arm.
+5. **Arm in front of the shirt**: with `armMask=skin` the arm is still found. With
+   `armMask=portrait` it isn't, and the forearm follows the hand.
+6. **Forearm occluder off** (`forearm=0`): the whole strap and every bead draw over the arm, which is
+   the failure case.
+7. **Both hands, both cameras**: each hand wears its watch and bracelet. On the mirrored webcam the
+   face still sits on the back of the wrist.
+
+### Glove (`/hand-rig?glove=1`)
+
+1. **Cover**: open hand, palm and back to the camera. No skin shows at the fingertips, the finger
+   sides or the knuckles; the cuff ends just past the wrist.
+2. **Fit**: spread, fist, point, OK sign. Each glove finger bends at the real knuckles and ends at
+   the real fingertip (tracked lengths). Switch fingers to "model proportions": the glove keeps its
+   own lengths.
+3. **Knit**: the stitch columns run along each finger and up the palm; the cuff is ribbed and
+   cream. Turn the hand: the knit catches the light softly (sheen).
+4. **Both hands**: each hand wears a glove (the left one mirrored). `gloveHands=left` gloves only
+   the left hand; the right shows the look material (or the rings when they are on).
+5. **Loss**: hide a gloved hand and bring it back. The glove returns on the same hand, snapped.
+
+### Tattoo (`/hand-rig?tattoo=1`)
+
+1. **Back of the hand to the camera**: the star sits between the wrist and the knuckles, upright
+   with the fingers up. The skin's shading and creases show through the ink.
+2. **Bend and spread**: make a fist, spread the fingers, bend the wrist. The art stays on the same
+   patch of skin and stretches with it, without sliding.
+3. **Turn the hand**: palm to the camera hides a back tattoo; edge-on, it fades out at the side of
+   the hand rather than smearing.
+4. **Palm** (`tattooAt=palm`): the art is on the palm. Curl the fingers over it: they cover it.
+5. **Mirroring** (`tattooArt=swallow`): show both hands, backs to the camera. The swallow faces the
+   same way on both and never reads mirrored. `tattooX` moves it toward the thumb on both hands.
+6. **Controls**: opacity, size, rotation (counter-clockwise), offsets and blend each change the art
+   live. `blend=normal` shows the art's own colours (the swallow's cream belly appears).
+7. **With the glove** (`glove=1&tattoo=1`): the art prints on the knit.