@rig-c/pose 0.26.0 → 0.28.0

package/README.md

@@ -107,7 +107,7 @@ A retarget of the cleaned dancer, with `--flip 90 --short 0.35 --bends clamp --l
 rig-pose retarget --rig out/rig --map map.json --pose dance.pose.json --out motion.json --name dance
 ```
 
-For every frame of the pose and every bone the map names, the segment that bone carries (an upper arm: shoulder → elbow) points somewhere in the picture. The bone is turned so its segment points the same way: the picture angle, flipped into rig-c's y-up world, minus the bone's angle at the rig's setup pose (read through rig-c's own poser), less the turn of the mapped bone above it — because a `rotate` key in rig-c adds to the setup rotation and a child turns with its parent. The travel of the pelvis midpoint, scaled by a length measured on both the rig and the pose, moves the hip. When a head chain carries the eye line (`r_eye → l_eye`), the eyes' vertical travel against `neck` in the reference, scaled into the rig's units, keys a vertical `translate` nod channel on the head bone ([#111](https://github.com/rig-c/pose/issues/111)), leaving its `rotate` (roll) track unchanged. The result is one animation in rig-c's motion format (`rigc-motion/1`): a `rotate` track per mapped bone, a `translate` track on the hip (and on the head when the reference nods), one key per pose frame — and, when the map lists alternate drawings, an `attachment` track per slot they switch (and a `translate` track per bone a set offsets).
+For every frame of the pose and every bone the map names, the segment that bone carries (an upper arm: shoulder → elbow) points somewhere in the picture. The bone is turned so its segment points the same way: the picture angle, flipped into rig-c's y-up world, minus the bone's angle at the rig's setup pose (read through rig-c's own poser), less the turn of the mapped bone above it — because a `rotate` key in rig-c adds to the setup rotation and a child turns with its parent. The travel of the pelvis midpoint, scaled by a length measured on both the rig and the pose, moves the hip. When a head chain carries the eye line (`r_eye → l_eye`), the eyes' vertical travel against `neck` in the reference, scaled into the rig's units, keys a vertical `translate` nod channel on the head bone ([#111](https://github.com/rig-c/pose/issues/111)) clamped within `±2%` of the setup neck length (`head − neck`, [#181](https://github.com/rig-c/pose/issues/181)), while any vertical residual removed by that clamp (`rawNod − dy`) is coupled in phase into `face.pitch`'s dial (`+2.8°/unit`, mapped through `face.pitch.range`) and the head bone's `rotate` track (`+1.6°/unit`, [#190](https://github.com/rig-c/pose/issues/190); gains calibrated against the hand-polished reference `p1`), adding `0°` on frames where the clamp does not act. The result is one animation in rig-c's motion format (`rigc-motion/1`): a `rotate` track per mapped bone, a `translate` track on the hip (and on the head when the reference nods), one key per pose frame — and, when the map lists alternate drawings, an `attachment` track per slot they switch (and a `translate` track per bone a set offsets).
 
 Nothing is written until every check passes:
 
@@ -315,7 +315,7 @@ Which bones carry which joints. You write it for your rig:
   ⚠️ **Seen from the front, an elbow's side is not a joint limit.** An elbow flexes toward the camera, and its picture falls on either side of the upper arm depending on how the arm is turned. On a real 13 s front-facing dance (313 frames, RTMPose), the right elbow sat on the "wrong" side of the rig's rest on 165 frames (median 23°), and the left on 200 (median 29°, up to 179° with the hands behind the head). Every one was a real elbow bend. The knees strayed at most 8.8° and 14.7°, which is pose noise around a straight leg. So for front-facing material, arms go `"free"`. Legs on `"rest"` with `--bends clamp` hold only while the knees stay near straight. On a second dance with knee bends (Toca Toca, rig-c/pose#27), that setting straightened the left knee for most of the dance: `leg_l1` was 48.3° off at p90, against 0.0–0.3° with free legs. Leave legs `"free"` unless the dance keeps its knees straight.
 - `mid_hip` and `mid_shoulder` are the midpoints of the two hips and the two shoulders.
 - `translate.scale` is one length measured on both: two points on the rig, two points in the pose (its median over the frames where both are good). Pick a long segment that does not foreshorten much.
-- `face` (optional, [#138](https://github.com/rig-c/pose/issues/138), [#164](https://github.com/rig-c/pose/issues/164), [#173](https://github.com/rig-c/pose/issues/173)): `{ "yaw"?: { "bone": "yaw", "range": [-15, 15], "sets"?: [{ "set": "T1", "above": 45 }] }, "pitch"?: { "bone": "pitch", "range": [-10, 15] }, "sets"?: [{ "set": "T1", "above": 45 }] }` (`min ≤ 0 ≤ max`, `min < max`). Reads per-frame yaw and pitch from `nose` against `r_eye + l_eye` (and `r_ear + l_ear` when present), maps 1:1 inside `0.8 × range` (`[-12, 12]°` on `[-15, 15]°`) and compresses only the overshoot above the knee into the remaining 20% of the range so a single outlier frame does not crush the rest of the clip, and keys the named dial bone's `rotate` track (`FACE`). When `sets` (`[{ "set": "<name>", "above": <deg> }]`, `0 < above ≤ 90`) is declared on `face.yaw.sets` or `face.sets`, each set enters on frames where `|yaw| ≥ above` (or the highest `above` satisfied when multiple sets are declared) and leaves only when `|yaw| < above − 10°` (`10°` hysteresis), emitting stepped slot `attachment` keys for the set's `slots` and `hides` (resolved from `alternates` in the map or `sets.json` (`rig-parts-sets/1`) beside `rig.json`; an unknown set is refused by name as `MAP_SET_EXISTS`).
+- `face` (optional, [#138](https://github.com/rig-c/pose/issues/138), [#164](https://github.com/rig-c/pose/issues/164), [#173](https://github.com/rig-c/pose/issues/173), [#189](https://github.com/rig-c/pose/issues/189), [#190](https://github.com/rig-c/pose/issues/190)): `{ "yaw"?: { "bone": "yaw", "range": [-15, 15], "sets"?: [{ "set": "T1", "above": 45 }] }, "pitch"?: { "bone": "pitch", "range": [-10, 15] }, "sets"?: [{ "set": "T1", "above": 45 }] }` (`min ≤ 0 ≤ max`, `min < max`). Reads per-frame yaw and pitch from `nose` against `r_eye + l_eye` (and `r_ear + l_ear` when present — where the eye span is foreshortened below `0.85 × median(de)` and the ear-chord offset `xea = (mid_ears − mid_eyes) · u / da` exceeds both `1.8 × |xne|` and `0.15`, yaw blends toward the ear chord `atan(2 × xea)` with weight `w = clamp((0.85 × median(de) − de) / ((0.85 − 0.80) × median(de)), 0, 1)` so a side gaze whose 2D nose sits near the eye midpoint reaches `atan(2 × xea)` below `0.80 × median(de)` and exits without a step), couples any vertical residual removed by the head nod channel's `±2%` neck-length clamp (`+2.8°/unit`, `0°` where un-clamped) into `pitch` in phase ([#190](https://github.com/rig-c/pose/issues/190)), maps 1:1 inside `0.8 × range` (`[-12, 12]°` on `[-15, 15]°`) and compresses only the overshoot above the knee into the remaining 20% of the range so a single outlier frame does not crush the rest of the clip, and keys the named dial bone's `rotate` track (`FACE`). When `sets` (`[{ "set": "<name>", "above": <deg> }]`, `0 < above ≤ 90`) is declared on `face.yaw.sets` or `face.sets`, each set enters on frames where `|yaw| ≥ above` (or the highest `above` satisfied when multiple sets are declared) and leaves only when `|yaw| < above − 10°` (`10°` hysteresis), emitting stepped slot `attachment` keys for the set's `slots` and `hides` (resolved from `alternates` in the map or `sets.json` (`rig-parts-sets/1`) beside `rig.json`; an unknown set is refused by name as `MAP_SET_EXISTS`).
 - `body` (optional, [#173](https://github.com/rig-c/pose/issues/173)): `{ "yaw"?: { "bone": "body_yaw", "range": [-30, 30], "sets"?: [{ "set": "T8", "above": 45 }] }, "sets"?: [{ "set": "T8", "above": 45 }] }` (`min ≤ 0 ≤ max`, `min < max`). Maps the per-frame body yaw read from the camera-normalised shoulder and hip width (`BODY`) into `range` with the same `0.8 × range` knee mapping as `face`, keying the named dial bone's `rotate` track, and switches any declared `sets` (`body.sets` or `body.yaw.sets`) with the same `above` / `above − 10°` hysteresis (overriding `face` sets and arm `alternates` on slots they share while active). Without `body` in the map, `BODY` and `--report`'s `body` still report the reading while leaving every track in `motion.json` unchanged.
 
 ### A joint's range: `range`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@rig-c/pose",
-  "version": "0.26.0",
+  "version": "0.28.0",
   "description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
   "type": "module",
   "bin": {

package/src/face.ts

@@ -123,6 +123,37 @@ function corrOf(a: readonly number[], b: readonly number[]): number {
   return den > 1e-9 ? r3(num / den) : 1;
 }
 
+/**
+ * Eye-span foreshortening entry threshold `de < EAR_CHORD_K_DE * median(de)` (`rig-c/pose#189`):
+ * derived from $\cos(31.79^\circ) = 0.85$ ($\approx 32^\circ$ head turn), the foreshortening angle
+ * beyond which the near-planar 2D nose point begins to sit trapped between the foreshortened eyes.
+ */
+export const EAR_CHORD_K_DE = 0.85;
+
+/**
+ * Eye-span foreshortening full-weight threshold `de <= EAR_CHORD_K_LO * median(de)` (`rig-c/pose#189`):
+ * derived from $\cos(36.87^\circ) = 0.80$ ($4/5$, $\approx 37^\circ$ head turn), where the foreshortening
+ * blend weight `w = clamp((EAR_CHORD_K_DE * de0 - de) / ((EAR_CHORD_K_DE - EAR_CHORD_K_LO) * de0), 0, 1)`
+ * reaches `1.0` so the exit from a foreshortened side gaze ramps smoothly instead of snapping at `EAR_CHORD_K_DE`.
+ */
+export const EAR_CHORD_K_LO = 0.8;
+
+/**
+ * Ear-chord to nose-cue disagreement ratio `|xea| > EAR_CHORD_K_RATIO * |xne|` (`rig-c/pose#189`):
+ * derived from the 95th-percentile of `|xea| / |xne|` on confident frames (`|de - median(de)| <= 15%`)
+ * above `EAR_CHORD_K_MIN`, measured at `p95 = 1.7352` on Dancer (`37` frames) and `p95 = 1.1937`
+ * (`p100 = 1.2867`) on Toca (`58` frames) -> `1.8`.
+ */
+export const EAR_CHORD_K_RATIO = 1.8;
+
+/**
+ * Minimum ear-to-eye-midpoint offset `|xea| > EAR_CHORD_K_MIN` (`rig-c/pose#189`):
+ * derived from the 100th-percentile (maximum) detector ear-placement noise `|xea - xne|` on
+ * confident frames (`|de - median(de)| <= 15%`), measured at `p99 = 0.0949, p100 = 0.1094` on Dancer
+ * (`231` frames) and `p99 = 0.1178, p100 = 0.1450` on Toca (`373` frames) -> `0.15` (`16.70°`).
+ */
+export const EAR_CHORD_K_MIN = 0.15;
+
 /**
  * Read per-frame yaw and pitch (degrees) from `nose`, `r_eye`, `l_eye`, and
  * (when both are present) `r_ear` and `l_ear`.
@@ -140,8 +171,10 @@ export function readFaceAngles(
 ): { ears: boolean; yaw: number[]; pitch: number[] } {
   const n = nose.length;
   const ears = rEar !== null && lEar !== null && rEar.length === n && lEar.length === n;
+  const des: number[] = [];
   const xne: number[] = [];
   const xna: number[] = [];
+  const xea: number[] = [];
   const yne: number[] = [];
   const yna: number[] = [];
 
@@ -153,14 +186,17 @@ export function readFaceAngles(
     const ey = El[1] - Er[1];
     const de = Math.hypot(ex, ey);
     if (!(de > 1e-9)) {
+      des.push(des[i - 1] ?? 0);
       xne.push(xne[i - 1] ?? 0);
       yne.push(yne[i - 1] ?? 0);
       if (ears) {
         xna.push(xna[i - 1] ?? 0);
+        xea.push(xea[i - 1] ?? 0);
         yna.push(yna[i - 1] ?? 0);
       }
       continue;
     }
+    des.push(de);
     const ux = ex / de;
     const uy = ey / de;
     const vx = -uy;
@@ -176,18 +212,31 @@ export function readFaceAngles(
       const may = 0.5 * (Ar[1] + Al[1]);
       const da = Math.max(de, (Al[0] - Ar[0]) * ux + (Al[1] - Ar[1]) * uy);
       xna.push(((max - N[0]) * ux + (may - N[1]) * uy) / da);
+      xea.push(((max - mex) * ux + (may - mey) * uy) / da);
       yna.push(((N[0] - max) * vx + (N[1] - may) * vy) / de);
     }
   }
 
+  const de0 = n > 0 ? median(des) : 0;
   const yne0 = median(yne);
   const yna0 = ears ? median(yna) : 0;
   const yaw: number[] = [];
   const pitch: number[] = [];
   for (let i = 0; i < n; i++) {
-    const ty = ears ? (2 * xne[i]! + xna[i]!) / 2 : 2 * xne[i]!;
+    const earChord =
+      ears &&
+      de0 > 1e-9 &&
+      des[i]! < EAR_CHORD_K_DE * de0 &&
+      Math.abs(xea[i]!) > EAR_CHORD_K_RATIO * Math.abs(xne[i]!) &&
+      Math.abs(xea[i]!) > EAR_CHORD_K_MIN;
+    const w = earChord
+      ? EAR_CHORD_K_DE > EAR_CHORD_K_LO
+        ? Math.max(0, Math.min(1, (EAR_CHORD_K_DE * de0 - des[i]!) / ((EAR_CHORD_K_DE - EAR_CHORD_K_LO) * de0)))
+        : 1
+      : 0;
+    const rawYaw = Math.atan(ears ? (2 * xne[i]! + xna[i]!) / 2 : 2 * xne[i]!) * DEG;
     const tp = ears ? (yne[i]! - yne0 + (yna[i]! - yna0)) / 2 : yne[i]! - yne0;
-    yaw.push(Math.atan(ty) * DEG);
+    yaw.push(w > 0 ? w * (Math.atan(2 * xea[i]!) * DEG) + (1 - w) * rawYaw : rawYaw);
     pitch.push(Math.atan(tp) * DEG);
   }
   return { ears, yaw, pitch };
@@ -244,18 +293,35 @@ export function readFace(
   lEye: readonly Pt[],
   rEar: readonly Pt[] | null,
   lEar: readonly Pt[] | null,
+  nodResidual?: readonly number[] | null,
+  pitchGain = 0,
 ): FaceReading {
   const { ears, yaw, pitch } = readFaceAngles(nose, rEye, lEye, rEar, lEar);
+  const coupledPitch =
+    nodResidual !== undefined && nodResidual !== null && nodResidual.length === pitch.length && pitchGain !== 0
+      ? pitch.map((p, i) => (nodResidual[i]! === 0 ? p : p + pitchGain * nodResidual[i]!))
+      : pitch;
+  const basePitch = face.pitch === undefined ? undefined : mapFaceAxis(pitch, face.pitch);
+  const mappedPitch =
+    face.pitch === undefined
+      ? undefined
+      : coupledPitch === pitch
+        ? basePitch!
+        : (() => {
+            const m = mapFaceAxis(coupledPitch, face.pitch);
+            const keys = m.keys.map((v, i) => (nodResidual![i]! === 0 ? basePitch!.keys[i]! : v));
+            return { ...m, mapped: spanOf(keys), corr: corrOf(m.raw, keys), keys };
+          })();
   return {
     ears,
     read: {
       yaw: spanOf(yaw),
-      pitch: spanOf(pitch),
+      pitch: spanOf(coupledPitch),
       rawYaw: yaw,
-      rawPitch: pitch,
+      rawPitch: coupledPitch,
     },
     ...(face.yaw === undefined ? {} : { yaw: mapFaceAxis(yaw, face.yaw) }),
-    ...(face.pitch === undefined ? {} : { pitch: mapFaceAxis(pitch, face.pitch) }),
+    ...(mappedPitch === undefined ? {} : { pitch: mappedPitch }),
   };
 }
 

package/src/retarget.ts

@@ -98,6 +98,30 @@ const DEG = 180 / Math.PI;
  */
 export const HELD_MISS = 0.01;
 
+/**
+ * Beat-nod phase-coupling gain (`°/unit`) from the vertical residual that the
+ * `±2%` neck-length clamp removes from `head.translate.y` (`rawNod - dy`) onto
+ * `map.face.pitch` (`pitch_dial.rotate`, `rig-c/pose#190`). Measured on the
+ * dancer reference hand pass (`hand/p1` HP2: `+2.8°/unit`).
+ */
+export const NOD_RESIDUAL_PITCH_GAIN = 2.8;
+
+/**
+ * Beat-nod phase-coupling gain (`°/unit`) from the vertical residual that the
+ * `±2%` neck-length clamp removes from `head.translate.y` (`rawNod - dy`) onto
+ * the head chain bone's `rotate` track (`head.rotate`, `rig-c/pose#190`).
+ * Measured on the dancer reference hand pass (`hand/p1` HP2: `+1.6°/unit`).
+ */
+export const NOD_RESIDUAL_HEAD_GAIN = 1.6;
+
+export interface HeadNodReading {
+  bone: string;
+  maxNod: number;
+  rawY: number[];
+  clampedY: number[];
+  residual: number[];
+}
+
 export interface RetargetOptions extends SwitchOptions {
   /** A joint with a score below this is not good on that frame (a joint with no score is). */
   minScore: number;
@@ -334,6 +358,8 @@ export interface Retarget {
   body: BodyReading | null;
   /** Continuous per-frame figure scale relative to frame 0 (`#163`, `#161`). */
   figureScale: FigureScaleReading;
+  /** Per-frame vertical head nod reading (`rawY`, `clampedY`, and clamped `residual`, `#111`, `#181`, `#190`). */
+  headNod: HeadNodReading | null;
 }
 
 export interface FigureScaleFrame {
@@ -1080,6 +1106,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
   let headNodBone: string | null = null;
   let headNod: Pt[] | null = null;
   let headNodTrack: MotionTrack | null = null;
+  let headNodReading: HeadNodReading | null = null;
   const posedOrigin = (b: string, i: number): Pt => {
     const own = rest.get(b)!;
     if (t !== null && b === t.bone) return [own.worldX + (travel[i]?.[0] ?? 0), own.worldY + (travel[i]?.[1] ?? 0)];
@@ -1271,6 +1298,8 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
           }
           return Math.max(-maxNod, Math.min(maxNod, v));
         });
+        const residual = rawNod.map((v, i) => v - dy[i]!);
+        headNodReading = { bone: hb, maxNod, rawY: rawNod, clampedY: dy, residual };
         if (dy.some((v) => Math.abs(r3(v)) > 0)) {
           const keys: MotionKey[] = [];
           const nod: Pt[] = [];
@@ -2094,6 +2123,31 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
   }
   const solvedKey = (b: string, i: number) => legPoses?.[i]!.solved.get(b);
 
+  // #190: couple the vertical residual removed by the neck-length clamp into head.rotate
+  // in phase (+NOD_RESIDUAL_HEAD_GAIN °/unit), adding 0 on frames where the clamp did not act.
+  if (headNodReading !== null) {
+    const hb = headNodReading.bone;
+    const tg = targets.get(hb);
+    if (tg !== undefined) {
+      const above = mappedAbove(hb);
+      const hc = map.chains.find((c) => c.bones.includes(hb));
+      const limits = hc?.range[hb];
+      const cov = hc !== undefined ? covered.get(hc.name) : undefined;
+      for (let i = 0; i < n; i++) {
+        const res = headNodReading.residual[i]!;
+        if (res === 0) continue;
+        let next = wrap(tg[i]! + NOD_RESIDUAL_HEAD_GAIN * res);
+        if (limits !== undefined && cov?.[i] !== true) {
+          const key = wrap((next - restOf.get(hb)!) - (above === null ? 0 : turn(above, i)));
+          if (overflow(key, limits) > RANGE_SLACK) {
+            next = wrap(next + Math.min(Math.max(key, limits[0]), limits[1]) - key);
+          }
+        }
+        tg[i] = next;
+      }
+    }
+  }
+
   // Keys: the bone's turn less the turn of the nearest mapped bone above it.
   const tracks: Array<MotionTrack | SlotTrack> = [];
   for (const c of map.chains) {
@@ -2129,7 +2183,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
       const h = heldOf('l_ear');
       return 'never' in h ? null : h.at;
     })();
-    face = readFace(map.face, series.get('nose')!, series.get('r_eye')!, series.get('l_eye')!, rEar, lEar);
+    face = readFace(map.face, series.get('nose')!, series.get('r_eye')!, series.get('l_eye')!, rEar, lEar, headNodReading?.residual ?? null, NOD_RESIDUAL_PITCH_GAIN);
     for (const axis of ['yaw', 'pitch'] as const) {
       const ax = face[axis];
       if (ax === undefined) continue;
@@ -2260,7 +2314,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
     animation: { duration: keyTimes[n - 1]!, tracks },
     times: keyTimes, targets, origin, holds, bends, short, endOn, flips, scale,
     switches: sw.switches, shown: sw.shown, distances: sw.distances, turns, turnsUnread, range, artRange,
-    legScale, scales, feet, planted: plantOn, stance, floor, blendHold, closure, ankles, pivot: pivotOf(rest, t?.bone ?? null, legs), raised, byHip, face, body, figureScale,
+    legScale, scales, feet, planted: plantOn, stance, floor, blendHold, closure, ankles, pivot: pivotOf(rest, t?.bone ?? null, legs), raised, byHip, face, body, figureScale, headNod: headNodReading,
   };
 }