@rig-c/pose 0.42.0 → 0.43.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -104,7 +104,7 @@ A retarget of the cleaned `dance-a` fixture finds the same jumps (`FEET`) as the
104
104
  ## The pelvis track: `rig-pose pelvis`
105
105
 
106
106
  ```sh
107
- rig-pose pelvis --pose dance.scan.json --frames frames/ --out dance.pelvis.json [--mode silhouette|waistband] [--person <id>]
107
+ rig-pose pelvis --pose dance.scan.json --frames frames/ --out dance.pelvis.json [--mode silhouette|waistband|below-cover] [--person <id>]
108
108
  rig-pose pelvis --pose dance.scan.json --pelvis-keys pelvis.keys.json --out dance.pelvis.json
109
109
  ```
110
110
 
@@ -114,12 +114,13 @@ A pose detector keeps little of a drawn pelvis tilt ([#288](https://github.com/r
114
114
 
115
115
  - **`silhouette`** (the default): the straightest edge of the drawn figure across the hip span, near the hips' height. An edge pixel is a figure pixel (a channel more than 64 from the background colour, the median of the frame's border) beside a background pixel, so a drawn outline and a filled shape both have one. Every straight line within ±40° of level and one hip width of the hips is scored by the edge pixels within 1 px of it; the best is read when its pixels cover at least 60 % of the span's columns, and fitted by least squares (one 3-MAD rejection pass) within 2.5 px rms. It reads a drawn pelvis line, or the edge of a drawn hip.
116
116
  - **`waistband`**: the top edge of a garment covering the hips (a skirt, shorts). The garment's colour is the median at the hips' midpoint, and must not be the background's. In each column of the span, the first row from 1.5 hip widths above the hips where three pixels in a row carry it (within 40 per channel) is the top edge, and the line is fitted through those tops on the same terms. The colour must **hold** on its patch: at least 75 % of the patch's pixels carry its median ([#293](https://github.com/rig-c/pose/issues/293)). A dithered or paletted garment (a 216-colour GIF, whose channels alternate between palette steps pixel by pixel) does not, so its pixels are read through the mean of a 5×5 window, which does. Hips on the band's top edge give a patch that is half garment and half ground, so the patch is also tried one and two patch radii (0.3 hip widths) below the hips, and the scan then runs down to that patch. The first patch that holds and is not the background gives the colour, trying each height as is and then through the window mean. If none holds, the midpoint's median is read as before. The `PELVIS` line counts the frames read through the window mean and those whose colour was found below the hips.
117
+ - **`below-cover`** ([#342](https://github.com/rig-c/pose/issues/342)): for a garment whose top is hidden under another — a sweater's hem lying over a skirt's waistband hangs level whatever the pelvis does, so the band-top reader reads the cover. It reads what the cover cannot hide, every colour through the 5×5 window mean, a pixel being garment when it is nearer the garment's colour than each leg's (sampled 0.7 of the way hip → knee) and the ground's: the **hem** (in each column of the span, the last garment row before three that are not; a line fitted through them), the **thigh roots** (along each hip → knee, where the garment stops for 3 px; the line from the right root to the left), and the waistband's band top only when the ground lies above it. The tilt is the mean of the cues; fewer than two, or two more than 8° apart, decline the frame by name (`too few cues below the cover`, `cues disagree`). A `CUES` line per frame prints every cue and the hip keypoints' line and sign beside the fit's; the hips are reported, not fitted (a detector's hips keep a fraction of a drawn tilt), and a `HIPS-SIGN` line names a frame where their sign is opposite the fit's. On the synthetic walk with shorts turned with the drawn pelvis under a level sweater (`sweater`), every frame reads within 1.5° of the drawn tilt; the `waistband` reader reads the sweater's level hem there. **Known limit:** a thigh drawn in front of the garment (raised toward the camera) is read as the hem — its outline over the cloth looks like the hem's edge, and the hem and that thigh's root agree on it — so those frames must be declined by hand (set their `pelvisTilt` to null, or leave them out of `--pelvis-keys`). Reading them is [#349](https://github.com/rig-c/pose/issues/349).
117
118
 
118
119
  Both pass over the forearms: pixels within 0.15 hip widths of a segment elbow → wrist (1.5 times that of a wrist, the hand) are not read, and a waistband top right under a forearm is not taken. A frame neither reader can read is **declined**: its `pelvisTilt` is `null` — marked, not guessed — and a `DECLINED` line names the frames and the reason (the pelvis blank, no garment, no straight edge across 60 % of the span, the hips side-on or missing). The `PELVIS` line counts the frames read and declined and gives the swing read.
119
120
 
120
121
  `--pelvis-keys` is the hand override where neither reader applies: `rig-pose-pelvis-keys/1`, `{ "spec": "rig-pose-pelvis-keys/1", "keys": [ { "frame", "deg" } ], "note"? }`. Frames are the sequence's and strictly increase, `deg` is in `pelvisTilt`'s sense; the tilt is linear between keys, the short way round, and held before the first and after the last. A frame may be fractional ([#294](https://github.com/rig-c/pose/issues/294)): a scan interpolated from drawn keys puts them between its frames — k × 4.75 for 8 keys over 38 frames — and a key rounded onto the grid moves up to half a frame and bends the tilt between. On that synthetic case (24 fps, a known tilt sampled at the keys' times), the tilt written on every frame is the keys' linear interpolation within 0.0005° (the writer's rounding), where the same keys rounded to whole frames miss it by up to 1.19° (`test/pelvis.test.ts`). A key outside the sequence's span, `[0, frames − 1]`, is refused by name. Fractional frames stay `/1`: they are a wider value of the same key, and a reader before #294 refuses them by name (`"frame" is number 4.75; it must be a frame index in [0, N]`) rather than misreading them. Anything malformed is refused by name (`PELVIS_KEYS`), and the source is `"keys"`. `--person` picks the person when there is more than one, and only that person is written.
121
122
 
122
- **The track** is `pelvisTilt` on every frame and `pelvisSource` (`"silhouette"`, `"waistband"` or `"keys"`) on the sequence, written as `rig-pose-sequence/1.1` ([below](#the-pose-rig-pose-sequence1)). `pelvisTilt` is Spine degrees — counter-clockwise, y up — of the direction from the figure's right hip to its left, the angle `retarget` reads off `r_hip → l_hip`: about 0 for a figure facing the camera, about ±180 facing away.
123
+ **The track** is `pelvisTilt` on every frame and `pelvisSource` (`"silhouette"`, `"waistband"`, `"below-cover"` or `"keys"`) on the sequence, written as `rig-pose-sequence/1.1` ([below](#the-pose-rig-pose-sequence1)). `pelvisTilt` is Spine degrees — counter-clockwise, y up — of the direction from the figure's right hip to its left, the angle `retarget` reads off `r_hip → l_hip`: about 0 for a figure facing the camera, about ±180 facing away.
123
124
 
124
125
  **`clean`** smooths `pelvisTilt` as its own track, not re-read from the hips: the quadratic Savitzky–Golay over the legs' window (`--legs`, the hips' window), circular with `--loop`, on the frames that carry a reading — a `null` stays `null`. It prints a `PELVIS` line with the largest change, and records `"pelvis": { "window" }` in the record's `params` (the `CLEAN` line ends `pelvisTilt smoothed as its own track over N frames`). A sequence with a track whose record lacks it — the track added after `clean` — is refused by name (`POSE_PELVIS`).
125
126
 
@@ -168,15 +169,15 @@ With `--draw-order depth` (off by default; without it nothing changes) `retarget
168
169
 
169
170
  - **The pair.** The legs: a leg is the map's chains naming its knee or ankle, their bones and the bones under them no other chain maps. Its slots are the slots on those bones. A slot of an alternate set (`alternates`) on none of them goes with the leg the set depicts, so a set keeps its own switch and takes the order of the leg it replaces. The legs are keyed when their slots overlap in the draw order, filling one unbroken run of the setup order. A slot of anything else drawn between them would be re-layered by the swap, and the setup order is not changed here. The pair is then not keyed, and the line names that slot. The arms across the trunk are a later card.
170
171
  - **The rule.** On each frame the leg whose depth is the lower is drawn behind: its slots first in the pair's run, in their setup order, then the other's. The depths must differ by at least a tie, the depth a flat chain can hold there. Inside the tie, or where a leg is not ordered, the order before holds. Before the first decided frame it is the setup order, or on a looped sequence the order the clip ends in. A key is written where the order changes.
171
- - **The joint the gait names** ([#336](https://github.com/rig-c/pose/issues/336), the owner's two statements). On a sequence cleaned with `--depth walk` a leg's depth is its ankle's (the hip at 0, plus the thigh's, plus the shin's), the tie √(1 − 0.95²) × (thigh + shin) of the rest lengths (46.83 px on the fixtures): in a walk the lifted foot is behind the supporting leg. With `--depth run` it is its knee's (the thigh's), the tie √(1 − 0.95²) × thigh (24.97 px): in a run the lifted leg is in front, its knee protruding. The limb mean (`layers`, above) says neither: a lifted leg's thigh forward and shin back cancel in it, and a walk whose thighs stay inside the band leaves it at most half a shin (35 px on the fixtures), under its tie of √(1 − 0.95²) × (thigh + shin ÷ 2) (35.89 px). A measured depth names no gait and keeps the limb mean and its tie ([#304](https://github.com/rig-c/pose/issues/304)).
172
+ - **The joint the gait names** ([#336](https://github.com/rig-c/pose/issues/336), the owner's two statements). On a sequence cleaned with `--depth walk` a leg's depth is its ankle's (the hip at 0, plus the thigh's, plus the shin's), the tie √(1 − 0.95²) × (thigh + shin) of the rest lengths (46.83 px on the fixtures): in a walk the lifted foot is behind the supporting leg. With `--depth run` it is its knee's (the thigh's), the tie √(1 − 0.95²) × thigh (24.96 px): in a run the lifted leg is in front, its knee protruding. The limb mean (`layers`, above) says neither: a lifted leg's thigh forward and shin back cancel in it, and a walk whose thighs stay inside the band leaves it at most half a shin (35 px on the fixtures), under its tie of √(1 − 0.95²) × (thigh + shin ÷ 2) (35.89 px). A measured depth names no gait and keeps the limb mean and its tie ([#304](https://github.com/rig-c/pose/issues/304)).
172
173
  - **No depth.** On a sequence without depth the flag is refused (`DRAWORDER_NO_DEPTH`): the order would come from nowhere, and a motion written without it would read as ordered.
173
174
  - **A walk cleaned before #336** ([#336](https://github.com/rig-c/pose/issues/336)). A derived depth whose foreshortened shin leans the way the trunk faces carries the sign table before #336, where a walk's lifted foot read forward; the ankle would draw that foot in front. The flag refuses it by name (`DEPTH_RECLEAN`), listing the frames, until it is cleaned again with this version. The table since leans the shin back in both gaits, so a record it wrote never matches. A run's signs did not change, so a run cleaned by an earlier release is read as it is, and so is a walk with no foreshortened shin (its signs read the same under both tables). Measured depth is not checked. The file format does not change.
174
175
 
175
176
  On #281's fixtures, retargeted onto a limb-cut rig whose legs overlap (the left leg drawn first; `test/draworder.test.ts`):
176
177
 
177
178
  - **The walk** (by the ankle): each of the 36 frames with one leg lifted draws that leg behind; the order changes on frames 3, 15, 27 and 39, where a lifted foot first goes back past the tie, and holds through the double support between. The generator orders the true ankles on 16 of the 48 frames, and the keyed order agrees on all 16.
178
- - **The run** (by the knee): the order changes on frames 2, 12, … 52. Of the 42 frames with one leg lifted, the 30 whose lifted knee protrudes (its thigh leaning forward past the band) draw that leg in front. The other 12 (frames 0–1, 10–11, …) are the early swing: the lifted thigh is still inside the band, the generator's knees are within the tie, and the order holds — the landing leg in front, as the generator's ankles have it. The generator orders the true knees on 42 of the 60 frames, and the keyed order agrees on all 42. On frame 12, on a rig that draws the right leg first, none of the 8,763 pixels the lifted right leg draws shows the standing leg, against 2,192 in the setup order.
179
- - **Read otherwise**: by the limb mean or by the ankle, the run draws the lifted leg behind on all 30 frames whose knee protrudes; by the limb mean or by the knee, the walk keys nothing (the setup order) and draws the lifted right leg in front on its 18 frames. Knees planted 10 px apart, inside the knee tie, hold the order before; without the tie they swap it.
179
+ - **The run** (by the knee): the order changes on frames 0, 10, … 50, where the other foot lands. The generator's swing thigh flexes forward from toe-off ([#343](https://github.com/rig-c/pose/issues/343)), so on each of the 42 frames with one leg lifted the lifted knee is in front of the supporting one, its thigh leaning forward past the band, and that leg is drawn in front. The generator orders the true knees on 42 of the 60 frames, and the keyed order agrees on all 42. On frame 12, on a rig that draws the right leg first, none of the 8,771 pixels the lifted right leg draws shows the standing leg, against 2,205 in the setup order.
180
+ - **Read otherwise**: by the limb mean or by the ankle, the run draws the lifted leg behind on 21 of its 42 lifted frames; by the limb mean or by the knee, the walk keys nothing (the setup order) and draws the lifted right leg in front on its 18 frames. Knees planted 10 px apart, inside the knee tie, hold the order before; without the tie they swap it.
180
181
 
181
182
  ## What it does
182
183
 
@@ -228,7 +229,7 @@ and what the run cannot decide for you it lists:
228
229
  - `REACH` — with `--raised-by-head`, or a `"by-hip"` set: a placed span whose target was out of the arm's reach, too far or too near. It names the frames, which way, and the shortfall in eye distances (hip widths for `"by-hip"`). The wrist was placed at the reach limit instead. This is not `ART-RANGE`: the rig's proportions were the limit, not the art's range. A too-near frame is placed at the limit like a too-far one, and graded at the placed distance.
229
230
  - `LEG-SCALE` — with `--leg-scale` ([below](#the-legs-foreshortening-and-planted-feet)): per leg segment, its straight length in px and as a share of `FEET`'s leg length, the spread of its picture length over it, the limits in px, the frames clamped at each limit, where its leg was free on clamped frames how far the clamps on that leg left the free ankle above or below the picture's (in the rig's leg at setup, thigh origin → ankle, and in `FEET`'s px; `legScale[].free` in the report, [#237](https://github.com/rig-c/pose/issues/237)), and (with `--plant-feet`) one `past the floor (support)` line per supporting run that asked under the floor ([#133](https://github.com/rig-c/pose/issues/133), `legScale[].support` in the report).
230
231
  - `DEPTH` — when the sequence carries depth ([above](#derived-depth-clean---depth)): where it came from, on how many frames the limbs were ordered front to back, and which limb was in front alone how often — `DEPTH layers from the sequence's derived depth (clean --depth run): the limbs ordered front to back on N of N frame(s); in front alone: r_arm a, l_arm b, r_leg c, l_leg d; tied in front on k (the report's layers)`.
231
- - `DRAWORDER` — with `--draw-order depth` ([above](#draw-order-from-depth---draw-order-depth)): the keys and the spans they key, per leg drawn behind, cut to 260 characters — `DRAWORDER from depth by the knee (run; tie 24.97 px), 6 key(s) over 3 span(s) (leg_r behind leg_l 2–11, 22–31, 42–51)`, naming the joint and its tie for a keyed pair ([#336](https://github.com/rig-c/pose/issues/336)) — and a line per pair not keyed, saying why.
232
+ - `DRAWORDER` — with `--draw-order depth` ([above](#draw-order-from-depth---draw-order-depth)): the keys and the spans they key, per leg drawn behind, cut to 260 characters — `DRAWORDER from depth by the knee (run; tie 24.96 px), 6 key(s) over 3 span(s) (leg_r behind leg_l 0–9, 20–29, 40–49)`, naming the joint and its tie for a keyed pair ([#336](https://github.com/rig-c/pose/issues/336)) — and a line per pair not keyed, saying why.
232
233
  - `PELVIS` — when the map has a pelvis chain (one bone, joints `r_hip → l_hip`) or the sequence a pelvis track ([above](#the-pelvis-track-rig-pose-pelvis)): where the pelvis bone's rotation came from and its swing — `PELVIS: from <source>, swing N°`, or `PELVIS: from hip line (no track)` — with the frames keyed from the hip line on a track that declined them.
233
234
  - `PIVOT` — when the map translates a bone and has both legs: the translated bone's origin against its thigh roots' midpoint at setup, in units and hip widths — the pivot every hip turn swings the legs about.
234
235
  - `FEET` — when the map has both legs: one `segment` line per mapped leg segment with its picture length ÷ straight length (min · median · max), the scale `--leg-scale` would ask before its clamp, printed with or without the flag so the minimum can be read before choosing the range ([#237](https://github.com/rig-c/pose/issues/237)); the supports and jumps read off the reference ankles (planted or not), and on the posed build, outside the jumps, the lowest ankle's vertical range and each support span's worst vertical range and horizontal drift of the supporting ankle, in leg lengths.
@@ -427,7 +428,7 @@ Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2
427
428
  - Joints are body-18 (`nose neck r_shoulder r_elbow r_wrist l_shoulder l_elbow l_wrist r_hip r_knee r_ankle l_hip l_knee l_ankle r_eye l_eye r_ear l_ear`); `r`/`l` are the **subject's** sides. `state` is `observed`, `occluded` or `missing`, as in the keypoints file, or `inferred` ([#63](https://github.com/rig-c/pose/issues/63)): a position the detector supplied but did not see. An `inferred` joint carries `at` (one without it is refused, `POSE_JOINT_AT`), and `retarget` keys it as it keys `observed`. An unlisted joint is missing.
428
429
  - Two differences from the keypoints file: the unit is `picture-px`, and `at` may lie outside the picture (a hand leaves the frame; the score says how much to believe it).
429
430
  - More than one person: `--person <id>`.
430
- - **The pelvis track** ([#288](https://github.com/rig-c/pose/issues/288)) is the minor `rig-pose-sequence/1.1`: a `pelvisTilt` on every frame (beside `t` and `people`; Spine degrees in (−180, 180] of the direction r_hip → l_hip, or `null` where the reading declined the frame) and `pelvisSource` on the sequence (`"silhouette"`, `"waistband"` or `"keys"`), written by `rig-pose pelvis` ([above](#the-pelvis-track-rig-pose-pelvis)). A sequence carrying them holds one person; a track on some frames only, a source without a track, or a track without a source is refused by name (`POSE_PELVIS`). The writer writes the minor the content needs — `/1.1` with a track, `/1` without — so a sequence without one stays readable by a reader of `/1`; a `/1` file carrying a 1.1 key is refused naming the minor (`POSE_SPEC`). The spec string is what the file records of the version that wrote it: a file of a newer minor than this reader's is read, and each key it does not know is refused with the minor the file says it was written as (`… the file says it was written as "rig-pose-sequence/1.3", a newer minor than this reader's "rig-pose-sequence/1.2" — read it with a rig-pose that reads that minor`). A reader of `/1` (rig-pose 0.37 and before) refuses a `/1.1` file by its spec (`POSE_SPEC`, naming `"rig-pose-sequence/1.1"`) and its keys (`POSE_KEYS`).
431
+ - **The pelvis track** ([#288](https://github.com/rig-c/pose/issues/288)) is the minor `rig-pose-sequence/1.1`: a `pelvisTilt` on every frame (beside `t` and `people`; Spine degrees in (−180, 180] of the direction r_hip → l_hip, or `null` where the reading declined the frame) and `pelvisSource` on the sequence (`"silhouette"`, `"waistband"`, `"below-cover"` or `"keys"`), written by `rig-pose pelvis` ([above](#the-pelvis-track-rig-pose-pelvis)). A sequence carrying them holds one person; a track on some frames only, a source without a track, or a track without a source is refused by name (`POSE_PELVIS`). The writer writes the minor the content needs — `/1.1` with a track, `/1` without — so a sequence without one stays readable by a reader of `/1`; a `/1` file carrying a 1.1 key is refused naming the minor (`POSE_SPEC`). The spec string is what the file records of the version that wrote it: a file of a newer minor than this reader's is read, and each key it does not know is refused with the minor the file says it was written as (`… the file says it was written as "rig-pose-sequence/1.3", a newer minor than this reader's "rig-pose-sequence/1.2" — read it with a rig-pose that reads that minor`). A reader of `/1` (rig-pose 0.37 and before) refuses a `/1.1` file by its spec (`POSE_SPEC`, naming `"rig-pose-sequence/1.1"`) and its keys (`POSE_KEYS`).
431
432
  - **Depth** ([#281](https://github.com/rig-c/pose/issues/281)) is the minor `rig-pose-sequence/1.2`: a `depth` on every frame (beside `t` and `people`), `{ "<segment>": { "ratio", "sign", "source" } }` — the segments `r_upper_arm`, `r_forearm`, `r_thigh`, `r_shin`, `l_upper_arm`, `l_forearm`, `l_thigh`, `l_shin`, each present where its two joints are placed on the frame (`{}` where none is); `ratio` a number ≥ 0, `sign` −1, 0 or 1, `source` `"derived"` (`clean --depth`, [above](#derived-depth-clean---depth)) or `"measured"` (a 3D detector). A minor includes the keys of the minors before it, so a sequence with depth and a pelvis track is `/1.2`, and one with the track alone stays `/1.1`. A sequence with depth holds one person; depth on some frames only, a malformed entry, an unknown segment, derived and measured depth together, and derived depth without the `cleaned` record's `"depth": { "gait" }` (or that record without derived depth on every frame) are refused by name (`POSE_DEPTH`); a `/1` or `/1.1` file carrying `depth` is refused naming the minor (`POSE_SPEC`).
432
433
  - `travel` (optional, [#92](https://github.com/rig-c/pose/issues/92)) is the clip's frame of reference. Its one value is `"in-place"`: a gait filmed without the figure travelling, such as a treadmill or reel walk, or a loop. `clean --loop` writes it, and it may be written by hand on any sequence. Any other value is refused by name (`POSE_TRAVEL`). Without it the figure's travel in the picture is its travel. It is a property of the sequence, not of a flag ([#242](https://github.com/rig-c/pose/issues/242)): with or without `--plant-feet`, retarget reads the supports in place, skips the camera-zoom normalisations (a loop has no zoom, and their rolling windows do not wrap) and reads the head nod without its detrend window or clamp. Only the following foot and the loop closure need `--plant-feet` ([below](#the-legs-foreshortening-and-planted-feet)).
433
434
  - `score` is whatever the pose model gives. RTMPose's goes above 1, so `--min-score` is a cut on that model's scale, not a probability.
package/cli.ts CHANGED
@@ -22,7 +22,7 @@ import { runRetarget, type RetargetArgs } from './src/run.ts';
22
22
  const HELP = `rig-pose ${pkg.version} — motion from a pose sequence onto a rig-c rig
23
23
 
24
24
  Usage:
25
- rig-pose pelvis --pose <pose.json> --out <pose.pelvis.json> (--frames <dir> [--mode silhouette|waistband] | --pelvis-keys <keys.json>) [--person <id>]
25
+ rig-pose pelvis --pose <pose.json> --out <pose.pelvis.json> (--frames <dir> [--mode silhouette|waistband|below-cover] | --pelvis-keys <keys.json>) [--person <id>]
26
26
  rig-pose clean --pose <pose.json> --out <pose.clean.json> [--loop] [--legs 5] [--rest 7] [--hold off] [--depth walk|run] [--coco] [--person <id>]
27
27
  rig-pose retarget --rig <rig> --map <map.json> --sequence <pose.json> --out <motion.json> [--animation dance] [flags]
28
28
  rig-pose convert --coco <coco17.json> --out <pose.json>
@@ -38,11 +38,15 @@ pelvis (before clean; rig-c/pose#288)
38
38
  r_hip → l_hip, about 0 facing the camera — or null on a frame the reading declined
39
39
  --frames <dir> the picture: one PNG per sequence frame, in file-name order, the sequence's size (no video;
40
40
  extract the frames first). The hip keypoints place the reading; their angle is not read
41
- --mode silhouette|waistband
41
+ --mode silhouette|waistband|below-cover
42
42
  silhouette (default): the straightest edge of the drawn figure across the hip span (between
43
43
  the two hip keypoints), within 40° of level and one hip width of the hips' height — a drawn
44
44
  pelvis line or the edge of a drawn hip. waistband: the top edge of a garment covering the hips
45
45
  (skirt, shorts) — the colour at the hips' midpoint, its first row in each column of the span.
46
+ below-cover: for a garment whose top is hidden under another (a sweater's hem over a skirt)
47
+ — the bottomwear's lower hem across the span, where each thigh leaves it, and the band's top
48
+ only where nothing covers it; the mean of those cues, declined when they differ by more
49
+ than 8° (a CUES line per frame; the hip keypoints are printed there, not fitted).
46
50
  Forearms (elbow → wrist) and hands are masked. A frame either cannot read (the pelvis blank,
47
51
  no garment, no straight edge over 60% of the span) is declined, named on a DECLINED line,
48
52
  and written null: marked, not guessed
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@rig-c/pose",
3
- "version": "0.42.0",
3
+ "version": "0.43.0",
4
4
  "description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
5
5
  "type": "module",
6
6
  "bin": {
package/src/pelvis.ts CHANGED
@@ -59,7 +59,7 @@ type Pt = [number, number];
59
59
  type RGB = [number, number, number];
60
60
 
61
61
  /** The two readers. */
62
- export const PELVIS_MODES = ['silhouette', 'waistband'] as const;
62
+ export const PELVIS_MODES = ['silhouette', 'waistband', 'below-cover'] as const;
63
63
  export type PelvisMode = (typeof PELVIS_MODES)[number];
64
64
  export const PELVIS_KEYS_SPEC = 'rig-pose-pelvis-keys/1';
65
65
 
@@ -85,13 +85,41 @@ export const MIN_SPAN = 6;
85
85
  export const CLIMB_SPREAD = 2;
86
86
  /** The band's top is climbed against at most this many garment colours (rig-c/pose#318): a paletted garment's own, or a painted one's fullest cells. */
87
87
  export const MAX_CLOTH = 16;
88
+ /** rig-c/pose#342: the below-cover cues read the same line when their tilts lie within this many degrees (max − min). */
89
+ export const CUE_SPREAD = 8;
90
+ /** ... a leg's colour is sampled this far along its thigh, hip → knee. */
91
+ export const LEG_AT = 0.7;
92
+ /** ... the hem and a thigh root are where the garment stops for this many px (rows, or along the thigh) in a row. */
93
+ export const LEAVE = 3;
88
94
 
89
95
  /**
90
96
  * One frame's reading: the tilt (Spine degrees, r_hip → l_hip), or why it was declined. A waistband
91
97
  * read through the window mean says so (`mean`), and one whose colour was found below the hips says
92
98
  * how far (`below`, px); one whose band top lay above the colour's edge in some columns says how many (`raised`, #318).
93
99
  */
94
- export type Reading = { tilt: number; cover: number; rms: number; mean?: true; below?: number; raised?: number } | { kind: string; why: string };
100
+ export type Reading = { tilt: number; cover: number; rms: number; mean?: true; below?: number; raised?: number; cues?: CueTilts } | { kind: string; why: string };
101
+
102
+ /**
103
+ * rig-c/pose#342: the below-cover cues of one frame, Spine degrees r_hip → l_hip; null where the cue was not read. `band` is
104
+ * "covered" when something other than the ground lies over the band's top. `hips` is the keypoints' line: reported, not fitted.
105
+ */
106
+ export interface CueTilts {
107
+ hem: number | null;
108
+ roots: number | null;
109
+ band: number | 'covered' | null;
110
+ hips: number;
111
+ }
112
+
113
+ /** rig-c/pose#342: a below-cover reading's figures and points (the picture's), and why each cue not read was not. */
114
+ export interface CueDetail {
115
+ tilts: CueTilts;
116
+ why: { hem?: string; roots?: string; band?: string };
117
+ garment: RGB;
118
+ legs: { r: RGB | null; l: RGB | null };
119
+ hem: { pts: Pt[]; fit: { m: number; q: number; rms: number } | null };
120
+ roots: { r: Pt | null; l: Pt | null; paths: Array<[Pt, Pt]> };
121
+ band: { m: number; q: number } | null;
122
+ }
95
123
 
96
124
  /** The frame as RGB over white, its background colour, and which pixels are masked (forearms, hands). */
97
125
  interface Frame {
@@ -194,7 +222,38 @@ export function readPelvis(plate: Plate, joints: Partial<Record<Body18, Joint>>,
194
222
  const f = frameOf(plate, arms, Math.max(3, ARM_RADIUS * span));
195
223
  if (x0 < 0 || x1 >= f.w || c[1] < 0 || c[1] >= f.h) return { kind: 'hips outside the frame', why: `the hips (${c[0].toFixed(1)}, ${c[1].toFixed(1)}) lie outside the ${f.w}×${f.h} frame` };
196
224
  const cols = x1 - x0 + 1;
197
- return mode === 'silhouette' ? silhouette(f, c, span, x0, x1, cols, rHip, lHip) : waistband(f, c, span, x0, x1, cols, rHip, lHip);
225
+ if (mode === 'silhouette') return silhouette(f, c, span, x0, x1, cols, rHip, lHip);
226
+ if (mode === 'waistband') return waistband(f, c, span, x0, x1, cols, rHip, lHip);
227
+ return belowCover(f, c, span, x0, x1, cols, rHip, lHip, pt).reading;
228
+ }
229
+
230
+ /**
231
+ * rig-c/pose#342: the below-cover reading of one frame with every cue's figures and points, for a picture of it. Null when
232
+ * the frame is declined before any cue is read (no hips, side-on, outside the frame); the reading is `readPelvis`'s.
233
+ */
234
+ export function readBelowCover(plate: Plate, joints: Partial<Record<Body18, Joint>>): { reading: Reading; detail: CueDetail | null } {
235
+ const reading = readPelvis(plate, joints, 'below-cover');
236
+ const pt = (n: Body18): Pt | null => {
237
+ const j = joints[n];
238
+ return j === undefined || j.state === 'missing' || j.at === undefined ? null : [j.at[0], j.at[1]];
239
+ };
240
+ const rHip = pt('r_hip');
241
+ const lHip = pt('l_hip');
242
+ if (rHip === null || lHip === null) return { reading, detail: null };
243
+ const span = Math.hypot(lHip[0] - rHip[0], lHip[1] - rHip[1]);
244
+ const x0 = Math.round(Math.min(rHip[0], lHip[0]));
245
+ const x1 = Math.round(Math.max(rHip[0], lHip[0]));
246
+ if (span < MIN_SPAN || x1 - x0 + 1 < MIN_SPAN) return { reading, detail: null };
247
+ const c: Pt = [(rHip[0] + lHip[0]) / 2, (rHip[1] + lHip[1]) / 2];
248
+ const arms: Array<[Pt, Pt]> = [];
249
+ for (const [e, w] of [['r_elbow', 'r_wrist'], ['l_elbow', 'l_wrist']] as const) {
250
+ const a = pt(e);
251
+ const b = pt(w);
252
+ if (a !== null && b !== null) arms.push([a, b]);
253
+ }
254
+ const f = frameOf(plate, arms, Math.max(3, ARM_RADIUS * span));
255
+ if (x0 < 0 || x1 >= f.w || c[1] < 0 || c[1] >= f.h) return { reading, detail: null };
256
+ return { reading, detail: belowCover(f, c, span, x0, x1, x1 - x0 + 1, rHip, lHip, pt).detail };
198
257
  }
199
258
 
200
259
  function silhouette(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt): Reading {
@@ -272,7 +331,8 @@ function meanOf(f: Frame): (x: number, y: number) => RGB | null {
272
331
  };
273
332
  }
274
333
 
275
- function waistband(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt): Reading {
334
+ /** `out.fit`, when given, receives the edge read (rig-c/pose#342 looks at what lies above it). */
335
+ function waistband(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt, out?: { fit?: { m: number; q: number } }): Reading {
276
336
  // The garment's colour: the median over the unmasked pixels of a patch at the hips' midpoint (or below it), as they are or through the window mean.
277
337
  const r = Math.max(2, 0.3 * span);
278
338
  const raw = (x: number, y: number): RGB | null => f.rgb(x, y);
@@ -341,6 +401,7 @@ function waistband(f: Frame, c: Pt, span: number, x0: number, x1: number, cols:
341
401
  if (Math.abs(Math.atan(fit.m)) * (180 / Math.PI) > 45) return { kind: 'waistband too steep', why: `the garment's top edge runs ${(Math.atan(fit.m) * (180 / Math.PI)).toFixed(1)}° from level; a waistband is within 45°` };
342
402
  const off = notTheBand(f, fit, tops, top, colourAt);
343
403
  if (off !== null) return off;
404
+ if (out !== undefined) out.fit = { m: fit.m, q: fit.q };
344
405
  return { tilt: tiltOf(fit.m, rHip, lHip), cover: share, rms: fit.rms, ...(isMean ? { mean: true as const } : {}), ...(below > 0 ? { below } : {}), ...(raised > 0 ? { raised } : {}) };
345
406
  }
346
407
 
@@ -532,6 +593,193 @@ function notTheBand(f: Frame, fit: { m: number; q: number }, tops: Pt[], top: nu
532
593
  };
533
594
  }
534
595
 
596
+ /**
597
+ * rig-c/pose#342: the pelvis read below a cover. A garment hanging over the waistband (a sweater's hem) hides the band's top
598
+ * and hangs level whatever the pelvis does, so the band-top reader reads the cover. What the cover cannot hide is read:
599
+ *
600
+ * - `hem`: the bottomwear's lower hem across the hip span — in each column, from the hips' row down, the last garment row
601
+ * before `LEAVE` rows that are not garment; a line fitted through them (≥ `MIN_COVER` of the columns);
602
+ * - `roots`: where each thigh leaves the garment — along hip → knee (keypoints), the last garment point before `LEAVE` px
603
+ * that are not; the line from the right root to the left;
604
+ * - `band`: the waistband reader's band top, used only when the ground lies above it (nothing covers it);
605
+ * - `hips`: the keypoints' line, reported but not fitted — a detector's hips keep a fraction of a drawn tilt (#288).
606
+ *
607
+ * Every colour is the window mean (`DITHER_RADIUS`); a pixel is garment when it is nearest the garment's colour among the
608
+ * garment's, each leg's (sampled `LEG_AT` along its thigh) and the ground's, so both shades of a cloth and the ink on it
609
+ * are garment and skin or ground under the hem are not. The tilt is the mean of the cues fitted (hem, roots, an
610
+ * uncovered band); fewer than two, or two that differ by more than `CUE_SPREAD`, decline the frame by name.
611
+ */
612
+ function belowCover(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt, pt: (n: Body18) => Pt | null): { reading: Reading; detail: CueDetail | null } {
613
+ const r = Math.max(2, 0.3 * span);
614
+ const mean = meanOf(f);
615
+ const disc = (at: Pt): RGB | null => {
616
+ const ps: RGB[] = [];
617
+ for (let y = Math.floor(at[1] - r); y <= Math.ceil(at[1] + r); y++) {
618
+ for (let x = Math.floor(at[0] - r); x <= Math.ceil(at[0] + r); x++) {
619
+ if (x < 0 || y < 0 || x >= f.w || y >= f.h || Math.hypot(x - at[0], y - at[1]) > r || f.masked(x, y)) continue;
620
+ const p = mean(x, y);
621
+ if (p !== null) ps.push(p);
622
+ }
623
+ }
624
+ return ps.length === 0 ? null : ([0, 1, 2].map((k) => median(ps.map((p) => p[k]!))) as RGB);
625
+ };
626
+ const holds = (at: Pt, g: RGB) => {
627
+ let n = 0;
628
+ let k = 0;
629
+ for (let y = Math.floor(at[1] - r); y <= Math.ceil(at[1] + r); y++) {
630
+ for (let x = Math.floor(at[0] - r); x <= Math.ceil(at[0] + r); x++) {
631
+ if (x < 0 || y < 0 || x >= f.w || y >= f.h || Math.hypot(x - at[0], y - at[1]) > r || f.masked(x, y)) continue;
632
+ const p = mean(x, y);
633
+ if (p === null) continue;
634
+ n++;
635
+ if (diff(p, g) <= GARMENT_DIFF) k++;
636
+ }
637
+ }
638
+ return n > 0 && k >= HOLD * n;
639
+ };
640
+ const mid = disc(c);
641
+ if (mid === null) return { reading: { kind: 'hips behind a forearm', why: 'the hips\' midpoint is covered by a forearm; no garment colour to read' }, detail: null };
642
+ let g: RGB = mid;
643
+ for (const drop of [0, r, 2 * r]) {
644
+ const at: Pt = [c[0], c[1] + drop];
645
+ const p = drop === 0 ? mid : disc(at);
646
+ if (p !== null && diff(p, f.bg) > FIGURE_DIFF && holds(at, p)) {
647
+ g = p;
648
+ break;
649
+ }
650
+ }
651
+ if (diff(g, f.bg) <= FIGURE_DIFF) return { reading: { kind: 'no garment over the hips', why: `the colour at the hips' midpoint (${g.map((v) => v.toFixed(0)).join(', ')}) is the background's (${f.bg.map((v) => v.toFixed(0)).join(', ')})` }, detail: null };
652
+ const knee = { r: pt('r_knee'), l: pt('l_knee') };
653
+ const hip = { r: rHip, l: lHip };
654
+ const legs = { r: null as RGB | null, l: null as RGB | null };
655
+ for (const s of ['r', 'l'] as const) {
656
+ const k = knee[s];
657
+ if (k === null) continue;
658
+ const L = disc([hip[s][0] + LEG_AT * (k[0] - hip[s][0]), hip[s][1] + LEG_AT * (k[1] - hip[s][1])]);
659
+ if (L !== null && diff(L, g) > GARMENT_DIFF) legs[s] = L;
660
+ }
661
+ const others = [f.bg, ...(legs.r !== null ? [legs.r] : []), ...(legs.l !== null ? [legs.l] : [])];
662
+ if (others.length === 1) return { reading: { kind: 'no leg below the garment', why: `neither thigh carries a colour of its own ${LEG_AT} of the way to its knee (${knee.r === null || knee.l === null ? 'a knee has no position' : 'both within ' + GARMENT_DIFF + ' of the garment\'s'}); nothing below the garment to tell its hem by` }, detail: null };
663
+ // Garment: nearest the garment's colour among it, the legs' and the ground's (a tie is garment). Null where every pixel of the window is masked.
664
+ const isGarment = (x: number, y: number): boolean | null => {
665
+ if (x < 0 || y < 0 || x >= f.w || y >= f.h || f.masked(x, y)) return null;
666
+ const p = mean(x, y);
667
+ if (p === null) return null;
668
+ return diff(p, g) <= Math.min(...others.map((o) => diff(p, o)));
669
+ };
670
+ const why: CueDetail['why'] = {};
671
+ // The hem.
672
+ const hemPts: Pt[] = [];
673
+ const yEnd = Math.min(f.h - 1, Math.floor(c[1] + 2.5 * span));
674
+ for (let x = x0; x <= x1; x++) {
675
+ let inside = false;
676
+ let last = -1;
677
+ let out = 0;
678
+ for (let y = Math.max(0, Math.round(c[1])); y <= yEnd; y++) {
679
+ const gm = isGarment(x, y);
680
+ if (gm === null) {
681
+ if (inside) break;
682
+ continue;
683
+ }
684
+ if (gm) {
685
+ inside = true;
686
+ last = y;
687
+ out = 0;
688
+ } else if (inside && ++out >= LEAVE) {
689
+ hemPts.push([x, last]);
690
+ break;
691
+ }
692
+ }
693
+ }
694
+ const hemFit = hemPts.length >= MIN_COVER * cols ? fitLine(hemPts) : null;
695
+ let hem: number | null = null;
696
+ if (hemFit === null) why.hem = hemPts.length < MIN_COVER * cols ? `the hem is found in ${hemPts.length} of the ${cols} columns between the hips (< ${100 * MIN_COVER}%)` : 'the hem rows do not fit a line';
697
+ else hem = tiltOf(hemFit.m, rHip, lHip);
698
+ // The thigh roots.
699
+ const root = { r: null as Pt | null, l: null as Pt | null };
700
+ const rootWhy: string[] = [];
701
+ const paths: Array<[Pt, Pt]> = [];
702
+ for (const s of ['r', 'l'] as const) {
703
+ const k = knee[s];
704
+ if (k === null) {
705
+ rootWhy.push(`${s}_knee has no position`);
706
+ continue;
707
+ }
708
+ const h = hip[s];
709
+ paths.push([h, k]);
710
+ const len = Math.hypot(k[0] - h[0], k[1] - h[1]);
711
+ const n = Math.ceil(len / 0.5);
712
+ let inside = false;
713
+ let last: Pt | null = null;
714
+ let out = 0;
715
+ let hidden = false;
716
+ for (let i = 0; i <= n; i++) {
717
+ const p: Pt = [h[0] + (i / n) * (k[0] - h[0]), h[1] + (i / n) * (k[1] - h[1])];
718
+ const gm = isGarment(Math.round(p[0]), Math.round(p[1]));
719
+ if (gm === null) {
720
+ if (inside) {
721
+ hidden = true;
722
+ break;
723
+ }
724
+ continue;
725
+ }
726
+ if (gm) {
727
+ inside = true;
728
+ last = p;
729
+ out = 0;
730
+ } else if (inside && ++out * (len / n) >= LEAVE) {
731
+ root[s] = last;
732
+ break;
733
+ }
734
+ }
735
+ if (root[s] === null) rootWhy.push(hidden ? `the ${s} thigh's root is behind a forearm` : inside ? `the ${s} thigh does not leave the garment before its knee` : `the ${s} thigh's path carries no garment`);
736
+ }
737
+ let roots: number | null = null;
738
+ if (root.r !== null && root.l !== null) {
739
+ if (Math.abs(root.l[0] - root.r[0]) < MIN_SPAN) rootWhy.push(`the roots are ${Math.abs(root.l[0] - root.r[0]).toFixed(1)} px apart across (< ${MIN_SPAN})`);
740
+ else roots = pictureAngle(root.r, root.l);
741
+ }
742
+ if (roots === null) why.roots = rootWhy.join('; ');
743
+ // The band's top, when nothing covers it.
744
+ const edge: { fit?: { m: number; q: number } } = {};
745
+ const wb = waistband(f, c, span, x0, x1, cols, rHip, lHip, edge);
746
+ let band: number | 'covered' | null = null;
747
+ if (!('tilt' in wb) || edge.fit === undefined) why.band = 'kind' in wb ? `${wb.kind}` : 'no edge';
748
+ else {
749
+ const above: RGB[] = [];
750
+ for (let x = x0; x <= x1; x++) {
751
+ const y = Math.round(edge.fit.m * x + edge.fit.q) - 3 * (DITHER_RADIUS + 1);
752
+ if (y < 0 || y >= f.h || f.masked(x, y)) continue;
753
+ const p = mean(x, y);
754
+ if (p !== null) above.push(p);
755
+ }
756
+ const a = above.length === 0 ? null : ([0, 1, 2].map((k) => median(above.map((p) => p[k]!))) as RGB);
757
+ if (a !== null && diff(a, f.bg) <= FIGURE_DIFF) band = wb.tilt;
758
+ else {
759
+ band = 'covered';
760
+ why.band = a === null ? 'nothing seen above the band\'s top' : `the band's top is covered (${a.map((v) => v.toFixed(0)).join(', ')} lies ${3 * (DITHER_RADIUS + 1)} px above it, not the ground)`;
761
+ }
762
+ }
763
+ const hips = pictureAngle(rHip, lHip);
764
+ const tilts: CueTilts = { hem, roots, band, hips };
765
+ const detail: CueDetail = { tilts, why, garment: g, legs, hem: { pts: hemPts, fit: hemFit === null ? null : { m: hemFit.m, q: hemFit.q, rms: hemFit.rms } }, roots: { ...root, paths }, band: typeof band === 'number' ? edge.fit! : null };
766
+ const fitted: Array<[string, number]> = [];
767
+ if (hem !== null) fitted.push(['hem', hem]);
768
+ if (roots !== null) fitted.push(['roots', roots]);
769
+ if (typeof band === 'number') fitted.push(['band', band]);
770
+ const fmt = (v: number) => `${v >= 0 ? '+' : ''}${v.toFixed(1)}°`;
771
+ if (fitted.length < 2) {
772
+ const missing = (['hem', 'roots', 'band'] as const).filter((k) => why[k] !== undefined).map((k) => `${k}: ${why[k]}`);
773
+ return { reading: { kind: 'too few cues below the cover', why: `${fitted.length} of the cues read (${fitted.map(([n, v]) => `${n} ${fmt(v)}`).join(', ') || 'none'}); ${missing.join('; ')}` }, detail };
774
+ }
775
+ const base = fitted[0]![1];
776
+ const about = fitted.map(([, v]) => base + wrap(v - base));
777
+ const spread = Math.max(...about) - Math.min(...about);
778
+ if (spread > CUE_SPREAD) return { reading: { kind: 'cues disagree', why: `${fitted.map(([n, v]) => `${n} ${fmt(v)}`).join(', ')}: ${spread.toFixed(1)}° apart (> ${CUE_SPREAD}°); they do not read one line (hips ${fmt(hips)}, not fitted)` }, detail };
779
+ const tilt = wrap(about.reduce((s, v) => s + v, 0) / about.length);
780
+ return { reading: { tilt, cover: hemPts.length / cols, rms: hemFit?.rms ?? 0, mean: true, cues: tilts }, detail };
781
+ }
782
+
535
783
  /**
536
784
  * `rig-pose-pelvis-keys/1`: `{ spec, keys: [{ frame, deg }], note? }` — frames of the
537
785
  * sequence, strictly increasing, within its span [0, frames − 1]; `deg` the tilt in
@@ -686,6 +934,17 @@ export function runPelvis(a: PelvisArgs, say: (l: string) => void): void {
686
934
  const tilt = readings.map((r) => ('tilt' in r ? r.tilt : null));
687
935
  const read = tilt.filter((v): v is number => v !== null);
688
936
  say(`rig-pose pelvis: ${n} frame(s) of person "${person}" from ${a.pose}, read by ${a.mode} off ${dir} (${files.length} PNG)`);
937
+ if (a.mode === 'below-cover') {
938
+ // rig-c/pose#342: every frame's cues, read or not, so a decline or a reading can be checked cue by cue.
939
+ const fmt = (v: number | 'covered' | null) => (v === null ? 'not read' : v === 'covered' ? 'covered' : `${v >= 0 ? '+' : ''}${v.toFixed(2)}°`);
940
+ readings.forEach((r, i) => {
941
+ if (!('tilt' in r) || r.cues === undefined) return;
942
+ // The hips stay out of the vote; their sign is printed beside the fit, and an opposite sign is warned.
943
+ const sign = (v: number) => (Math.abs(v) < 0.5 ? '0' : v > 0 ? '+' : '−');
944
+ say(` CUES frame ${i}: hem ${fmt(r.cues.hem)}, roots ${fmt(r.cues.roots)}, band ${fmt(r.cues.band)}, hips ${fmt(r.cues.hips)} (not fitted) → ${fmt(r.tilt)}; hips sign ${sign(r.cues.hips)}, fit sign ${sign(r.tilt)}`);
945
+ if (sign(r.cues.hips) !== '0' && sign(r.tilt) !== '0' && sign(r.cues.hips) !== sign(r.tilt)) say(` HIPS-SIGN frame ${i}: the hip keypoints tilt ${fmt(r.cues.hips)}, opposite the fit's ${fmt(r.tilt)} (the hips are not fitted)`);
946
+ });
947
+ }
689
948
  for (const d of declinedRuns(readings)) say(` DECLINED frame${d.from === d.to ? ` ${d.from}` : `s ${d.from}–${d.to}`} (${d.to - d.from + 1}): ${d.kind} — ${d.same ? d.why : `frame ${d.from}: ${d.why}; the figures differ frame to frame`}`);
690
949
  if (read.length > 0) {
691
950
  const covers = readings.flatMap((r) => ('tilt' in r ? [r.cover] : []));
package/src/pose.ts CHANGED
@@ -80,9 +80,10 @@ export type DepthGait = (typeof DEPTH_GAITS)[number];
80
80
  /**
81
81
  * Where a sequence's `pelvisTilt` came from (rig-c/pose#288): `silhouette` — the
82
82
  * straight edge of the drawn figure across the hips; `waistband` — the top edge
83
- * of a garment covering them; `keys` — a hand file (`--pelvis-keys`).
83
+ * of a garment covering them; `below-cover` — the cues under a garment hanging
84
+ * over the waistband (rig-c/pose#342); `keys` — a hand file (`--pelvis-keys`).
84
85
  */
85
- export const PELVIS_SOURCES = ['silhouette', 'waistband', 'keys'] as const;
86
+ export const PELVIS_SOURCES = ['silhouette', 'waistband', 'below-cover', 'keys'] as const;
86
87
  export type PelvisSource = (typeof PELVIS_SOURCES)[number];
87
88
  export const POSE_SPACE = { units: 'picture-px', origin: 'top-left', y: 'down' } as const;
88
89
  /**
package/src/retarget.ts CHANGED
@@ -76,7 +76,7 @@ import type { Body18 } from './joints.ts';
76
76
  import { isBody18, VIRTUAL } from './joints.ts';
77
77
  import type { Chain, JointMap, Rest, RigSets, RigSlots } from './map.ts';
78
78
  import { aimAngle, aimEnds, bodyYawSets, descends, faceYawSets, knownSetsOf, partBone, pointParts, rigPoint } from './map.ts';
79
- import type { Joint, Travel } from './pose.ts';
79
+ import type { DepthGait, Joint, Travel } from './pose.ts';
80
80
  import { TRAVEL_IN_PLACE } from './pose.ts';
81
81
  import { Problems, Refusal } from './problems.ts';
82
82
  import type { ArtRange, AttachmentKey, ByHip, Raised, ReachShort, SetRuns, SlotTrack, Switch, SwitchOptions } from './switch.ts';
@@ -188,6 +188,35 @@ export interface RetargetOptions extends SwitchOptions {
188
188
  * frame they cover they decide the support. Absent, nothing changes.
189
189
  */
190
190
  contactKeys?: ContactKeys;
191
+ /**
192
+ * rig-c/pose#344: the gait the sequence was cleaned as (its `cleaned.params.depth.gait`, `clean --depth`). An
193
+ * in-place walk anchors the upper body (`TorsoAnchor`): the pelvis sways under it. Absent, or a run, nothing changes.
194
+ */
195
+ gait?: DepthGait;
196
+ }
197
+
198
+ /**
199
+ * rig-c/pose#344: in an in-place walk the upper body is the anchor and the pelvis sways under it. The pelvis bone
200
+ * carries the torso (its child), so its turn and its travel would swing the chest around it; the first spine bone
201
+ * (`bone`: the pelvis bone's child on the way to the trunk chain's bone) is keyed with the counter: a rotate of minus
202
+ * the pelvis's turn (none when it is the trunk bone itself, whose aim is already in the world), and a translate that
203
+ * puts its origin back where the pelvis's turn moved it from, up and down, and sideways where it carries `point`
204
+ * (the trunk chain's aim bone: the top of the trunk) on the drawn neck's own sideways path: about the world x `x`
205
+ * (the setup x plus the pelvis's mean travel), the picture's `joint` x less its mean, times `factor` — the pelvis's
206
+ * sideways sway in the rig against the drawing's (`joints[0]`, mid_hip), each its standard deviation over the frames,
207
+ * so the neck sways against the pelvis as the drawing's does; the figure's scale where the drawing's pelvis does not
208
+ * sway. The pelvis's own keys are as read; `neck` is the carried path's peak-to-peak and `sway` the first spine bone's
209
+ * sideways counter's, world units.
210
+ */
211
+ export interface TorsoAnchor {
212
+ gait: DepthGait;
213
+ bone: string;
214
+ point: string;
215
+ joint: string;
216
+ x: number;
217
+ factor: number;
218
+ neck: number;
219
+ sway: number;
191
220
  }
192
221
 
193
222
  /**
@@ -208,6 +237,8 @@ export interface PelvisReading {
208
237
  hipLine: number;
209
238
  /** The frames keyed from the hip line on a sequence that carries a track: the frames its reading declined. */
210
239
  declined: Array<[number, number]>;
240
+ /** rig-c/pose#344: the torso held over the swaying pelvis, in an in-place walk only. */
241
+ anchor?: TorsoAnchor;
211
242
  }
212
243
 
213
244
  export interface Hold {
@@ -904,6 +935,32 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
904
935
  }
905
936
  p.throwIfAny();
906
937
 
938
+ // rig-c/pose#344: an in-place walk holds the upper body over the swaying pelvis, on the first spine bone.
939
+ const anchorOf = ((): { bone: string; trunk: string; point: string; pelvis: string; joints: [string, string] } | null => {
940
+ if (opts.gait !== 'walk' || opts.travel !== TRAVEL_IN_PLACE || pelvisChain === null) return null;
941
+ const pelvisBone = pelvisChain.bones[0]!;
942
+ const trunk = map.chains.find((c) => !c.reach && c.joints[0] === 'mid_hip' && descends(rest, c.bones[0]!, pelvisBone));
943
+ if (trunk === undefined) return null;
944
+ const tb = trunk.bones[0]!;
945
+ let first = tb;
946
+ while (rest.get(first)!.parent !== pelvisBone) first = rest.get(first)!.parent!;
947
+ // The aim's upper end (aimEnds: `neck` and `chest>neck` alike end at the neck), a rig point on the trunk bone or below it.
948
+ const aim = trunk.aim[0]!;
949
+ const top = aim === 'x' ? null : aimEnds(tb, aim)[1];
950
+ const onTrunk = top !== null && pointParts(top).every((part) => {
951
+ const b = partBone(part);
952
+ return rest.has(b) && (b === tb || descends(rest, b, tb));
953
+ });
954
+ const point = onTrunk ? top! : tb;
955
+ return { bone: first, trunk: tb, point, pelvis: pelvisBone, joints: [trunk.joints[0]!, trunk.joints[1]!] };
956
+ })();
957
+ if (anchorOf !== null && anchorOf.bone !== anchorOf.trunk && !targets.has(anchorOf.bone)) {
958
+ // Not in a chain: its world turn is held at setup, so the bones below it are keyed against none.
959
+ const a0 = restAngle(rest, anchorOf.bone, 'x');
960
+ restOf.set(anchorOf.bone, a0);
961
+ targets.set(anchorOf.bone, Array.from({ length: n }, () => a0));
962
+ }
963
+
907
964
  // The short hold, on every segment the pose aims directly (not a reach): its
908
965
  // picture length against its own median, on the points the angle was read from.
909
966
  // A horizontal line across the body (`isAcross`: pelvis, shoulders, head line)
@@ -1509,7 +1566,43 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
1509
1566
  const r = turnAbove(parent, i) / DEG;
1510
1567
  const ox = own.worldX - rest.get(parent)!.worldX;
1511
1568
  const oy = own.worldY - rest.get(parent)!.worldY;
1512
- return [po[0] + Math.cos(r) * ox - Math.sin(r) * oy + nod[0], po[1] + Math.sin(r) * ox + Math.cos(r) * oy + nod[1]];
1569
+ const at: Pt = [po[0] + Math.cos(r) * ox - Math.sin(r) * oy + nod[0], po[1] + Math.sin(r) * ox + Math.cos(r) * oy + nod[1]];
1570
+ if (anchorOf === null || b !== anchorOf.bone || anchorRaw) return at;
1571
+ const s = anchorShift(i, at, po, oy);
1572
+ return [at[0] + s[0], at[1] + s[1]];
1573
+ };
1574
+ /**
1575
+ * rig-c/pose#344: the world shift that holds the first spine bone over the pelvis: up and down, its origin where
1576
+ * the pelvis's travel alone puts it (the pelvis's turn taken out); sideways, the trunk's top point on `anchorPath`:
1577
+ * the drawn neck's sideways path about its mean, scaled by the pelvis's rig ÷ drawn sideways sway (`TorsoAnchor`).
1578
+ */
1579
+ let anchorRaw = false;
1580
+ let anchorX: number | null = null;
1581
+ let anchorFactor = 1;
1582
+ let anchorPath: number[] | null = null;
1583
+ const anchorShift = (i: number, at: Pt, po: Pt, oy: number): Pt => {
1584
+ const a = anchorOf!;
1585
+ if (anchorPath === null) {
1586
+ anchorRaw = true;
1587
+ const hipX = Array.from({ length: n }, (_, k) => posedOrigin(a.pelvis, k)[0] - rest.get(a.pelvis)!.worldX);
1588
+ anchorRaw = false;
1589
+ const mean = (xs: readonly number[]) => xs.reduce((s, x) => s + x, 0) / xs.length;
1590
+ const sd = (xs: readonly number[]) => {
1591
+ const m = mean(xs);
1592
+ return Math.sqrt(xs.reduce((s, x) => s + (x - m) ** 2, 0) / xs.length);
1593
+ };
1594
+ const drawnHip = series.get(a.joints[0])!.map((p) => p[0]);
1595
+ const drawnNeck = series.get(a.joints[1])!.map((p) => p[0]);
1596
+ const sdHip = sd(drawnHip);
1597
+ anchorFactor = sdHip > 1e-6 ? sd(hipX) / sdHip : (scale?.factor ?? 1);
1598
+ anchorX = rigPoint(rest, a.point)![0] + mean(hipX);
1599
+ const m = mean(drawnNeck);
1600
+ anchorPath = drawnNeck.map((x) => anchorX! + (x - m) * anchorFactor);
1601
+ }
1602
+ anchorRaw = true;
1603
+ const top = posedPoint(a.point, i);
1604
+ anchorRaw = false;
1605
+ return [anchorPath[i]! - top[0], po[1] + oy - at[1]];
1513
1606
  };
1514
1607
  /** A rig point (a bone, `name:tip`, `a+b`) posed on frame i the same way. */
1515
1608
  const posedPoint = (spec: string, i: number): Pt => {
@@ -2999,6 +3092,42 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
2999
3092
  }
3000
3093
  }
3001
3094
  }
3095
+ // rig-c/pose#344: the first spine bone's counter — its rotate when it is not the trunk bone, and its translate.
3096
+ let anchor: TorsoAnchor | null = null;
3097
+ if (anchorOf !== null) {
3098
+ const a = anchorOf;
3099
+ if (a.bone !== a.trunk) {
3100
+ const above = mappedAbove(a.bone);
3101
+ let prev = 0;
3102
+ const keys: MotionKey[] = [];
3103
+ for (let i = 0; i < n; i++) {
3104
+ const raw = turn(a.bone, i) - (above === null ? 0 : turn(above, i));
3105
+ prev = i === 0 ? wrap(raw) : prev + wrap(raw - prev);
3106
+ keys.push({ t: keyTimes[i]!, v: [r3(prev)] });
3107
+ }
3108
+ tracks.push({ bone: a.bone, property: 'rotate', keys });
3109
+ }
3110
+ const pr = rest.get(a.pelvis)!;
3111
+ const det = pr.a * pr.d - pr.b * pr.c;
3112
+ const keys: MotionKey[] = [];
3113
+ const xs: number[] = [];
3114
+ for (let i = 0; i < n; i++) {
3115
+ const held = posedOrigin(a.bone, i);
3116
+ anchorRaw = true;
3117
+ const free = posedOrigin(a.bone, i);
3118
+ anchorRaw = false;
3119
+ const wx = held[0] - free[0];
3120
+ const wy = held[1] - free[1];
3121
+ xs.push(wx);
3122
+ // Into the pelvis's turned frame, then through its setup matrix: the translate is in the parent's local axes.
3123
+ const r = -turnAbove(a.pelvis, i) / DEG;
3124
+ const ux = Math.cos(r) * wx - Math.sin(r) * wy;
3125
+ const uy = Math.sin(r) * wx + Math.cos(r) * wy;
3126
+ keys.push({ t: keyTimes[i]!, v: [r3((pr.d * ux - pr.b * uy) / det), r3((-pr.c * ux + pr.a * uy) / det)] });
3127
+ }
3128
+ tracks.push({ bone: a.bone, property: 'translate', keys });
3129
+ anchor = { gait: 'walk', bone: a.bone, point: a.point, joint: a.joints[1], x: r3(anchorX!), factor: r6(anchorFactor), neck: r3(Math.max(...anchorPath!) - Math.min(...anchorPath!)), sway: r3(Math.max(...xs) - Math.min(...xs)) };
3130
+ }
3002
3131
  let face: FaceReading | null = null;
3003
3132
  if (map.face !== undefined) {
3004
3133
  const rEar = series.get('r_ear') ?? (() => {
@@ -3136,6 +3265,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
3136
3265
 
3137
3266
  const pivot = pivotOf(rest, t?.bone ?? null, legs);
3138
3267
  const pelvis = pelvisReadingOf(pelvisChain, opts.pelvis, track, fromTrack, tracks);
3268
+ if (pelvis !== null && anchor !== null) pelvis.anchor = anchor;
3139
3269
  const scaleFit = t === null || scale === null || origin === null ? null : scaleFitOf({
3140
3270
  joints: t.scale.pose, joint: t.joint, bone: t.bone, lengths: scaleLengths, factor: scale.factor,
3141
3271
  series, good, legLength: feet?.legLength ?? null, legStraight, rigLeg,
@@ -3187,10 +3317,16 @@ export function pelvisLine(r: PelvisReading, frames: number): string {
3187
3317
  const where = r.bone === null ? '' : `bone "${r.bone}", chain "${r.chain}"`;
3188
3318
  if (r.bone === null) return `PELVIS: not keyed — the sequence carries a ${r.trackSource} track (${frames - r.declined.reduce((s, [a, b]) => s + b - a + 1, 0)} of ${frames} frames read) and the map has no pelvis chain (one bone, joints r_hip → l_hip); no bone turns with the pelvis`;
3189
3319
  const swing = r.swing === null ? '' : `, swing ${r.swing.toFixed(2)}°`;
3190
- if (r.trackSource === null) return `PELVIS: from hip line (no track)${swing} (${where}; every frame from r_hip → l_hip)`;
3191
- if (r.track === 0) return `PELVIS: from hip line (the ${r.trackSource} track read none of the ${frames} frames)${swing} (${where}; every frame from r_hip → l_hip)`;
3320
+ if (r.trackSource === null) return `PELVIS: from hip line (no track)${swing} (${where}; every frame from r_hip → l_hip)${anchorClause(r.anchor)}`;
3321
+ if (r.track === 0) return `PELVIS: from hip line (the ${r.trackSource} track read none of the ${frames} frames)${swing} (${where}; every frame from r_hip → l_hip)${anchorClause(r.anchor)}`;
3192
3322
  const rest = r.hipLine === 0 ? `all ${frames} frames from the track` : `${r.track} of ${frames} frames from the track, the hip line on the ${r.hipLine} it declined (frames ${spans(r.declined)})`;
3193
- return `PELVIS: from ${r.source}${swing} (${where}; ${rest})`;
3323
+ return `PELVIS: from ${r.source}${swing} (${where}; ${rest})${anchorClause(r.anchor)}`;
3324
+ }
3325
+
3326
+ /** rig-c/pose#344: the PELVIS line's clause for the torso held over the pelvis. */
3327
+ function anchorClause(a: TorsoAnchor | undefined): string {
3328
+ if (a === undefined) return '';
3329
+ return `; torso anchored (in-place ${a.gait}): bone "${a.bone}" keyed with the pelvis's counter, "${a.point}" carried on the drawn ${a.joint}'s sideways path about x ${a.x.toFixed(2)} (× ${a.factor.toFixed(4)}, the pelvis's rig ÷ drawn sway; ${a.neck.toFixed(2)} peak to peak, the counter ${a.sway.toFixed(2)}), the pelvis swaying under it`;
3194
3330
  }
3195
3331
 
3196
3332
  /** A reach bone's origin within this many world units of the anchor's, with nothing above it moved, is aimed along the pair: no REACH line. */
package/src/run.ts CHANGED
@@ -168,7 +168,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
168
168
  const slots = slotsOf(doc0);
169
169
  resolveMap(map, rest, a.map, slots, input.sets);
170
170
 
171
- const r = retarget(rest, map, frames, times, { ...a, ...(seq.travel !== undefined ? { travel: seq.travel } : {}), ...(seq.cleaned!.params.loop ? { loop: true } : {}), ...(seq.pelvisSource !== undefined ? { pelvis: { source: seq.pelvisSource, tilt: seq.frames.map((f) => f.pelvisTilt ?? null) } } : {}), ...(contactKeys !== null ? { contactKeys } : {}) }, slots, input.sets);
171
+ const r = retarget(rest, map, frames, times, { ...a, ...(seq.travel !== undefined ? { travel: seq.travel } : {}), ...(seq.cleaned!.params.loop ? { loop: true } : {}), ...(seq.pelvisSource !== undefined ? { pelvis: { source: seq.pelvisSource, tilt: seq.frames.map((f) => f.pelvisTilt ?? null) } } : {}), ...(contactKeys !== null ? { contactKeys } : {}), ...(seq.cleaned!.params.depth !== undefined ? { gait: seq.cleaned!.params.depth.gait } : {}) }, slots, input.sets);
172
172
  if (fragment.length > 0 && ((r.face?.sets?.length ?? 0) > 0 || (r.body?.sets?.length ?? 0) > 0)) {
173
173
  const known = knownSetsOf(map, input.sets);
174
174
  for (const s of [...(r.face?.sets ?? []), ...(r.body?.sets ?? [])]) {