@rig-c/pose 0.42.0 → 0.43.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -7
- package/cli.ts +6 -2
- package/package.json +1 -1
- package/src/pelvis.ts +263 -4
- package/src/pose.ts +3 -2
- package/src/retarget.ts +141 -5
- package/src/run.ts +1 -1
package/README.md
CHANGED
|
@@ -104,7 +104,7 @@ A retarget of the cleaned `dance-a` fixture finds the same jumps (`FEET`) as the
|
|
|
104
104
|
## The pelvis track: `rig-pose pelvis`
|
|
105
105
|
|
|
106
106
|
```sh
|
|
107
|
-
rig-pose pelvis --pose dance.scan.json --frames frames/ --out dance.pelvis.json [--mode silhouette|waistband] [--person <id>]
|
|
107
|
+
rig-pose pelvis --pose dance.scan.json --frames frames/ --out dance.pelvis.json [--mode silhouette|waistband|below-cover] [--person <id>]
|
|
108
108
|
rig-pose pelvis --pose dance.scan.json --pelvis-keys pelvis.keys.json --out dance.pelvis.json
|
|
109
109
|
```
|
|
110
110
|
|
|
@@ -114,12 +114,13 @@ A pose detector keeps little of a drawn pelvis tilt ([#288](https://github.com/r
|
|
|
114
114
|
|
|
115
115
|
- **`silhouette`** (the default): the straightest edge of the drawn figure across the hip span, near the hips' height. An edge pixel is a figure pixel (a channel more than 64 from the background colour, the median of the frame's border) beside a background pixel, so a drawn outline and a filled shape both have one. Every straight line within ±40° of level and one hip width of the hips is scored by the edge pixels within 1 px of it; the best is read when its pixels cover at least 60 % of the span's columns, and fitted by least squares (one 3-MAD rejection pass) within 2.5 px rms. It reads a drawn pelvis line, or the edge of a drawn hip.
|
|
116
116
|
- **`waistband`**: the top edge of a garment covering the hips (a skirt, shorts). The garment's colour is the median at the hips' midpoint, and must not be the background's. In each column of the span, the first row from 1.5 hip widths above the hips where three pixels in a row carry it (within 40 per channel) is the top edge, and the line is fitted through those tops on the same terms. The colour must **hold** on its patch: at least 75 % of the patch's pixels carry its median ([#293](https://github.com/rig-c/pose/issues/293)). A dithered or paletted garment (a 216-colour GIF, whose channels alternate between palette steps pixel by pixel) does not, so its pixels are read through the mean of a 5×5 window, which does. Hips on the band's top edge give a patch that is half garment and half ground, so the patch is also tried one and two patch radii (0.3 hip widths) below the hips, and the scan then runs down to that patch. The first patch that holds and is not the background gives the colour, trying each height as is and then through the window mean. If none holds, the midpoint's median is read as before. The `PELVIS` line counts the frames read through the window mean and those whose colour was found below the hips.
|
|
117
|
+
- **`below-cover`** ([#342](https://github.com/rig-c/pose/issues/342)): for a garment whose top is hidden under another — a sweater's hem lying over a skirt's waistband hangs level whatever the pelvis does, so the band-top reader reads the cover. It reads what the cover cannot hide, every colour through the 5×5 window mean, a pixel being garment when it is nearer the garment's colour than each leg's (sampled 0.7 of the way hip → knee) and the ground's: the **hem** (in each column of the span, the last garment row before three that are not; a line fitted through them), the **thigh roots** (along each hip → knee, where the garment stops for 3 px; the line from the right root to the left), and the waistband's band top only when the ground lies above it. The tilt is the mean of the cues; fewer than two, or two more than 8° apart, decline the frame by name (`too few cues below the cover`, `cues disagree`). A `CUES` line per frame prints every cue and the hip keypoints' line and sign beside the fit's; the hips are reported, not fitted (a detector's hips keep a fraction of a drawn tilt), and a `HIPS-SIGN` line names a frame where their sign is opposite the fit's. On the synthetic walk with shorts turned with the drawn pelvis under a level sweater (`sweater`), every frame reads within 1.5° of the drawn tilt; the `waistband` reader reads the sweater's level hem there. **Known limit:** a thigh drawn in front of the garment (raised toward the camera) is read as the hem — its outline over the cloth looks like the hem's edge, and the hem and that thigh's root agree on it — so those frames must be declined by hand (set their `pelvisTilt` to null, or leave them out of `--pelvis-keys`). Reading them is [#349](https://github.com/rig-c/pose/issues/349).
|
|
117
118
|
|
|
118
119
|
Both pass over the forearms: pixels within 0.15 hip widths of a segment elbow → wrist (1.5 times that of a wrist, the hand) are not read, and a waistband top right under a forearm is not taken. A frame neither reader can read is **declined**: its `pelvisTilt` is `null` — marked, not guessed — and a `DECLINED` line names the frames and the reason (the pelvis blank, no garment, no straight edge across 60 % of the span, the hips side-on or missing). The `PELVIS` line counts the frames read and declined and gives the swing read.
|
|
119
120
|
|
|
120
121
|
`--pelvis-keys` is the hand override where neither reader applies: `rig-pose-pelvis-keys/1`, `{ "spec": "rig-pose-pelvis-keys/1", "keys": [ { "frame", "deg" } ], "note"? }`. Frames are the sequence's and strictly increase, `deg` is in `pelvisTilt`'s sense; the tilt is linear between keys, the short way round, and held before the first and after the last. A frame may be fractional ([#294](https://github.com/rig-c/pose/issues/294)): a scan interpolated from drawn keys puts them between its frames — k × 4.75 for 8 keys over 38 frames — and a key rounded onto the grid moves up to half a frame and bends the tilt between. On that synthetic case (24 fps, a known tilt sampled at the keys' times), the tilt written on every frame is the keys' linear interpolation within 0.0005° (the writer's rounding), where the same keys rounded to whole frames miss it by up to 1.19° (`test/pelvis.test.ts`). A key outside the sequence's span, `[0, frames − 1]`, is refused by name. Fractional frames stay `/1`: they are a wider value of the same key, and a reader before #294 refuses them by name (`"frame" is number 4.75; it must be a frame index in [0, N]`) rather than misreading them. Anything malformed is refused by name (`PELVIS_KEYS`), and the source is `"keys"`. `--person` picks the person when there is more than one, and only that person is written.
|
|
121
122
|
|
|
122
|
-
**The track** is `pelvisTilt` on every frame and `pelvisSource` (`"silhouette"`, `"waistband"` or `"keys"`) on the sequence, written as `rig-pose-sequence/1.1` ([below](#the-pose-rig-pose-sequence1)). `pelvisTilt` is Spine degrees — counter-clockwise, y up — of the direction from the figure's right hip to its left, the angle `retarget` reads off `r_hip → l_hip`: about 0 for a figure facing the camera, about ±180 facing away.
|
|
123
|
+
**The track** is `pelvisTilt` on every frame and `pelvisSource` (`"silhouette"`, `"waistband"`, `"below-cover"` or `"keys"`) on the sequence, written as `rig-pose-sequence/1.1` ([below](#the-pose-rig-pose-sequence1)). `pelvisTilt` is Spine degrees — counter-clockwise, y up — of the direction from the figure's right hip to its left, the angle `retarget` reads off `r_hip → l_hip`: about 0 for a figure facing the camera, about ±180 facing away.
|
|
123
124
|
|
|
124
125
|
**`clean`** smooths `pelvisTilt` as its own track, not re-read from the hips: the quadratic Savitzky–Golay over the legs' window (`--legs`, the hips' window), circular with `--loop`, on the frames that carry a reading — a `null` stays `null`. It prints a `PELVIS` line with the largest change, and records `"pelvis": { "window" }` in the record's `params` (the `CLEAN` line ends `pelvisTilt smoothed as its own track over N frames`). A sequence with a track whose record lacks it — the track added after `clean` — is refused by name (`POSE_PELVIS`).
|
|
125
126
|
|
|
@@ -168,15 +169,15 @@ With `--draw-order depth` (off by default; without it nothing changes) `retarget
|
|
|
168
169
|
|
|
169
170
|
- **The pair.** The legs: a leg is the map's chains naming its knee or ankle, their bones and the bones under them no other chain maps. Its slots are the slots on those bones. A slot of an alternate set (`alternates`) on none of them goes with the leg the set depicts, so a set keeps its own switch and takes the order of the leg it replaces. The legs are keyed when their slots overlap in the draw order, filling one unbroken run of the setup order. A slot of anything else drawn between them would be re-layered by the swap, and the setup order is not changed here. The pair is then not keyed, and the line names that slot. The arms across the trunk are a later card.
|
|
170
171
|
- **The rule.** On each frame the leg whose depth is the lower is drawn behind: its slots first in the pair's run, in their setup order, then the other's. The depths must differ by at least a tie, the depth a flat chain can hold there. Inside the tie, or where a leg is not ordered, the order before holds. Before the first decided frame it is the setup order, or on a looped sequence the order the clip ends in. A key is written where the order changes.
|
|
171
|
-
- **The joint the gait names** ([#336](https://github.com/rig-c/pose/issues/336), the owner's two statements). On a sequence cleaned with `--depth walk` a leg's depth is its ankle's (the hip at 0, plus the thigh's, plus the shin's), the tie √(1 − 0.95²) × (thigh + shin) of the rest lengths (46.83 px on the fixtures): in a walk the lifted foot is behind the supporting leg. With `--depth run` it is its knee's (the thigh's), the tie √(1 − 0.95²) × thigh (24.
|
|
172
|
+
- **The joint the gait names** ([#336](https://github.com/rig-c/pose/issues/336), the owner's two statements). On a sequence cleaned with `--depth walk` a leg's depth is its ankle's (the hip at 0, plus the thigh's, plus the shin's), the tie √(1 − 0.95²) × (thigh + shin) of the rest lengths (46.83 px on the fixtures): in a walk the lifted foot is behind the supporting leg. With `--depth run` it is its knee's (the thigh's), the tie √(1 − 0.95²) × thigh (24.96 px): in a run the lifted leg is in front, its knee protruding. The limb mean (`layers`, above) says neither: a lifted leg's thigh forward and shin back cancel in it, and a walk whose thighs stay inside the band leaves it at most half a shin (35 px on the fixtures), under its tie of √(1 − 0.95²) × (thigh + shin ÷ 2) (35.89 px). A measured depth names no gait and keeps the limb mean and its tie ([#304](https://github.com/rig-c/pose/issues/304)).
|
|
172
173
|
- **No depth.** On a sequence without depth the flag is refused (`DRAWORDER_NO_DEPTH`): the order would come from nowhere, and a motion written without it would read as ordered.
|
|
173
174
|
- **A walk cleaned before #336** ([#336](https://github.com/rig-c/pose/issues/336)). A derived depth whose foreshortened shin leans the way the trunk faces carries the sign table before #336, where a walk's lifted foot read forward; the ankle would draw that foot in front. The flag refuses it by name (`DEPTH_RECLEAN`), listing the frames, until it is cleaned again with this version. The table since leans the shin back in both gaits, so a record it wrote never matches. A run's signs did not change, so a run cleaned by an earlier release is read as it is, and so is a walk with no foreshortened shin (its signs read the same under both tables). Measured depth is not checked. The file format does not change.
|
|
174
175
|
|
|
175
176
|
On #281's fixtures, retargeted onto a limb-cut rig whose legs overlap (the left leg drawn first; `test/draworder.test.ts`):
|
|
176
177
|
|
|
177
178
|
- **The walk** (by the ankle): each of the 36 frames with one leg lifted draws that leg behind; the order changes on frames 3, 15, 27 and 39, where a lifted foot first goes back past the tie, and holds through the double support between. The generator orders the true ankles on 16 of the 48 frames, and the keyed order agrees on all 16.
|
|
178
|
-
- **The run** (by the knee): the order changes on frames
|
|
179
|
-
- **Read otherwise**: by the limb mean or by the ankle, the run draws the lifted leg behind on
|
|
179
|
+
- **The run** (by the knee): the order changes on frames 0, 10, … 50, where the other foot lands. The generator's swing thigh flexes forward from toe-off ([#343](https://github.com/rig-c/pose/issues/343)), so on each of the 42 frames with one leg lifted the lifted knee is in front of the supporting one, its thigh leaning forward past the band, and that leg is drawn in front. The generator orders the true knees on 42 of the 60 frames, and the keyed order agrees on all 42. On frame 12, on a rig that draws the right leg first, none of the 8,771 pixels the lifted right leg draws shows the standing leg, against 2,205 in the setup order.
|
|
180
|
+
- **Read otherwise**: by the limb mean or by the ankle, the run draws the lifted leg behind on 21 of its 42 lifted frames; by the limb mean or by the knee, the walk keys nothing (the setup order) and draws the lifted right leg in front on its 18 frames. Knees planted 10 px apart, inside the knee tie, hold the order before; without the tie they swap it.
|
|
180
181
|
|
|
181
182
|
## What it does
|
|
182
183
|
|
|
@@ -228,7 +229,7 @@ and what the run cannot decide for you it lists:
|
|
|
228
229
|
- `REACH` — with `--raised-by-head`, or a `"by-hip"` set: a placed span whose target was out of the arm's reach, too far or too near. It names the frames, which way, and the shortfall in eye distances (hip widths for `"by-hip"`). The wrist was placed at the reach limit instead. This is not `ART-RANGE`: the rig's proportions were the limit, not the art's range. A too-near frame is placed at the limit like a too-far one, and graded at the placed distance.
|
|
229
230
|
- `LEG-SCALE` — with `--leg-scale` ([below](#the-legs-foreshortening-and-planted-feet)): per leg segment, its straight length in px and as a share of `FEET`'s leg length, the spread of its picture length over it, the limits in px, the frames clamped at each limit, where its leg was free on clamped frames how far the clamps on that leg left the free ankle above or below the picture's (in the rig's leg at setup, thigh origin → ankle, and in `FEET`'s px; `legScale[].free` in the report, [#237](https://github.com/rig-c/pose/issues/237)), and (with `--plant-feet`) one `past the floor (support)` line per supporting run that asked under the floor ([#133](https://github.com/rig-c/pose/issues/133), `legScale[].support` in the report).
|
|
230
231
|
- `DEPTH` — when the sequence carries depth ([above](#derived-depth-clean---depth)): where it came from, on how many frames the limbs were ordered front to back, and which limb was in front alone how often — `DEPTH layers from the sequence's derived depth (clean --depth run): the limbs ordered front to back on N of N frame(s); in front alone: r_arm a, l_arm b, r_leg c, l_leg d; tied in front on k (the report's layers)`.
|
|
231
|
-
- `DRAWORDER` — with `--draw-order depth` ([above](#draw-order-from-depth---draw-order-depth)): the keys and the spans they key, per leg drawn behind, cut to 260 characters — `DRAWORDER from depth by the knee (run; tie 24.
|
|
232
|
+
- `DRAWORDER` — with `--draw-order depth` ([above](#draw-order-from-depth---draw-order-depth)): the keys and the spans they key, per leg drawn behind, cut to 260 characters — `DRAWORDER from depth by the knee (run; tie 24.96 px), 6 key(s) over 3 span(s) (leg_r behind leg_l 0–9, 20–29, 40–49)`, naming the joint and its tie for a keyed pair ([#336](https://github.com/rig-c/pose/issues/336)) — and a line per pair not keyed, saying why.
|
|
232
233
|
- `PELVIS` — when the map has a pelvis chain (one bone, joints `r_hip → l_hip`) or the sequence a pelvis track ([above](#the-pelvis-track-rig-pose-pelvis)): where the pelvis bone's rotation came from and its swing — `PELVIS: from <source>, swing N°`, or `PELVIS: from hip line (no track)` — with the frames keyed from the hip line on a track that declined them.
|
|
233
234
|
- `PIVOT` — when the map translates a bone and has both legs: the translated bone's origin against its thigh roots' midpoint at setup, in units and hip widths — the pivot every hip turn swings the legs about.
|
|
234
235
|
- `FEET` — when the map has both legs: one `segment` line per mapped leg segment with its picture length ÷ straight length (min · median · max), the scale `--leg-scale` would ask before its clamp, printed with or without the flag so the minimum can be read before choosing the range ([#237](https://github.com/rig-c/pose/issues/237)); the supports and jumps read off the reference ankles (planted or not), and on the posed build, outside the jumps, the lowest ankle's vertical range and each support span's worst vertical range and horizontal drift of the supporting ankle, in leg lengths.
|
|
@@ -427,7 +428,7 @@ Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2
|
|
|
427
428
|
- Joints are body-18 (`nose neck r_shoulder r_elbow r_wrist l_shoulder l_elbow l_wrist r_hip r_knee r_ankle l_hip l_knee l_ankle r_eye l_eye r_ear l_ear`); `r`/`l` are the **subject's** sides. `state` is `observed`, `occluded` or `missing`, as in the keypoints file, or `inferred` ([#63](https://github.com/rig-c/pose/issues/63)): a position the detector supplied but did not see. An `inferred` joint carries `at` (one without it is refused, `POSE_JOINT_AT`), and `retarget` keys it as it keys `observed`. An unlisted joint is missing.
|
|
428
429
|
- Two differences from the keypoints file: the unit is `picture-px`, and `at` may lie outside the picture (a hand leaves the frame; the score says how much to believe it).
|
|
429
430
|
- More than one person: `--person <id>`.
|
|
430
|
-
- **The pelvis track** ([#288](https://github.com/rig-c/pose/issues/288)) is the minor `rig-pose-sequence/1.1`: a `pelvisTilt` on every frame (beside `t` and `people`; Spine degrees in (−180, 180] of the direction r_hip → l_hip, or `null` where the reading declined the frame) and `pelvisSource` on the sequence (`"silhouette"`, `"waistband"` or `"keys"`), written by `rig-pose pelvis` ([above](#the-pelvis-track-rig-pose-pelvis)). A sequence carrying them holds one person; a track on some frames only, a source without a track, or a track without a source is refused by name (`POSE_PELVIS`). The writer writes the minor the content needs — `/1.1` with a track, `/1` without — so a sequence without one stays readable by a reader of `/1`; a `/1` file carrying a 1.1 key is refused naming the minor (`POSE_SPEC`). The spec string is what the file records of the version that wrote it: a file of a newer minor than this reader's is read, and each key it does not know is refused with the minor the file says it was written as (`… the file says it was written as "rig-pose-sequence/1.3", a newer minor than this reader's "rig-pose-sequence/1.2" — read it with a rig-pose that reads that minor`). A reader of `/1` (rig-pose 0.37 and before) refuses a `/1.1` file by its spec (`POSE_SPEC`, naming `"rig-pose-sequence/1.1"`) and its keys (`POSE_KEYS`).
|
|
431
|
+
- **The pelvis track** ([#288](https://github.com/rig-c/pose/issues/288)) is the minor `rig-pose-sequence/1.1`: a `pelvisTilt` on every frame (beside `t` and `people`; Spine degrees in (−180, 180] of the direction r_hip → l_hip, or `null` where the reading declined the frame) and `pelvisSource` on the sequence (`"silhouette"`, `"waistband"`, `"below-cover"` or `"keys"`), written by `rig-pose pelvis` ([above](#the-pelvis-track-rig-pose-pelvis)). A sequence carrying them holds one person; a track on some frames only, a source without a track, or a track without a source is refused by name (`POSE_PELVIS`). The writer writes the minor the content needs — `/1.1` with a track, `/1` without — so a sequence without one stays readable by a reader of `/1`; a `/1` file carrying a 1.1 key is refused naming the minor (`POSE_SPEC`). The spec string is what the file records of the version that wrote it: a file of a newer minor than this reader's is read, and each key it does not know is refused with the minor the file says it was written as (`… the file says it was written as "rig-pose-sequence/1.3", a newer minor than this reader's "rig-pose-sequence/1.2" — read it with a rig-pose that reads that minor`). A reader of `/1` (rig-pose 0.37 and before) refuses a `/1.1` file by its spec (`POSE_SPEC`, naming `"rig-pose-sequence/1.1"`) and its keys (`POSE_KEYS`).
|
|
431
432
|
- **Depth** ([#281](https://github.com/rig-c/pose/issues/281)) is the minor `rig-pose-sequence/1.2`: a `depth` on every frame (beside `t` and `people`), `{ "<segment>": { "ratio", "sign", "source" } }` — the segments `r_upper_arm`, `r_forearm`, `r_thigh`, `r_shin`, `l_upper_arm`, `l_forearm`, `l_thigh`, `l_shin`, each present where its two joints are placed on the frame (`{}` where none is); `ratio` a number ≥ 0, `sign` −1, 0 or 1, `source` `"derived"` (`clean --depth`, [above](#derived-depth-clean---depth)) or `"measured"` (a 3D detector). A minor includes the keys of the minors before it, so a sequence with depth and a pelvis track is `/1.2`, and one with the track alone stays `/1.1`. A sequence with depth holds one person; depth on some frames only, a malformed entry, an unknown segment, derived and measured depth together, and derived depth without the `cleaned` record's `"depth": { "gait" }` (or that record without derived depth on every frame) are refused by name (`POSE_DEPTH`); a `/1` or `/1.1` file carrying `depth` is refused naming the minor (`POSE_SPEC`).
|
|
432
433
|
- `travel` (optional, [#92](https://github.com/rig-c/pose/issues/92)) is the clip's frame of reference. Its one value is `"in-place"`: a gait filmed without the figure travelling, such as a treadmill or reel walk, or a loop. `clean --loop` writes it, and it may be written by hand on any sequence. Any other value is refused by name (`POSE_TRAVEL`). Without it the figure's travel in the picture is its travel. It is a property of the sequence, not of a flag ([#242](https://github.com/rig-c/pose/issues/242)): with or without `--plant-feet`, retarget reads the supports in place, skips the camera-zoom normalisations (a loop has no zoom, and their rolling windows do not wrap) and reads the head nod without its detrend window or clamp. Only the following foot and the loop closure need `--plant-feet` ([below](#the-legs-foreshortening-and-planted-feet)).
|
|
433
434
|
- `score` is whatever the pose model gives. RTMPose's goes above 1, so `--min-score` is a cut on that model's scale, not a probability.
|
package/cli.ts
CHANGED
|
@@ -22,7 +22,7 @@ import { runRetarget, type RetargetArgs } from './src/run.ts';
|
|
|
22
22
|
const HELP = `rig-pose ${pkg.version} — motion from a pose sequence onto a rig-c rig
|
|
23
23
|
|
|
24
24
|
Usage:
|
|
25
|
-
rig-pose pelvis --pose <pose.json> --out <pose.pelvis.json> (--frames <dir> [--mode silhouette|waistband] | --pelvis-keys <keys.json>) [--person <id>]
|
|
25
|
+
rig-pose pelvis --pose <pose.json> --out <pose.pelvis.json> (--frames <dir> [--mode silhouette|waistband|below-cover] | --pelvis-keys <keys.json>) [--person <id>]
|
|
26
26
|
rig-pose clean --pose <pose.json> --out <pose.clean.json> [--loop] [--legs 5] [--rest 7] [--hold off] [--depth walk|run] [--coco] [--person <id>]
|
|
27
27
|
rig-pose retarget --rig <rig> --map <map.json> --sequence <pose.json> --out <motion.json> [--animation dance] [flags]
|
|
28
28
|
rig-pose convert --coco <coco17.json> --out <pose.json>
|
|
@@ -38,11 +38,15 @@ pelvis (before clean; rig-c/pose#288)
|
|
|
38
38
|
r_hip → l_hip, about 0 facing the camera — or null on a frame the reading declined
|
|
39
39
|
--frames <dir> the picture: one PNG per sequence frame, in file-name order, the sequence's size (no video;
|
|
40
40
|
extract the frames first). The hip keypoints place the reading; their angle is not read
|
|
41
|
-
--mode silhouette|waistband
|
|
41
|
+
--mode silhouette|waistband|below-cover
|
|
42
42
|
silhouette (default): the straightest edge of the drawn figure across the hip span (between
|
|
43
43
|
the two hip keypoints), within 40° of level and one hip width of the hips' height — a drawn
|
|
44
44
|
pelvis line or the edge of a drawn hip. waistband: the top edge of a garment covering the hips
|
|
45
45
|
(skirt, shorts) — the colour at the hips' midpoint, its first row in each column of the span.
|
|
46
|
+
below-cover: for a garment whose top is hidden under another (a sweater's hem over a skirt)
|
|
47
|
+
— the bottomwear's lower hem across the span, where each thigh leaves it, and the band's top
|
|
48
|
+
only where nothing covers it; the mean of those cues, declined when they differ by more
|
|
49
|
+
than 8° (a CUES line per frame; the hip keypoints are printed there, not fitted).
|
|
46
50
|
Forearms (elbow → wrist) and hands are masked. A frame either cannot read (the pelvis blank,
|
|
47
51
|
no garment, no straight edge over 60% of the span) is declined, named on a DECLINED line,
|
|
48
52
|
and written null: marked, not guessed
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@rig-c/pose",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.43.0",
|
|
4
4
|
"description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/src/pelvis.ts
CHANGED
|
@@ -59,7 +59,7 @@ type Pt = [number, number];
|
|
|
59
59
|
type RGB = [number, number, number];
|
|
60
60
|
|
|
61
61
|
/** The two readers. */
|
|
62
|
-
export const PELVIS_MODES = ['silhouette', 'waistband'] as const;
|
|
62
|
+
export const PELVIS_MODES = ['silhouette', 'waistband', 'below-cover'] as const;
|
|
63
63
|
export type PelvisMode = (typeof PELVIS_MODES)[number];
|
|
64
64
|
export const PELVIS_KEYS_SPEC = 'rig-pose-pelvis-keys/1';
|
|
65
65
|
|
|
@@ -85,13 +85,41 @@ export const MIN_SPAN = 6;
|
|
|
85
85
|
export const CLIMB_SPREAD = 2;
|
|
86
86
|
/** The band's top is climbed against at most this many garment colours (rig-c/pose#318): a paletted garment's own, or a painted one's fullest cells. */
|
|
87
87
|
export const MAX_CLOTH = 16;
|
|
88
|
+
/** rig-c/pose#342: the below-cover cues read the same line when their tilts lie within this many degrees (max − min). */
|
|
89
|
+
export const CUE_SPREAD = 8;
|
|
90
|
+
/** ... a leg's colour is sampled this far along its thigh, hip → knee. */
|
|
91
|
+
export const LEG_AT = 0.7;
|
|
92
|
+
/** ... the hem and a thigh root are where the garment stops for this many px (rows, or along the thigh) in a row. */
|
|
93
|
+
export const LEAVE = 3;
|
|
88
94
|
|
|
89
95
|
/**
|
|
90
96
|
* One frame's reading: the tilt (Spine degrees, r_hip → l_hip), or why it was declined. A waistband
|
|
91
97
|
* read through the window mean says so (`mean`), and one whose colour was found below the hips says
|
|
92
98
|
* how far (`below`, px); one whose band top lay above the colour's edge in some columns says how many (`raised`, #318).
|
|
93
99
|
*/
|
|
94
|
-
export type Reading = { tilt: number; cover: number; rms: number; mean?: true; below?: number; raised?: number } | { kind: string; why: string };
|
|
100
|
+
export type Reading = { tilt: number; cover: number; rms: number; mean?: true; below?: number; raised?: number; cues?: CueTilts } | { kind: string; why: string };
|
|
101
|
+
|
|
102
|
+
/**
|
|
103
|
+
* rig-c/pose#342: the below-cover cues of one frame, Spine degrees r_hip → l_hip; null where the cue was not read. `band` is
|
|
104
|
+
* "covered" when something other than the ground lies over the band's top. `hips` is the keypoints' line: reported, not fitted.
|
|
105
|
+
*/
|
|
106
|
+
export interface CueTilts {
|
|
107
|
+
hem: number | null;
|
|
108
|
+
roots: number | null;
|
|
109
|
+
band: number | 'covered' | null;
|
|
110
|
+
hips: number;
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
/** rig-c/pose#342: a below-cover reading's figures and points (the picture's), and why each cue not read was not. */
|
|
114
|
+
export interface CueDetail {
|
|
115
|
+
tilts: CueTilts;
|
|
116
|
+
why: { hem?: string; roots?: string; band?: string };
|
|
117
|
+
garment: RGB;
|
|
118
|
+
legs: { r: RGB | null; l: RGB | null };
|
|
119
|
+
hem: { pts: Pt[]; fit: { m: number; q: number; rms: number } | null };
|
|
120
|
+
roots: { r: Pt | null; l: Pt | null; paths: Array<[Pt, Pt]> };
|
|
121
|
+
band: { m: number; q: number } | null;
|
|
122
|
+
}
|
|
95
123
|
|
|
96
124
|
/** The frame as RGB over white, its background colour, and which pixels are masked (forearms, hands). */
|
|
97
125
|
interface Frame {
|
|
@@ -194,7 +222,38 @@ export function readPelvis(plate: Plate, joints: Partial<Record<Body18, Joint>>,
|
|
|
194
222
|
const f = frameOf(plate, arms, Math.max(3, ARM_RADIUS * span));
|
|
195
223
|
if (x0 < 0 || x1 >= f.w || c[1] < 0 || c[1] >= f.h) return { kind: 'hips outside the frame', why: `the hips (${c[0].toFixed(1)}, ${c[1].toFixed(1)}) lie outside the ${f.w}×${f.h} frame` };
|
|
196
224
|
const cols = x1 - x0 + 1;
|
|
197
|
-
|
|
225
|
+
if (mode === 'silhouette') return silhouette(f, c, span, x0, x1, cols, rHip, lHip);
|
|
226
|
+
if (mode === 'waistband') return waistband(f, c, span, x0, x1, cols, rHip, lHip);
|
|
227
|
+
return belowCover(f, c, span, x0, x1, cols, rHip, lHip, pt).reading;
|
|
228
|
+
}
|
|
229
|
+
|
|
230
|
+
/**
|
|
231
|
+
* rig-c/pose#342: the below-cover reading of one frame with every cue's figures and points, for a picture of it. Null when
|
|
232
|
+
* the frame is declined before any cue is read (no hips, side-on, outside the frame); the reading is `readPelvis`'s.
|
|
233
|
+
*/
|
|
234
|
+
export function readBelowCover(plate: Plate, joints: Partial<Record<Body18, Joint>>): { reading: Reading; detail: CueDetail | null } {
|
|
235
|
+
const reading = readPelvis(plate, joints, 'below-cover');
|
|
236
|
+
const pt = (n: Body18): Pt | null => {
|
|
237
|
+
const j = joints[n];
|
|
238
|
+
return j === undefined || j.state === 'missing' || j.at === undefined ? null : [j.at[0], j.at[1]];
|
|
239
|
+
};
|
|
240
|
+
const rHip = pt('r_hip');
|
|
241
|
+
const lHip = pt('l_hip');
|
|
242
|
+
if (rHip === null || lHip === null) return { reading, detail: null };
|
|
243
|
+
const span = Math.hypot(lHip[0] - rHip[0], lHip[1] - rHip[1]);
|
|
244
|
+
const x0 = Math.round(Math.min(rHip[0], lHip[0]));
|
|
245
|
+
const x1 = Math.round(Math.max(rHip[0], lHip[0]));
|
|
246
|
+
if (span < MIN_SPAN || x1 - x0 + 1 < MIN_SPAN) return { reading, detail: null };
|
|
247
|
+
const c: Pt = [(rHip[0] + lHip[0]) / 2, (rHip[1] + lHip[1]) / 2];
|
|
248
|
+
const arms: Array<[Pt, Pt]> = [];
|
|
249
|
+
for (const [e, w] of [['r_elbow', 'r_wrist'], ['l_elbow', 'l_wrist']] as const) {
|
|
250
|
+
const a = pt(e);
|
|
251
|
+
const b = pt(w);
|
|
252
|
+
if (a !== null && b !== null) arms.push([a, b]);
|
|
253
|
+
}
|
|
254
|
+
const f = frameOf(plate, arms, Math.max(3, ARM_RADIUS * span));
|
|
255
|
+
if (x0 < 0 || x1 >= f.w || c[1] < 0 || c[1] >= f.h) return { reading, detail: null };
|
|
256
|
+
return { reading, detail: belowCover(f, c, span, x0, x1, x1 - x0 + 1, rHip, lHip, pt).detail };
|
|
198
257
|
}
|
|
199
258
|
|
|
200
259
|
function silhouette(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt): Reading {
|
|
@@ -272,7 +331,8 @@ function meanOf(f: Frame): (x: number, y: number) => RGB | null {
|
|
|
272
331
|
};
|
|
273
332
|
}
|
|
274
333
|
|
|
275
|
-
|
|
334
|
+
/** `out.fit`, when given, receives the edge read (rig-c/pose#342 looks at what lies above it). */
|
|
335
|
+
function waistband(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt, out?: { fit?: { m: number; q: number } }): Reading {
|
|
276
336
|
// The garment's colour: the median over the unmasked pixels of a patch at the hips' midpoint (or below it), as they are or through the window mean.
|
|
277
337
|
const r = Math.max(2, 0.3 * span);
|
|
278
338
|
const raw = (x: number, y: number): RGB | null => f.rgb(x, y);
|
|
@@ -341,6 +401,7 @@ function waistband(f: Frame, c: Pt, span: number, x0: number, x1: number, cols:
|
|
|
341
401
|
if (Math.abs(Math.atan(fit.m)) * (180 / Math.PI) > 45) return { kind: 'waistband too steep', why: `the garment's top edge runs ${(Math.atan(fit.m) * (180 / Math.PI)).toFixed(1)}° from level; a waistband is within 45°` };
|
|
342
402
|
const off = notTheBand(f, fit, tops, top, colourAt);
|
|
343
403
|
if (off !== null) return off;
|
|
404
|
+
if (out !== undefined) out.fit = { m: fit.m, q: fit.q };
|
|
344
405
|
return { tilt: tiltOf(fit.m, rHip, lHip), cover: share, rms: fit.rms, ...(isMean ? { mean: true as const } : {}), ...(below > 0 ? { below } : {}), ...(raised > 0 ? { raised } : {}) };
|
|
345
406
|
}
|
|
346
407
|
|
|
@@ -532,6 +593,193 @@ function notTheBand(f: Frame, fit: { m: number; q: number }, tops: Pt[], top: nu
|
|
|
532
593
|
};
|
|
533
594
|
}
|
|
534
595
|
|
|
596
|
+
/**
|
|
597
|
+
* rig-c/pose#342: the pelvis read below a cover. A garment hanging over the waistband (a sweater's hem) hides the band's top
|
|
598
|
+
* and hangs level whatever the pelvis does, so the band-top reader reads the cover. What the cover cannot hide is read:
|
|
599
|
+
*
|
|
600
|
+
* - `hem`: the bottomwear's lower hem across the hip span — in each column, from the hips' row down, the last garment row
|
|
601
|
+
* before `LEAVE` rows that are not garment; a line fitted through them (≥ `MIN_COVER` of the columns);
|
|
602
|
+
* - `roots`: where each thigh leaves the garment — along hip → knee (keypoints), the last garment point before `LEAVE` px
|
|
603
|
+
* that are not; the line from the right root to the left;
|
|
604
|
+
* - `band`: the waistband reader's band top, used only when the ground lies above it (nothing covers it);
|
|
605
|
+
* - `hips`: the keypoints' line, reported but not fitted — a detector's hips keep a fraction of a drawn tilt (#288).
|
|
606
|
+
*
|
|
607
|
+
* Every colour is the window mean (`DITHER_RADIUS`); a pixel is garment when it is nearest the garment's colour among the
|
|
608
|
+
* garment's, each leg's (sampled `LEG_AT` along its thigh) and the ground's, so both shades of a cloth and the ink on it
|
|
609
|
+
* are garment and skin or ground under the hem are not. The tilt is the mean of the cues fitted (hem, roots, an
|
|
610
|
+
* uncovered band); fewer than two, or two that differ by more than `CUE_SPREAD`, decline the frame by name.
|
|
611
|
+
*/
|
|
612
|
+
function belowCover(f: Frame, c: Pt, span: number, x0: number, x1: number, cols: number, rHip: Pt, lHip: Pt, pt: (n: Body18) => Pt | null): { reading: Reading; detail: CueDetail | null } {
|
|
613
|
+
const r = Math.max(2, 0.3 * span);
|
|
614
|
+
const mean = meanOf(f);
|
|
615
|
+
const disc = (at: Pt): RGB | null => {
|
|
616
|
+
const ps: RGB[] = [];
|
|
617
|
+
for (let y = Math.floor(at[1] - r); y <= Math.ceil(at[1] + r); y++) {
|
|
618
|
+
for (let x = Math.floor(at[0] - r); x <= Math.ceil(at[0] + r); x++) {
|
|
619
|
+
if (x < 0 || y < 0 || x >= f.w || y >= f.h || Math.hypot(x - at[0], y - at[1]) > r || f.masked(x, y)) continue;
|
|
620
|
+
const p = mean(x, y);
|
|
621
|
+
if (p !== null) ps.push(p);
|
|
622
|
+
}
|
|
623
|
+
}
|
|
624
|
+
return ps.length === 0 ? null : ([0, 1, 2].map((k) => median(ps.map((p) => p[k]!))) as RGB);
|
|
625
|
+
};
|
|
626
|
+
const holds = (at: Pt, g: RGB) => {
|
|
627
|
+
let n = 0;
|
|
628
|
+
let k = 0;
|
|
629
|
+
for (let y = Math.floor(at[1] - r); y <= Math.ceil(at[1] + r); y++) {
|
|
630
|
+
for (let x = Math.floor(at[0] - r); x <= Math.ceil(at[0] + r); x++) {
|
|
631
|
+
if (x < 0 || y < 0 || x >= f.w || y >= f.h || Math.hypot(x - at[0], y - at[1]) > r || f.masked(x, y)) continue;
|
|
632
|
+
const p = mean(x, y);
|
|
633
|
+
if (p === null) continue;
|
|
634
|
+
n++;
|
|
635
|
+
if (diff(p, g) <= GARMENT_DIFF) k++;
|
|
636
|
+
}
|
|
637
|
+
}
|
|
638
|
+
return n > 0 && k >= HOLD * n;
|
|
639
|
+
};
|
|
640
|
+
const mid = disc(c);
|
|
641
|
+
if (mid === null) return { reading: { kind: 'hips behind a forearm', why: 'the hips\' midpoint is covered by a forearm; no garment colour to read' }, detail: null };
|
|
642
|
+
let g: RGB = mid;
|
|
643
|
+
for (const drop of [0, r, 2 * r]) {
|
|
644
|
+
const at: Pt = [c[0], c[1] + drop];
|
|
645
|
+
const p = drop === 0 ? mid : disc(at);
|
|
646
|
+
if (p !== null && diff(p, f.bg) > FIGURE_DIFF && holds(at, p)) {
|
|
647
|
+
g = p;
|
|
648
|
+
break;
|
|
649
|
+
}
|
|
650
|
+
}
|
|
651
|
+
if (diff(g, f.bg) <= FIGURE_DIFF) return { reading: { kind: 'no garment over the hips', why: `the colour at the hips' midpoint (${g.map((v) => v.toFixed(0)).join(', ')}) is the background's (${f.bg.map((v) => v.toFixed(0)).join(', ')})` }, detail: null };
|
|
652
|
+
const knee = { r: pt('r_knee'), l: pt('l_knee') };
|
|
653
|
+
const hip = { r: rHip, l: lHip };
|
|
654
|
+
const legs = { r: null as RGB | null, l: null as RGB | null };
|
|
655
|
+
for (const s of ['r', 'l'] as const) {
|
|
656
|
+
const k = knee[s];
|
|
657
|
+
if (k === null) continue;
|
|
658
|
+
const L = disc([hip[s][0] + LEG_AT * (k[0] - hip[s][0]), hip[s][1] + LEG_AT * (k[1] - hip[s][1])]);
|
|
659
|
+
if (L !== null && diff(L, g) > GARMENT_DIFF) legs[s] = L;
|
|
660
|
+
}
|
|
661
|
+
const others = [f.bg, ...(legs.r !== null ? [legs.r] : []), ...(legs.l !== null ? [legs.l] : [])];
|
|
662
|
+
if (others.length === 1) return { reading: { kind: 'no leg below the garment', why: `neither thigh carries a colour of its own ${LEG_AT} of the way to its knee (${knee.r === null || knee.l === null ? 'a knee has no position' : 'both within ' + GARMENT_DIFF + ' of the garment\'s'}); nothing below the garment to tell its hem by` }, detail: null };
|
|
663
|
+
// Garment: nearest the garment's colour among it, the legs' and the ground's (a tie is garment). Null where every pixel of the window is masked.
|
|
664
|
+
const isGarment = (x: number, y: number): boolean | null => {
|
|
665
|
+
if (x < 0 || y < 0 || x >= f.w || y >= f.h || f.masked(x, y)) return null;
|
|
666
|
+
const p = mean(x, y);
|
|
667
|
+
if (p === null) return null;
|
|
668
|
+
return diff(p, g) <= Math.min(...others.map((o) => diff(p, o)));
|
|
669
|
+
};
|
|
670
|
+
const why: CueDetail['why'] = {};
|
|
671
|
+
// The hem.
|
|
672
|
+
const hemPts: Pt[] = [];
|
|
673
|
+
const yEnd = Math.min(f.h - 1, Math.floor(c[1] + 2.5 * span));
|
|
674
|
+
for (let x = x0; x <= x1; x++) {
|
|
675
|
+
let inside = false;
|
|
676
|
+
let last = -1;
|
|
677
|
+
let out = 0;
|
|
678
|
+
for (let y = Math.max(0, Math.round(c[1])); y <= yEnd; y++) {
|
|
679
|
+
const gm = isGarment(x, y);
|
|
680
|
+
if (gm === null) {
|
|
681
|
+
if (inside) break;
|
|
682
|
+
continue;
|
|
683
|
+
}
|
|
684
|
+
if (gm) {
|
|
685
|
+
inside = true;
|
|
686
|
+
last = y;
|
|
687
|
+
out = 0;
|
|
688
|
+
} else if (inside && ++out >= LEAVE) {
|
|
689
|
+
hemPts.push([x, last]);
|
|
690
|
+
break;
|
|
691
|
+
}
|
|
692
|
+
}
|
|
693
|
+
}
|
|
694
|
+
const hemFit = hemPts.length >= MIN_COVER * cols ? fitLine(hemPts) : null;
|
|
695
|
+
let hem: number | null = null;
|
|
696
|
+
if (hemFit === null) why.hem = hemPts.length < MIN_COVER * cols ? `the hem is found in ${hemPts.length} of the ${cols} columns between the hips (< ${100 * MIN_COVER}%)` : 'the hem rows do not fit a line';
|
|
697
|
+
else hem = tiltOf(hemFit.m, rHip, lHip);
|
|
698
|
+
// The thigh roots.
|
|
699
|
+
const root = { r: null as Pt | null, l: null as Pt | null };
|
|
700
|
+
const rootWhy: string[] = [];
|
|
701
|
+
const paths: Array<[Pt, Pt]> = [];
|
|
702
|
+
for (const s of ['r', 'l'] as const) {
|
|
703
|
+
const k = knee[s];
|
|
704
|
+
if (k === null) {
|
|
705
|
+
rootWhy.push(`${s}_knee has no position`);
|
|
706
|
+
continue;
|
|
707
|
+
}
|
|
708
|
+
const h = hip[s];
|
|
709
|
+
paths.push([h, k]);
|
|
710
|
+
const len = Math.hypot(k[0] - h[0], k[1] - h[1]);
|
|
711
|
+
const n = Math.ceil(len / 0.5);
|
|
712
|
+
let inside = false;
|
|
713
|
+
let last: Pt | null = null;
|
|
714
|
+
let out = 0;
|
|
715
|
+
let hidden = false;
|
|
716
|
+
for (let i = 0; i <= n; i++) {
|
|
717
|
+
const p: Pt = [h[0] + (i / n) * (k[0] - h[0]), h[1] + (i / n) * (k[1] - h[1])];
|
|
718
|
+
const gm = isGarment(Math.round(p[0]), Math.round(p[1]));
|
|
719
|
+
if (gm === null) {
|
|
720
|
+
if (inside) {
|
|
721
|
+
hidden = true;
|
|
722
|
+
break;
|
|
723
|
+
}
|
|
724
|
+
continue;
|
|
725
|
+
}
|
|
726
|
+
if (gm) {
|
|
727
|
+
inside = true;
|
|
728
|
+
last = p;
|
|
729
|
+
out = 0;
|
|
730
|
+
} else if (inside && ++out * (len / n) >= LEAVE) {
|
|
731
|
+
root[s] = last;
|
|
732
|
+
break;
|
|
733
|
+
}
|
|
734
|
+
}
|
|
735
|
+
if (root[s] === null) rootWhy.push(hidden ? `the ${s} thigh's root is behind a forearm` : inside ? `the ${s} thigh does not leave the garment before its knee` : `the ${s} thigh's path carries no garment`);
|
|
736
|
+
}
|
|
737
|
+
let roots: number | null = null;
|
|
738
|
+
if (root.r !== null && root.l !== null) {
|
|
739
|
+
if (Math.abs(root.l[0] - root.r[0]) < MIN_SPAN) rootWhy.push(`the roots are ${Math.abs(root.l[0] - root.r[0]).toFixed(1)} px apart across (< ${MIN_SPAN})`);
|
|
740
|
+
else roots = pictureAngle(root.r, root.l);
|
|
741
|
+
}
|
|
742
|
+
if (roots === null) why.roots = rootWhy.join('; ');
|
|
743
|
+
// The band's top, when nothing covers it.
|
|
744
|
+
const edge: { fit?: { m: number; q: number } } = {};
|
|
745
|
+
const wb = waistband(f, c, span, x0, x1, cols, rHip, lHip, edge);
|
|
746
|
+
let band: number | 'covered' | null = null;
|
|
747
|
+
if (!('tilt' in wb) || edge.fit === undefined) why.band = 'kind' in wb ? `${wb.kind}` : 'no edge';
|
|
748
|
+
else {
|
|
749
|
+
const above: RGB[] = [];
|
|
750
|
+
for (let x = x0; x <= x1; x++) {
|
|
751
|
+
const y = Math.round(edge.fit.m * x + edge.fit.q) - 3 * (DITHER_RADIUS + 1);
|
|
752
|
+
if (y < 0 || y >= f.h || f.masked(x, y)) continue;
|
|
753
|
+
const p = mean(x, y);
|
|
754
|
+
if (p !== null) above.push(p);
|
|
755
|
+
}
|
|
756
|
+
const a = above.length === 0 ? null : ([0, 1, 2].map((k) => median(above.map((p) => p[k]!))) as RGB);
|
|
757
|
+
if (a !== null && diff(a, f.bg) <= FIGURE_DIFF) band = wb.tilt;
|
|
758
|
+
else {
|
|
759
|
+
band = 'covered';
|
|
760
|
+
why.band = a === null ? 'nothing seen above the band\'s top' : `the band's top is covered (${a.map((v) => v.toFixed(0)).join(', ')} lies ${3 * (DITHER_RADIUS + 1)} px above it, not the ground)`;
|
|
761
|
+
}
|
|
762
|
+
}
|
|
763
|
+
const hips = pictureAngle(rHip, lHip);
|
|
764
|
+
const tilts: CueTilts = { hem, roots, band, hips };
|
|
765
|
+
const detail: CueDetail = { tilts, why, garment: g, legs, hem: { pts: hemPts, fit: hemFit === null ? null : { m: hemFit.m, q: hemFit.q, rms: hemFit.rms } }, roots: { ...root, paths }, band: typeof band === 'number' ? edge.fit! : null };
|
|
766
|
+
const fitted: Array<[string, number]> = [];
|
|
767
|
+
if (hem !== null) fitted.push(['hem', hem]);
|
|
768
|
+
if (roots !== null) fitted.push(['roots', roots]);
|
|
769
|
+
if (typeof band === 'number') fitted.push(['band', band]);
|
|
770
|
+
const fmt = (v: number) => `${v >= 0 ? '+' : ''}${v.toFixed(1)}°`;
|
|
771
|
+
if (fitted.length < 2) {
|
|
772
|
+
const missing = (['hem', 'roots', 'band'] as const).filter((k) => why[k] !== undefined).map((k) => `${k}: ${why[k]}`);
|
|
773
|
+
return { reading: { kind: 'too few cues below the cover', why: `${fitted.length} of the cues read (${fitted.map(([n, v]) => `${n} ${fmt(v)}`).join(', ') || 'none'}); ${missing.join('; ')}` }, detail };
|
|
774
|
+
}
|
|
775
|
+
const base = fitted[0]![1];
|
|
776
|
+
const about = fitted.map(([, v]) => base + wrap(v - base));
|
|
777
|
+
const spread = Math.max(...about) - Math.min(...about);
|
|
778
|
+
if (spread > CUE_SPREAD) return { reading: { kind: 'cues disagree', why: `${fitted.map(([n, v]) => `${n} ${fmt(v)}`).join(', ')}: ${spread.toFixed(1)}° apart (> ${CUE_SPREAD}°); they do not read one line (hips ${fmt(hips)}, not fitted)` }, detail };
|
|
779
|
+
const tilt = wrap(about.reduce((s, v) => s + v, 0) / about.length);
|
|
780
|
+
return { reading: { tilt, cover: hemPts.length / cols, rms: hemFit?.rms ?? 0, mean: true, cues: tilts }, detail };
|
|
781
|
+
}
|
|
782
|
+
|
|
535
783
|
/**
|
|
536
784
|
* `rig-pose-pelvis-keys/1`: `{ spec, keys: [{ frame, deg }], note? }` — frames of the
|
|
537
785
|
* sequence, strictly increasing, within its span [0, frames − 1]; `deg` the tilt in
|
|
@@ -686,6 +934,17 @@ export function runPelvis(a: PelvisArgs, say: (l: string) => void): void {
|
|
|
686
934
|
const tilt = readings.map((r) => ('tilt' in r ? r.tilt : null));
|
|
687
935
|
const read = tilt.filter((v): v is number => v !== null);
|
|
688
936
|
say(`rig-pose pelvis: ${n} frame(s) of person "${person}" from ${a.pose}, read by ${a.mode} off ${dir} (${files.length} PNG)`);
|
|
937
|
+
if (a.mode === 'below-cover') {
|
|
938
|
+
// rig-c/pose#342: every frame's cues, read or not, so a decline or a reading can be checked cue by cue.
|
|
939
|
+
const fmt = (v: number | 'covered' | null) => (v === null ? 'not read' : v === 'covered' ? 'covered' : `${v >= 0 ? '+' : ''}${v.toFixed(2)}°`);
|
|
940
|
+
readings.forEach((r, i) => {
|
|
941
|
+
if (!('tilt' in r) || r.cues === undefined) return;
|
|
942
|
+
// The hips stay out of the vote; their sign is printed beside the fit, and an opposite sign is warned.
|
|
943
|
+
const sign = (v: number) => (Math.abs(v) < 0.5 ? '0' : v > 0 ? '+' : '−');
|
|
944
|
+
say(` CUES frame ${i}: hem ${fmt(r.cues.hem)}, roots ${fmt(r.cues.roots)}, band ${fmt(r.cues.band)}, hips ${fmt(r.cues.hips)} (not fitted) → ${fmt(r.tilt)}; hips sign ${sign(r.cues.hips)}, fit sign ${sign(r.tilt)}`);
|
|
945
|
+
if (sign(r.cues.hips) !== '0' && sign(r.tilt) !== '0' && sign(r.cues.hips) !== sign(r.tilt)) say(` HIPS-SIGN frame ${i}: the hip keypoints tilt ${fmt(r.cues.hips)}, opposite the fit's ${fmt(r.tilt)} (the hips are not fitted)`);
|
|
946
|
+
});
|
|
947
|
+
}
|
|
689
948
|
for (const d of declinedRuns(readings)) say(` DECLINED frame${d.from === d.to ? ` ${d.from}` : `s ${d.from}–${d.to}`} (${d.to - d.from + 1}): ${d.kind} — ${d.same ? d.why : `frame ${d.from}: ${d.why}; the figures differ frame to frame`}`);
|
|
690
949
|
if (read.length > 0) {
|
|
691
950
|
const covers = readings.flatMap((r) => ('tilt' in r ? [r.cover] : []));
|
package/src/pose.ts
CHANGED
|
@@ -80,9 +80,10 @@ export type DepthGait = (typeof DEPTH_GAITS)[number];
|
|
|
80
80
|
/**
|
|
81
81
|
* Where a sequence's `pelvisTilt` came from (rig-c/pose#288): `silhouette` — the
|
|
82
82
|
* straight edge of the drawn figure across the hips; `waistband` — the top edge
|
|
83
|
-
* of a garment covering them; `
|
|
83
|
+
* of a garment covering them; `below-cover` — the cues under a garment hanging
|
|
84
|
+
* over the waistband (rig-c/pose#342); `keys` — a hand file (`--pelvis-keys`).
|
|
84
85
|
*/
|
|
85
|
-
export const PELVIS_SOURCES = ['silhouette', 'waistband', 'keys'] as const;
|
|
86
|
+
export const PELVIS_SOURCES = ['silhouette', 'waistband', 'below-cover', 'keys'] as const;
|
|
86
87
|
export type PelvisSource = (typeof PELVIS_SOURCES)[number];
|
|
87
88
|
export const POSE_SPACE = { units: 'picture-px', origin: 'top-left', y: 'down' } as const;
|
|
88
89
|
/**
|
package/src/retarget.ts
CHANGED
|
@@ -76,7 +76,7 @@ import type { Body18 } from './joints.ts';
|
|
|
76
76
|
import { isBody18, VIRTUAL } from './joints.ts';
|
|
77
77
|
import type { Chain, JointMap, Rest, RigSets, RigSlots } from './map.ts';
|
|
78
78
|
import { aimAngle, aimEnds, bodyYawSets, descends, faceYawSets, knownSetsOf, partBone, pointParts, rigPoint } from './map.ts';
|
|
79
|
-
import type { Joint, Travel } from './pose.ts';
|
|
79
|
+
import type { DepthGait, Joint, Travel } from './pose.ts';
|
|
80
80
|
import { TRAVEL_IN_PLACE } from './pose.ts';
|
|
81
81
|
import { Problems, Refusal } from './problems.ts';
|
|
82
82
|
import type { ArtRange, AttachmentKey, ByHip, Raised, ReachShort, SetRuns, SlotTrack, Switch, SwitchOptions } from './switch.ts';
|
|
@@ -188,6 +188,35 @@ export interface RetargetOptions extends SwitchOptions {
|
|
|
188
188
|
* frame they cover they decide the support. Absent, nothing changes.
|
|
189
189
|
*/
|
|
190
190
|
contactKeys?: ContactKeys;
|
|
191
|
+
/**
|
|
192
|
+
* rig-c/pose#344: the gait the sequence was cleaned as (its `cleaned.params.depth.gait`, `clean --depth`). An
|
|
193
|
+
* in-place walk anchors the upper body (`TorsoAnchor`): the pelvis sways under it. Absent, or a run, nothing changes.
|
|
194
|
+
*/
|
|
195
|
+
gait?: DepthGait;
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
/**
|
|
199
|
+
* rig-c/pose#344: in an in-place walk the upper body is the anchor and the pelvis sways under it. The pelvis bone
|
|
200
|
+
* carries the torso (its child), so its turn and its travel would swing the chest around it; the first spine bone
|
|
201
|
+
* (`bone`: the pelvis bone's child on the way to the trunk chain's bone) is keyed with the counter: a rotate of minus
|
|
202
|
+
* the pelvis's turn (none when it is the trunk bone itself, whose aim is already in the world), and a translate that
|
|
203
|
+
* puts its origin back where the pelvis's turn moved it from, up and down, and sideways where it carries `point`
|
|
204
|
+
* (the trunk chain's aim bone: the top of the trunk) on the drawn neck's own sideways path: about the world x `x`
|
|
205
|
+
* (the setup x plus the pelvis's mean travel), the picture's `joint` x less its mean, times `factor` — the pelvis's
|
|
206
|
+
* sideways sway in the rig against the drawing's (`joints[0]`, mid_hip), each its standard deviation over the frames,
|
|
207
|
+
* so the neck sways against the pelvis as the drawing's does; the figure's scale where the drawing's pelvis does not
|
|
208
|
+
* sway. The pelvis's own keys are as read; `neck` is the carried path's peak-to-peak and `sway` the first spine bone's
|
|
209
|
+
* sideways counter's, world units.
|
|
210
|
+
*/
|
|
211
|
+
export interface TorsoAnchor {
|
|
212
|
+
gait: DepthGait;
|
|
213
|
+
bone: string;
|
|
214
|
+
point: string;
|
|
215
|
+
joint: string;
|
|
216
|
+
x: number;
|
|
217
|
+
factor: number;
|
|
218
|
+
neck: number;
|
|
219
|
+
sway: number;
|
|
191
220
|
}
|
|
192
221
|
|
|
193
222
|
/**
|
|
@@ -208,6 +237,8 @@ export interface PelvisReading {
|
|
|
208
237
|
hipLine: number;
|
|
209
238
|
/** The frames keyed from the hip line on a sequence that carries a track: the frames its reading declined. */
|
|
210
239
|
declined: Array<[number, number]>;
|
|
240
|
+
/** rig-c/pose#344: the torso held over the swaying pelvis, in an in-place walk only. */
|
|
241
|
+
anchor?: TorsoAnchor;
|
|
211
242
|
}
|
|
212
243
|
|
|
213
244
|
export interface Hold {
|
|
@@ -904,6 +935,32 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
904
935
|
}
|
|
905
936
|
p.throwIfAny();
|
|
906
937
|
|
|
938
|
+
// rig-c/pose#344: an in-place walk holds the upper body over the swaying pelvis, on the first spine bone.
|
|
939
|
+
const anchorOf = ((): { bone: string; trunk: string; point: string; pelvis: string; joints: [string, string] } | null => {
|
|
940
|
+
if (opts.gait !== 'walk' || opts.travel !== TRAVEL_IN_PLACE || pelvisChain === null) return null;
|
|
941
|
+
const pelvisBone = pelvisChain.bones[0]!;
|
|
942
|
+
const trunk = map.chains.find((c) => !c.reach && c.joints[0] === 'mid_hip' && descends(rest, c.bones[0]!, pelvisBone));
|
|
943
|
+
if (trunk === undefined) return null;
|
|
944
|
+
const tb = trunk.bones[0]!;
|
|
945
|
+
let first = tb;
|
|
946
|
+
while (rest.get(first)!.parent !== pelvisBone) first = rest.get(first)!.parent!;
|
|
947
|
+
// The aim's upper end (aimEnds: `neck` and `chest>neck` alike end at the neck), a rig point on the trunk bone or below it.
|
|
948
|
+
const aim = trunk.aim[0]!;
|
|
949
|
+
const top = aim === 'x' ? null : aimEnds(tb, aim)[1];
|
|
950
|
+
const onTrunk = top !== null && pointParts(top).every((part) => {
|
|
951
|
+
const b = partBone(part);
|
|
952
|
+
return rest.has(b) && (b === tb || descends(rest, b, tb));
|
|
953
|
+
});
|
|
954
|
+
const point = onTrunk ? top! : tb;
|
|
955
|
+
return { bone: first, trunk: tb, point, pelvis: pelvisBone, joints: [trunk.joints[0]!, trunk.joints[1]!] };
|
|
956
|
+
})();
|
|
957
|
+
if (anchorOf !== null && anchorOf.bone !== anchorOf.trunk && !targets.has(anchorOf.bone)) {
|
|
958
|
+
// Not in a chain: its world turn is held at setup, so the bones below it are keyed against none.
|
|
959
|
+
const a0 = restAngle(rest, anchorOf.bone, 'x');
|
|
960
|
+
restOf.set(anchorOf.bone, a0);
|
|
961
|
+
targets.set(anchorOf.bone, Array.from({ length: n }, () => a0));
|
|
962
|
+
}
|
|
963
|
+
|
|
907
964
|
// The short hold, on every segment the pose aims directly (not a reach): its
|
|
908
965
|
// picture length against its own median, on the points the angle was read from.
|
|
909
966
|
// A horizontal line across the body (`isAcross`: pelvis, shoulders, head line)
|
|
@@ -1509,7 +1566,43 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
1509
1566
|
const r = turnAbove(parent, i) / DEG;
|
|
1510
1567
|
const ox = own.worldX - rest.get(parent)!.worldX;
|
|
1511
1568
|
const oy = own.worldY - rest.get(parent)!.worldY;
|
|
1512
|
-
|
|
1569
|
+
const at: Pt = [po[0] + Math.cos(r) * ox - Math.sin(r) * oy + nod[0], po[1] + Math.sin(r) * ox + Math.cos(r) * oy + nod[1]];
|
|
1570
|
+
if (anchorOf === null || b !== anchorOf.bone || anchorRaw) return at;
|
|
1571
|
+
const s = anchorShift(i, at, po, oy);
|
|
1572
|
+
return [at[0] + s[0], at[1] + s[1]];
|
|
1573
|
+
};
|
|
1574
|
+
/**
|
|
1575
|
+
* rig-c/pose#344: the world shift that holds the first spine bone over the pelvis: up and down, its origin where
|
|
1576
|
+
* the pelvis's travel alone puts it (the pelvis's turn taken out); sideways, the trunk's top point on `anchorPath`:
|
|
1577
|
+
* the drawn neck's sideways path about its mean, scaled by the pelvis's rig ÷ drawn sideways sway (`TorsoAnchor`).
|
|
1578
|
+
*/
|
|
1579
|
+
let anchorRaw = false;
|
|
1580
|
+
let anchorX: number | null = null;
|
|
1581
|
+
let anchorFactor = 1;
|
|
1582
|
+
let anchorPath: number[] | null = null;
|
|
1583
|
+
const anchorShift = (i: number, at: Pt, po: Pt, oy: number): Pt => {
|
|
1584
|
+
const a = anchorOf!;
|
|
1585
|
+
if (anchorPath === null) {
|
|
1586
|
+
anchorRaw = true;
|
|
1587
|
+
const hipX = Array.from({ length: n }, (_, k) => posedOrigin(a.pelvis, k)[0] - rest.get(a.pelvis)!.worldX);
|
|
1588
|
+
anchorRaw = false;
|
|
1589
|
+
const mean = (xs: readonly number[]) => xs.reduce((s, x) => s + x, 0) / xs.length;
|
|
1590
|
+
const sd = (xs: readonly number[]) => {
|
|
1591
|
+
const m = mean(xs);
|
|
1592
|
+
return Math.sqrt(xs.reduce((s, x) => s + (x - m) ** 2, 0) / xs.length);
|
|
1593
|
+
};
|
|
1594
|
+
const drawnHip = series.get(a.joints[0])!.map((p) => p[0]);
|
|
1595
|
+
const drawnNeck = series.get(a.joints[1])!.map((p) => p[0]);
|
|
1596
|
+
const sdHip = sd(drawnHip);
|
|
1597
|
+
anchorFactor = sdHip > 1e-6 ? sd(hipX) / sdHip : (scale?.factor ?? 1);
|
|
1598
|
+
anchorX = rigPoint(rest, a.point)![0] + mean(hipX);
|
|
1599
|
+
const m = mean(drawnNeck);
|
|
1600
|
+
anchorPath = drawnNeck.map((x) => anchorX! + (x - m) * anchorFactor);
|
|
1601
|
+
}
|
|
1602
|
+
anchorRaw = true;
|
|
1603
|
+
const top = posedPoint(a.point, i);
|
|
1604
|
+
anchorRaw = false;
|
|
1605
|
+
return [anchorPath[i]! - top[0], po[1] + oy - at[1]];
|
|
1513
1606
|
};
|
|
1514
1607
|
/** A rig point (a bone, `name:tip`, `a+b`) posed on frame i the same way. */
|
|
1515
1608
|
const posedPoint = (spec: string, i: number): Pt => {
|
|
@@ -2999,6 +3092,42 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
2999
3092
|
}
|
|
3000
3093
|
}
|
|
3001
3094
|
}
|
|
3095
|
+
// rig-c/pose#344: the first spine bone's counter — its rotate when it is not the trunk bone, and its translate.
|
|
3096
|
+
let anchor: TorsoAnchor | null = null;
|
|
3097
|
+
if (anchorOf !== null) {
|
|
3098
|
+
const a = anchorOf;
|
|
3099
|
+
if (a.bone !== a.trunk) {
|
|
3100
|
+
const above = mappedAbove(a.bone);
|
|
3101
|
+
let prev = 0;
|
|
3102
|
+
const keys: MotionKey[] = [];
|
|
3103
|
+
for (let i = 0; i < n; i++) {
|
|
3104
|
+
const raw = turn(a.bone, i) - (above === null ? 0 : turn(above, i));
|
|
3105
|
+
prev = i === 0 ? wrap(raw) : prev + wrap(raw - prev);
|
|
3106
|
+
keys.push({ t: keyTimes[i]!, v: [r3(prev)] });
|
|
3107
|
+
}
|
|
3108
|
+
tracks.push({ bone: a.bone, property: 'rotate', keys });
|
|
3109
|
+
}
|
|
3110
|
+
const pr = rest.get(a.pelvis)!;
|
|
3111
|
+
const det = pr.a * pr.d - pr.b * pr.c;
|
|
3112
|
+
const keys: MotionKey[] = [];
|
|
3113
|
+
const xs: number[] = [];
|
|
3114
|
+
for (let i = 0; i < n; i++) {
|
|
3115
|
+
const held = posedOrigin(a.bone, i);
|
|
3116
|
+
anchorRaw = true;
|
|
3117
|
+
const free = posedOrigin(a.bone, i);
|
|
3118
|
+
anchorRaw = false;
|
|
3119
|
+
const wx = held[0] - free[0];
|
|
3120
|
+
const wy = held[1] - free[1];
|
|
3121
|
+
xs.push(wx);
|
|
3122
|
+
// Into the pelvis's turned frame, then through its setup matrix: the translate is in the parent's local axes.
|
|
3123
|
+
const r = -turnAbove(a.pelvis, i) / DEG;
|
|
3124
|
+
const ux = Math.cos(r) * wx - Math.sin(r) * wy;
|
|
3125
|
+
const uy = Math.sin(r) * wx + Math.cos(r) * wy;
|
|
3126
|
+
keys.push({ t: keyTimes[i]!, v: [r3((pr.d * ux - pr.b * uy) / det), r3((-pr.c * ux + pr.a * uy) / det)] });
|
|
3127
|
+
}
|
|
3128
|
+
tracks.push({ bone: a.bone, property: 'translate', keys });
|
|
3129
|
+
anchor = { gait: 'walk', bone: a.bone, point: a.point, joint: a.joints[1], x: r3(anchorX!), factor: r6(anchorFactor), neck: r3(Math.max(...anchorPath!) - Math.min(...anchorPath!)), sway: r3(Math.max(...xs) - Math.min(...xs)) };
|
|
3130
|
+
}
|
|
3002
3131
|
let face: FaceReading | null = null;
|
|
3003
3132
|
if (map.face !== undefined) {
|
|
3004
3133
|
const rEar = series.get('r_ear') ?? (() => {
|
|
@@ -3136,6 +3265,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
3136
3265
|
|
|
3137
3266
|
const pivot = pivotOf(rest, t?.bone ?? null, legs);
|
|
3138
3267
|
const pelvis = pelvisReadingOf(pelvisChain, opts.pelvis, track, fromTrack, tracks);
|
|
3268
|
+
if (pelvis !== null && anchor !== null) pelvis.anchor = anchor;
|
|
3139
3269
|
const scaleFit = t === null || scale === null || origin === null ? null : scaleFitOf({
|
|
3140
3270
|
joints: t.scale.pose, joint: t.joint, bone: t.bone, lengths: scaleLengths, factor: scale.factor,
|
|
3141
3271
|
series, good, legLength: feet?.legLength ?? null, legStraight, rigLeg,
|
|
@@ -3187,10 +3317,16 @@ export function pelvisLine(r: PelvisReading, frames: number): string {
|
|
|
3187
3317
|
const where = r.bone === null ? '' : `bone "${r.bone}", chain "${r.chain}"`;
|
|
3188
3318
|
if (r.bone === null) return `PELVIS: not keyed — the sequence carries a ${r.trackSource} track (${frames - r.declined.reduce((s, [a, b]) => s + b - a + 1, 0)} of ${frames} frames read) and the map has no pelvis chain (one bone, joints r_hip → l_hip); no bone turns with the pelvis`;
|
|
3189
3319
|
const swing = r.swing === null ? '' : `, swing ${r.swing.toFixed(2)}°`;
|
|
3190
|
-
if (r.trackSource === null) return `PELVIS: from hip line (no track)${swing} (${where}; every frame from r_hip → l_hip)`;
|
|
3191
|
-
if (r.track === 0) return `PELVIS: from hip line (the ${r.trackSource} track read none of the ${frames} frames)${swing} (${where}; every frame from r_hip → l_hip)`;
|
|
3320
|
+
if (r.trackSource === null) return `PELVIS: from hip line (no track)${swing} (${where}; every frame from r_hip → l_hip)${anchorClause(r.anchor)}`;
|
|
3321
|
+
if (r.track === 0) return `PELVIS: from hip line (the ${r.trackSource} track read none of the ${frames} frames)${swing} (${where}; every frame from r_hip → l_hip)${anchorClause(r.anchor)}`;
|
|
3192
3322
|
const rest = r.hipLine === 0 ? `all ${frames} frames from the track` : `${r.track} of ${frames} frames from the track, the hip line on the ${r.hipLine} it declined (frames ${spans(r.declined)})`;
|
|
3193
|
-
return `PELVIS: from ${r.source}${swing} (${where}; ${rest})`;
|
|
3323
|
+
return `PELVIS: from ${r.source}${swing} (${where}; ${rest})${anchorClause(r.anchor)}`;
|
|
3324
|
+
}
|
|
3325
|
+
|
|
3326
|
+
/** rig-c/pose#344: the PELVIS line's clause for the torso held over the pelvis. */
|
|
3327
|
+
function anchorClause(a: TorsoAnchor | undefined): string {
|
|
3328
|
+
if (a === undefined) return '';
|
|
3329
|
+
return `; torso anchored (in-place ${a.gait}): bone "${a.bone}" keyed with the pelvis's counter, "${a.point}" carried on the drawn ${a.joint}'s sideways path about x ${a.x.toFixed(2)} (× ${a.factor.toFixed(4)}, the pelvis's rig ÷ drawn sway; ${a.neck.toFixed(2)} peak to peak, the counter ${a.sway.toFixed(2)}), the pelvis swaying under it`;
|
|
3194
3330
|
}
|
|
3195
3331
|
|
|
3196
3332
|
/** A reach bone's origin within this many world units of the anchor's, with nothing above it moved, is aimed along the pair: no REACH line. */
|
package/src/run.ts
CHANGED
|
@@ -168,7 +168,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
|
|
|
168
168
|
const slots = slotsOf(doc0);
|
|
169
169
|
resolveMap(map, rest, a.map, slots, input.sets);
|
|
170
170
|
|
|
171
|
-
const r = retarget(rest, map, frames, times, { ...a, ...(seq.travel !== undefined ? { travel: seq.travel } : {}), ...(seq.cleaned!.params.loop ? { loop: true } : {}), ...(seq.pelvisSource !== undefined ? { pelvis: { source: seq.pelvisSource, tilt: seq.frames.map((f) => f.pelvisTilt ?? null) } } : {}), ...(contactKeys !== null ? { contactKeys } : {}) }, slots, input.sets);
|
|
171
|
+
const r = retarget(rest, map, frames, times, { ...a, ...(seq.travel !== undefined ? { travel: seq.travel } : {}), ...(seq.cleaned!.params.loop ? { loop: true } : {}), ...(seq.pelvisSource !== undefined ? { pelvis: { source: seq.pelvisSource, tilt: seq.frames.map((f) => f.pelvisTilt ?? null) } } : {}), ...(contactKeys !== null ? { contactKeys } : {}), ...(seq.cleaned!.params.depth !== undefined ? { gait: seq.cleaned!.params.depth.gait } : {}) }, slots, input.sets);
|
|
172
172
|
if (fragment.length > 0 && ((r.face?.sets?.length ?? 0) > 0 || (r.body?.sets?.length ?? 0) > 0)) {
|
|
173
173
|
const known = knownSetsOf(map, input.sets);
|
|
174
174
|
for (const s of [...(r.face?.sets ?? []), ...(r.body?.sets ?? [])]) {
|