@rig-c/pose 0.17.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -10
- package/cli.ts +18 -12
- package/package.json +1 -1
- package/src/clean.ts +7 -3
- package/src/legs.ts +246 -60
- package/src/map.ts +49 -3
- package/src/pose.ts +15 -3
- package/src/reduce.ts +167 -7
- package/src/retarget.ts +118 -37
- package/src/run.ts +202 -27
- package/src/switch.ts +31 -8
package/README.md
CHANGED
|
@@ -35,7 +35,7 @@ The steps, in this order, each printed with its figures:
|
|
|
35
35
|
1. **Spikes.** A joint more than 0.2 trunk lengths from its 5-frame median takes the median. The trunk is neck → mid-hip, the median over the clip. Each replacement is listed by frame and joint (`SPIKE`). Without `--loop`, a clip end cuts the window of the first and last two frames. There the median is a neighbour's value, and a joint moving steadily would read as a spike. Those frames are **not tested**, and the `SPIKES` line and the record name them. Reflecting the window does not help: the median of `[x2, x1, x0, x1, x2]` is still `x1`.
|
|
36
36
|
2. **Occlusion.** A joint whose state is not `observed` (or `inferred`), or whose score is under 0.3, takes its 3-frame median (`OCCLUDED`).
|
|
37
37
|
3. **Smoothing.** A quadratic Savitzky–Golay per joint: the legs (both hips, knees and ankles) over `--legs` frames (default 5), every other joint over `--rest` (default 7). A window is 1 (none) or odd and at least 5. 3 is refused because a quadratic through 3 points passes through all three. Without `--loop`, the window is cut at the ends. Where fewer than three frames have a position, the degree drops to fit the ones that do.
|
|
38
|
-
4. **Floor.** Only with `--loop`: the whole figure moves in y by one constant, so that the lowest ankle over the cycle is where it was before smoothing.
|
|
38
|
+
4. **Floor.** Only with `--loop`: the whole figure moves in y by one constant, so that the lowest ankle over the cycle is where it was before smoothing. A loop is also written with `"travel": "in-place"` ([#92](https://github.com/rig-c/pose/issues/92), the `TRAVEL` line): one cycle of a gait, repeated, does not travel, whatever its supporting foot does in the picture ([below](#the-pose-rig-pose-sequence1)).
|
|
39
39
|
5. **Hold** ([#91](https://github.com/rig-c/pose/issues/91)). A joint that has stopped is written at one position (`HOLD`), because detector noise on a still joint is not motion, and no smoothing window removes it without also smearing the movement into the stop. Below.
|
|
40
40
|
|
|
41
41
|
A **median** is taken per coordinate (x and y apart), over the window's frames on which the joint has a position, its own frame included, whatever its state. An even count takes the mean of the middle two. **`--loop`** says the sequence is one cycle, such as a gait from strike to strike. Every window then wraps, the last frame running into the first, so nothing is untested; a cycle shorter than its widest window is refused (`CLEAN_LOOP`). **Nothing is invented.** A joint with no position on a frame gets none: `retarget` holds it and prints a `HOLD` line. States and scores are written as they were read. `--coco` reads COCO-17 (below) and converts it first. `--person` picks the person when there is more than one, and only that person is written.
|
|
@@ -134,17 +134,18 @@ and what the run cannot decide for you it lists:
|
|
|
134
134
|
- `RANGE` — a chain's `range` ([below](#a-joints-range-range)): a bone whose rotate key the pose asks past its range, outside every span of a set depicting its chain, is clamped at the limit; one line per run, with the frames, the seconds and the largest overflow. The demand for an alternate.
|
|
135
135
|
- `ART-RANGE` — inside a set's span: a bone with a `drawn` angle in the set, turned from it past its limits — the set's `turn` for that bone (held at the limit) or, without one, its chain's `range` (the bone follows the pose). Listed as the pose asked it either way: the demand for another drawing of that set.
|
|
136
136
|
- `RAISED` — with `--raised-by-head` ([below](#alternate-drawings-alternates)): one line per span of a set that draws an arm raised, per chain. It gives the frames and the median eye distances (the rig's in units, the reference's in px). It gives the wrist's offset from the head centre in eye distances: the median of the reference's, of where the wrist was placed, of the rig's on the pose's angles, and of the rig's as keyed. It gives the worst |rig − reference| before and after, the worst |after − placed|, and the frames whose upper arm the set's `turn` held. `RAISED none` when the flag is given and no such set showed.
|
|
137
|
-
- `
|
|
137
|
+
- `BY-HIP` — with a set's `"place": "by-hip"` ([below](#alternate-drawings-alternates)): the `RAISED` line against the hip, one per span of such a set per chain it places. The unit is the hip width, the offset is from the mid-hip. `BY-HIP none` when the map has such a set and none of its spans showed.
|
|
138
|
+
- `REACH` — with `--raised-by-head`, or a `"by-hip"` set: a placed span whose target was out of the arm's reach, too far or too near. It names the frames, which way, and the shortfall in eye distances (hip widths for `"by-hip"`). The wrist was placed at the reach limit instead. This is not `ART-RANGE`: the rig's proportions were the limit, not the art's range. A too-near frame is placed at the limit like a too-far one, and graded at the placed distance.
|
|
138
139
|
- `LEG-SCALE` — with `--leg-scale` ([below](#the-legs-foreshortening-and-planted-feet)): per leg segment, its straight length, the spread of its picture length over it, and the frames clamped at each limit.
|
|
139
140
|
- `PIVOT` — when the map translates a bone and has both legs: the translated bone's origin against its thigh roots' midpoint at setup, in units and hip widths — the pivot every hip turn swings the legs about.
|
|
140
141
|
- `FEET` — when the map has both legs: the supports and jumps read off the reference ankles (planted or not), and on the posed build, outside the jumps, the lowest ankle's vertical range and each support span's worst vertical range and horizontal drift of the supporting ankle, in leg lengths.
|
|
141
|
-
- `STANCE` — with `--stance` ([below](#the-legs-foreshortening-and-planted-feet)): one line per run of frames planted both (with the stretch keyed, its largest k) or
|
|
142
|
+
- `STANCE` — with `--stance` ([below](#the-legs-foreshortening-and-planted-feet)): one line per run of frames planted both (with the stretch keyed, its largest k), lifted, or past the tolerance below (the pose's height kept, [#114](https://github.com/rig-c/pose/issues/114)), with the free side, the frames and the tolerance; and one per run whose stretch fell past its bounds (`past the bounds`), with the k the floor needed, the bound it had and how far the foot was left from it; and, where the leg's own range set a lower bound below 0.92, one `bound from the leg's own range` line with each side's lowest.
|
|
142
143
|
- `SHEAR` — with `--leg-scale`: every scaled bone and every bone under one, how far its art is sheared and how long its world axes are against setup. A reading.
|
|
143
144
|
- `IDLE`, `IDLE-DROPPED`, `IDLE-KEPT-OUT`, `MERGE` — the rig's idle against the new animation, and what `--carry-idle` and `--merge` put back ([below](#the-idle-what-the-retarget-drops)).
|
|
144
145
|
- `REDUCE` — with `--reduce` ([below](#few-keys---reduce)): per bone, its keys before and after and its largest error, with the frame; then the total.
|
|
145
146
|
- `TURN` — a line across the body (hips, shoulders, eyes) shorter in the picture than 0.5 of its median length, or reversed, for at least 0.15 s: the body or the head turned. A reading of the pose, made whether or not the map uses the line; it changes no key. The demand for a turned-body drawing, as `END-ON` is for a foreshortened limb.
|
|
146
147
|
|
|
147
|
-
`--report <file>` writes all of it as JSON (`rig-pose-report/1`): `spec`, `rigc` (`{ version, entry }`: the rig-c that gated it), `rig`, `animation`, `frames`, `duration`, `cleaned` (the sequence's `cleaned` record, [above](#clean-first-rig-pose-clean)), `options` (every figure the run used, `switchTolerance`, `switchMin`, `switchAt`, `legScale`, `plantFeet`, `stance`, `floor` and `raisedByHead` included), `scale`, `holds`, `short`, `endOn`, `bends`, `flips`, `gate` (rig-c's gate summary, a string), `landings`, `posture`, `folds`, `legScale`, `pivot`, `feet` (`planted`, `legLength`, `supports`, `jumps`, `steps`, `changes`, `floor`, `shifts`, `blend`, and the posed `reading`; a jump is an inclusive `[first, last]` pair of frames; `floor` is the world y every support span was planted on and `shifts` holds one `{ side, frames, shift }` per support span in world units, `null` and empty without `--plant-feet`; `blend` is `{ bounds, runs: [{ side, frames, k: { min, max } }], over: [{ side, frames, needed, left }] }`, the supporting leg stretched on the blend frames, `null` without `--plant-feet`), `stance` (`tolerance`, `release`, `minSeconds`, `bounds`, `runs`: `{ state, side, frames, maxHeight, k, absorbed? }`, `over`: `{ side, frames, needed, left }`; `null` without `--stance`), `shear`, `idleDropped`, `idleKeptOut` and `merged` ([below](#the-idle-what-the-retarget-drops)), `reduce` (the tolerances and each row of `REDUCE`, [below](#few-keys---reduce)), and the demand lists. The fields a run is graded by:
|
|
148
|
+
`--report <file>` writes all of it as JSON (`rig-pose-report/1`): `spec`, `rigc` (`{ version, entry }`: the rig-c that gated it), `rig`, `animation`, `frames`, `duration`, `cleaned` (the sequence's `cleaned` record, [above](#clean-first-rig-pose-clean)), `options` (every figure the run used, `switchTolerance`, `switchMin`, `switchAt`, `legScale`, `plantFeet`, `stance`, `floor` and `raisedByHead` included), `scale`, `holds`, `short`, `endOn`, `bends`, `flips`, `gate` (rig-c's gate summary, a string), `landings`, `posture`, `folds`, `legScale`, `pivot`, `feet` (`planted`, `legLength`, `supports`, `jumps`, `steps`, `changes`, `lowered`, `floor`, `shifts`, `blend`, `closure`, and the posed `reading`; a jump is an inclusive `[first, last]` pair of frames; `lowered` holds `{ frames, was, now, cause? }` runs (`cause`, where `now` is `null`: `released` or `higher`), present only with `--plant-feet` on a sequence not declared in place ([#113](https://github.com/rig-c/pose/issues/113)); `floor` is the world y every support span was planted on and `shifts` holds one `{ side, frames, shift }` per support span in world units, `null` and empty without `--plant-feet`; `blend` is `{ bounds, runs: [{ side, frames, k: { min, max } }], over: [{ side, frames, needed, left }] }`, the supporting leg stretched on the blend frames, `null` without `--plant-feet`; `closure` is `{ drift, frames }`, the hip's x drift closed over an in-place loop, `null` unless the sequence is `"in-place"` and `--plant-feet` is given), `stance` (`tolerance`, `release`, `minSeconds`, `bounds`, `runs`: `{ state, side, frames, maxHeight, maxDepth?, k, absorbed? }` (`state` is `both`, `lifted` or `below`; `maxDepth`, on a `below` run, is the free ankle's largest distance below the supporting one in leg lengths), `over`: `{ side, frames, needed, left }`; `null` without `--stance`), `shear`, `idleDropped`, `idleKeptOut` and `merged` ([below](#the-idle-what-the-retarget-drops)), `reduce` (the tolerances and each row of `REDUCE`, [below](#few-keys---reduce)), and the demand lists. The fields a run is graded by:
|
|
148
149
|
|
|
149
150
|
| field | each entry |
|
|
150
151
|
| --- | --- |
|
|
@@ -161,8 +162,9 @@ The demand lists:
|
|
|
161
162
|
- `endOn` — above, under the short hold.
|
|
162
163
|
- `turns: [{ line, joints, frames, seconds, minFraction, reversed }]` — `line` is `hip`, `shoulder` or `eye`.
|
|
163
164
|
- `range: [{ chain, bone, frames, seconds, maxOverflow, key, limits }]` — `key` is the rotate key the pose asked for on the frame of the largest overflow, before the clamp.
|
|
164
|
-
- `artRange: [{ set, chain, bone, frames, seconds, maxOverflow, turn, limits, clamped, shortfall? }]` — `turn` is the signed art turn the pose asked on that frame; `clamped` says whether the set's `turn` held the bone there. `shortfall` (with `--raised-by-head`) marks a held upper arm whose re-aimed forearm left the wrist short of its placed point: the most, in eye distances, with `turn` and `maxOverflow` the upper arm's on that frame.
|
|
165
|
+
- `artRange: [{ set, chain, bone, frames, seconds, maxOverflow, turn, limits, clamped, shortfall?, anchor? }]` — `turn` is the signed art turn the pose asked on that frame; `clamped` says whether the set's `turn` held the bone there. `shortfall` (with `--raised-by-head`, or a `"by-hip"` set) marks a held upper arm whose re-aimed forearm left the wrist short of its placed point: the most, in eye distances (in hip widths with `anchor: "hip"`), with `turn` and `maxOverflow` the upper arm's on that frame.
|
|
165
166
|
- `raised: [{ set, chain, frames, seconds, eyes: { rig, reference }, reference, placed, before, after, worst: { before, after, placed }, short, upperHeld, reach: { min, max, farthest } }]` — with `--raised-by-head`. `reference`, `placed`, `before` and `after` are the wrist's offset from the head centre on each frame of the span, `[x, y]` in eye distances, x right and y up. `eyes` holds each side's median eye distance over the span. `worst.before` and `worst.after` are the largest |rig − reference| in x and in y; `worst.placed` is the largest |after − placed|. `short` lists the `REACH` runs, each `{ frames, how, shortfall: { max, median } }`, where `how` is `far`, `near` or `missed`. `upperHeld` lists the frames whose upper arm the set's `turn` held. `reach` gives |l0 − l1|, l0 + l1 and the farthest target from the shoulder, in world units.
|
|
167
|
+
- `byHip: [{ set, chain, frames, seconds, hips: { rig, reference }, reference, placed, before, after, worst: { before, after, placed }, short, upperHeld, reach: { min, max, farthest } }]` — Present only when a set of the map declares `place`; a run without one writes no `byHip` key. Each span of a `"place": "by-hip"` set on each chain it places ([#93](https://github.com/rig-c/pose/issues/93)): `raised`'s fields against the hip, the offsets from the mid-hip in hip widths, `hips` each side's median hip width over the span, `short` in hip widths.
|
|
166
168
|
|
|
167
169
|
Runs in `turns`, `range` and `artRange` merge across a gap of one frame, as `endOn` does; `seconds` runs from the first frame's key time to the end of the last frame.
|
|
168
170
|
|
|
@@ -202,10 +204,12 @@ rig-c's `scalex` multiplies the bone's x axis, and a child with the `normal` inh
|
|
|
202
204
|
- The supporting foot is the one whose reference ankle is lower in the picture. It changes only when the other ankle has been at least **0.02 leg lengths** lower for at least **0.1 s** (timed as `END-ON` runs are); a leg length is each side's median picture length hip → ankle, and the switch compares against their mean.
|
|
203
205
|
- The new support is held where it is on the frame it takes over, and the hip blends from the old derivation to the new over **3 frames**.
|
|
204
206
|
- A **jump** — both reference ankles **and** mid_hip more than **0.05** leg lengths above their **rolling floor**, the 95th-percentile picture y over ±1.0 s around the frame, for at least **2 frames** — releases the plant: the hip follows the pose's travel from where it was, and re-plants on landing (blended the same way). The body has to lift: an ankle's picture y drifts with depth, and under one floor for the whole sequence HQ's dancer standing still (frames 0–29 and 277–312, mid_hip moving 10 and 14 px) read as jumping, 152–155 of 313 frames in all.
|
|
207
|
+
- **In place** ([#92](https://github.com/rig-c/pose/issues/92)). On a sequence with `"travel": "in-place"`, the supporting foot of a figure that does not travel moves back in the picture while the body stays. Holding it where it planted turns that slide into the body's travel, so the hip walks off and never comes back. Under the declaration each plant's anchor follows its own ankle's picture x, scaled into the rig, from the frame it planted, and keeps its y on the one floor. No step releases a foot, since a foot that follows its track has no plant to drag. Every foot may take support, and the lower one does, by the rules above. The loop is then closed: the hip's x on frame i moves by −drift · i / (n − 1), where drift is its x on the last frame less its x on the first. The last frame's hip x is then the first's, and the loop carries no net travel. The y is the floor's and is not touched. `FEET` prints the drift closed, in place of the steps line, and the report's `feet.closure` carries `{ drift, frames }` (`null` otherwise). On the tests' synthetic in-place loop (40 frames, each supporting ankle sliding back 2 px a frame, the hips still), the hip moves less than 10⁻⁹ px. The same joints without the declaration put it 60 px forward over the loop. With a slide of 0.04 leg lengths a frame, both feet are released as stepping and never re-plant without the declaration, and the supports alternate with it. Jumps are read as before.
|
|
205
208
|
- A **step** releases the plant too: while the supporting ankle moves more than **0.03** leg lengths a frame, the other foot takes support if it is planted (not itself released or moving), and otherwise the hip follows the pose's travel. The released ankle may support again once it has moved less than **0.01** leg lengths a frame for **2 frames**. On the dancer the plant held the supporting ankle through a real step (frames 176–181, 579 → 718 px) and threw the hip sideways.
|
|
206
|
-
- **
|
|
207
|
-
-
|
|
208
|
-
-
|
|
209
|
+
- **Read from the lower ankle** ([#113](https://github.com/rig-c/pose/issues/113)). On a sequence not declared `"in-place"`, a foot whose ankle is at least **0.02 leg lengths** higher in the picture than the other's never takes support: not when a plant starts, and not when a step releases the other foot. Before, a step handed support to the only foot free to take it, even a folded, lifted one, and the hip was then placed under a foot in the air. A supporting foot that the other ankle has been that much lower than for **0.1 s** hands support to it when it may take it, as before; when the lower foot may not (it is released and not yet still, or moving), the supporting foot is released, and the hip follows the pose's travel until a foot may take support. Steps and jumps are read as before, and under `"in-place"` #92's reading stands. `FEET support read from the lower ankle` names every run of frames whose support differs from the reading before #113, the side before and after (`none`: no foot supports), and for each `none` why: `released`, the supporting foot let go because the lower ankle could not take support (released by a step, or moving), or the only foot free to take support was the higher one. The report's `feet.lowered` holds them as `{ frames, was, now, cause }` (absent in place). On the tests' synthetic step under a lifted leg (the lifted ankle 0.2 leg lengths up, the grounded one stepping 0.05 leg lengths on frame 10 and still from 11), the lifted leg took support on frames 10–13 before; now frames 10–11 have no supporting foot and the grounded one supports from 12, when it is still again.
|
|
210
|
+
- **One floor** ([#85](https://github.com/rig-c/pose/issues/85)). A 2D rig has no depth, so its floor is one line, and a planted foot is on it. The picture disagrees: a step toward the camera puts the foot lower in the frame, and a camera residual puts a whole span higher, so a plant held where the picture puts each span leaves the floor stepping between spans. Every support span's supporting ankle is therefore moved in y onto one floor, the hip and everything above with it. A span's *planted height* is the world y (on the rig, posed by rig-c) its supporting ankle would be held at; the floor is the median of the spans' planted heights, or **`--floor <y>`** (world units, y up; only with `--plant-feet`, exit 2 without it). Each span's *shift* is floor − its planted height. A change of support still blends the hip over the 3 frames, now between two spans on the floor, and a planted foot does not leave the floor: on each blend frame the **supporting** leg is stretched along its chain about its root, by the factor `--stance` uses, k = (root y − floor y) / (root y − ankle y), held within **[0.92, 1.08]**, its lower bound on the frame the supporting leg's own range as `--stance`'s ([#95](https://github.com/rig-c/pose/issues/95); where that went below 0.92, a `FEET bound from the leg's own range` line, the report's `feet.blend.range`, and each blend run's `lower`). A frame that needs more is held at the bound, the rest left to the blend, and listed `FEET past the bounds` with the k needed and how far the foot was left from the floor. An ankle already within 0.01 units (the landing check's origin tolerance) is not stretched. The stretched scales are scale targets, so the landing check holds them; `--stance` then stretches the free leg, a different chain, to the supporting ankle. A jump follows the pose's travel from the hip it left, which is on the floor, and lands on the floor, held the same way over its first 3 planted frames. `--stance` is unchanged: it stretches the free leg to the supporting ankle, which is on the floor. On the tests' synthetic stick, three spans 0, 6 and 1 px apart in the picture (0, 9 and 1.5 units on the rig) plant within 0.0001 units of one floor. With the shift forced to 0 they stay 9 units apart; without the stretch, the supporting ankle leaves the floor by 6.75 units on the first blend frame.
|
|
211
|
+
- `FEET` lists the supports, the jumps and the steps (each with the side, its frames, who took support and where it re-planted; in place, the loop's closure instead), and with `--plant-feet` the floor: its world y and where it came from, each span's shift in units and in that side's rig leg at setup (thigh origin → ankle), and the shifts' range, then one line per run of blend frames whose supporting leg was stretched (`blend held on the floor`) and per run `past the bounds`. The report carries them under `feet` (`supports`, `jumps`, `steps`, `changes`, `floor`, `shifts`, `blend`, `closure`).
|
|
212
|
+
- **`--stance <leg lengths>`** (only with `--plant-feet`, which it is refused without; off by default) plants **both** feet when the free one is near the floor ([#60](https://github.com/rig-c/pose/issues/60)). On a frame with a supporting side (so outside every jump), the free ankle's height above the supporting one is read in the picture, in the free side's leg length — the leg length `FEET` uses. The tolerance is a **distance** ([#114](https://github.com/rig-c/pose/issues/114)): within it, above the supporting ankle or below it, the frame is **planted both**: every bone of the free leg's chain has its `scalex` key multiplied by k = (root y − floor y) / (root y − ankle y) on the posed rig, the root being the chain's first bone's origin and the floor the supporting ankle, so the foot reaches the floor. A free ankle within the tolerance below the supporting one is planted too: its k is under 1, since a foot slightly under the floor is the same fault as one floating above it. A free ankle **past the tolerance below** the supporting one is not a stance but a support error (the foot read as supporting is the one in the air), so that frame is never planted both and keeps the pose's height; it is named on a `STANCE past the tolerance below` line with the side, the frames and the largest distance, and read `"below"` in the report's `stance.runs`. Before #114 the gap was signed, so any free ankle below the supporting one counted as within, and a lifted foot read as supporting had the grounded leg shrunk up to it. On the tests' synthetic stance, a free ankle 0.5 leg lengths below is read below and not planted; one 0.03 below is planted with k under 1. k is held within **[0.92, 1.08]**, except that the lower bound comes from the leg's own range in the sequence ([#95](https://github.com/rig-c/pose/issues/95)): per side, r = the leg's picture length hip → ankle on the frame ÷ its straight length (the 95th percentile of that length, as `--leg-scale` reads a segment's), and the frame's lower bound is min(0.92, r), so the rig leg may fold as far as the picture folds it on that frame and never less far than 0.92. The upper bound stays 1.08, since a picture leg does not read longer than straight. A sequence whose legs never fold below 0.92 keeps exactly [0.92, 1.08]. Where the range set a bound below 0.92 on a planted-both frame, one `STANCE bound from the leg's own range` line gives each side's lowest, as does the report's `stance.range`. A frame past its bound is listed `past the bounds` with the bound it had, held at the bound and not forced. Past the tolerance above, the foot is **lifted** and keeps the pose's height. The state is read with **hysteresis** and a **shortest run** ([#87](https://github.com/rig-c/pose/issues/87)), on the distance: a planted foot leaves the stance when its distance from the supporting one goes past the tolerance, and one past it is planted again only under **0.8** of it (both figures are on the `STANCE` line). A change of state takes effect only when its condition holds for **0.1 s** (by the frames' times). A shorter crossing with the other state on both sides of it keeps their state and is listed `absorbed` with its frames and the height that crossed (`absorbed: true` in the report's `stance.runs`, beside the run that took it in). One that runs to the end of a support span is taken, since nothing of the other state follows it. A threshold read frame by frame turned noise at the threshold into a pop: one frame at 0.0708 against 0.07 dropped the stretch for one frame and the foot jumped up and back. The stretched scales are the scale targets, so the landing check holds them. Measured with `--stance 0.07` and HQ's flags (`--smooth 7 --flip 90 --short 0.35 --bends clamp --leg-scale 0.8,1.05 --plant-feet --carry-idle`): on HQ's dancer, frames 94–175 outside the jumps, the free ankle's height above the supporting one on the posed build went from median 18.3 and p90 25.4 units to 0.0 and 0.0. On a second sequence (Toca, 492 frames at 30 fps) the 29 frames labelled by eye as both feet flat, outside the jumps, read 0.0 and 0.0. Of the 6 labelled one foot lifted, 5 stay above the tolerance; the sixth (frame 360) was 0.042 leg lengths up in the picture, within the tolerance, and so planted; without `--stance` it stood 6.5 units up, also under the tolerance (22.6 units there). That is 83 %, under the 90 % the card asked.
|
|
209
213
|
- The map needs one chain ending at each ankle whose last bone is aimed `"x"` (the ankle is that bone's tip), and a `translate` the legs hang from (`LEGS_MAP`). A `range`, or a set's `turn` or `offsets`, on a bone between the hip and an ankle is refused (`LEGS_RANGE`): the plant is derived before the range clamp, and a clamp after it would move the planted foot.
|
|
210
214
|
|
|
211
215
|
Both are checked on the gated build: every scale lands, each ankle lands where the keys put it, and the hip's origin where the plant put it. The figures were fixed in #47 before the dancer was measured; the defaults stay off until they pass.
|
|
@@ -244,7 +248,10 @@ The retarget writes one key per pose frame on every mapped bone. That is faithfu
|
|
|
244
248
|
- **Kept regardless:** the first and last frame; the frames either side of a switch boundary, on the chains the switch changes; and the first and last frame of every `FLIP` spread, `SHORT` span and `RANGE` run, on that bone.
|
|
245
249
|
- **The landing check holds the dense targets** with the tolerance added (0.01° + `deg`; 0.01 + `units`), and it is posed by rig-c as always.
|
|
246
250
|
- **On HQ's dancer** (`--smooth 7 --flip 90 --short 0.35 --bends clamp`; 313 frames, 11 mapped bones): `--reduce 1 --reduce-hip 1` takes the rotate keys from 3443 to 588 (5.86×), 296 of the spans curved, and the hip's translate keys from 313 to 96. The gate is green. rig-c's poser lands every bone within 0.9998° of the dense target (head) and the hip within 0.901 units. At 0.5° the keys drop 3.92×; at 2°, 8.38×.
|
|
247
|
-
- **
|
|
251
|
+
- **It runs last, on the final keys** (rig-c/pose#94): after `--plant-feet`'s plant, its one floor, the blend and `--stance` stretches and the in-place closure. Their keys are reduced like any other, and every landing check is run again on the reduced motion.
|
|
252
|
+
- A scaled bone's `scalex` track is thinned within 5e-5 of its setup length: half of the scale landing check's 1e-4, so a reduced curve cannot fail that check.
|
|
253
|
+
- A mapped bone under a scaled bone (keyed through its parent's posed matrix by `--leg-scale`) is solved again through its parent as reduced. Its rotate is then thinned with the error read as the world aim it lands on, and its `scalex` with the error read as the world length it lands on. Its world error is its own fit error, as down any other chain.
|
|
254
|
+
- Each ankle is held to the plant's tolerance plus the most the reduction may move it: the hip's tolerance (with `--reduce-hip`), plus the leg from the hip to the ankle (each bone's setup offset from its parent and the last bone's length) times the largest scale asked, times the angle tolerance in radians plus 5e-5. The ankle's `LAND` line prints that tolerance.
|
|
248
255
|
|
|
249
256
|
Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2` bad invocation. `rig-pose --help` has every flag.
|
|
250
257
|
|
|
@@ -268,6 +275,7 @@ Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2
|
|
|
268
275
|
- Joints are body-18 (`nose neck r_shoulder r_elbow r_wrist l_shoulder l_elbow l_wrist r_hip r_knee r_ankle l_hip l_knee l_ankle r_eye l_eye r_ear l_ear`); `r`/`l` are the **subject's** sides. `state` is `observed`, `occluded` or `missing`, as in the keypoints file, or `inferred` ([#63](https://github.com/rig-c/pose/issues/63)): a position the detector supplied but did not see. An `inferred` joint carries `at` (one without it is refused, `POSE_JOINT_AT`), and `retarget` keys it as it keys `observed`. An unlisted joint is missing.
|
|
269
276
|
- Two differences from the keypoints file: the unit is `picture-px`, and `at` may lie outside the picture (a hand leaves the frame; the score says how much to believe it).
|
|
270
277
|
- More than one person: `--person <id>`.
|
|
278
|
+
- `travel` (optional, [#92](https://github.com/rig-c/pose/issues/92)) is the clip's frame of reference. Its one value is `"in-place"`: a gait filmed without the figure travelling, such as a treadmill or reel walk, or a loop. `clean --loop` writes it, and it may be written by hand on any sequence. Any other value is refused by name (`POSE_TRAVEL`). Without it the figure's travel in the picture is its travel. It changes only `--plant-feet` ([below](#the-legs-foreshortening-and-planted-feet)).
|
|
271
279
|
- `score` is whatever the pose model gives. RTMPose's goes above 1, so `--min-score` is a cut on that model's scale, not a probability.
|
|
272
280
|
|
|
273
281
|
**From COCO-17** — the layout most pose models and datasets use — `rig-pose clean --coco in.json --out pose.json`, which converts and cleans in one step. `rig-pose convert --coco` converts only, and its output still needs `clean`. `retarget --coco` is refused (`POSE_NOT_CLEANED`), naming the `clean --coco` command. The input is `{ "format": "coco17-sequence/1", "width", "height", "source", "frames": [{ "t", "keypoints" }] }`, keypoints in COCO's order as 17 `[x, y, score]` or 51 numbers, or `null` for a frame with nobody. The mapping: COCO's `left_*`/`right_*` → body-18's `l_*`/`r_*` (both are the subject's sides), a score of 0 → `missing`, and body-18's `neck`, which COCO lacks, is the shoulders' midpoint scored the lower of the two (OpenPose's definition).
|
|
@@ -338,7 +346,8 @@ Bone rotation cannot carry every pose: an arm behind the head, a forearm toward
|
|
|
338
346
|
- `chains`: the map's chains whose pose the set depicts; it is selected by them. `pose`: the key skeleton the art was painted from, body-18 (or `mid_hip`/`mid_shoulder`) in picture px, y down, any scale and place — only its directions are read; every joint the listed chains use must be in it.
|
|
339
347
|
- `drawn` (optional): each bone's world angle (Spine degrees) as painted. It changes no key: while a set shows, a bone is keyed to the pose as always, and the art turns with it. What it gives is a figure: the art turn, how far each bone is from the angle its art was painted at (`artTurn`, and `ART-RANGE` past the bone's range).
|
|
340
348
|
- `offsets` (optional): a translate per bone, in its parent's axes, held while the set shows and 0 elsewhere (stepped keys), so the joints as painted are the pivots. A bone `translate` already keys (the hip) cannot take one.
|
|
341
|
-
-
|
|
349
|
+
- `place` (optional): `"by-hip"` places a hand that meets the body by its offset from the hip ([below](#alternate-drawings-alternates)).
|
|
350
|
+
- Each name resolves against the built rig and the map, and a miss is refused by name, listing what exists: `MAP_ALT_SLOT`, `MAP_ALT_ATTACHMENT`, `MAP_ALT_HIDES`, `MAP_ALT_CHAIN`, `MAP_ALT_JOINT`, `MAP_ALT_POSE`, `MAP_ALT_DRAWN`, `MAP_ALT_OFFSET` (a bone that is not a bone of the set's chains) and `MAP_ALT_PLACE`.
|
|
342
351
|
|
|
343
352
|
**When one shows.** Per frame and per set, the distance is the largest difference, in degrees, between a segment of its chains as the retarget asked for it (after holds, the short hold, the flip guard and the hinge check; before any range clamp) and the same segment in its key skeleton. On each frame the nearest set within `--switch-tolerance` (default 25°) shows, ties going to the map's order; a run of one set shorter than `--switch-min` (default 0.25 s) is dropped and the base art shows there. Both figures were fixed before a run ([#6](https://github.com/rig-c/pose/issues/6)). Sets compete only with those they share a slot (shown or hidden) or an offset bone with, so a drawing of each arm can show at once; two that compete never overlap.
|
|
344
353
|
|
|
@@ -366,6 +375,16 @@ Either way the `SWITCH` line ends with `moved on …, off …`: each boundary's
|
|
|
366
375
|
- **The solve.** Two bones on the rig's own lengths put the wrist on that point, the elbow bending to the side the pose's does. The set's `turn` is part of the solve. When it holds the upper arm, the forearm is re-aimed from the held elbow at the placed wrist rather than held on its own. `ART-RANGE` lists where the re-aimed forearm still cannot reach within `turn`, in two cases. One is a forearm past its own `turn`, held there. The other is a held upper arm whose re-aimed forearm leaves the wrist more than 0.01 eye distances short of its placed point; 0.01 is the float and rounding scale. That case is listed on the upper arm, with the shortfall in eye distances. The solved angles become the targets, which the keys and the landing check follow.
|
|
367
376
|
- Frames outside those spans, and sets that draw no arm raised, are keyed as without the flag, byte for byte. A raised chain must have two bones, each aimed `"x"` or at one point, and the eye points must not turn with it (`RAISED_CHAIN`).
|
|
368
377
|
|
|
378
|
+
**A hand that meets the body: `"place": "by-hip"`.** The same mismatch, pointed at the hip. On a clasped-hands set the pose's angles aim the forearms the right way, and the rig's arm lengths put the clasp lower or higher on the body than the video has it ([#93](https://github.com/rig-c/pose/issues/93)). A set may ask for its hands to be placed against the hip instead, `"place": "by-hip"` (off by default; no flag). It is the solve above, with the head's measures swapped for the hip's:
|
|
379
|
+
|
|
380
|
+
- **The key skeleton carries both hips.** A set with `"by-hip"` must have `r_hip` and `l_hip` in its `pose`: the map's author supplies them, where the hips were when the drawing was painted. A set without them is refused by name (`MAP_ALT_PLACE`, naming the missing hip); nothing is guessed from the chains' joints.
|
|
381
|
+
- **Which chains.** A chain of the set is placed when it ends at a wrist (`r_wrist`, `l_wrist`) that the set's `pose` puts within one hip width of the mid-hip: |wrist − (r_hip + l_hip) / 2| ≤ |r_hip − l_hip|, read on the key skeleton. A set asking for `"by-hip"` with no such chain is refused, naming each wrist's distance in hip widths, and so is a `place` other than `"by-hip"`, a `pose` whose hips meet, and a `pose` without a chain's wrist (`MAP_ALT_PLACE`; the missing wrist is also `MAP_ALT_JOINT`, as for any set).
|
|
382
|
+
- **The unit is the hip width**: on each side, its median over the span. The reference's is |r_hip − l_hip|. The rig's is the distance between its two hip points, posed from the targets of the bones above as the eyes are: the two points a chain carrying the hip line (`r_hip` → `l_hip`, either way round) names with an `"a>b"` aim, a pelvis; else the origins of the first bones of the two chains starting at `r_hip` and at `l_hip`, the legs. A map with neither is refused (`PLACE_HIP`), and so is a span over which either median is 0 (`PLACE_HIPS`).
|
|
383
|
+
- The target is the rig's mid-hip + (reference wrist − reference mid-hip) × (rig hip width ÷ reference hip width), the picture's y turned up, per frame. The order of solving, the reach, the two-bone solve inside the set's `turn` and `ART-RANGE` are those of `--raised-by-head`, with every eye distance a hip width (a held arm's shortfall too, its `artRange` entry marked `anchor: "hip"`), and the mid-hip where the head centre is.
|
|
384
|
+
- **Out of reach below the shoulder.** The reach limit is taken on the ray from the mid-hip through the target. A hand at the hips sits on the far side of the mid-hip from the shoulder on most rigs, so that ray usually runs away from the arm and never enters its reach. The wrist then goes to the reach's point nearest the target, on the line from the shoulder (`missed` in `REACH`).
|
|
385
|
+
- A chain both `--raised-by-head` and `"by-hip"` would place is refused (`PLACE_BOTH`). A placed chain must have two bones, each aimed `"x"` or at one point, and the hip points must not turn with it (`PLACE_CHAIN`).
|
|
386
|
+
- Frames outside the set's spans, the chains it does not place, and every map without `"place"` are keyed as before, byte for byte. The report's `byHip` carries each span.
|
|
387
|
+
|
|
369
388
|
## Compare: the rig's render against the video
|
|
370
389
|
|
|
371
390
|
```sh
|
package/cli.ts
CHANGED
|
@@ -30,7 +30,8 @@ clean (required before retarget; rig-c/pose#67)
|
|
|
30
30
|
--coco read --pose as coco17-sequence/1 and convert it first (neck = the shoulders' midpoint)
|
|
31
31
|
--person <id> the person to clean when the sequence holds more than one; only that person is written
|
|
32
32
|
--loop the sequence is one cycle (a gait's strike to strike): every window wraps, last frame
|
|
33
|
-
to first, and the floor is re-pinned
|
|
33
|
+
to first, and the floor is re-pinned; the sequence is written with "travel": "in-place"
|
|
34
|
+
(rig-c/pose#92)
|
|
34
35
|
--legs <n> Savitzky–Golay window over the legs — both hips, knees and ankles (default: 5)
|
|
35
36
|
--rest <n> Savitzky–Golay window over every other joint (default: 7)
|
|
36
37
|
n is 1 (none), or odd and ≥ 5: a quadratic through 3 points passes through all three
|
|
@@ -104,14 +105,18 @@ retarget
|
|
|
104
105
|
supporting ankle moved in y onto the median of the spans' planted heights (world y on
|
|
105
106
|
the rig, posed by rig-c), the hip and everything above with it; FEET lists each span's
|
|
106
107
|
shift. On the 3 blend frames the supporting leg's scalex is stretched (k within
|
|
107
|
-
[0.92, 1.08], past it listed) so its ankle stays on the floor (rig-c/pose#85)
|
|
108
|
+
[0.92, 1.08], past it listed) so its ankle stays on the floor (rig-c/pose#85).
|
|
109
|
+
On a sequence with "travel": "in-place" (rig-c/pose#92) each supporting foot follows its
|
|
110
|
+
own ankle's picture x, scaled, instead of being held, no step releases it, and the hip's x
|
|
111
|
+
drift from the first frame to the last is spread back over the frames, closing the loop
|
|
108
112
|
--floor <y> with --plant-feet: the floor's world y (rig units, y up) instead of the median
|
|
109
113
|
--stance <leg lengths> with --plant-feet: on a frame outside the jumps, a free ankle within this many
|
|
110
|
-
leg lengths (FEET's) of the supporting one in the picture
|
|
111
|
-
scalex keys stretched by k = (root y − floor y) / (root y − ankle y) until
|
|
112
|
-
reaches the floor, raised or lowered, k within [0.92,
|
|
113
|
-
|
|
114
|
-
the
|
|
114
|
+
leg lengths (FEET's) of the supporting one in the picture, above or below, is planted
|
|
115
|
+
too — its leg's scalex keys stretched by k = (root y − floor y) / (root y − ankle y) until
|
|
116
|
+
the foot reaches the floor, raised or lowered, k within [min(0.92, the leg's own fold on
|
|
117
|
+
the frame), 1.08] (#95; past it, reported, not forced); above the tolerance the foot is
|
|
118
|
+
lifted and keeps the pose's height; below it, it is reported (past the tolerance below,
|
|
119
|
+
#114) and not planted (STANCE lines; default: off; 0.07 is the card's, rig-c/pose#60)
|
|
115
120
|
--raised-by-head while a set whose arm is drawn raised shows (its "drawn" upper arm within 30° of straight
|
|
116
121
|
up), that arm's wrist goes where the reference's is against the head, in eye distances:
|
|
117
122
|
rig head centre + (reference wrist − reference head centre) × rig eye distance ÷ the
|
|
@@ -141,7 +146,10 @@ retarget
|
|
|
141
146
|
each bone against its parent's reduced error, so errors do not pile up down a chain.
|
|
142
147
|
Kept regardless: the first and last frame, the frames either side of a switch boundary
|
|
143
148
|
on the chains it changes, and the edges of every FLIP spread, SHORT span and RANGE run
|
|
144
|
-
(REDUCE lines; default: off).
|
|
149
|
+
(REDUCE lines; default: off). It runs last, on the final keys: with --leg-scale,
|
|
150
|
+
--plant-feet and --stance, a scaled bone's scalex is thinned within 5e-5 of its setup
|
|
151
|
+
length, a bone under one is re-solved through its parent as reduced, and each ankle
|
|
152
|
+
is held to what the reduced bones above it may move it by (rig-c/pose#94)
|
|
145
153
|
--reduce-hip <units> with --reduce, thin the hip's translate keys too, its world origin within units
|
|
146
154
|
of the dense keys (default: the hip stays dense)
|
|
147
155
|
--switch-tolerance <d> a set of alternate drawings (the map's "alternates") may show on a frame where
|
|
@@ -155,7 +163,7 @@ retarget
|
|
|
155
163
|
--build <dir> keep the gated build (skeleton.json, .atlas, .model.json) here
|
|
156
164
|
--render <dir> keep rigc's render of the animation (frames, contact.png, folds.json) here
|
|
157
165
|
--report <file> write the run's measurements (holds, short, endOn, flips, bends, landings, folds,
|
|
158
|
-
switches, uncovered, turns, range, artRange, raised,
|
|
166
|
+
switches, uncovered, turns, range, artRange, raised, byHip,
|
|
159
167
|
idleDropped, idleKeptOut, merged, reduce) as JSON
|
|
160
168
|
|
|
161
169
|
It writes nothing until rigc's gate is green with the animation in, every mapped bone lands on
|
|
@@ -321,9 +329,6 @@ function reduceOf(f: Map<string, string>): { reduce?: number; reduceHip?: number
|
|
|
321
329
|
return {};
|
|
322
330
|
}
|
|
323
331
|
const reduce = num(f, 'reduce', 0, (n) => n > 0 && n < 180, 'degrees in (0, 180)');
|
|
324
|
-
for (const other of ['leg-scale', 'plant-feet'] as const) {
|
|
325
|
-
if (f.has(other) && f.get(other) !== 'off') throw new Usage(`--reduce is given with --${other}; their keys land a bone through a scaled parent or an ankle's lever, which the reduction's tolerance does not bound (rig-c/pose#32 round 1)`);
|
|
326
|
-
}
|
|
327
332
|
return f.has('reduce-hip') ? { reduce, reduceHip: num(f, 'reduce-hip', 0, (n) => n > 0, 'rig units > 0') } : { reduce };
|
|
328
333
|
}
|
|
329
334
|
|
|
@@ -424,6 +429,7 @@ function runClean(argv: string[], say: (l: string) => void): void {
|
|
|
424
429
|
if (params.loop) {
|
|
425
430
|
if (c.floor === null) say(' FLOOR no ankle has a position on any frame; not re-pinned');
|
|
426
431
|
else say(` FLOOR the lowest ankle over the cycle: y ${fmt(c.floor.before)} before smoothing, ${fmt(c.floor.after)} after; every joint moved ${c.floor.shift >= 0 ? '+' : ''}${fmt(c.floor.shift, 3)} px in y`);
|
|
432
|
+
say(' TRAVEL "in-place" written (rig-c/pose#92): a loop is one cycle repeated, so the figure does not travel; retarget --plant-feet holds the hip\'s x over the loop and lets the supporting foot follow its own track');
|
|
427
433
|
}
|
|
428
434
|
const jit = (v: number | null) => (v === null ? 'not measured' : `${fmt(v, 3)}`);
|
|
429
435
|
say(` JITTER mean |second difference| of y over ${c.triples} joint-frame triple(s): ${jit(c.record.jitter.before)} → ${jit(c.record.jitter.after)} px/f²${c.record.jitter.before !== null && c.record.jitter.after !== null && c.record.jitter.before > 0 ? ` (${fmt(100 * (1 - c.record.jitter.after / c.record.jitter.before), 1)}% lower)` : ''}`);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@rig-c/pose",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.18.0",
|
|
4
4
|
"description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/src/clean.ts
CHANGED
|
@@ -23,7 +23,10 @@
|
|
|
23
23
|
* `rest`. A window of 1 is none.
|
|
24
24
|
* 4. **Floor.** With `loop` only: the whole figure is moved in y by one
|
|
25
25
|
* constant, so the lowest ankle over the cycle (the largest y of either
|
|
26
|
-
* ankle on any frame) is at its height before smoothing.
|
|
26
|
+
* ankle on any frame) is at its height before smoothing. A loop is also
|
|
27
|
+
* written with `travel: "in-place"` (rig-c/pose#92): one cycle of a gait,
|
|
28
|
+
* repeated, does not travel, whatever its supporting foot does in the
|
|
29
|
+
* picture.
|
|
27
30
|
* 5. **Hold** (#91, unless `hold` is "off"). A joint that has stopped is
|
|
28
31
|
* written at one position: detector noise on a still joint is not motion.
|
|
29
32
|
* `holdStill` below states the test and the write.
|
|
@@ -68,7 +71,7 @@ import type { CleanRecord, CleanSeam, CleanSpike, CleanUnheld, Joint, PoseSequen
|
|
|
68
71
|
export type Unheld = CleanUnheld;
|
|
69
72
|
/** Two holds of one joint back to back (#91): the second's first frame, and the step between their medians. */
|
|
70
73
|
export type Seam = CleanSeam;
|
|
71
|
-
import { CLEAN_SPEC, personTrack, sortedJoints } from './pose.ts';
|
|
74
|
+
import { CLEAN_SPEC, personTrack, sortedJoints, TRAVEL_IN_PLACE } from './pose.ts';
|
|
72
75
|
import { Problems } from './problems.ts';
|
|
73
76
|
|
|
74
77
|
/** A spike is a joint this many trunk lengths or more from its median. */
|
|
@@ -693,7 +696,8 @@ export function clean(seq: PoseSequence, params: CleanParams, wanted: string | u
|
|
|
693
696
|
});
|
|
694
697
|
const { cleaned: _was, ...rest } = seq;
|
|
695
698
|
return {
|
|
696
|
-
|
|
699
|
+
// #92: a loop is one cycle of a gait repeated in place, so it carries no travel of its own.
|
|
700
|
+
seq: { ...rest, ...(params.loop ? { travel: TRAVEL_IN_PLACE } : {}), cleaned: record, frames },
|
|
697
701
|
record,
|
|
698
702
|
person,
|
|
699
703
|
trunk: { length: trunk, frames: lengths.length },
|