@rig-c/pose 0.15.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +66 -9
- package/cli.ts +70 -7
- package/package.json +1 -1
- package/src/clean.ts +339 -7
- package/src/legs.ts +211 -15
- package/src/pose.ts +307 -21
- package/src/retarget.ts +351 -41
- package/src/run.ts +63 -10
- package/src/switch.ts +164 -0
- package/src/verify.ts +29 -4
package/README.md
CHANGED
|
@@ -25,10 +25,10 @@ The map names **chains**: each limb as bones that descend from one another, pare
|
|
|
25
25
|
## Clean first: `rig-pose clean`
|
|
26
26
|
|
|
27
27
|
```sh
|
|
28
|
-
rig-pose clean --pose dance.scan.json --out dance.pose.json [--loop] [--legs 5] [--rest 7] [--hips-inferred]
|
|
28
|
+
rig-pose clean --pose dance.scan.json --out dance.pose.json [--loop] [--legs 5] [--rest 7] [--hips-inferred] [--hold off] [--hold-speed 0.015] [--hold-reach 0.03]
|
|
29
29
|
```
|
|
30
30
|
|
|
31
|
-
`retarget` reads only a sequence that `clean` wrote ([#67](https://github.com/rig-c/pose/issues/67)). A pose model reads each frame alone, so a hanging wrist moves ±20 px a frame and a cloak-hidden leg jumps 100+ px, and every key carries that shake. `clean` takes it out once, at the sequence, where it can be inspected. A sequence without the record is refused by name (`POSE_NOT_CLEANED`), and the message is the `clean` command to run on that file. There is no flag to skip the step. Windows of 1 (`--legs 1 --rest 1`) are a legitimate run that smooths nothing, and they are recorded as such.
|
|
31
|
+
`retarget` reads only a sequence that `clean` wrote ([#67](https://github.com/rig-c/pose/issues/67)). Where the video's camera moves, the order is **scan → clean → retarget** ([#80](https://github.com/rig-c/pose/issues/80)): a zoom step is a jump of the picture, and `clean`'s spike and smoothing steps would read it as motion of the body, so `tools/scan.py` takes the camera out of the raw sequence first and `clean` reads what it writes. A pose model reads each frame alone, so a hanging wrist moves ±20 px a frame and a cloak-hidden leg jumps 100+ px, and every key carries that shake. `clean` takes it out once, at the sequence, where it can be inspected. A sequence without the record is refused by name (`POSE_NOT_CLEANED`), and the message is the `clean` command to run on that file. There is no flag to skip the step. Windows of 1 (`--legs 1 --rest 1`) are a legitimate run that smooths nothing, and they are recorded as such.
|
|
32
32
|
|
|
33
33
|
The steps, in this order, each printed with its figures:
|
|
34
34
|
|
|
@@ -36,6 +36,7 @@ The steps, in this order, each printed with its figures:
|
|
|
36
36
|
2. **Occlusion.** A joint whose state is not `observed` (or `inferred`), or whose score is under 0.3, takes its 3-frame median (`OCCLUDED`).
|
|
37
37
|
3. **Smoothing.** A quadratic Savitzky–Golay per joint: the legs (both hips, knees and ankles) over `--legs` frames (default 5), every other joint over `--rest` (default 7). A window is 1 (none) or odd and at least 5. 3 is refused because a quadratic through 3 points passes through all three. Without `--loop`, the window is cut at the ends. Where fewer than three frames have a position, the degree drops to fit the ones that do.
|
|
38
38
|
4. **Floor.** Only with `--loop`: the whole figure moves in y by one constant, so that the lowest ankle over the cycle is where it was before smoothing.
|
|
39
|
+
5. **Hold** ([#91](https://github.com/rig-c/pose/issues/91)). A joint that has stopped is written at one position (`HOLD`), because detector noise on a still joint is not motion, and no smoothing window removes it without also smearing the movement into the stop. Below.
|
|
39
40
|
|
|
40
41
|
A **median** is taken per coordinate (x and y apart), over the window's frames on which the joint has a position, its own frame included, whatever its state. An even count takes the mean of the middle two. **`--loop`** says the sequence is one cycle, such as a gait from strike to strike. Every window then wraps, the last frame running into the first, so nothing is untested; a cycle shorter than its widest window is refused (`CLEAN_LOOP`). **Nothing is invented.** A joint with no position on a frame gets none: `retarget` holds it and prints a `HOLD` line. States and scores are written as they were read. `--coco` reads COCO-17 (below) and converts it first. `--person` picks the person when there is more than one, and only that person is written.
|
|
41
42
|
|
|
@@ -50,7 +51,43 @@ The output carries the record, here the dancer's from the table below:
|
|
|
50
51
|
|
|
51
52
|
`jitter` is the mean |second difference| of y, in px/f², over every joint and every three consecutive frames that have a position, wrapping with `--loop`. It is measured on the sequence as read and on the cleaned positions. `retarget` prints the record as its `CLEAN` line and copies it into `--report`.
|
|
52
53
|
|
|
53
|
-
**`--hips-inferred`** ([#63](https://github.com/rig-c/pose/issues/63)) declares that the hips are not seen: under a skirt or a cloak the pose model still places them, but where it puts them is its guess. After the
|
|
54
|
+
**`--hips-inferred`** ([#63](https://github.com/rig-c/pose/issues/63)) declares that the hips are not seen: under a skirt or a cloak the pose model still places them, but where it puts them is its guess. After the steps, `r_hip` and `l_hip` are marked `inferred` on every frame where they have a position (`INFERRED`), and the record's params carry `"hipsInferred": { "by": "declared" }` — the person running `clean` said so. Positions and scores are unchanged, so `retarget`'s keys are the same with the flag and without it. What changes is the `POSTURE` line, which does not read a hip tilt off inferred hips (below), and the `CLEAN` line, which ends `hips inferred (declared at clean)`.
|
|
55
|
+
|
|
56
|
+
**The stillness hold** ([#91](https://github.com/rig-c/pose/issues/91)) runs on the cleaned positions, after the smoothing and the floor, **joint by joint**: stillness is a property of a joint, so a hand that has stopped is held while the elbow beside it still settles. A joint's speed on a frame is its movement from the frame before, in trunk lengths, taken as a 5-frame median. A **hold** of a joint is a span of at least 0.5 s (the first frame's time to the end of the last) on every frame of which its median speed is under `--hold-speed` (default 0.015 trunk lengths a frame), and over which it stays within `--hold-reach` (default 0.03 trunk lengths) of its own span median. Speed alone cannot tell a slow drift from stillness, because a drift accumulates. For each joint the span is chosen greedily, left to right: from a still frame it grows one frame at a time while the reach holds on the grown span, its median read again each time, and stops at the first frame that breaks it. If what it reached lasts 0.5 s it is a hold and the search resumes after it; if not, the search moves one frame on. A span does not cross a loop's seam.
|
|
57
|
+
|
|
58
|
+
On every frame of a hold the joint takes its per-coordinate median over the span. It blends in linearly over the 6 frames before it (the 6th stays as cleaned, the frame before the span is ⅚ of the way) and out over the 6 frames after it, mirrored. A blend frame inside another hold of the same joint is not touched; where two blends of one joint meet, the larger weight wins. Where one hold of a joint ends on frame k and the next starts on k + 1 + g, with a gap g of under 6 frames (a **seam**: the reach broke while the joint was still moving slowly, or a few frames between two holds failed a test), the two medians differ, and the joint would jump between them. A seam cross-fades over 6 + g frames, symmetric about the gap's centre: frames k − 2 … k + 3 + g take 1/(2(6 + g)), 3/(2(6 + g)), … of the way from the first median to the second (for g = 0: 1/12, 3/12, … 11/12, the blend's sixths placed symmetric about the seam), so no frame moves more than 1/(6 + g) of the step. The cross-fade replaces the blends on the gap's frames. Each seam is printed (`SEAM`) and recorded with the joint, the second hold's first frame, the step in trunk lengths and the gap. A hold that carries a cross-fade has a non-zero jitter after: the ramp's own curvature, printed as `(ramp)`, not noise. A joint that fails keeps its cleaned track. A joint with no position on a frame gets none, and states and scores are kept.
|
|
59
|
+
|
|
60
|
+
Each hold is printed (`HOLD`, joint by joint, with its largest speed and reach) and recorded with the joint's jitter inside it, before and after. Where some joints are held, every run of the same frames (the held spans joined where they overlap or touch) on which a joint has no hold is named with the test it fails there and its figure (`UNHELD`, and the record's `unheld`): `speed` (its largest median speed), else `reach` (its largest distance from its median over the run), else `short` (it is still, but for under 0.5 s; the figure is its seconds), else `span` (it passes all three, and its spans broke elsewhere).
|
|
61
|
+
|
|
62
|
+
```json
|
|
63
|
+
"params": { "loop": false, "legs": 5, "rest": 7, "hold": { "speed": 0.015, "reach": 0.03, "minSeconds": 0.5, "blend": 6 } },
|
|
64
|
+
"holds": [{ "joint": "nose", "frames": [first, last], "jitter": { "before": before, "after": 0 } }, …],
|
|
65
|
+
"seams": [{ "joint": "neck", "frame": second hold's first frame, "step": trunk lengths, "gap": frames }, …],
|
|
66
|
+
"unheld": [{ "joint": "l_elbow", "frames": [first, last], "test": "reach", "value": figure }, …]
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The `CLEAN` line names, for each region, the joints held over which frames and the frames not held with their test and figure and the seams cross-faded (`HOLD frames a–b: held nose, neck, … a–b; seams cross-faded: neck f (step …); not held: l_elbow c–d (reach … > 0.03)`), or says `no HOLD`. `--hold off` turns the step off: the positions are those of a clean without it, and the record carries `"hold": "off"` and none of `holds`, `seams` and `unheld`. A record written before #91 names none of them, and is read as it was.
|
|
70
|
+
|
|
71
|
+
**Measured hips: `tools/pelvis.py`** ([#86](https://github.com/rig-c/pose/issues/86), not shipped) reads the pelvis line where the eye reads it under a skirt, off the skirt's waistband, and runs after `clean`: scan → clean → pelvis → retarget. It takes the video and a sequence `clean` wrote without `--hips-inferred`. It refuses by name, writing nothing:
|
|
72
|
+
|
|
73
|
+
- a sequence with no `cleaned` record (`PELVIS_NOT_CLEANED`);
|
|
74
|
+
- one whose hips are already marked (`PELVIS_HIPS_INFERRED`);
|
|
75
|
+
- a frame count other than the video's (`PELVIS_FRAMES`);
|
|
76
|
+
- a clip on which no frame gives a reading (`PELVIS_NO_READING`).
|
|
77
|
+
|
|
78
|
+
On each frame it reads the top edge of the dark region at the waist. The region is set by the hips and the frame's trunk length. The tool keeps the edge's flattest high run, since a flaring skirt's sides are steep and are not the pelvis line, and fits a line to it.
|
|
79
|
+
|
|
80
|
+
A frame gives no reading, declined with the reason, when:
|
|
81
|
+
|
|
82
|
+
- a wrist lies in front of the waist (the picture under it is not the skirt's dark);
|
|
83
|
+
- the edge is too broken or too far from a line;
|
|
84
|
+
- the reading is over 25° from level, or a lone outlier: further, per frame, than the misread guard allows from both the read before it and the reading after it.
|
|
85
|
+
|
|
86
|
+
The tool never draws a line through a hand in front. A declined frame does not fall back to the detector's hip line, which is the value known to be wrong there. If two reads lie close in time on either side, the frame takes the interpolation between them. Otherwise it holds the earlier read to the end of its run, so the written angle changes only at a read; at the clip's start, with no earlier read, it holds the later one.
|
|
87
|
+
|
|
88
|
+
Both hips are marked `inferred` on every frame and turned about their midpoint to the frame's angle, keeping the midpoint and the width. The record says which frame is which: `"hipsInferred": { "by": "measured", "method", "read", "interpolated", "held": [[first, last], …], "declined": [{ "frames": [first, last], "why" }] }`. Every frame is in exactly one of `read`, `interpolated` and `held`, and `declined` lists exactly the interpolated and held frames. Runs are frame indices inside the sequence, increasing and disjoint. A malformed record is refused by name (`POSE_CLEANED`).
|
|
89
|
+
|
|
90
|
+
`tools/pelvis_selftest.py` is its synthetic check. A planted wrist on a hand in front of the waist reads "covered", and with the hand moved away the edge's angle is read again. The misread guard, the interpolation and the hold are checked frame by frame. Its mutant (`--no-cover-check`) must read a line through the hand, and the check asserts that it does.
|
|
54
91
|
|
|
55
92
|
**Measured** on HQ's references ([#67](https://github.com/rig-c/pose/issues/67)'s bars):
|
|
56
93
|
|
|
@@ -91,11 +128,13 @@ and what the run cannot decide for you it lists:
|
|
|
91
128
|
- `FLIP` — a segment that turns more than `--flip` degrees in one frame (with `--flip <deg>`; off by default) is spread over the frames around the jump; one line per spread, with the chain, the segment, the jump, the frames and h. Below.
|
|
92
129
|
- `BEND` — with `--bends clamp`, a hinge bent the wrong way is straightened on that frame instead of refusing.
|
|
93
130
|
- `FOLD` — `rigc render --folds` over the new animation: every mesh that folds, and when. A reading, not a gate.
|
|
94
|
-
- `POSTURE` — when the map has a pelvis chain (joints `r_hip`, `l_hip`) and a `translate`: the hip line's tilt range, and the sway — the x-range of mid_hip − mid_shoulder — in rig units and in shoulder widths, the pose's beside the posed rig's. A reading, not a gate. A frame on which either hip is `inferred` is left out of the tilt. When that is every frame, as after `clean --hips-inferred`, the line is `hip tilt not read: hips inferred (declared at clean)` and the report's `posture` has `"tilt": null` with `"tiltWhy"`; on some frames only, the tilt is read over the others and the line counts the frames left out (`posture.hipsInferred`). The source is read from the clean record, or is `source not stated` when the joints carry the state and the record does not. The sway reads the hips as before.
|
|
131
|
+
- `POSTURE` — when the map has a pelvis chain (joints `r_hip`, `l_hip`) and a `translate`: the hip line's tilt range, and the sway — the x-range of mid_hip − mid_shoulder — in rig units and in shoulder widths, the pose's beside the posed rig's. A reading, not a gate. A frame on which either hip is `inferred` is left out of the tilt. When that is every frame, as after `clean --hips-inferred`, the line is `hip tilt not read: hips inferred (declared at clean)` and the report's `posture` has `"tilt": null` with `"tiltWhy"`; on some frames only, the tilt is read over the others and the line counts the frames left out (`posture.hipsInferred`). The source is read from the clean record, or is `source not stated` when the joints carry the state and the record does not. With a `measured` record ([#86](https://github.com/rig-c/pose/issues/86)), the read, interpolated and held frames are the measurement, so the tilt is read on all three. The line adds `tilt measured (<method>) on N frame(s): R read, I interpolated, H held (held: <spans>)`. The report's `posture` carries `tiltMeasured: { method, frames, read, interpolated, held, heldSpans }`. The sway reads the hips as before.
|
|
95
132
|
- `SWITCH` — with `alternates` in the map ([below](#alternate-drawings-alternates)): each span on which a set of alternate drawings shows, its slots, its frames, how far the pose was from the set's key skeleton, how far each bone with a `drawn` angle was turned from it, and where each of its boundaries was moved to (`moved`, [below](#alternate-drawings-alternates)).
|
|
96
133
|
- `UNCOVERED` — with `alternates` in the map: every fold on a frame where no switch took the folding mesh off screen. This is the list of poses still to paint.
|
|
97
134
|
- `RANGE` — a chain's `range` ([below](#a-joints-range-range)): a bone whose rotate key the pose asks past its range, outside every span of a set depicting its chain, is clamped at the limit; one line per run, with the frames, the seconds and the largest overflow. The demand for an alternate.
|
|
98
135
|
- `ART-RANGE` — inside a set's span: a bone with a `drawn` angle in the set, turned from it past its limits — the set's `turn` for that bone (held at the limit) or, without one, its chain's `range` (the bone follows the pose). Listed as the pose asked it either way: the demand for another drawing of that set.
|
|
136
|
+
- `RAISED` — with `--raised-by-head` ([below](#alternate-drawings-alternates)): one line per span of a set that draws an arm raised, per chain. It gives the frames and the median eye distances (the rig's in units, the reference's in px). It gives the wrist's offset from the head centre in eye distances: the median of the reference's, of where the wrist was placed, of the rig's on the pose's angles, and of the rig's as keyed. It gives the worst |rig − reference| before and after, the worst |after − placed|, and the frames whose upper arm the set's `turn` held. `RAISED none` when the flag is given and no such set showed.
|
|
137
|
+
- `REACH` — with `--raised-by-head`: a raised span whose target was out of the arm's reach, too far or too near. It names the frames, which way, and the shortfall in eye distances. The wrist was placed at the reach limit instead. This is not `ART-RANGE`: the rig's proportions were the limit, not the art's range. A too-near frame is placed at the limit like a too-far one, and graded at the placed distance.
|
|
99
138
|
- `LEG-SCALE` — with `--leg-scale` ([below](#the-legs-foreshortening-and-planted-feet)): per leg segment, its straight length, the spread of its picture length over it, and the frames clamped at each limit.
|
|
100
139
|
- `PIVOT` — when the map translates a bone and has both legs: the translated bone's origin against its thigh roots' midpoint at setup, in units and hip widths — the pivot every hip turn swings the legs about.
|
|
101
140
|
- `FEET` — when the map has both legs: the supports and jumps read off the reference ankles (planted or not), and on the posed build, outside the jumps, the lowest ankle's vertical range and each support span's worst vertical range and horizontal drift of the supporting ankle, in leg lengths.
|
|
@@ -105,7 +144,7 @@ and what the run cannot decide for you it lists:
|
|
|
105
144
|
- `REDUCE` — with `--reduce` ([below](#few-keys---reduce)): per bone, its keys before and after and its largest error, with the frame; then the total.
|
|
106
145
|
- `TURN` — a line across the body (hips, shoulders, eyes) shorter in the picture than 0.5 of its median length, or reversed, for at least 0.15 s: the body or the head turned. A reading of the pose, made whether or not the map uses the line; it changes no key. The demand for a turned-body drawing, as `END-ON` is for a foreshortened limb.
|
|
107
146
|
|
|
108
|
-
`--report <file>` writes all of it as JSON (`rig-pose-report/1`): `options` (every figure the run used, `switchTolerance`, `switchMin`, `switchAt`, `legScale`, `plantFeet` and `
|
|
147
|
+
`--report <file>` writes all of it as JSON (`rig-pose-report/1`): `spec`, `rigc` (`{ version, entry }`: the rig-c that gated it), `rig`, `animation`, `frames`, `duration`, `cleaned` (the sequence's `cleaned` record, [above](#clean-first-rig-pose-clean)), `options` (every figure the run used, `switchTolerance`, `switchMin`, `switchAt`, `legScale`, `plantFeet`, `stance`, `floor` and `raisedByHead` included), `scale`, `holds`, `short`, `endOn`, `bends`, `flips`, `gate` (rig-c's gate summary, a string), `landings`, `posture`, `folds`, `legScale`, `pivot`, `feet` (`planted`, `legLength`, `supports`, `jumps`, `steps`, `changes`, `floor`, `shifts`, `blend`, and the posed `reading`; a jump is an inclusive `[first, last]` pair of frames; `floor` is the world y every support span was planted on and `shifts` holds one `{ side, frames, shift }` per support span in world units, `null` and empty without `--plant-feet`; `blend` is `{ bounds, runs: [{ side, frames, k: { min, max } }], over: [{ side, frames, needed, left }] }`, the supporting leg stretched on the blend frames, `null` without `--plant-feet`), `stance` (`tolerance`, `release`, `minSeconds`, `bounds`, `runs`: `{ state, side, frames, maxHeight, k, absorbed? }`, `over`: `{ side, frames, needed, left }`; `null` without `--stance`), `shear`, `idleDropped`, `idleKeptOut` and `merged` ([below](#the-idle-what-the-retarget-drops)), `reduce` (the tolerances and each row of `REDUCE`, [below](#few-keys---reduce)), and the demand lists. The fields a run is graded by:
|
|
109
148
|
|
|
110
149
|
| field | each entry |
|
|
111
150
|
| --- | --- |
|
|
@@ -122,7 +161,8 @@ The demand lists:
|
|
|
122
161
|
- `endOn` — above, under the short hold.
|
|
123
162
|
- `turns: [{ line, joints, frames, seconds, minFraction, reversed }]` — `line` is `hip`, `shoulder` or `eye`.
|
|
124
163
|
- `range: [{ chain, bone, frames, seconds, maxOverflow, key, limits }]` — `key` is the rotate key the pose asked for on the frame of the largest overflow, before the clamp.
|
|
125
|
-
- `artRange: [{ set, chain, bone, frames, seconds, maxOverflow, turn, limits, clamped }]` — `turn` is the signed art turn the pose asked on that frame; `clamped` says whether the set's `turn` held the bone there.
|
|
164
|
+
- `artRange: [{ set, chain, bone, frames, seconds, maxOverflow, turn, limits, clamped, shortfall? }]` — `turn` is the signed art turn the pose asked on that frame; `clamped` says whether the set's `turn` held the bone there. `shortfall` (with `--raised-by-head`) marks a held upper arm whose re-aimed forearm left the wrist short of its placed point: the most, in eye distances, with `turn` and `maxOverflow` the upper arm's on that frame.
|
|
165
|
+
- `raised: [{ set, chain, frames, seconds, eyes: { rig, reference }, reference, placed, before, after, worst: { before, after, placed }, short, upperHeld, reach: { min, max, farthest } }]` — with `--raised-by-head`. `reference`, `placed`, `before` and `after` are the wrist's offset from the head centre on each frame of the span, `[x, y]` in eye distances, x right and y up. `eyes` holds each side's median eye distance over the span. `worst.before` and `worst.after` are the largest |rig − reference| in x and in y; `worst.placed` is the largest |after − placed|. `short` lists the `REACH` runs, each `{ frames, how, shortfall: { max, median } }`, where `how` is `far`, `near` or `missed`. `upperHeld` lists the frames whose upper arm the set's `turn` held. `reach` gives |l0 − l1|, l0 + l1 and the farthest target from the shoulder, in world units.
|
|
126
166
|
|
|
127
167
|
Runs in `turns`, `range` and `artRange` merge across a gap of one frame, as `endOn` does; `seconds` runs from the first frame's key time to the end of the last frame.
|
|
128
168
|
|
|
@@ -163,8 +203,9 @@ rig-c's `scalex` multiplies the bone's x axis, and a child with the `normal` inh
|
|
|
163
203
|
- The new support is held where it is on the frame it takes over, and the hip blends from the old derivation to the new over **3 frames**.
|
|
164
204
|
- A **jump** — both reference ankles **and** mid_hip more than **0.05** leg lengths above their **rolling floor**, the 95th-percentile picture y over ±1.0 s around the frame, for at least **2 frames** — releases the plant: the hip follows the pose's travel from where it was, and re-plants on landing (blended the same way). The body has to lift: an ankle's picture y drifts with depth, and under one floor for the whole sequence HQ's dancer standing still (frames 0–29 and 277–312, mid_hip moving 10 and 14 px) read as jumping, 152–155 of 313 frames in all.
|
|
165
205
|
- A **step** releases the plant too: while the supporting ankle moves more than **0.03** leg lengths a frame, the other foot takes support if it is planted (not itself released or moving), and otherwise the hip follows the pose's travel. The released ankle may support again once it has moved less than **0.01** leg lengths a frame for **2 frames**. On the dancer the plant held the supporting ankle through a real step (frames 176–181, 579 → 718 px) and threw the hip sideways.
|
|
166
|
-
-
|
|
167
|
-
-
|
|
206
|
+
- **One floor** ([#85](https://github.com/rig-c/pose/issues/85)). A 2D rig has no depth, so its floor is one line, and a planted foot is on it. The picture disagrees: a step toward the camera puts the foot lower in the frame, and a camera residual puts a whole span higher, so a plant held where the picture puts each span leaves the floor stepping between spans. Every support span's supporting ankle is therefore moved in y onto one floor, the hip and everything above with it. A span's *planted height* is the world y (on the rig, posed by rig-c) its supporting ankle would be held at; the floor is the median of the spans' planted heights, or **`--floor <y>`** (world units, y up; only with `--plant-feet`, exit 2 without it). Each span's *shift* is floor − its planted height. A change of support still blends the hip over the 3 frames, now between two spans on the floor, and a planted foot does not leave the floor: on each blend frame the **supporting** leg is stretched along its chain about its root, by the factor `--stance` uses, k = (root y − floor y) / (root y − ankle y), held within **[0.92, 1.08]**. A frame that needs more is held at the bound, the rest left to the blend, and listed `FEET past the bounds` with the k needed and how far the foot was left from the floor. An ankle already within 0.01 units (the landing check's origin tolerance) is not stretched. The stretched scales are scale targets, so the landing check holds them; `--stance` then stretches the free leg, a different chain, to the supporting ankle. A jump follows the pose's travel from the hip it left, which is on the floor, and lands on the floor, held the same way over its first 3 planted frames. `--stance` is unchanged: it stretches the free leg to the supporting ankle, which is on the floor. On the tests' synthetic stick, three spans 0, 6 and 1 px apart in the picture (0, 9 and 1.5 units on the rig) plant within 0.0001 units of one floor. With the shift forced to 0 they stay 9 units apart; without the stretch, the supporting ankle leaves the floor by 6.75 units on the first blend frame.
|
|
207
|
+
- `FEET` lists the supports, the jumps and the steps (each with the side, its frames, who took support and where it re-planted), and with `--plant-feet` the floor: its world y and where it came from, each span's shift in units and in that side's rig leg at setup (thigh origin → ankle), and the shifts' range, then one line per run of blend frames whose supporting leg was stretched (`blend held on the floor`) and per run `past the bounds`. The report carries them under `feet` (`supports`, `jumps`, `steps`, `changes`, `floor`, `shifts`, `blend`).
|
|
208
|
+
- **`--stance <leg lengths>`** (only with `--plant-feet`, which it is refused without; off by default) plants **both** feet when the free one is near the floor ([#60](https://github.com/rig-c/pose/issues/60)). On a frame with a supporting side (so outside every jump), the free ankle's height above the supporting one is read in the picture, in the free side's leg length — the leg length `FEET` uses. Within the tolerance the frame is **planted both**: every bone of the free leg's chain has its `scalex` key multiplied by k = (root y − floor y) / (root y − ankle y) on the posed rig, the root being the chain's first bone's origin and the floor the supporting ankle, so the foot reaches the floor. A free ankle below the supporting one is planted too: its k is under 1, since a foot under the floor is the same fault as one floating above it. k is held within **[0.92, 1.08]**, and a frame past those bounds is listed `past the bounds`, held at the bound and not forced. Above the tolerance the foot is **lifted** and keeps the pose's height. The state is read with **hysteresis** and a **shortest run** ([#87](https://github.com/rig-c/pose/issues/87)): a planted foot is lifted when its height goes past the tolerance, and a lifted one is planted again only under **0.8** of it (both figures are on the `STANCE` line). A change of state takes effect only when its condition holds for **0.1 s** (by the frames' times). A shorter crossing with the other state on both sides of it keeps their state and is listed `absorbed` with its frames and the height that crossed (`absorbed: true` in the report's `stance.runs`, beside the run that took it in). One that runs to the end of a support span is taken, since nothing of the other state follows it. A threshold read frame by frame turned noise at the threshold into a pop: one frame at 0.0708 against 0.07 dropped the stretch for one frame and the foot jumped up and back. The stretched scales are the scale targets, so the landing check holds them. Measured with `--stance 0.07` and HQ's flags (`--smooth 7 --flip 90 --short 0.35 --bends clamp --leg-scale 0.8,1.05 --plant-feet --carry-idle`): on HQ's dancer, frames 94–175 outside the jumps, the free ankle's height above the supporting one on the posed build went from median 18.3 and p90 25.4 units to 0.0 and 0.0. On a second sequence (Toca, 492 frames at 30 fps) the 29 frames labelled by eye as both feet flat, outside the jumps, read 0.0 and 0.0. Of the 6 labelled one foot lifted, 5 stay above the tolerance; the sixth (frame 360) was 0.042 leg lengths up in the picture, within the tolerance, and so planted; without `--stance` it stood 6.5 units up, also under the tolerance (22.6 units there). That is 83 %, under the 90 % the card asked.
|
|
168
209
|
- The map needs one chain ending at each ankle whose last bone is aimed `"x"` (the ankle is that bone's tip), and a `translate` the legs hang from (`LEGS_MAP`). A `range`, or a set's `turn` or `offsets`, on a bone between the hip and an ankle is refused (`LEGS_RANGE`): the plant is derived before the range clamp, and a clamp after it would move the planted foot.
|
|
169
210
|
|
|
170
211
|
Both are checked on the gated build: every scale lands, each ankle lands where the keys put it, and the hip's origin where the plant put it. The figures were fixed in #47 before the dancer was measured; the defaults stay off until they pass.
|
|
@@ -232,7 +273,13 @@ Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2
|
|
|
232
273
|
**From COCO-17** — the layout most pose models and datasets use — `rig-pose clean --coco in.json --out pose.json`, which converts and cleans in one step. `rig-pose convert --coco` converts only, and its output still needs `clean`. `retarget --coco` is refused (`POSE_NOT_CLEANED`), naming the `clean --coco` command. The input is `{ "format": "coco17-sequence/1", "width", "height", "source", "frames": [{ "t", "keypoints" }] }`, keypoints in COCO's order as 17 `[x, y, score]` or 51 numbers, or `null` for a frame with nobody. The mapping: COCO's `left_*`/`right_*` → body-18's `l_*`/`r_*` (both are the subject's sides), a score of 0 → `missing`, and body-18's `neck`, which COCO lacks, is the shoulders' midpoint scored the lower of the two (OpenPose's definition).
|
|
233
274
|
|
|
234
275
|
|
|
235
|
-
**The camera** ([#63](https://github.com/rig-c/pose/issues/63)). A sequence may carry an optional `camera`, one `{ "s", "tx", "ty", "inliers" }` per frame: the picture's scale and translation against frame 0's, so a point p₀ of frame 0's picture appears at `s · p₀ + (tx, ty)` on that frame. Its joints are then already normalised by it, `p′ = (p − t) / s`, and `inliers` is how many background features the fit kept. `s` must be > 0, `tx` and `ty` finite, `inliers` an integer ≥ 0, and there is one entry per frame; anything else is refused by name (`POSE_CAMERA`). The retarget does not apply it again: it prints a `CAMERA` line with the range of `s` and the three frames where it changes fastest. With `--plant-feet` and no `camera`, the `CAMERA` line says the floor may step with the picture, since a zoom or a pan in the video moves the floor with it. An optional `cameraSource` string says where the camera came from: `"tracked"
|
|
276
|
+
**The camera** ([#63](https://github.com/rig-c/pose/issues/63)). A sequence may carry an optional `camera`, one `{ "s", "tx", "ty", "inliers" }` per frame: the picture's scale and translation against frame 0's, so a point p₀ of frame 0's picture appears at `s · p₀ + (tx, ty)` on that frame. Its joints are then already normalised by it, `p′ = (p − t) / s`, and `inliers` is how many background features the fit kept. `s` must be > 0, `tx` and `ty` finite, `inliers` an integer ≥ 0, and there is one entry per frame; anything else is refused by name (`POSE_CAMERA`). The retarget does not apply it again: it prints a `CAMERA` line with the range of `s` and the three frames where it changes fastest. With `--plant-feet` and no `camera`, the `CAMERA` line says the floor may step with the picture, since a zoom or a pan in the video moves the floor with it. An optional `cameraSource` string says where the camera came from: `"tracked"`, `"hand-labelled, N keys"` or `"hand windows, floor-snapped: …"`. It is refused without a `camera`, or when it is not a non-empty string (`POSE_CAMERA_SOURCE`), and the `CAMERA` line starts with it (`CAMERA tracked: …`; a sequence with a `camera` and no `cameraSource` prints `CAMERA (source not stated): …`). `tools/scan.py` (not shipped) writes the normalised sequence. It reads the sequence before `clean` does (scan → clean → retarget): a sequence that already carries a `cleaned` record is refused by name (`SCAN_AFTER_CLEAN`), since its cleaning was done with the camera's steps still in it. The camera can come three ways, exactly one per run:
|
|
277
|
+
|
|
278
|
+
- **Tracked** (`--video`), where the background has texture to follow. The camera is read off the video's background, and the sequence is written with `"cameraSource": "tracked"`. A frame whose fit keeps too few background features is refused by name, and nothing is written.
|
|
279
|
+
- **A hand file** (`--camera-file`), where the background cannot be tracked and the camera's values are known or judged by hand. The file is `rig-pose-camera/1`: `{ "spec": "rig-pose-camera/1", "keys": [ { "frame", "s", "tx", "ty" } ], "note"? }`. Its frames are integers of the pose and strictly increase, `s` is > 0, and `tx` and `ty` are finite. Values are linear between keys and held before the first key and after the last. `inliers` is 0 on every frame, since no feature was fitted, and anything malformed is refused by name (`CAMERA_FILE`). The source is `"hand-labelled, N keys"`.
|
|
280
|
+
- **Hand windows, snapped on the floor** (`--camera-windows a-b,c-d,…`, [#82](https://github.com/rig-c/pose/issues/82)), where the background cannot be tracked and the camera zooms in steps with a full-body figure on the floor. The windows are the frames of each zoom, by eye. Each step is measured on the floor, the one background feature every full-body clip has. An ankle is planted on a frame when it is still both ways, moving less than the retarget's stillness threshold (0.01 leg lengths a frame, in one picture leg: the clip median of hip → ankle) from the previous frame and to the next (through `bun tools/supports.ts`, beside the retarget's own jumps from `supportsOf`). Jump frames are not excluded: the detector reads them on the uncorrected picture, so they carry the very camera steps the windows remove, and an airborne ankle cannot hold still over three frames. A frame read inside a detector jump is named in the source (`r 187 inside detector jump 187-193`), not refused. For each window and each ankle, y before and y after are the ankle's median picture y on the 3 planted frames nearest the window on each side, searched within 15 frames and never past a neighbouring window, and its ratio about the picture centre c is `(y_after − c_y) / (y_before − c_y)`. Where both ankles read, the step is the mean of the two, and their disagreement is always written. Where one reads, the window says which ankle alone. A window no ankle reads is refused by name (`CAMERA_WINDOWS`), and so are windows that are not `a ≤ b` inside the pose's frames, that overlap, or that leave no frame between them; nothing is written. s is 1 before the first window, multiplies by each step, is linear across a window and is held between windows, and `t = c (1 − s)`, so no pan is read. The slow drift between windows (a push-in) is held, not fitted: that is open. The source is `"hand windows, floor-snapped: a-b (r+l, l/r x.x %; r 83,84,85|97,100,104; l …), c-d (r alone; r …)"` (`no window` for none): each window names the frames every median stood on, before and after the bar, and the `CAMERA` line prints it. The step moves with those frames: a 3-frame median moved ±2 % with the frames picked on a reference clip, which over three windows decided whether 37 or 107 of its 265 support frames left the floor band ([#82](https://github.com/rig-c/pose/issues/82)). `--plot` draws both ankles' y with the windows shaded and the frames each window was read on marked, where a wrong window shows.
|
|
281
|
+
|
|
282
|
+
`tools/scan_selftest.py` is the synthetic check of all three.
|
|
236
283
|
|
|
237
284
|
**The older name.** `rig-c-pose-frames/0`, written by the extraction jobs this package's demos were made from, is the same file under another spec string. It has `space.units` "painting-px" and an `fps` key; each frame's `t` is the time read. It is read as it is.
|
|
238
285
|
## The map: `rig-pose-map/1`
|
|
@@ -309,6 +356,16 @@ Either way the `SWITCH` line ends with `moved on …, off …`: each boundary's
|
|
|
309
356
|
**A stopgap window: `frames`.** A set may state `"frames": [[first, last], …]`: it shows only inside those windows, and on them it is chosen over any set within tolerance that has no window. It is for two sets whose key skeletons the angles cannot tell apart. Measured on HQ's dancer, the hands clasped behind the back and the hands low in front differ by only 2–21°, and the wrist and hand-point scores do not separate them either; the cue that might is #33's depth state. It is recorded in the map as a stopgap, not a selection rule. A window **restricts** a set; it never forces one. Where the set does not show inside its window, the run says why in a `FRAMES` line: the stretch, and whether the pose there was past the tolerance (with the largest distance), or within it but the run was dropped.
|
|
310
357
|
|
|
311
358
|
**How far a set's art may turn: `turn`.** A drawing reads right only near the angle it was painted at. A set may state, per bone with a `drawn` angle, how far its art may turn from it, `"turn": { "sleeve_r0": [-20, 20] }`. While the set shows, the bone is held within it: its target becomes drawn + clamp(wrap(target − drawn), min, max), each bone on its own world angle, so a forearm under a held upper arm keeps its own target. The landing check measures against the held angle. The overflow the pose asked for is listed as `ART-RANGE`. A `turn` on a bone the set has no `drawn` angle for is refused (`MAP_ALT_TURN`).
|
|
359
|
+
|
|
360
|
+
**A raised arm placed against the head: `--raised-by-head`.** The pose's angles put a hand where it is against the dancer's own shoulder. On a rig whose shoulders sit nearer its head, or whose head is larger, the same angles put a hand raised beside the head on top of it. With `--raised-by-head` (off by default, [#62](https://github.com/rig-c/pose/issues/62)) a set draws an arm **raised** when its `drawn` angle for the chain's first bone (the upper arm) is within 30° of straight up (a world angle of 60°–120°). While that set shows, the arm's wrist goes where the reference's is **against the head**:
|
|
361
|
+
|
|
362
|
+
- The unit is the **eye distance**: on each side, its median over the span. That is the reference's |r_eye − l_eye|, and on the rig the distance between the two points the head chain's `"a>b"` aim names. The head centre is the eyes' midpoint on each frame. The head chain is the one carrying the eye line (r_eye → l_eye, either way round). The flag is refused without one, or with one not aimed `"a>b"` (`RAISED_HEAD`).
|
|
363
|
+
- The target is rig head centre + (reference wrist − reference head centre) × (rig eye distance ÷ reference eye distance), the picture's y turned up, per frame.
|
|
364
|
+
- **The order of solving.** The arm is solved last of the bones. By then the hip's travel, every reach, the head and every range clamp are final, and the arm moves neither the head nor its own shoulder. On each frame the rig's eye points and the shoulder are posed forward from those targets, as a reach's origin is: each rest offset is turned by the nearest mapped bone above.
|
|
365
|
+
- **Reach.** The arm reaches the annulus about the shoulder from |l0 − l1| to l0 + l1. Here l0 runs from the upper arm's origin to the forearm's, and l1 from there to the wrist (the forearm's tip, or the point it aims at). A target outside it is replaced by the reach limit along the head-relative direction. That is the point inside the annulus, on the ray from the head centre through the target, nearest the target. Where the ray crosses the annulus twice, the nearer stretch wins, and a tie goes to the point nearer the head. A ray that never enters it gives the annulus point nearest the target. Every such frame is listed in `REACH`. A too-near frame is treated as a too-far one: the wrist is placed at the reach limit, and the distance the rule grades it at is the placed one. That is |placed − head centre|, not the reference's, which the arm cannot fold to.
|
|
366
|
+
- **The solve.** Two bones on the rig's own lengths put the wrist on that point, the elbow bending to the side the pose's does. The set's `turn` is part of the solve. When it holds the upper arm, the forearm is re-aimed from the held elbow at the placed wrist rather than held on its own. `ART-RANGE` lists where the re-aimed forearm still cannot reach within `turn`, in two cases. One is a forearm past its own `turn`, held there. The other is a held upper arm whose re-aimed forearm leaves the wrist more than 0.01 eye distances short of its placed point; 0.01 is the float and rounding scale. That case is listed on the upper arm, with the shortfall in eye distances. The solved angles become the targets, which the keys and the landing check follow.
|
|
367
|
+
- Frames outside those spans, and sets that draw no arm raised, are keyed as without the flag, byte for byte. A raised chain must have two bones, each aimed `"x"` or at one point, and the eye points must not turn with it (`RAISED_CHAIN`).
|
|
368
|
+
|
|
312
369
|
## Compare: the rig's render against the video
|
|
313
370
|
|
|
314
371
|
```sh
|
package/cli.ts
CHANGED
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
*/
|
|
9
9
|
import { readFileSync, writeFileSync } from 'node:fs';
|
|
10
10
|
import pkg from './package.json' with { type: 'json' };
|
|
11
|
-
import { clean, cleanLine, untestedText, LEGS, OCCLUSION_SCORE, OCCLUSION_WINDOW, SPIKE_TRUNKS, SPIKE_WINDOW, windowProblem } from './src/clean.ts';
|
|
11
|
+
import { clean, cleanLine, holdRegions, unheldText, untestedText, HOLD_BLEND, HOLD_MIN_SECONDS, HOLD_REACH, HOLD_SPEED, HOLD_SPEED_WINDOW, LEGS, OCCLUSION_SCORE, OCCLUSION_WINDOW, SPIKE_TRUNKS, SPIKE_WINDOW, windowProblem } from './src/clean.ts';
|
|
12
12
|
import { convertCoco17, readPoseSequence, writePoseSequence } from './src/pose.ts';
|
|
13
13
|
import { parseWindows } from './src/compare.ts';
|
|
14
14
|
import { lineOf, Refusal } from './src/problems.ts';
|
|
@@ -18,7 +18,7 @@ import { runRetarget, type RetargetArgs } from './src/run.ts';
|
|
|
18
18
|
const HELP = `rig-pose ${pkg.version} — motion from a pose sequence onto a rig-c rig
|
|
19
19
|
|
|
20
20
|
Usage:
|
|
21
|
-
rig-pose clean --pose <pose.json> --out <pose.clean.json> [--loop] [--legs 5] [--rest 7] [--coco] [--person <id>]
|
|
21
|
+
rig-pose clean --pose <pose.json> --out <pose.clean.json> [--loop] [--legs 5] [--rest 7] [--hold off] [--coco] [--person <id>]
|
|
22
22
|
rig-pose retarget --rig <rig> --map <map.json> --pose <pose.json> --out <motion.json> [--name dance] [flags]
|
|
23
23
|
rig-pose convert --coco <coco17.json> --out <pose.json>
|
|
24
24
|
rig-pose compare --reference <pose.json> --render <pose.json> --map <map.json> [flags]
|
|
@@ -38,6 +38,9 @@ clean (required before retarget; rig-c/pose#67)
|
|
|
38
38
|
position the detector supplied but did not see — on every frame where they have one, and
|
|
39
39
|
the record says so (hipsInferred: { by: "declared" }). Positions are unchanged, so retarget's
|
|
40
40
|
keys are too; its POSTURE line does not read the hip tilt ("hip tilt not read: hips inferred")
|
|
41
|
+
--hold off turn the stillness hold off (rig-c/pose#91); recorded as hold: "off"
|
|
42
|
+
--hold-speed <n> the hold's speed limit, trunk lengths per frame (default: 0.015)
|
|
43
|
+
--hold-reach <n> how far a held joint may lie from its span median, trunk lengths (default: 0.03)
|
|
41
44
|
|
|
42
45
|
In order, each step printed with its figures: (1) spikes — a joint more than 0.2 trunk lengths
|
|
43
46
|
(neck → mid_hip, the median over the clip) from its 5-frame median takes the median, each listed by
|
|
@@ -48,6 +51,12 @@ clean (required before retarget; rig-c/pose#67)
|
|
|
48
51
|
A median is per coordinate over the window's frames where the joint has a position. A joint with no
|
|
49
52
|
position on a frame is left without one (retarget holds it). States and scores are written as read.
|
|
50
53
|
Jitter: the mean |second difference| of y over every joint, px/f², before and after.
|
|
54
|
+
(5) the stillness hold (rig-c/pose#91), per joint, on the cleaned positions: a span of ≥ 0.5 s where the
|
|
55
|
+
joint's 5-frame median speed (its movement from the frame before, in trunk lengths) is under --hold-speed and
|
|
56
|
+
it stays within --hold-reach of its own span median is written as that median, blended in linearly over the
|
|
57
|
+
6 frames before and out over the 6 after (two holds of a joint back to back, or under 6 frames apart, cross-fade
|
|
58
|
+
over the 6 frames plus the gap, centred on it, SEAM); each hold is printed (HOLD) and recorded with its jitter before and
|
|
59
|
+
after, and the joints not held where others are are named with the test each fails (UNHELD).
|
|
51
60
|
|
|
52
61
|
retarget
|
|
53
62
|
--rig <path> the rig spec rigc builds from: a rig.json, or a directory holding one
|
|
@@ -91,13 +100,30 @@ retarget
|
|
|
91
100
|
rolling floor — the 95th-percentile y over ±1 s — for ≥ 2 frames) follows the pose's
|
|
92
101
|
travel and re-plants on landing; a supporting ankle moving > 0.03 leg lengths a frame
|
|
93
102
|
releases its plant to the other foot, or to the pose's travel, until it is still
|
|
94
|
-
(< 0.01 for 2 frames) (FEET lines)
|
|
103
|
+
(< 0.01 for 2 frames) (FEET lines). Every support span is planted on one floor: its
|
|
104
|
+
supporting ankle moved in y onto the median of the spans' planted heights (world y on
|
|
105
|
+
the rig, posed by rig-c), the hip and everything above with it; FEET lists each span's
|
|
106
|
+
shift. On the 3 blend frames the supporting leg's scalex is stretched (k within
|
|
107
|
+
[0.92, 1.08], past it listed) so its ankle stays on the floor (rig-c/pose#85)
|
|
108
|
+
--floor <y> with --plant-feet: the floor's world y (rig units, y up) instead of the median
|
|
95
109
|
--stance <leg lengths> with --plant-feet: on a frame outside the jumps, a free ankle within this many
|
|
96
110
|
leg lengths (FEET's) of the supporting one in the picture is planted too — its leg's
|
|
97
111
|
scalex keys stretched by k = (root y − floor y) / (root y − ankle y) until the foot
|
|
98
112
|
reaches the floor, raised or lowered, k within [0.92, 1.08] (past it, reported, not forced); above
|
|
99
113
|
it the foot is lifted and keeps the pose's height (STANCE lines; default: off; 0.07 is
|
|
100
114
|
the card's, rig-c/pose#60)
|
|
115
|
+
--raised-by-head while a set whose arm is drawn raised shows (its "drawn" upper arm within 30° of straight
|
|
116
|
+
up), that arm's wrist goes where the reference's is against the head, in eye distances:
|
|
117
|
+
rig head centre + (reference wrist − reference head centre) × rig eye distance ÷ the
|
|
118
|
+
reference's, per frame; each eye distance is its median over the span. The head centre is
|
|
119
|
+
the eyes' midpoint; the rig's eyes are the two points the head chain's "a>b" aim names
|
|
120
|
+
(refused without one, RAISED_HEAD), posed from the targets of the bones above. A target out
|
|
121
|
+
of the arm's reach, too far or too near, goes to the reach limit on the ray from the head
|
|
122
|
+
centre toward it and is graded at that placed distance (REACH lines, in eye distances). A
|
|
123
|
+
two-bone solve on the rig's own lengths keeps the pose's elbow side; when the set's "turn"
|
|
124
|
+
holds the upper arm the forearm is re-aimed at the wrist. ART-RANGE: a forearm still past
|
|
125
|
+
its turn, or a held arm leaving the wrist > 0.01 eye distances short of its placed point
|
|
126
|
+
(the shortfall listed). RAISED lines (default: off; rig-c/pose#62)
|
|
101
127
|
--idle <name> the rig's animation read as its idle (default: idle). Every idle track the new
|
|
102
128
|
animation does not key, on any channel, is listed as IDLE-DROPPED (a blink, a
|
|
103
129
|
breath); one it keys is IDLE-KEPT-OUT, naming the track that keys it, and so is a
|
|
@@ -129,7 +155,7 @@ retarget
|
|
|
129
155
|
--build <dir> keep the gated build (skeleton.json, .atlas, .model.json) here
|
|
130
156
|
--render <dir> keep rigc's render of the animation (frames, contact.png, folds.json) here
|
|
131
157
|
--report <file> write the run's measurements (holds, short, endOn, flips, bends, landings, folds,
|
|
132
|
-
switches, uncovered, turns, range, artRange,
|
|
158
|
+
switches, uncovered, turns, range, artRange, raised,
|
|
133
159
|
idleDropped, idleKeptOut, merged, reduce) as JSON
|
|
134
160
|
|
|
135
161
|
It writes nothing until rigc's gate is green with the animation in, every mapped bone lands on
|
|
@@ -228,7 +254,7 @@ function legScaleOf(f: Map<string, string>): [number, number] | 'off' {
|
|
|
228
254
|
}
|
|
229
255
|
|
|
230
256
|
function retargetArgs(argv: string[]): RetargetArgs {
|
|
231
|
-
const f = parseFlags(argv, ['rig', 'map', 'pose', 'out', 'name', 'person', 'base-motion', 'images', 'min-score', 'smooth', 'flip', 'short', 'bends', 'bend-tolerance', 'switch-tolerance', 'switch-min', 'switch-at', 'profile', 'build', 'render', 'report', 'leg-scale', 'idle', 'merge', 'reduce', 'reduce-hip', 'stance'], ['coco', 'plant-feet', 'carry-idle']);
|
|
257
|
+
const f = parseFlags(argv, ['rig', 'map', 'pose', 'out', 'name', 'person', 'base-motion', 'images', 'min-score', 'smooth', 'flip', 'short', 'bends', 'bend-tolerance', 'switch-tolerance', 'switch-min', 'switch-at', 'profile', 'build', 'render', 'report', 'leg-scale', 'idle', 'merge', 'reduce', 'reduce-hip', 'stance', 'floor'], ['coco', 'plant-feet', 'carry-idle', 'raised-by-head']);
|
|
232
258
|
const bends = f.get('bends') ?? 'refuse';
|
|
233
259
|
if (bends !== 'refuse' && bends !== 'clamp') throw new Usage(`--bends is "${bends}"; it must be refuse or clamp`);
|
|
234
260
|
const switchAt = f.get('switch-at') ?? 'first';
|
|
@@ -254,6 +280,8 @@ function retargetArgs(argv: string[]): RetargetArgs {
|
|
|
254
280
|
legScale: legScaleOf(f),
|
|
255
281
|
plantFeet: f.has('plant-feet'),
|
|
256
282
|
stance: stanceOf(f),
|
|
283
|
+
...floorOf(f),
|
|
284
|
+
raisedByHead: f.has('raised-by-head'),
|
|
257
285
|
bends,
|
|
258
286
|
bendTolerance: num(f, 'bend-tolerance', 0, (n) => n >= 0 && n < 180, 'degrees in [0, 180)'),
|
|
259
287
|
switchTolerance: num(f, 'switch-tolerance', 25, (n) => n >= 0 && n < 180, 'degrees in [0, 180)'),
|
|
@@ -278,6 +306,14 @@ function stanceOf(f: Map<string, string>): number | 'off' {
|
|
|
278
306
|
return v;
|
|
279
307
|
}
|
|
280
308
|
|
|
309
|
+
/** --floor (rig-c/pose#85): the floor's world y, only with --plant-feet; absent, the median of the spans' planted heights. */
|
|
310
|
+
function floorOf(f: Map<string, string>): { floor?: number } {
|
|
311
|
+
if (!f.has('floor')) return {};
|
|
312
|
+
const v = num(f, 'floor', 0, () => true, 'a world y in rig units (a finite number)');
|
|
313
|
+
if (!f.has('plant-feet')) throw new Usage(`--floor is "${f.get('floor')}" without --plant-feet; it is the floor --plant-feet plants every support span on`);
|
|
314
|
+
return { floor: v };
|
|
315
|
+
}
|
|
316
|
+
|
|
281
317
|
/** --reduce and --reduce-hip (rig-c/pose#32): off unless given; the hip only with the bones. */
|
|
282
318
|
function reduceOf(f: Map<string, string>): { reduce?: number; reduceHip?: number } {
|
|
283
319
|
if (!f.has('reduce')) {
|
|
@@ -321,6 +357,21 @@ function compareArgs(argv: string[]): CompareArgs {
|
|
|
321
357
|
};
|
|
322
358
|
}
|
|
323
359
|
|
|
360
|
+
/** The stillness hold's flags (#91): `--hold off`, or the thresholds (defaults 0.015 and 0.03). */
|
|
361
|
+
function holdOf(f: Map<string, string>): { speed: number; reach: number } | 'off' {
|
|
362
|
+
const v = f.get('hold');
|
|
363
|
+
if (v !== undefined && v !== 'off') throw new Usage(`--hold is "${v}"; the hold runs by default, and the only value is "off"`);
|
|
364
|
+
if (v === 'off') {
|
|
365
|
+
const given = ['hold-speed', 'hold-reach'].filter((k) => f.has(k));
|
|
366
|
+
if (given.length > 0) throw new Usage(`${given.map((k) => `--${k}`).join(' and ')} with --hold off: the hold does not run, so ${given.length === 1 ? 'it changes' : 'they change'} nothing — drop ${given.length === 1 ? 'it' : 'them'} or the --hold off`);
|
|
367
|
+
return 'off';
|
|
368
|
+
}
|
|
369
|
+
return {
|
|
370
|
+
speed: num(f, 'hold-speed', HOLD_SPEED, (n) => n > 0, 'a number > 0 (trunk lengths per frame)'),
|
|
371
|
+
reach: num(f, 'hold-reach', HOLD_REACH, (n) => n > 0, 'a number > 0 (trunk lengths)'),
|
|
372
|
+
};
|
|
373
|
+
}
|
|
374
|
+
|
|
324
375
|
/** A window flag of clean: 1, or odd and ≥ 5 (`windowProblem`). */
|
|
325
376
|
function windowOf(f: Map<string, string>, name: string, dflt: number): number {
|
|
326
377
|
const v = f.get(name);
|
|
@@ -350,10 +401,10 @@ function runsOf(frames: number[]): string {
|
|
|
350
401
|
|
|
351
402
|
/** `rig-pose clean` (#67): read, clean, print each step's figures, write. */
|
|
352
403
|
function runClean(argv: string[], say: (l: string) => void): void {
|
|
353
|
-
const f = parseFlags(argv, ['pose', 'out', 'legs', 'rest', 'person'], ['loop', 'coco', 'hips-inferred']);
|
|
404
|
+
const f = parseFlags(argv, ['pose', 'out', 'legs', 'rest', 'person', 'hold', 'hold-speed', 'hold-reach'], ['loop', 'coco', 'hips-inferred']);
|
|
354
405
|
const input = need(f, 'pose');
|
|
355
406
|
const out = need(f, 'out');
|
|
356
|
-
const params = { loop: f.has('loop'), legs: windowOf(f, 'legs', 5), rest: windowOf(f, 'rest', 7), hipsInferred: f.has('hips-inferred') };
|
|
407
|
+
const params = { loop: f.has('loop'), legs: windowOf(f, 'legs', 5), rest: windowOf(f, 'rest', 7), hipsInferred: f.has('hips-inferred'), hold: holdOf(f) };
|
|
357
408
|
const text = readFileSync(input, 'utf8');
|
|
358
409
|
const seq = f.has('coco') ? convertCoco17(text, input) : readPoseSequence(text, input);
|
|
359
410
|
const c = clean(seq, params, f.get('person'), input);
|
|
@@ -376,6 +427,18 @@ function runClean(argv: string[], say: (l: string) => void): void {
|
|
|
376
427
|
}
|
|
377
428
|
const jit = (v: number | null) => (v === null ? 'not measured' : `${fmt(v, 3)}`);
|
|
378
429
|
say(` JITTER mean |second difference| of y over ${c.triples} joint-frame triple(s): ${jit(c.record.jitter.before)} → ${jit(c.record.jitter.after)} px/f²${c.record.jitter.before !== null && c.record.jitter.after !== null && c.record.jitter.before > 0 ? ` (${fmt(100 * (1 - c.record.jitter.after / c.record.jitter.before), 1)}% lower)` : ''}`);
|
|
430
|
+
if (params.hold === 'off') say(' HOLD off (--hold off): no stillness hold');
|
|
431
|
+
else {
|
|
432
|
+
const h = params.hold;
|
|
433
|
+
const span = (r: [number, number] | null, what: string) => (r === null ? `nothing ${what}` : `${what} ${r[0] === r[1] ? `frame ${r[0]}` : `frames ${r[0]}–${r[1]}`}`);
|
|
434
|
+
for (const x of c.holds) say(` HOLD joint "${x.joint}" frames ${x.frames[0]}–${x.frames[1]} (${tOf(x.frames[0])}–${tOf(x.frames[1])}, ${fmt(x.seconds, 3)} s): ${HOLD_SPEED_WINDOW}-frame median speed under ${h.speed} trunk/frame (largest ${fmt(x.speed, 4)}), within ${h.reach} trunk of its span median (largest ${fmt(x.reach, 4)}); written as that median, blended in over ${span(x.blendIn, 'before').replace('nothing before', 'nothing before it')}, out over ${span(x.blendOut, 'after').replace('nothing after', 'nothing after it')}; jitter inside ${jit(x.jitter.before)} → ${jit(x.jitter.after)} px/f²${x.jitter.after !== null && x.jitter.after > 0 ? ' (ramp)' : ''}`);
|
|
435
|
+
for (const x of c.seams) say(` SEAM joint "${x.joint}" frame ${x.frame} (${tOf(x.frame)}): two holds ${x.gap === 0 ? 'back to back' : `${x.gap} frame(s) apart`}, their medians ${fmt(x.step, 4)} trunk (${fmt(x.step * c.trunk.length)} px) apart; cross-faded over frames ${x.frame - x.gap - HOLD_BLEND / 2}–${x.frame + HOLD_BLEND / 2 - 1}, no frame moving more than 1/${HOLD_BLEND + x.gap} of it`);
|
|
436
|
+
for (const [a, b] of holdRegions(c.holds)) {
|
|
437
|
+
const not = c.unheld.filter((u) => u.frames[0] >= a && u.frames[1] <= b);
|
|
438
|
+
if (not.length > 0) say(` UNHELD frames ${a}–${b}, where other joints are held: ${not.map(unheldText({ ...h, minSeconds: HOLD_MIN_SECONDS })).join(', ')}`);
|
|
439
|
+
}
|
|
440
|
+
say(` HOLDS ${c.holds.length} found over ${new Set(c.holds.map((x) => x.joint)).size} joint(s) (per joint: a span ≥ ${HOLD_MIN_SECONDS} s still by speed < ${h.speed} trunk/frame and reach ≤ ${h.reach} trunk; ${HOLD_BLEND}-frame blend in and out)`);
|
|
441
|
+
}
|
|
379
442
|
if (c.inferred !== null) say(` INFERRED r_hip on ${c.inferred.r_hip} frame(s), l_hip on ${c.inferred.l_hip} frame(s) of ${n} marked "inferred" (every frame where it has a position): declared at clean (--hips-inferred); positions and scores unchanged`);
|
|
380
443
|
writeFileSync(out, writePoseSequence(c.seq));
|
|
381
444
|
say(` CLEAN ${cleanLine(c.record)}`);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@rig-c/pose",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.17.0",
|
|
4
4
|
"description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|