@rig-c/pose 0.14.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -12
- package/cli.ts +22 -4
- package/package.json +1 -1
- package/src/clean.ts +41 -6
- package/src/pose.ts +34 -9
- package/src/retarget.ts +303 -23
- package/src/run.ts +9 -5
- package/src/switch.ts +164 -0
- package/src/verify.ts +49 -10
package/README.md
CHANGED
|
@@ -25,7 +25,7 @@ The map names **chains**: each limb as bones that descend from one another, pare
|
|
|
25
25
|
## Clean first: `rig-pose clean`
|
|
26
26
|
|
|
27
27
|
```sh
|
|
28
|
-
rig-pose clean --pose dance.scan.json --out dance.pose.json [--loop] [--legs 5] [--rest 7]
|
|
28
|
+
rig-pose clean --pose dance.scan.json --out dance.pose.json [--loop] [--legs 5] [--rest 7] [--hips-inferred]
|
|
29
29
|
```
|
|
30
30
|
|
|
31
31
|
`retarget` reads only a sequence that `clean` wrote ([#67](https://github.com/rig-c/pose/issues/67)). A pose model reads each frame alone, so a hanging wrist moves ±20 px a frame and a cloak-hidden leg jumps 100+ px, and every key carries that shake. `clean` takes it out once, at the sequence, where it can be inspected. A sequence without the record is refused by name (`POSE_NOT_CLEANED`), and the message is the `clean` command to run on that file. There is no flag to skip the step. Windows of 1 (`--legs 1 --rest 1`) are a legitimate run that smooths nothing, and they are recorded as such.
|
|
@@ -33,7 +33,7 @@ rig-pose clean --pose dance.scan.json --out dance.pose.json [--loop] [--legs 5]
|
|
|
33
33
|
The steps, in this order, each printed with its figures:
|
|
34
34
|
|
|
35
35
|
1. **Spikes.** A joint more than 0.2 trunk lengths from its 5-frame median takes the median. The trunk is neck → mid-hip, the median over the clip. Each replacement is listed by frame and joint (`SPIKE`). Without `--loop`, a clip end cuts the window of the first and last two frames. There the median is a neighbour's value, and a joint moving steadily would read as a spike. Those frames are **not tested**, and the `SPIKES` line and the record name them. Reflecting the window does not help: the median of `[x2, x1, x0, x1, x2]` is still `x1`.
|
|
36
|
-
2. **Occlusion.** A joint whose state is not `observed
|
|
36
|
+
2. **Occlusion.** A joint whose state is not `observed` (or `inferred`), or whose score is under 0.3, takes its 3-frame median (`OCCLUDED`).
|
|
37
37
|
3. **Smoothing.** A quadratic Savitzky–Golay per joint: the legs (both hips, knees and ankles) over `--legs` frames (default 5), every other joint over `--rest` (default 7). A window is 1 (none) or odd and at least 5. 3 is refused because a quadratic through 3 points passes through all three. Without `--loop`, the window is cut at the ends. Where fewer than three frames have a position, the degree drops to fit the ones that do.
|
|
38
38
|
4. **Floor.** Only with `--loop`: the whole figure moves in y by one constant, so that the lowest ankle over the cycle is where it was before smoothing.
|
|
39
39
|
|
|
@@ -50,6 +50,8 @@ The output carries the record, here the dancer's from the table below:
|
|
|
50
50
|
|
|
51
51
|
`jitter` is the mean |second difference| of y, in px/f², over every joint and every three consecutive frames that have a position, wrapping with `--loop`. It is measured on the sequence as read and on the cleaned positions. `retarget` prints the record as its `CLEAN` line and copies it into `--report`.
|
|
52
52
|
|
|
53
|
+
**`--hips-inferred`** ([#63](https://github.com/rig-c/pose/issues/63)) declares that the hips are not seen: under a skirt or a cloak the pose model still places them, but where it puts them is its guess. After the four steps, `r_hip` and `l_hip` are marked `inferred` on every frame where they have a position (`INFERRED`), and the record's params carry `"hipsInferred": { "by": "declared" }` — the person running `clean` said so. Positions and scores are unchanged, so `retarget`'s keys are the same with the flag and without it. What changes is the `POSTURE` line, which does not read a hip tilt off inferred hips (below), and the `CLEAN` line, which ends `hips inferred (declared at clean)`.
|
|
54
|
+
|
|
53
55
|
**Measured** on HQ's references ([#67](https://github.com/rig-c/pose/issues/67)'s bars):
|
|
54
56
|
|
|
55
57
|
| Clip | Run | Jitter, px/f² | Against |
|
|
@@ -89,11 +91,13 @@ and what the run cannot decide for you it lists:
|
|
|
89
91
|
- `FLIP` — a segment that turns more than `--flip` degrees in one frame (with `--flip <deg>`; off by default) is spread over the frames around the jump; one line per spread, with the chain, the segment, the jump, the frames and h. Below.
|
|
90
92
|
- `BEND` — with `--bends clamp`, a hinge bent the wrong way is straightened on that frame instead of refusing.
|
|
91
93
|
- `FOLD` — `rigc render --folds` over the new animation: every mesh that folds, and when. A reading, not a gate.
|
|
92
|
-
- `POSTURE` — when the map has a pelvis chain (joints `r_hip`, `l_hip`) and a `translate`: the hip line's tilt range, and the sway — the x-range of mid_hip − mid_shoulder — in rig units and in shoulder widths, the pose's beside the posed rig's. A reading, not a gate.
|
|
94
|
+
- `POSTURE` — when the map has a pelvis chain (joints `r_hip`, `l_hip`) and a `translate`: the hip line's tilt range, and the sway — the x-range of mid_hip − mid_shoulder — in rig units and in shoulder widths, the pose's beside the posed rig's. A reading, not a gate. A frame on which either hip is `inferred` is left out of the tilt. When that is every frame, as after `clean --hips-inferred`, the line is `hip tilt not read: hips inferred (declared at clean)` and the report's `posture` has `"tilt": null` with `"tiltWhy"`; on some frames only, the tilt is read over the others and the line counts the frames left out (`posture.hipsInferred`). The source is read from the clean record, or is `source not stated` when the joints carry the state and the record does not. The sway reads the hips as before.
|
|
93
95
|
- `SWITCH` — with `alternates` in the map ([below](#alternate-drawings-alternates)): each span on which a set of alternate drawings shows, its slots, its frames, how far the pose was from the set's key skeleton, how far each bone with a `drawn` angle was turned from it, and where each of its boundaries was moved to (`moved`, [below](#alternate-drawings-alternates)).
|
|
94
96
|
- `UNCOVERED` — with `alternates` in the map: every fold on a frame where no switch took the folding mesh off screen. This is the list of poses still to paint.
|
|
95
97
|
- `RANGE` — a chain's `range` ([below](#a-joints-range-range)): a bone whose rotate key the pose asks past its range, outside every span of a set depicting its chain, is clamped at the limit; one line per run, with the frames, the seconds and the largest overflow. The demand for an alternate.
|
|
96
98
|
- `ART-RANGE` — inside a set's span: a bone with a `drawn` angle in the set, turned from it past its limits — the set's `turn` for that bone (held at the limit) or, without one, its chain's `range` (the bone follows the pose). Listed as the pose asked it either way: the demand for another drawing of that set.
|
|
99
|
+
- `RAISED` — with `--raised-by-head` ([below](#alternate-drawings-alternates)): one line per span of a set that draws an arm raised, per chain. It gives the frames and the median eye distances (the rig's in units, the reference's in px). It gives the wrist's offset from the head centre in eye distances: the median of the reference's, of where the wrist was placed, of the rig's on the pose's angles, and of the rig's as keyed. It gives the worst |rig − reference| before and after, the worst |after − placed|, and the frames whose upper arm the set's `turn` held. `RAISED none` when the flag is given and no such set showed.
|
|
100
|
+
- `REACH` — with `--raised-by-head`: a raised span whose target was out of the arm's reach, too far or too near. It names the frames, which way, and the shortfall in eye distances. The wrist was placed at the reach limit instead. This is not `ART-RANGE`: the rig's proportions were the limit, not the art's range. A too-near frame is placed at the limit like a too-far one, and graded at the placed distance.
|
|
97
101
|
- `LEG-SCALE` — with `--leg-scale` ([below](#the-legs-foreshortening-and-planted-feet)): per leg segment, its straight length, the spread of its picture length over it, and the frames clamped at each limit.
|
|
98
102
|
- `PIVOT` — when the map translates a bone and has both legs: the translated bone's origin against its thigh roots' midpoint at setup, in units and hip widths — the pivot every hip turn swings the legs about.
|
|
99
103
|
- `FEET` — when the map has both legs: the supports and jumps read off the reference ankles (planted or not), and on the posed build, outside the jumps, the lowest ankle's vertical range and each support span's worst vertical range and horizontal drift of the supporting ankle, in leg lengths.
|
|
@@ -103,7 +107,7 @@ and what the run cannot decide for you it lists:
|
|
|
103
107
|
- `REDUCE` — with `--reduce` ([below](#few-keys---reduce)): per bone, its keys before and after and its largest error, with the frame; then the total.
|
|
104
108
|
- `TURN` — a line across the body (hips, shoulders, eyes) shorter in the picture than 0.5 of its median length, or reversed, for at least 0.15 s: the body or the head turned. A reading of the pose, made whether or not the map uses the line; it changes no key. The demand for a turned-body drawing, as `END-ON` is for a foreshortened limb.
|
|
105
109
|
|
|
106
|
-
`--report <file>` writes all of it as JSON (`rig-pose-report/1`): `options` (every figure the run used, `switchTolerance`, `switchMin`, `switchAt`, `legScale`, `plantFeet` and `
|
|
110
|
+
`--report <file>` writes all of it as JSON (`rig-pose-report/1`): `options` (every figure the run used, `switchTolerance`, `switchMin`, `switchAt`, `legScale`, `plantFeet`, `stance` and `raisedByHead` included), `scale`, `holds`, `short`, `endOn`, `bends`, `flips`, `gate`, `landings`, `posture`, `folds`, `legScale`, `pivot`, `feet` (`planted`, `legLength`, `floor`, `supports`, `jumps`, `changes`, and the posed `reading`), `stance` (`tolerance`, `bounds`, `runs`: `{ state, side, frames, maxHeight, k }`, `over`: `{ side, frames, needed, left }`; `null` without `--stance`), `shear`, `idleDropped`, `idleKeptOut` and `merged` ([below](#the-idle-what-the-retarget-drops)), `reduce` (the tolerances and each row of `REDUCE`, [below](#few-keys---reduce)), and the demand lists. The fields a run is graded by:
|
|
107
111
|
|
|
108
112
|
| field | each entry |
|
|
109
113
|
| --- | --- |
|
|
@@ -120,7 +124,8 @@ The demand lists:
|
|
|
120
124
|
- `endOn` — above, under the short hold.
|
|
121
125
|
- `turns: [{ line, joints, frames, seconds, minFraction, reversed }]` — `line` is `hip`, `shoulder` or `eye`.
|
|
122
126
|
- `range: [{ chain, bone, frames, seconds, maxOverflow, key, limits }]` — `key` is the rotate key the pose asked for on the frame of the largest overflow, before the clamp.
|
|
123
|
-
- `artRange: [{ set, chain, bone, frames, seconds, maxOverflow, turn, limits, clamped }]` — `turn` is the signed art turn the pose asked on that frame; `clamped` says whether the set's `turn` held the bone there.
|
|
127
|
+
- `artRange: [{ set, chain, bone, frames, seconds, maxOverflow, turn, limits, clamped, shortfall? }]` — `turn` is the signed art turn the pose asked on that frame; `clamped` says whether the set's `turn` held the bone there. `shortfall` (with `--raised-by-head`) marks a held upper arm whose re-aimed forearm left the wrist short of its placed point: the most, in eye distances, with `turn` and `maxOverflow` the upper arm's on that frame.
|
|
128
|
+
- `raised: [{ set, chain, frames, seconds, eyes: { rig, reference }, reference, placed, before, after, worst: { before, after, placed }, short, upperHeld, reach: { min, max, farthest } }]` — with `--raised-by-head`. `reference`, `placed`, `before` and `after` are the wrist's offset from the head centre on each frame of the span, `[x, y]` in eye distances, x right and y up. `eyes` holds each side's median eye distance over the span. `worst.before` and `worst.after` are the largest |rig − reference| in x and in y; `worst.placed` is the largest |after − placed|. `short` lists the `REACH` runs, each `{ frames, how, shortfall: { max, median } }`, where `how` is `far`, `near` or `missed`. `upperHeld` lists the frames whose upper arm the set's `turn` held. `reach` gives |l0 − l1|, l0 + l1 and the farthest target from the shoulder, in world units.
|
|
124
129
|
|
|
125
130
|
Runs in `turns`, `range` and `artRange` merge across a gap of one frame, as `endOn` does; `seconds` runs from the first frame's key time to the end of the last frame.
|
|
126
131
|
|
|
@@ -182,7 +187,7 @@ A rig's motion spec usually has an `idle` — a blink, a breath, hair and hem sw
|
|
|
182
187
|
|
|
183
188
|
Every bone still has to land with the carried tracks in. A carried translate above the translated bone moves its origin off the pose's travel, and the run refuses with `POSE_LANDS`, naming the carried tracks above the bone that missed.
|
|
184
189
|
|
|
185
|
-
On HQ's dancer (`--flip 90 --short 0.35 --bends clamp --carry-idle`) the idle has 26 tracks. Kept out: the four arm rotates, which the dance keys, and
|
|
190
|
+
On HQ's dancer (`--flip 90 --short 0.35 --bends clamp --carry-idle`) the idle has 26 tracks. Kept out: the four arm rotates, which the dance keys, and a control bone's scalex above the chest, the neck's rotate and a head control's rotate, which sit above the mapped chest, head and arms — carried, they missed the chest, the head and the arms by 0.16°–1.37°. Carried: the other 19, the blink (an eyes group's scaley) and the breath (the chest control's translatey) among them. The gate is green, every bone lands (worst 0.00167°), and rig-c's poser shows the blink at every cycle (scaleY 0.12 at 2.37 s, 5.70 s, 9.08 s and 12.41 s, against the idle's at t mod 3.333 s within 8.9e-6) and the breath's offset (5.02 at 0.833, 4.167, 7.5, 10.833 s; 0 at 2.5, 5.833, 9.167, 12.5 s; 1.035 at the cut, 13 s).
|
|
186
191
|
|
|
187
192
|
**`--merge <file>`** merges a timed track fragment into the new animation, at absolute times from its start — blinks read off the video, for example:
|
|
188
193
|
|
|
@@ -222,7 +227,7 @@ Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2
|
|
|
222
227
|
```
|
|
223
228
|
|
|
224
229
|
- `t` is seconds and strictly increases; the animation's keys are at `t − t₀`.
|
|
225
|
-
- Joints are body-18 (`nose neck r_shoulder r_elbow r_wrist l_shoulder l_elbow l_wrist r_hip r_knee r_ankle l_hip l_knee l_ankle r_eye l_eye r_ear l_ear`); `r`/`l` are the **subject's** sides. `state` is `observed`, `occluded` or `missing`, as in the keypoints file. An unlisted joint is missing.
|
|
230
|
+
- Joints are body-18 (`nose neck r_shoulder r_elbow r_wrist l_shoulder l_elbow l_wrist r_hip r_knee r_ankle l_hip l_knee l_ankle r_eye l_eye r_ear l_ear`); `r`/`l` are the **subject's** sides. `state` is `observed`, `occluded` or `missing`, as in the keypoints file, or `inferred` ([#63](https://github.com/rig-c/pose/issues/63)): a position the detector supplied but did not see. An `inferred` joint carries `at` (one without it is refused, `POSE_JOINT_AT`), and `retarget` keys it as it keys `observed`. An unlisted joint is missing.
|
|
226
231
|
- Two differences from the keypoints file: the unit is `picture-px`, and `at` may lie outside the picture (a hand leaves the frame; the score says how much to believe it).
|
|
227
232
|
- More than one person: `--person <id>`.
|
|
228
233
|
- `score` is whatever the pose model gives. RTMPose's goes above 1, so `--min-score` is a cut on that model's scale, not a probability.
|
|
@@ -273,16 +278,18 @@ Bone rotation cannot carry every pose: an arm behind the head, a forearm toward
|
|
|
273
278
|
|
|
274
279
|
```json
|
|
275
280
|
"alternates": [
|
|
276
|
-
{ "name": "
|
|
277
|
-
"slots": { "
|
|
281
|
+
{ "name": "raised_arms",
|
|
282
|
+
"slots": { "upperarm_r_raised": "raised/upperarm_r", "forearm_r_raised": "raised/forearm_r", "torso": "raised/torso" },
|
|
278
283
|
"hides": ["upperarm_r", "forearm_r"],
|
|
279
284
|
"chains": ["arm_r", "arm_l"],
|
|
280
|
-
"pose": { "r_shoulder": [
|
|
281
|
-
"l_shoulder": [
|
|
282
|
-
"drawn": { "sleeve_r0":
|
|
285
|
+
"pose": { "r_shoulder": [400, 600], "r_elbow": [300, 450], "r_wrist": [450, 400],
|
|
286
|
+
"l_shoulder": [600, 600], "l_elbow": [700, 450], "l_wrist": [550, 400] },
|
|
287
|
+
"drawn": { "sleeve_r0": 120, "sleeve_r1": 30 },
|
|
283
288
|
"offsets": { "sleeve_r1": [4, -2] } } ]
|
|
284
289
|
```
|
|
285
290
|
|
|
291
|
+
(The names, coordinates and angles above are illustrative.)
|
|
292
|
+
|
|
286
293
|
- `slots`: what each slot of the set shows while it shows; each attachment must be in that slot under some skin of the rig, as rig-c built it. `hides`: the slots that are empty while it shows. A slot's place in the draw order is the rig's own, so a drawing that must sit in front of the torso is an attachment in a slot that is: a switch is attachment keys only, and no draw-order key is written.
|
|
287
294
|
- `chains`: the map's chains whose pose the set depicts; it is selected by them. `pose`: the key skeleton the art was painted from, body-18 (or `mid_hip`/`mid_shoulder`) in picture px, y down, any scale and place — only its directions are read; every joint the listed chains use must be in it.
|
|
288
295
|
- `drawn` (optional): each bone's world angle (Spine degrees) as painted. It changes no key: while a set shows, a bone is keyed to the pose as always, and the art turns with it. What it gives is a figure: the art turn, how far each bone is from the angle its art was painted at (`artTurn`, and `ART-RANGE` past the bone's range).
|
|
@@ -305,6 +312,16 @@ Either way the `SWITCH` line ends with `moved on …, off …`: each boundary's
|
|
|
305
312
|
**A stopgap window: `frames`.** A set may state `"frames": [[first, last], …]`: it shows only inside those windows, and on them it is chosen over any set within tolerance that has no window. It is for two sets whose key skeletons the angles cannot tell apart. Measured on HQ's dancer, the hands clasped behind the back and the hands low in front differ by only 2–21°, and the wrist and hand-point scores do not separate them either; the cue that might is #33's depth state. It is recorded in the map as a stopgap, not a selection rule. A window **restricts** a set; it never forces one. Where the set does not show inside its window, the run says why in a `FRAMES` line: the stretch, and whether the pose there was past the tolerance (with the largest distance), or within it but the run was dropped.
|
|
306
313
|
|
|
307
314
|
**How far a set's art may turn: `turn`.** A drawing reads right only near the angle it was painted at. A set may state, per bone with a `drawn` angle, how far its art may turn from it, `"turn": { "sleeve_r0": [-20, 20] }`. While the set shows, the bone is held within it: its target becomes drawn + clamp(wrap(target − drawn), min, max), each bone on its own world angle, so a forearm under a held upper arm keeps its own target. The landing check measures against the held angle. The overflow the pose asked for is listed as `ART-RANGE`. A `turn` on a bone the set has no `drawn` angle for is refused (`MAP_ALT_TURN`).
|
|
315
|
+
|
|
316
|
+
**A raised arm placed against the head: `--raised-by-head`.** The pose's angles put a hand where it is against the dancer's own shoulder. On a rig whose shoulders sit nearer its head, or whose head is larger, the same angles put a hand raised beside the head on top of it. With `--raised-by-head` (off by default, [#62](https://github.com/rig-c/pose/issues/62)) a set draws an arm **raised** when its `drawn` angle for the chain's first bone (the upper arm) is within 30° of straight up (a world angle of 60°–120°). While that set shows, the arm's wrist goes where the reference's is **against the head**:
|
|
317
|
+
|
|
318
|
+
- The unit is the **eye distance**: on each side, its median over the span. That is the reference's |r_eye − l_eye|, and on the rig the distance between the two points the head chain's `"a>b"` aim names. The head centre is the eyes' midpoint on each frame. The head chain is the one carrying the eye line (r_eye → l_eye, either way round). The flag is refused without one, or with one not aimed `"a>b"` (`RAISED_HEAD`).
|
|
319
|
+
- The target is rig head centre + (reference wrist − reference head centre) × (rig eye distance ÷ reference eye distance), the picture's y turned up, per frame.
|
|
320
|
+
- **The order of solving.** The arm is solved last of the bones. By then the hip's travel, every reach, the head and every range clamp are final, and the arm moves neither the head nor its own shoulder. On each frame the rig's eye points and the shoulder are posed forward from those targets, as a reach's origin is: each rest offset is turned by the nearest mapped bone above.
|
|
321
|
+
- **Reach.** The arm reaches the annulus about the shoulder from |l0 − l1| to l0 + l1. Here l0 runs from the upper arm's origin to the forearm's, and l1 from there to the wrist (the forearm's tip, or the point it aims at). A target outside it is replaced by the reach limit along the head-relative direction. That is the point inside the annulus, on the ray from the head centre through the target, nearest the target. Where the ray crosses the annulus twice, the nearer stretch wins, and a tie goes to the point nearer the head. A ray that never enters it gives the annulus point nearest the target. Every such frame is listed in `REACH`. A too-near frame is treated as a too-far one: the wrist is placed at the reach limit, and the distance the rule grades it at is the placed one. That is |placed − head centre|, not the reference's, which the arm cannot fold to.
|
|
322
|
+
- **The solve.** Two bones on the rig's own lengths put the wrist on that point, the elbow bending to the side the pose's does. The set's `turn` is part of the solve. When it holds the upper arm, the forearm is re-aimed from the held elbow at the placed wrist rather than held on its own. `ART-RANGE` lists where the re-aimed forearm still cannot reach within `turn`, in two cases. One is a forearm past its own `turn`, held there. The other is a held upper arm whose re-aimed forearm leaves the wrist more than 0.01 eye distances short of its placed point; 0.01 is the float and rounding scale. That case is listed on the upper arm, with the shortfall in eye distances. The solved angles become the targets, which the keys and the landing check follow.
|
|
323
|
+
- Frames outside those spans, and sets that draw no arm raised, are keyed as without the flag, byte for byte. A raised chain must have two bones, each aimed `"x"` or at one point, and the eye points must not turn with it (`RAISED_CHAIN`).
|
|
324
|
+
|
|
308
325
|
## Compare: the rig's render against the video
|
|
309
326
|
|
|
310
327
|
```sh
|
package/cli.ts
CHANGED
|
@@ -34,6 +34,10 @@ clean (required before retarget; rig-c/pose#67)
|
|
|
34
34
|
--legs <n> Savitzky–Golay window over the legs — both hips, knees and ankles (default: 5)
|
|
35
35
|
--rest <n> Savitzky–Golay window over every other joint (default: 7)
|
|
36
36
|
n is 1 (none), or odd and ≥ 5: a quadratic through 3 points passes through all three
|
|
37
|
+
--hips-inferred declare the hips inferred (rig-c/pose#63): r_hip and l_hip are marked "inferred" — a
|
|
38
|
+
position the detector supplied but did not see — on every frame where they have one, and
|
|
39
|
+
the record says so (hipsInferred: { by: "declared" }). Positions are unchanged, so retarget's
|
|
40
|
+
keys are too; its POSTURE line does not read the hip tilt ("hip tilt not read: hips inferred")
|
|
37
41
|
|
|
38
42
|
In order, each step printed with its figures: (1) spikes — a joint more than 0.2 trunk lengths
|
|
39
43
|
(neck → mid_hip, the median over the clip) from its 5-frame median takes the median, each listed by
|
|
@@ -94,6 +98,18 @@ retarget
|
|
|
94
98
|
reaches the floor, raised or lowered, k within [0.92, 1.08] (past it, reported, not forced); above
|
|
95
99
|
it the foot is lifted and keeps the pose's height (STANCE lines; default: off; 0.07 is
|
|
96
100
|
the card's, rig-c/pose#60)
|
|
101
|
+
--raised-by-head while a set whose arm is drawn raised shows (its "drawn" upper arm within 30° of straight
|
|
102
|
+
up), that arm's wrist goes where the reference's is against the head, in eye distances:
|
|
103
|
+
rig head centre + (reference wrist − reference head centre) × rig eye distance ÷ the
|
|
104
|
+
reference's, per frame; each eye distance is its median over the span. The head centre is
|
|
105
|
+
the eyes' midpoint; the rig's eyes are the two points the head chain's "a>b" aim names
|
|
106
|
+
(refused without one, RAISED_HEAD), posed from the targets of the bones above. A target out
|
|
107
|
+
of the arm's reach, too far or too near, goes to the reach limit on the ray from the head
|
|
108
|
+
centre toward it and is graded at that placed distance (REACH lines, in eye distances). A
|
|
109
|
+
two-bone solve on the rig's own lengths keeps the pose's elbow side; when the set's "turn"
|
|
110
|
+
holds the upper arm the forearm is re-aimed at the wrist. ART-RANGE: a forearm still past
|
|
111
|
+
its turn, or a held arm leaving the wrist > 0.01 eye distances short of its placed point
|
|
112
|
+
(the shortfall listed). RAISED lines (default: off; rig-c/pose#62)
|
|
97
113
|
--idle <name> the rig's animation read as its idle (default: idle). Every idle track the new
|
|
98
114
|
animation does not key, on any channel, is listed as IDLE-DROPPED (a blink, a
|
|
99
115
|
breath); one it keys is IDLE-KEPT-OUT, naming the track that keys it, and so is a
|
|
@@ -125,7 +141,7 @@ retarget
|
|
|
125
141
|
--build <dir> keep the gated build (skeleton.json, .atlas, .model.json) here
|
|
126
142
|
--render <dir> keep rigc's render of the animation (frames, contact.png, folds.json) here
|
|
127
143
|
--report <file> write the run's measurements (holds, short, endOn, flips, bends, landings, folds,
|
|
128
|
-
switches, uncovered, turns, range, artRange,
|
|
144
|
+
switches, uncovered, turns, range, artRange, raised,
|
|
129
145
|
idleDropped, idleKeptOut, merged, reduce) as JSON
|
|
130
146
|
|
|
131
147
|
It writes nothing until rigc's gate is green with the animation in, every mapped bone lands on
|
|
@@ -224,7 +240,7 @@ function legScaleOf(f: Map<string, string>): [number, number] | 'off' {
|
|
|
224
240
|
}
|
|
225
241
|
|
|
226
242
|
function retargetArgs(argv: string[]): RetargetArgs {
|
|
227
|
-
const f = parseFlags(argv, ['rig', 'map', 'pose', 'out', 'name', 'person', 'base-motion', 'images', 'min-score', 'smooth', 'flip', 'short', 'bends', 'bend-tolerance', 'switch-tolerance', 'switch-min', 'switch-at', 'profile', 'build', 'render', 'report', 'leg-scale', 'idle', 'merge', 'reduce', 'reduce-hip', 'stance'], ['coco', 'plant-feet', 'carry-idle']);
|
|
243
|
+
const f = parseFlags(argv, ['rig', 'map', 'pose', 'out', 'name', 'person', 'base-motion', 'images', 'min-score', 'smooth', 'flip', 'short', 'bends', 'bend-tolerance', 'switch-tolerance', 'switch-min', 'switch-at', 'profile', 'build', 'render', 'report', 'leg-scale', 'idle', 'merge', 'reduce', 'reduce-hip', 'stance'], ['coco', 'plant-feet', 'carry-idle', 'raised-by-head']);
|
|
228
244
|
const bends = f.get('bends') ?? 'refuse';
|
|
229
245
|
if (bends !== 'refuse' && bends !== 'clamp') throw new Usage(`--bends is "${bends}"; it must be refuse or clamp`);
|
|
230
246
|
const switchAt = f.get('switch-at') ?? 'first';
|
|
@@ -250,6 +266,7 @@ function retargetArgs(argv: string[]): RetargetArgs {
|
|
|
250
266
|
legScale: legScaleOf(f),
|
|
251
267
|
plantFeet: f.has('plant-feet'),
|
|
252
268
|
stance: stanceOf(f),
|
|
269
|
+
raisedByHead: f.has('raised-by-head'),
|
|
253
270
|
bends,
|
|
254
271
|
bendTolerance: num(f, 'bend-tolerance', 0, (n) => n >= 0 && n < 180, 'degrees in [0, 180)'),
|
|
255
272
|
switchTolerance: num(f, 'switch-tolerance', 25, (n) => n >= 0 && n < 180, 'degrees in [0, 180)'),
|
|
@@ -346,10 +363,10 @@ function runsOf(frames: number[]): string {
|
|
|
346
363
|
|
|
347
364
|
/** `rig-pose clean` (#67): read, clean, print each step's figures, write. */
|
|
348
365
|
function runClean(argv: string[], say: (l: string) => void): void {
|
|
349
|
-
const f = parseFlags(argv, ['pose', 'out', 'legs', 'rest', 'person'], ['loop', 'coco']);
|
|
366
|
+
const f = parseFlags(argv, ['pose', 'out', 'legs', 'rest', 'person'], ['loop', 'coco', 'hips-inferred']);
|
|
350
367
|
const input = need(f, 'pose');
|
|
351
368
|
const out = need(f, 'out');
|
|
352
|
-
const params = { loop: f.has('loop'), legs: windowOf(f, 'legs', 5), rest: windowOf(f, 'rest', 7) };
|
|
369
|
+
const params = { loop: f.has('loop'), legs: windowOf(f, 'legs', 5), rest: windowOf(f, 'rest', 7), hipsInferred: f.has('hips-inferred') };
|
|
353
370
|
const text = readFileSync(input, 'utf8');
|
|
354
371
|
const seq = f.has('coco') ? convertCoco17(text, input) : readPoseSequence(text, input);
|
|
355
372
|
const c = clean(seq, params, f.get('person'), input);
|
|
@@ -372,6 +389,7 @@ function runClean(argv: string[], say: (l: string) => void): void {
|
|
|
372
389
|
}
|
|
373
390
|
const jit = (v: number | null) => (v === null ? 'not measured' : `${fmt(v, 3)}`);
|
|
374
391
|
say(` JITTER mean |second difference| of y over ${c.triples} joint-frame triple(s): ${jit(c.record.jitter.before)} → ${jit(c.record.jitter.after)} px/f²${c.record.jitter.before !== null && c.record.jitter.after !== null && c.record.jitter.before > 0 ? ` (${fmt(100 * (1 - c.record.jitter.after / c.record.jitter.before), 1)}% lower)` : ''}`);
|
|
392
|
+
if (c.inferred !== null) say(` INFERRED r_hip on ${c.inferred.r_hip} frame(s), l_hip on ${c.inferred.l_hip} frame(s) of ${n} marked "inferred" (every frame where it has a position): declared at clean (--hips-inferred); positions and scores unchanged`);
|
|
375
393
|
writeFileSync(out, writePoseSequence(c.seq));
|
|
376
394
|
say(` CLEAN ${cleanLine(c.record)}`);
|
|
377
395
|
say(` wrote ${out}`);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@rig-c/pose",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.16.0",
|
|
4
4
|
"description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/src/clean.ts
CHANGED
|
@@ -37,7 +37,15 @@
|
|
|
37
37
|
* median of on its own frame, and a value made from its neighbours would be a
|
|
38
38
|
* guess; `retarget` holds it, and says so with a HOLD line. A joint's state and
|
|
39
39
|
* score are the detector's and are written as they were read: this step moves
|
|
40
|
-
* positions only.
|
|
40
|
+
* positions only — with one stated exception, `hipsInferred` below.
|
|
41
|
+
*
|
|
42
|
+
* `hipsInferred` (rig-c/pose#63, `--hips-inferred`): after the four steps, r_hip
|
|
43
|
+
* and l_hip are marked `inferred` on every frame where they have a position,
|
|
44
|
+
* and the record says who said so (`{ by: 'declared' }`, the person running
|
|
45
|
+
* clean). Positions and scores are not touched by it, so retarget's keys are
|
|
46
|
+
* the same with and without it; what changes is POSTURE, which does not read a
|
|
47
|
+
* hip tilt off inferred hips. The occlusion step reads `inferred` as it reads
|
|
48
|
+
* `observed`: a cleaned file cleaned again moves its hips the same way.
|
|
41
49
|
*
|
|
42
50
|
* The smoothing fits the frames of the window where the joint has a position;
|
|
43
51
|
* where fewer than three are there, the polynomial's degree drops to fit the
|
|
@@ -75,8 +83,13 @@ export interface CleanParams {
|
|
|
75
83
|
legs: number;
|
|
76
84
|
/** Savitzky–Golay window over every other joint, frames: 1 (none), or odd and ≥ 5. */
|
|
77
85
|
rest: number;
|
|
86
|
+
/** Mark r_hip and l_hip `inferred` wherever they have a position (#63): the person running clean declares it. */
|
|
87
|
+
hipsInferred?: boolean;
|
|
78
88
|
}
|
|
79
89
|
|
|
90
|
+
/** The joints `--hips-inferred` marks. */
|
|
91
|
+
export const HIPS: readonly Body18[] = ['r_hip', 'l_hip'];
|
|
92
|
+
|
|
80
93
|
export type Pt = [number, number];
|
|
81
94
|
/** One joint's positions over the frames; null where it has none. */
|
|
82
95
|
export type Series = Array<Pt | null>;
|
|
@@ -116,6 +129,8 @@ export interface Cleaned {
|
|
|
116
129
|
floor: Floor | null;
|
|
117
130
|
/** How many (joint, three-frame) runs the jitter was read over: before, after. */
|
|
118
131
|
triples: number;
|
|
132
|
+
/** With `hipsInferred`: how many joint-frames were marked `inferred`, per hip. */
|
|
133
|
+
inferred: { r_hip: number; l_hip: number } | null;
|
|
119
134
|
}
|
|
120
135
|
|
|
121
136
|
/**
|
|
@@ -323,10 +338,10 @@ export function clean(seq: PoseSequence, params: CleanParams, wanted: string | u
|
|
|
323
338
|
if (q === null) return;
|
|
324
339
|
const j = track[i]![name]!;
|
|
325
340
|
const low = j.score !== undefined && j.score < OCCLUSION_SCORE;
|
|
326
|
-
if (j.state === 'observed' && !low) return;
|
|
341
|
+
if ((j.state === 'observed' || j.state === 'inferred') && !low) return;
|
|
327
342
|
const m = medianAt(xs, i, OCCLUSION_WINDOW, params.loop)!;
|
|
328
343
|
out[i] = m;
|
|
329
|
-
occluded.push({ joint: name, frame: i, moved: Math.hypot(m[0] - q[0], m[1] - q[1]), why: j.state !== 'observed' ? j.state : `score ${j.score!.toFixed(3)} < ${OCCLUSION_SCORE}` });
|
|
344
|
+
occluded.push({ joint: name, frame: i, moved: Math.hypot(m[0] - q[0], m[1] - q[1]), why: j.state !== 'observed' && j.state !== 'inferred' ? j.state : `score ${j.score!.toFixed(3)} < ${OCCLUSION_SCORE}` });
|
|
330
345
|
});
|
|
331
346
|
unhidden.set(name, out);
|
|
332
347
|
}
|
|
@@ -366,17 +381,23 @@ export function clean(seq: PoseSequence, params: CleanParams, wanted: string | u
|
|
|
366
381
|
const after = jitterOf([...smoothed.values()], params.loop);
|
|
367
382
|
const record: CleanRecord = {
|
|
368
383
|
spec: CLEAN_SPEC,
|
|
369
|
-
params: { loop: params.loop, legs: params.legs, rest: params.rest },
|
|
384
|
+
params: { loop: params.loop, legs: params.legs, rest: params.rest, ...(params.hipsInferred === true ? { hipsInferred: { by: 'declared' as const } } : {}) },
|
|
370
385
|
spikes: { found: spikes.map((s) => ({ frame: s.frame, joint: s.joint })), untested },
|
|
371
386
|
jitter: { before: before.value, after: after.value },
|
|
372
387
|
};
|
|
388
|
+
const inferred = params.hipsInferred === true ? { r_hip: 0, l_hip: 0 } : null;
|
|
373
389
|
const frames = seq.frames.map((f, i) => {
|
|
374
390
|
const q = f.people.find((x) => x.id === person);
|
|
375
391
|
if (q === undefined) return { t: f.t, people: [] };
|
|
376
392
|
const joints: Partial<Record<Body18, Joint>> = {};
|
|
377
393
|
for (const [name, j] of Object.entries(q.joints) as Array<[Body18, Joint]>) {
|
|
378
394
|
const moved = smoothed.get(name)?.[i] ?? null;
|
|
379
|
-
|
|
395
|
+
const out: Joint = moved === null ? { ...j } : { ...j, at: [moved[0], moved[1]] };
|
|
396
|
+
if (inferred !== null && (name === 'r_hip' || name === 'l_hip') && out.state !== 'missing' && out.at !== undefined) {
|
|
397
|
+
out.state = 'inferred';
|
|
398
|
+
inferred[name]++;
|
|
399
|
+
}
|
|
400
|
+
joints[name] = out;
|
|
380
401
|
}
|
|
381
402
|
return { t: f.t, people: [{ id: person, joints: sortedJoints(joints) }] };
|
|
382
403
|
});
|
|
@@ -392,6 +413,7 @@ export function clean(seq: PoseSequence, params: CleanParams, wanted: string | u
|
|
|
392
413
|
untested,
|
|
393
414
|
floor,
|
|
394
415
|
triples: before.triples,
|
|
416
|
+
inferred,
|
|
395
417
|
};
|
|
396
418
|
}
|
|
397
419
|
|
|
@@ -409,5 +431,18 @@ export function cleanLine(r: CleanRecord): string {
|
|
|
409
431
|
const shown = found.slice(0, 8).map((s) => `frame ${s.frame} ${s.joint}`).join(', ');
|
|
410
432
|
const spikes = `${found.length === 0 ? 'no spike' : `${found.length} spike(s) replaced (${shown}${found.length > 8 ? `, … ${found.length - 8} more` : ''})`}, ${untestedText(r.spikes.untested)}`;
|
|
411
433
|
const jit = (v: number | null) => (v === null ? 'not measured' : v.toFixed(2));
|
|
412
|
-
|
|
434
|
+
const hips = r.params.hipsInferred === undefined ? '' : `; hips inferred (${inferredSource(r)})`;
|
|
435
|
+
return `${r.spec}: ${windows}, ${loop ? 'loop' : 'no loop'}; ${spikes}; jitter (mean |second difference| of y) ${jit(r.jitter.before)} → ${jit(r.jitter.after)} px/f²${hips}`;
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
/**
|
|
439
|
+
* Where a sequence's `inferred` hips come from, as the lines say it: "declared
|
|
440
|
+
* at clean" when the record says `--hips-inferred` marked them; "source not
|
|
441
|
+
* stated" when the joints carry the state and the record does not say why
|
|
442
|
+
* (a file whose states were written by hand or by another tool).
|
|
443
|
+
*/
|
|
444
|
+
export function inferredSource(r: CleanRecord | undefined): string {
|
|
445
|
+
const h = r?.params.hipsInferred;
|
|
446
|
+
if (h === undefined) return 'source not stated';
|
|
447
|
+
return h.by === 'declared' ? 'declared at clean' : h.by;
|
|
413
448
|
}
|
package/src/pose.ts
CHANGED
|
@@ -23,7 +23,13 @@ import { isObject, parseJson, Problems, refuseUnknownKeys, typeName } from './pr
|
|
|
23
23
|
|
|
24
24
|
export const POSE_SPEC = 'rig-pose-sequence/1';
|
|
25
25
|
export const POSE_SPACE = { units: 'picture-px', origin: 'top-left', y: 'down' } as const;
|
|
26
|
-
|
|
26
|
+
/**
|
|
27
|
+
* `inferred` (rig-c/pose#63): a position the detector supplied but did not see
|
|
28
|
+
* — it carries `at` like `observed`, and retarget keys it the same; what it
|
|
29
|
+
* changes is what a reading may claim from it (POSTURE does not read a hip tilt
|
|
30
|
+
* off inferred hips).
|
|
31
|
+
*/
|
|
32
|
+
const STATES = ['observed', 'occluded', 'missing', 'inferred'] as const;
|
|
27
33
|
export type JointState = (typeof STATES)[number];
|
|
28
34
|
|
|
29
35
|
export interface Joint {
|
|
@@ -62,9 +68,18 @@ export interface CleanSpike {
|
|
|
62
68
|
frame: number;
|
|
63
69
|
joint: Body18;
|
|
64
70
|
}
|
|
71
|
+
/**
|
|
72
|
+
* Who said the hips are `inferred` (#63): `declared` — the person running
|
|
73
|
+
* `rig-pose clean --hips-inferred` said so. A measured producer would record
|
|
74
|
+
* itself under another `by`; this reader reads only `declared` today.
|
|
75
|
+
*/
|
|
76
|
+
export interface HipsInferred {
|
|
77
|
+
by: 'declared';
|
|
78
|
+
}
|
|
65
79
|
export interface CleanRecord {
|
|
66
80
|
spec: typeof CLEAN_SPEC;
|
|
67
|
-
|
|
81
|
+
/** `hipsInferred` is present only when clean marked the hips (`--hips-inferred`). */
|
|
82
|
+
params: { loop: boolean; legs: number; rest: number; hipsInferred?: HipsInferred };
|
|
68
83
|
/**
|
|
69
84
|
* `found`: every joint the spike step replaced, by frame (index into `frames`)
|
|
70
85
|
* and joint. `untested`: the frames the spike step did not test, because a
|
|
@@ -219,10 +234,10 @@ function readCamera(p: Problems, where: string, camera: unknown, frames: number
|
|
|
219
234
|
return out;
|
|
220
235
|
}
|
|
221
236
|
|
|
222
|
-
/** `cleaned`: { spec, params: { loop, legs, rest }, spikes: { found: [{ frame, joint }], untested: [frame] }, jitter: { before, after } }. */
|
|
237
|
+
/** `cleaned`: { spec, params: { loop, legs, rest, hipsInferred? }, spikes: { found: [{ frame, joint }], untested: [frame] }, jitter: { before, after } }. */
|
|
223
238
|
function readCleaned(p: Problems, where: string, c: unknown, frames: number | null): CleanRecord | undefined {
|
|
224
239
|
const cw = `${where} "cleaned"`;
|
|
225
|
-
const shape = '{ spec, params: { loop, legs, rest }, spikes: { found: [{ frame, joint }], untested: [frame] }, jitter: { before, after } }';
|
|
240
|
+
const shape = '{ spec, params: { loop, legs, rest, hipsInferred? }, spikes: { found: [{ frame, joint }], untested: [frame] }, jitter: { before, after } }';
|
|
226
241
|
if (!isObject(c)) {
|
|
227
242
|
p.add('POSE_CLEANED', `${cw} is ${typeName(c)}; it is the record rig-pose clean writes, ${shape}`);
|
|
228
243
|
return undefined;
|
|
@@ -234,9 +249,17 @@ function readCleaned(p: Problems, where: string, c: unknown, frames: number | nu
|
|
|
234
249
|
const window = (v: unknown) => Number.isInteger(v) && (v === 1 || ((v as number) >= 5 && (v as number) % 2 === 1));
|
|
235
250
|
if ('params' in c) {
|
|
236
251
|
const q = c.params;
|
|
237
|
-
if (!isObject(q)) p.add('POSE_CLEANED', `${cw} "params" is ${typeName(q)}; it must be { loop, legs, rest }`);
|
|
252
|
+
if (!isObject(q)) p.add('POSE_CLEANED', `${cw} "params" is ${typeName(q)}; it must be { loop, legs, rest, hipsInferred? }`);
|
|
238
253
|
else {
|
|
239
|
-
refuseUnknownKeys(p, 'POSE_CLEANED', `${cw} "params"`, q, ['loop', 'legs', 'rest']);
|
|
254
|
+
refuseUnknownKeys(p, 'POSE_CLEANED', `${cw} "params"`, q, ['loop', 'legs', 'rest', 'hipsInferred']);
|
|
255
|
+
if ('hipsInferred' in q) {
|
|
256
|
+
const h = q.hipsInferred;
|
|
257
|
+
if (!isObject(h)) p.add('POSE_CLEANED', `${cw} "params" "hipsInferred" is ${typeName(h)}; it must be { by: "declared" }`);
|
|
258
|
+
else {
|
|
259
|
+
refuseUnknownKeys(p, 'POSE_CLEANED', `${cw} "params" "hipsInferred"`, h, ['by']);
|
|
260
|
+
if (h.by !== 'declared') p.add('POSE_CLEANED', `${cw} "params" "hipsInferred" "by" is ${typeName(h.by)}; this reader reads "declared" (the person running rig-pose clean --hips-inferred said so)`);
|
|
261
|
+
}
|
|
262
|
+
}
|
|
240
263
|
if (typeof q.loop !== 'boolean') p.add('POSE_CLEANED', `${cw} "params" "loop" is ${typeName(q.loop)}; it must be true or false`);
|
|
241
264
|
for (const k of ['legs', 'rest'] as const) if (!window(q[k])) p.add('POSE_CLEANED', `${cw} "params" "${k}" is ${typeName(q[k])}; it must be 1 (none) or an odd whole number ≥ 5`);
|
|
242
265
|
}
|
|
@@ -277,11 +300,11 @@ function readCleaned(p: Problems, where: string, c: unknown, frames: number | nu
|
|
|
277
300
|
}
|
|
278
301
|
}
|
|
279
302
|
if (p.list.length > before) return undefined;
|
|
280
|
-
const q = c.params as { loop: boolean; legs: number; rest: number };
|
|
303
|
+
const q = c.params as { loop: boolean; legs: number; rest: number; hipsInferred?: HipsInferred };
|
|
281
304
|
const j = c.jitter as { before: number | null; after: number | null };
|
|
282
305
|
return {
|
|
283
306
|
spec: CLEAN_SPEC,
|
|
284
|
-
params: { loop: q.loop, legs: q.legs, rest: q.rest },
|
|
307
|
+
params: { loop: q.loop, legs: q.legs, rest: q.rest, ...(q.hipsInferred !== undefined ? { hipsInferred: { by: q.hipsInferred.by } } : {}) },
|
|
285
308
|
spikes: {
|
|
286
309
|
found: ((c.spikes as { found: Array<{ frame: number; joint: Body18 }> }).found).map((s) => ({ frame: s.frame, joint: s.joint })),
|
|
287
310
|
untested: [...(c.spikes as { untested: number[] }).untested],
|
|
@@ -324,6 +347,8 @@ function readJoints(p: Problems, pw: string, joints: unknown): Partial<Record<Bo
|
|
|
324
347
|
}
|
|
325
348
|
} else if (state === 'observed') {
|
|
326
349
|
p.add('POSE_JOINT_AT', `${jw} is "observed" and has no "at"; an observed joint has a position`);
|
|
350
|
+
} else if (state === 'inferred') {
|
|
351
|
+
p.add('POSE_JOINT_AT', `${jw} is "inferred" and has no "at"; an inferred joint is a position the detector supplied but did not see, so it has one`);
|
|
327
352
|
}
|
|
328
353
|
if ('score' in j) {
|
|
329
354
|
if (typeof j.score !== 'number' || !Number.isFinite(j.score)) p.add('POSE_JOINT_SCORE', `${jw} "score" is ${typeName(j.score)}; it must be a finite number`);
|
|
@@ -474,6 +499,6 @@ export function writePoseSequence(seq: PoseSequence): string {
|
|
|
474
499
|
const camera = seq.camera?.map((c) => ({ s: round(c.s, 6), tx: round(c.tx, 3), ty: round(c.ty, 3), inliers: c.inliers }));
|
|
475
500
|
const c = seq.cleaned;
|
|
476
501
|
const jit = (v: number | null) => (v === null ? null : round(v, 4));
|
|
477
|
-
const cleaned = c === undefined ? undefined : { spec: c.spec, params: { loop: c.params.loop, legs: c.params.legs, rest: c.params.rest }, spikes: { found: c.spikes.found.map((s) => ({ frame: s.frame, joint: s.joint })), untested: [...c.spikes.untested] }, jitter: { before: jit(c.jitter.before), after: jit(c.jitter.after) } };
|
|
502
|
+
const cleaned = c === undefined ? undefined : { spec: c.spec, params: { loop: c.params.loop, legs: c.params.legs, rest: c.params.rest, ...(c.params.hipsInferred !== undefined ? { hipsInferred: { by: c.params.hipsInferred.by } } : {}) }, spikes: { found: c.spikes.found.map((s) => ({ frame: s.frame, joint: s.joint })), untested: [...c.spikes.untested] }, jitter: { before: jit(c.jitter.before), after: jit(c.jitter.after) } };
|
|
478
503
|
return `${JSON.stringify({ spec: seq.spec, space: seq.space, width: seq.width, height: seq.height, source: seq.source, ...(seq.cameraSource !== undefined ? { cameraSource: seq.cameraSource } : {}), ...(camera !== undefined ? { camera } : {}), ...(cleaned !== undefined ? { cleaned } : {}), frames }, null, 2)}\n`;
|
|
479
504
|
}
|
package/src/retarget.ts
CHANGED
|
@@ -50,6 +50,14 @@
|
|
|
50
50
|
* Inside a span nothing is clamped, but a bone with a `drawn` angle is read
|
|
51
51
|
* against it (`artTurn`, and `artRange` where it leaves its range).
|
|
52
52
|
*
|
|
53
|
+
* With `raisedByHead` (rig-c/pose#62), after the range clamp and before the art
|
|
54
|
+
* turn is read: inside each span of a set that draws an arm raised, the arm is
|
|
55
|
+
* solved last, so its wrist lands where the reference's is against the head, in
|
|
56
|
+
* eye distances (each side's median over the span). The head and the shoulder
|
|
57
|
+
* are posed from the final targets of the bones above. A target out of reach
|
|
58
|
+
* goes to the reach limit along the head-relative direction (REACH); the set's
|
|
59
|
+
* `turn` is part of the solve, and only a forearm it still holds is ART-RANGE.
|
|
60
|
+
*
|
|
53
61
|
* The lines across the body — hips, shoulders, eyes — are read from the pose
|
|
54
62
|
* after holds, whether the map uses them or not: a line under
|
|
55
63
|
* `TURN_FRACTION` of its median picture length, or reversed (pointing more
|
|
@@ -63,16 +71,26 @@ import { screenToSpineDegrees } from 'rig-c/transform';
|
|
|
63
71
|
import type { Body18 } from './joints.ts';
|
|
64
72
|
import { isBody18, VIRTUAL } from './joints.ts';
|
|
65
73
|
import type { Chain, JointMap, Rest, RigSlots } from './map.ts';
|
|
66
|
-
import { aimAngle, descends, rigPoint } from './map.ts';
|
|
74
|
+
import { aimAngle, aimEnds, descends, partBone, pointParts, rigPoint } from './map.ts';
|
|
67
75
|
import type { Joint } from './pose.ts';
|
|
68
76
|
import { Problems, Refusal } from './problems.ts';
|
|
69
|
-
import type { ArtRange, SlotTrack, Switch, SwitchOptions } from './switch.ts';
|
|
70
|
-
import { clampArt, coveredChains, measureArt, overflow, RANGE_SLACK, runsOf, selectSwitches } from './switch.ts';
|
|
77
|
+
import type { ArtRange, Raised, ReachShort, SlotTrack, Switch, SwitchOptions } from './switch.ts';
|
|
78
|
+
import { clampArt, coveredChains, isRaised, measureArt, overflow, RANGE_SLACK, reachLimit, runsOf, selectSwitches, twoBone } from './switch.ts';
|
|
71
79
|
import type { Feet, LegScale, PathPose, Posing, Side, Stance } from './legs.ts';
|
|
72
80
|
import { isLegSegment, legScaleOf, legsOf, pathFrom, plantHip, posePath, rootOf, SIDES, stanceOf, supportsOf, tipOf } from './legs.ts';
|
|
73
81
|
|
|
74
82
|
const DEG = 180 / Math.PI;
|
|
75
83
|
|
|
84
|
+
/**
|
|
85
|
+
* --raised-by-head (#62): a held upper arm whose re-aimed forearm leaves the
|
|
86
|
+
* wrist farther than this from its placed point, in eye distances, "cannot
|
|
87
|
+
* reach within turn" — ART-RANGE. It sits at the float and rounding scale, so
|
|
88
|
+
* any real miss counts: the solve itself closes to ~1e-12, and the writer's
|
|
89
|
+
* 0.001° rounding of two keys moves a wrist by ≤ 2 × 1.75e-5 × (l0 + l1), which
|
|
90
|
+
* is 0.0004 eye distances on the tests' stick rig (a 130-unit arm, 12-unit eyes).
|
|
91
|
+
*/
|
|
92
|
+
export const HELD_MISS = 0.01;
|
|
93
|
+
|
|
76
94
|
export interface RetargetOptions extends SwitchOptions {
|
|
77
95
|
/** A joint with a score below this is not good on that frame (a joint with no score is). */
|
|
78
96
|
minScore: number;
|
|
@@ -90,6 +108,8 @@ export interface RetargetOptions extends SwitchOptions {
|
|
|
90
108
|
plantFeet?: boolean;
|
|
91
109
|
/** With plantFeet: a free ankle within this many leg lengths of the floor plants both feet, its leg stretched to reach it (`legs.ts` stanceOf, #60); 'off' (the default) plants one. */
|
|
92
110
|
stance?: number | 'off';
|
|
111
|
+
/** While a set whose arm is drawn raised shows, its wrist is placed against the rig's head in eye distances (rig-c/pose#62); false (the default) keeps the pose's angles. */
|
|
112
|
+
raisedByHead?: boolean;
|
|
93
113
|
}
|
|
94
114
|
|
|
95
115
|
export interface Hold {
|
|
@@ -272,6 +292,8 @@ export interface Retarget {
|
|
|
272
292
|
ankles: { bones: Record<Side, string>; at: Record<Side, Pt[]> } | null;
|
|
273
293
|
/** The translated bone's origin against its thigh roots' midpoint, at setup. */
|
|
274
294
|
pivot: Pivot | null;
|
|
295
|
+
/** With --raised-by-head: each raised set's span on each raised chain, its wrist placed against the head. */
|
|
296
|
+
raised: Raised[];
|
|
275
297
|
}
|
|
276
298
|
|
|
277
299
|
/** The hip's pivot against the thigh roots, at setup (rig-c/pose#47). */
|
|
@@ -368,6 +390,11 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
368
390
|
const endOf = (i: number): number => (i + 1 < n ? keyTimes[i + 1]! : n > 1 ? 2 * keyTimes[i]! - keyTimes[i - 1]! : keyTimes[i]!);
|
|
369
391
|
const secondsOf = (run: [number, number]): number => Math.round((endOf(run[1]) - keyTimes[run[0]]!) * 1e6) / 1e6;
|
|
370
392
|
|
|
393
|
+
// --raised-by-head (#62) measures against the head in eye distances: the map's head chain must carry the
|
|
394
|
+
// eye line with an "a>b" aim, whose two ends are the rig's eyes.
|
|
395
|
+
const head = opts.raisedByHead === true ? headOf(map) : null;
|
|
396
|
+
if (head !== null && 'problem' in head) p.add('RAISED_HEAD', head.problem);
|
|
397
|
+
|
|
371
398
|
// Every point the map reads, held where it is not good.
|
|
372
399
|
const needed = new Set<string>();
|
|
373
400
|
for (const c of map.chains) c.joints.forEach((j) => needed.add(j));
|
|
@@ -713,29 +740,52 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
713
740
|
for (let a = rest.get(b)!.parent; a !== null; a = rest.get(a)!.parent) d++;
|
|
714
741
|
return d;
|
|
715
742
|
};
|
|
743
|
+
/** The world turn of `b`'s nearest mapped bone-or-ancestor, degrees, on frame i, by the targets as they stand. */
|
|
744
|
+
const turnAbove = (b: string, i: number): number => {
|
|
745
|
+
for (let a: string | null = b; a !== null; a = rest.get(a)!.parent) {
|
|
746
|
+
const tg = targets.get(a);
|
|
747
|
+
if (tg !== undefined) return tg[i]! - restOf.get(a)!;
|
|
748
|
+
}
|
|
749
|
+
return 0;
|
|
750
|
+
};
|
|
751
|
+
/**
|
|
752
|
+
* Where the poser will put `b`'s origin on frame i: forward from the
|
|
753
|
+
* translated bone (moved by its travel) or the root (at setup), each rest
|
|
754
|
+
* offset turned by the nearest mapped bone above — the key formula's own
|
|
755
|
+
* assumption (unit scale, no shear), which the landing check holds to account.
|
|
756
|
+
*/
|
|
757
|
+
const posedOrigin = (b: string, i: number): Pt => {
|
|
758
|
+
const own = rest.get(b)!;
|
|
759
|
+
if (t !== null && b === t.bone) return [own.worldX + (travel[i]?.[0] ?? 0), own.worldY + (travel[i]?.[1] ?? 0)];
|
|
760
|
+
if (own.parent === null) return [own.worldX, own.worldY];
|
|
761
|
+
const parent = own.parent;
|
|
762
|
+
const po = posedOrigin(parent, i);
|
|
763
|
+
const r = turnAbove(parent, i) / DEG;
|
|
764
|
+
const ox = own.worldX - rest.get(parent)!.worldX;
|
|
765
|
+
const oy = own.worldY - rest.get(parent)!.worldY;
|
|
766
|
+
return [po[0] + Math.cos(r) * ox - Math.sin(r) * oy, po[1] + Math.sin(r) * ox + Math.cos(r) * oy];
|
|
767
|
+
};
|
|
768
|
+
/** A rig point (a bone, `name:tip`, `a+b`) posed on frame i the same way. */
|
|
769
|
+
const posedPoint = (spec: string, i: number): Pt => {
|
|
770
|
+
let x = 0;
|
|
771
|
+
let y = 0;
|
|
772
|
+
const parts = pointParts(spec);
|
|
773
|
+
for (const part of parts) {
|
|
774
|
+
const name = partBone(part);
|
|
775
|
+
const o = posedOrigin(name, i);
|
|
776
|
+
const bone = rest.get(name)!;
|
|
777
|
+
const l = part.endsWith(':tip') ? bone.length : 0;
|
|
778
|
+
const r = turnAbove(name, i) / DEG;
|
|
779
|
+
x += o[0] + l * (Math.cos(r) * bone.a - Math.sin(r) * bone.c);
|
|
780
|
+
y += o[1] + l * (Math.sin(r) * bone.a + Math.cos(r) * bone.c);
|
|
781
|
+
}
|
|
782
|
+
return [x / parts.length, y / parts.length];
|
|
783
|
+
};
|
|
716
784
|
const reaches = map.chains.filter((c) => c.reach).sort((x, y) => depth(x.bones[0]!) - depth(y.bones[0]!));
|
|
717
785
|
let aimReach: ((c: Chain) => void) | null = null;
|
|
718
786
|
if (reaches.length > 0 && t !== null && scale !== null) {
|
|
719
787
|
const s = scale.factor;
|
|
720
788
|
const tb = rest.get(t.bone)!;
|
|
721
|
-
/** The world turn of `b`'s nearest mapped bone-or-ancestor, degrees, on frame i. */
|
|
722
|
-
const turnAbove = (b: string, i: number): number => {
|
|
723
|
-
for (let a: string | null = b; a !== null; a = rest.get(a)!.parent) {
|
|
724
|
-
const tg = targets.get(a);
|
|
725
|
-
if (tg !== undefined) return tg[i]! - restOf.get(a)!;
|
|
726
|
-
}
|
|
727
|
-
return 0;
|
|
728
|
-
};
|
|
729
|
-
/** Where the poser will put `b`'s origin on frame i: forward from the translated bone. */
|
|
730
|
-
const posedOrigin = (b: string, i: number): Pt => {
|
|
731
|
-
if (b === t.bone) return [tb.worldX + travel[i]![0], tb.worldY + travel[i]![1]];
|
|
732
|
-
const parent = rest.get(b)!.parent!;
|
|
733
|
-
const po = posedOrigin(parent, i);
|
|
734
|
-
const r = turnAbove(parent, i) / DEG;
|
|
735
|
-
const ox = rest.get(b)!.worldX - rest.get(parent)!.worldX;
|
|
736
|
-
const oy = rest.get(b)!.worldY - rest.get(parent)!.worldY;
|
|
737
|
-
return [po[0] + Math.cos(r) * ox - Math.sin(r) * oy, po[1] + Math.sin(r) * ox + Math.cos(r) * oy];
|
|
738
|
-
};
|
|
739
789
|
aimReach = (c: Chain): void => {
|
|
740
790
|
const b = c.bones[0]!;
|
|
741
791
|
const anchor = series.get(c.joints[0]!)!;
|
|
@@ -793,8 +843,214 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
793
843
|
}
|
|
794
844
|
}
|
|
795
845
|
p.throwIfAny();
|
|
796
|
-
|
|
846
|
+
|
|
847
|
+
// --raised-by-head (#62, HQ's ruling): while a set whose arm is drawn raised shows, its wrist goes where
|
|
848
|
+
// the reference's is against the head, in eye distances (each side's median over the span). Solved last
|
|
849
|
+
// of the bones: every bone above the arm, the head and the trunk's reach included, has its final target
|
|
850
|
+
// (range clamp done), so the head and the shoulder are posed from them frame by frame; the arm moves
|
|
851
|
+
// neither. A target out of reach is placed at the reach limit along the head-relative direction
|
|
852
|
+
// (`reachLimit`, a REACH line). The set's `turn` is part of the solve: a held upper arm has its forearm
|
|
853
|
+
// re-aimed at the placed wrist, and only a forearm the turn still holds is ART-RANGE.
|
|
854
|
+
const raised: Raised[] = [];
|
|
855
|
+
const raisedArt: Array<{ sw: Switch; entries: ArtRange[] }> = [];
|
|
856
|
+
const raisedJobs: Array<{ sw: Switch; chain: Chain; wrist: (i: number) => Pt; before: Pt[]; reference: Pt[]; placed: Pt[]; refEye: number; rigEye: number; short: Array<{ i: number; how: 'far' | 'near' | 'missed'; shortfall: number }>; upperHeld: number[]; l0: number; l1: number; farthest: number; eyePoints: [string, string] }> = [];
|
|
857
|
+
if (head !== null && !('problem' in head)) {
|
|
858
|
+
const eyesOn = (i: number): { centre: Pt; dist: number } => {
|
|
859
|
+
const a = posedPoint(head.points[0], i);
|
|
860
|
+
const b = posedPoint(head.points[1], i);
|
|
861
|
+
return { centre: [(a[0] + b[0]) / 2, (a[1] + b[1]) / 2], dist: Math.hypot(b[0] - a[0], b[1] - a[1]) };
|
|
862
|
+
};
|
|
863
|
+
const headBones = new Set([...head.points.flatMap((x) => pointParts(x).map(partBone))]);
|
|
864
|
+
const ra = series.get(head.joints[0])!;
|
|
865
|
+
const rb = series.get(head.joints[1])!;
|
|
866
|
+
const refCentre = (i: number): Pt => [(ra[i]![0] + rb[i]![0]) / 2, (ra[i]![1] + rb[i]![1]) / 2];
|
|
867
|
+
for (const s of sw.switches) {
|
|
868
|
+
const set = map.alternates.find((a) => a.name === s.set)!;
|
|
869
|
+
const art: ArtRange[] = [];
|
|
870
|
+
for (const cn of s.chains) {
|
|
871
|
+
const c = map.chains.find((x) => x.name === cn)!;
|
|
872
|
+
const drawn = set.drawn[c.bones[0]!];
|
|
873
|
+
if (drawn === undefined || !isRaised(drawn)) continue;
|
|
874
|
+
const label = `--raised-by-head: set "${set.name}" draws chain "${c.name}" raised (bone "${c.bones[0]}" at ${drawn}°)`;
|
|
875
|
+
if (c.bones.length !== 2) {
|
|
876
|
+
p.add('RAISED_CHAIN', `${label}, and the chain has ${c.bones.length} bone(s); the wrist is placed by a two-bone solve, so it must have two (upper arm, forearm)`);
|
|
877
|
+
continue;
|
|
878
|
+
}
|
|
879
|
+
if (c.reach || c.aim.some((x) => x.includes('>'))) {
|
|
880
|
+
p.add('RAISED_CHAIN', `${label}, and it aims ${JSON.stringify(c.aim)}${c.reach ? ' with "reach"' : ''}; the solve turns each bone about its own origin toward the next point, so each must aim "x" or at one point`);
|
|
881
|
+
continue;
|
|
882
|
+
}
|
|
883
|
+
const [b0, b1] = c.bones as [string, string];
|
|
884
|
+
const moved = [...headBones].filter((h) => h === b0 || h === b1 || descends(rest, h, b0));
|
|
885
|
+
if (moved.length > 0) {
|
|
886
|
+
p.add('RAISED_CHAIN', `${label}, and the head's eye point ${moved.map((h) => `"${h}"`).join(', ')} turns with it; the wrist cannot be placed against a head the arm moves`);
|
|
887
|
+
continue;
|
|
888
|
+
}
|
|
889
|
+
const wristSpec = c.aim[1] === 'x' ? `${b1}:tip` : c.aim[1]!;
|
|
890
|
+
if (c.aim[1] === 'x' && !(rest.get(b1)!.length > 1e-9)) {
|
|
891
|
+
p.add('RAISED_CHAIN', `${label}, and its last bone "${b1}" aims "x" with no length; its wrist (the tip) is its origin, so there is nothing to place`);
|
|
892
|
+
continue;
|
|
893
|
+
}
|
|
894
|
+
const wristOf = (i: number): Pt => posedPoint(wristSpec, i);
|
|
895
|
+
const refWrist = series.get(c.joints[2]!)!;
|
|
896
|
+
// The unit, each side's median over the span (HQ's ruling).
|
|
897
|
+
const span: number[] = [];
|
|
898
|
+
for (let i = s.frames[0]; i <= s.frames[1]; i++) span.push(i);
|
|
899
|
+
const refEye = median(span.map((i) => Math.hypot(rb[i]![0] - ra[i]![0], rb[i]![1] - ra[i]![1])));
|
|
900
|
+
const rigEye = median(span.map((i) => eyesOn(i).dist));
|
|
901
|
+
if (!(refEye > 1e-9) || !(rigEye > 1e-9)) {
|
|
902
|
+
p.add('RAISED_EYES', `${label}: over frames ${s.frames[0]}–${s.frames[1]} the median eye distance is ${refEye > 1e-9 ? `${rigEye} units on the rig (points "${head.points[0]}", "${head.points[1]}")` : `${refEye}px in the reference (${head.joints[0]}, ${head.joints[1]})`}; an eye distance of 0 is no unit`);
|
|
903
|
+
continue;
|
|
904
|
+
}
|
|
905
|
+
// The rig's own lengths, and each bone's aim against the vector it carries, at setup.
|
|
906
|
+
const S0 = rigPoint(rest, b0)!;
|
|
907
|
+
const E0 = rigPoint(rest, b1)!;
|
|
908
|
+
const W0 = rigPoint(rest, wristSpec)!;
|
|
909
|
+
const l0 = Math.hypot(E0[0] - S0[0], E0[1] - S0[1]);
|
|
910
|
+
const l1 = Math.hypot(W0[0] - E0[0], W0[1] - E0[1]);
|
|
911
|
+
const d0 = wrap(Math.atan2(E0[1] - S0[1], E0[0] - S0[0]) * DEG - restOf.get(b0)!);
|
|
912
|
+
const d1 = wrap(Math.atan2(W0[1] - E0[1], W0[0] - E0[0]) * DEG - restOf.get(b1)!);
|
|
913
|
+
const t0 = targets.get(b0)!;
|
|
914
|
+
const t1 = targets.get(b1)!;
|
|
915
|
+
const turn0 = set.turn[b0];
|
|
916
|
+
const turn1 = set.turn[b1];
|
|
917
|
+
const job: (typeof raisedJobs)[number] = { sw: s, chain: c, wrist: wristOf, before: [], reference: [], placed: [], refEye, rigEye, short: [], upperHeld: [], l0, l1, farthest: 0, eyePoints: head.points };
|
|
918
|
+
/** Per frame of the span: the forearm's art turn the re-aim asked, and how far past its turn. */
|
|
919
|
+
const foreAsked: number[] = [];
|
|
920
|
+
const foreOver: number[] = [];
|
|
921
|
+
/** Per frame of the span: the upper arm's art turn the solve asked, how far past its turn, and — where the turn held it — how far the wrist ends from its placed point, eye distances (0 elsewhere). */
|
|
922
|
+
const upperAsked: number[] = [];
|
|
923
|
+
const upperOver: number[] = [];
|
|
924
|
+
const heldMiss: number[] = [];
|
|
925
|
+
for (const i of span) {
|
|
926
|
+
const h = eyesOn(i);
|
|
927
|
+
const rc = refCentre(i);
|
|
928
|
+
// The reference's wrist from its head centre, in its eye distances, y turned up.
|
|
929
|
+
const ref: Pt = [(refWrist[i]![0] - rc[0]) / refEye, -(refWrist[i]![1] - rc[1]) / refEye];
|
|
930
|
+
const w = wristOf(i);
|
|
931
|
+
job.reference.push(ref);
|
|
932
|
+
job.before.push([(w[0] - h.centre[0]) / rigEye, (w[1] - h.centre[1]) / rigEye]);
|
|
933
|
+
const target: Pt = [h.centre[0] + ref[0] * rigEye, h.centre[1] + ref[1] * rigEye];
|
|
934
|
+
const shoulder = posedOrigin(b0, i);
|
|
935
|
+
job.farthest = Math.max(job.farthest, Math.hypot(target[0] - shoulder[0], target[1] - shoulder[1]));
|
|
936
|
+
const place = reachLimit(h.centre, target, shoulder, l0, l1);
|
|
937
|
+
const P = place.point;
|
|
938
|
+
job.placed.push([(P[0] - h.centre[0]) / rigEye, (P[1] - h.centre[1]) / rigEye]);
|
|
939
|
+
if (place.how !== 'in') job.short.push({ i, how: place.how, shortfall: Math.hypot(target[0] - P[0], target[1] - P[1]) / rigEye });
|
|
940
|
+
// The pose's bend side, read on the vectors the two bones carry.
|
|
941
|
+
const side: 1 | -1 = wrap(t1[i]! + d1 - (t0[i]! + d0)) >= 0 ? 1 : -1;
|
|
942
|
+
const sol = twoBone(shoulder, P, l0, l1, side);
|
|
943
|
+
if (sol.upper === null) {
|
|
944
|
+
foreAsked.push(wrap(t1[i]! - (set.drawn[b1] ?? 0)));
|
|
945
|
+
foreOver.push(0);
|
|
946
|
+
upperAsked.push(0);
|
|
947
|
+
upperOver.push(0);
|
|
948
|
+
heldMiss.push(0);
|
|
949
|
+
continue;
|
|
950
|
+
}
|
|
951
|
+
let a0 = wrap(sol.upper - d0);
|
|
952
|
+
let a1 = wrap(sol.fore - d1);
|
|
953
|
+
// The set's turn holds the upper arm; the forearm is then re-aimed at the placed wrist from where the held upper arm puts the elbow.
|
|
954
|
+
let held = false;
|
|
955
|
+
const tt = turn0 === undefined ? 0 : wrap(a0 - set.drawn[b0]!);
|
|
956
|
+
upperAsked.push(tt);
|
|
957
|
+
upperOver.push(turn0 === undefined ? 0 : overflow(tt, turn0));
|
|
958
|
+
if (turn0 !== undefined) {
|
|
959
|
+
const cl = Math.min(Math.max(tt, turn0[0]), turn0[1]);
|
|
960
|
+
if (Math.abs(cl - tt) > RANGE_SLACK) {
|
|
961
|
+
held = true;
|
|
962
|
+
a0 = wrap(set.drawn[b0]! + cl);
|
|
963
|
+
job.upperHeld.push(i);
|
|
964
|
+
const r = (a0 + d0) / DEG;
|
|
965
|
+
const E: Pt = [shoulder[0] + l0 * Math.cos(r), shoulder[1] + l0 * Math.sin(r)];
|
|
966
|
+
a1 = wrap(Math.atan2(P[1] - E[1], P[0] - E[0]) * DEG - d1);
|
|
967
|
+
}
|
|
968
|
+
}
|
|
969
|
+
// A forearm the turn still holds: the re-aim failed within the art's range — ART-RANGE.
|
|
970
|
+
const asked = turn1 === undefined ? 0 : wrap(a1 - set.drawn[b1]!);
|
|
971
|
+
const over = turn1 === undefined ? 0 : overflow(asked, turn1);
|
|
972
|
+
foreAsked.push(asked);
|
|
973
|
+
foreOver.push(over);
|
|
974
|
+
if (turn1 !== undefined && over > RANGE_SLACK) a1 = wrap(set.drawn[b1]! + Math.min(Math.max(asked, turn1[0]), turn1[1]));
|
|
975
|
+
// Where the held arm leaves the wrist: the re-aimed forearm points at the placed point, but from a held
|
|
976
|
+
// elbow its length need not reach it (rule 3, "cannot reach within turn").
|
|
977
|
+
const r0 = (a0 + d0) / DEG;
|
|
978
|
+
const r1 = (a1 + d1) / DEG;
|
|
979
|
+
const W: Pt = [shoulder[0] + l0 * Math.cos(r0) + l1 * Math.cos(r1), shoulder[1] + l0 * Math.sin(r0) + l1 * Math.sin(r1)];
|
|
980
|
+
heldMiss.push(held ? Math.hypot(W[0] - P[0], W[1] - P[1]) / rigEye : 0);
|
|
981
|
+
t0[i] = a0;
|
|
982
|
+
t1[i] = a1;
|
|
983
|
+
}
|
|
984
|
+
// A held upper arm whose re-aimed forearm leaves the wrist more than HELD_MISS short of its placed point: ART-RANGE on the upper arm, with the shortfall.
|
|
985
|
+
if (turn0 !== undefined) {
|
|
986
|
+
for (const [a, b] of runsOf(heldMiss.map((m) => m > HELD_MISS), RANGE_GAP)) {
|
|
987
|
+
let worst = a;
|
|
988
|
+
for (let k = a; k <= b; k++) if (heldMiss[k]! > heldMiss[worst]!) worst = k;
|
|
989
|
+
const first = s.frames[0] + a;
|
|
990
|
+
const last = s.frames[0] + b;
|
|
991
|
+
art.push({ set: set.name, chain: c.name, bone: b0, frames: [first, last], seconds: r6(endOf(last) - keyTimes[first]!), maxOverflow: r3(upperOver[worst]!), turn: r3(upperAsked[worst]!), limits: turn0, clamped: true, shortfall: r3(heldMiss[worst]!) });
|
|
992
|
+
}
|
|
993
|
+
}
|
|
994
|
+
if (turn1 !== undefined) {
|
|
995
|
+
for (const [a, b] of runsOf(foreOver.map((o) => o > RANGE_SLACK), RANGE_GAP)) {
|
|
996
|
+
let worst = a;
|
|
997
|
+
for (let k = a; k <= b; k++) if (foreOver[k]! > foreOver[worst]!) worst = k;
|
|
998
|
+
const first = s.frames[0] + a;
|
|
999
|
+
const last = s.frames[0] + b;
|
|
1000
|
+
art.push({ set: set.name, chain: c.name, bone: b1, frames: [first, last], seconds: r6(endOf(last) - keyTimes[first]!), maxOverflow: r3(foreOver[worst]!), turn: r3(foreAsked[worst]!), limits: turn1, clamped: true });
|
|
1001
|
+
}
|
|
1002
|
+
}
|
|
1003
|
+
raisedJobs.push(job);
|
|
1004
|
+
}
|
|
1005
|
+
art.sort((x, y) => x.frames[0] - y.frames[0]);
|
|
1006
|
+
if (art.length > 0) raisedArt.push({ sw: s, entries: art });
|
|
1007
|
+
}
|
|
1008
|
+
}
|
|
1009
|
+
p.throwIfAny();
|
|
1010
|
+
// The art turn of every span; a raised chain's bones were held by the solve, and only its re-aimed forearm is listed (above).
|
|
1011
|
+
const measured = measureArt(sw.switches, map, targets, endOf, keyTimes, RANGE_GAP);
|
|
1012
|
+
const solvedBy = (e: ArtRange, s: Switch) => raisedJobs.some((j) => j.sw === s && j.chain.name === e.chain);
|
|
1013
|
+
const artRange: ArtRange[] = [];
|
|
1014
|
+
for (const s of sw.switches) {
|
|
1015
|
+
const own = measured.filter((e) => e.set === s.set && e.frames[0] >= s.frames[0] && e.frames[1] <= s.frames[1]);
|
|
1016
|
+
artRange.push(...own.filter((e) => !solvedBy(e, s)), ...(raisedArt.find((x) => x.sw === s)?.entries ?? []));
|
|
1017
|
+
}
|
|
797
1018
|
clampArt(sw.switches, map, targets);
|
|
1019
|
+
// Where each raised wrist lands as keyed: after the solve and the set's `turn`.
|
|
1020
|
+
for (const job of raisedJobs) {
|
|
1021
|
+
const after: Pt[] = [];
|
|
1022
|
+
for (let i = job.sw.frames[0]; i <= job.sw.frames[1]; i++) {
|
|
1023
|
+
const a = posedPoint(job.eyePoints[0], i);
|
|
1024
|
+
const b = posedPoint(job.eyePoints[1], i);
|
|
1025
|
+
const w = job.wrist(i);
|
|
1026
|
+
after.push([(w[0] - (a[0] + b[0]) / 2) / job.rigEye, (w[1] - (a[1] + b[1]) / 2) / job.rigEye]);
|
|
1027
|
+
}
|
|
1028
|
+
const worst = (rig: readonly Pt[]): Pt => [Math.max(...rig.map((v, k) => Math.abs(v[0] - job.reference[k]![0]))), Math.max(...rig.map((v, k) => Math.abs(v[1] - job.reference[k]![1])))].map(r6) as Pt;
|
|
1029
|
+
const round = (vs: readonly Pt[]): Pt[] => vs.map((v) => [r6(v[0]), r6(v[1])]);
|
|
1030
|
+
const short: ReachShort[] = [];
|
|
1031
|
+
for (let k = 0; k < job.short.length;) {
|
|
1032
|
+
let m = k;
|
|
1033
|
+
while (m + 1 < job.short.length && job.short[m + 1]!.i === job.short[m]!.i + 1 && job.short[m + 1]!.how === job.short[k]!.how) m++;
|
|
1034
|
+
const fs = job.short.slice(k, m + 1).map((x) => x.shortfall);
|
|
1035
|
+
short.push({ frames: [job.short[k]!.i, job.short[m]!.i], how: job.short[k]!.how, shortfall: { max: r6(Math.max(...fs)), median: r6(median(fs)) } });
|
|
1036
|
+
k = m + 1;
|
|
1037
|
+
}
|
|
1038
|
+
raised.push({
|
|
1039
|
+
set: job.sw.set,
|
|
1040
|
+
chain: job.chain.name,
|
|
1041
|
+
frames: job.sw.frames,
|
|
1042
|
+
seconds: secondsOf(job.sw.frames),
|
|
1043
|
+
eyes: { rig: r6(job.rigEye), reference: r6(job.refEye) },
|
|
1044
|
+
reference: round(job.reference),
|
|
1045
|
+
placed: round(job.placed),
|
|
1046
|
+
before: round(job.before),
|
|
1047
|
+
after: round(after),
|
|
1048
|
+
worst: { before: worst(job.before), after: worst(after), placed: r6(Math.max(...after.map((v, k) => Math.hypot(v[0] - job.placed[k]![0], v[1] - job.placed[k]![1])))) },
|
|
1049
|
+
short,
|
|
1050
|
+
upperHeld: job.upperHeld,
|
|
1051
|
+
reach: { min: r6(Math.abs(job.l0 - job.l1)), max: r6(job.l0 + job.l1), farthest: r6(job.farthest) },
|
|
1052
|
+
});
|
|
1053
|
+
}
|
|
798
1054
|
|
|
799
1055
|
// The legs on their final targets: a bone under a scaled bone is keyed through
|
|
800
1056
|
// its parent's posed matrix (legs.ts), and each ankle's world position is read.
|
|
@@ -854,10 +1110,34 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
|
|
|
854
1110
|
animation: { duration: keyTimes[n - 1]!, tracks },
|
|
855
1111
|
times: keyTimes, targets, origin, holds, bends, short, endOn, flips, scale,
|
|
856
1112
|
switches: sw.switches, shown: sw.shown, distances: sw.distances, turns, turnsUnread, range, artRange,
|
|
857
|
-
legScale, scales, feet, planted: plantOn, stance, ankles, pivot: pivotOf(rest, t?.bone ?? null, legs),
|
|
1113
|
+
legScale, scales, feet, planted: plantOn, stance, ankles, pivot: pivotOf(rest, t?.bone ?? null, legs), raised,
|
|
858
1114
|
};
|
|
859
1115
|
}
|
|
860
1116
|
|
|
1117
|
+
/** The head as --raised-by-head reads it: the chain carrying the eye line, its two pose joints and the rig's two eye points. */
|
|
1118
|
+
interface Head {
|
|
1119
|
+
chain: string;
|
|
1120
|
+
joints: [string, string];
|
|
1121
|
+
points: [string, string];
|
|
1122
|
+
}
|
|
1123
|
+
|
|
1124
|
+
/** The map's head chain (the one carrying r_eye → l_eye, either way round) and its "a>b" aim; or why there is none to read. */
|
|
1125
|
+
function headOf(map: JointMap): Head | { problem: string } {
|
|
1126
|
+
const lead = '--raised-by-head places a raised arm\'s wrist against the head in eye distances, read on the rig between the two points the head chain\'s "a>b" aim names';
|
|
1127
|
+
for (const c of map.chains) {
|
|
1128
|
+
for (let k = 0; k + 1 < c.joints.length; k++) {
|
|
1129
|
+
const a = c.joints[k]!;
|
|
1130
|
+
const b = c.joints[k + 1]!;
|
|
1131
|
+
if (!((a === 'r_eye' && b === 'l_eye') || (a === 'l_eye' && b === 'r_eye'))) continue;
|
|
1132
|
+
const aim = c.aim[k]!;
|
|
1133
|
+
if (!aim.includes('>')) return { problem: `${lead}; chain "${c.name}" carries ${a} → ${b} on bone "${c.bones[k]}", and aims it ${JSON.stringify(aim)}, not "a>b" between the rig's eye points` };
|
|
1134
|
+
const [pa, pb] = aimEnds(c.bones[k]!, aim);
|
|
1135
|
+
return { chain: c.name, joints: [a, b], points: [pa, pb] };
|
|
1136
|
+
}
|
|
1137
|
+
}
|
|
1138
|
+
return { problem: `${lead}; no chain of the map carries the eye line (joints r_eye → l_eye), so the rig has no eye points to measure` };
|
|
1139
|
+
}
|
|
1140
|
+
|
|
861
1141
|
type ShortSpan = Omit<Short, 'chain' | 'bone' | 'joints'>;
|
|
862
1142
|
|
|
863
1143
|
/**
|
package/src/run.ts
CHANGED
|
@@ -8,13 +8,13 @@ import { cpSync, existsSync, mkdtempSync, readFileSync, rmSync, statSync, writeF
|
|
|
8
8
|
import { tmpdir } from 'node:os';
|
|
9
9
|
import { dirname, join, resolve } from 'node:path';
|
|
10
10
|
import { readMap, resolveMap, type JointMap, type Rest } from './map.ts';
|
|
11
|
-
import { cleanLine } from './clean.ts';
|
|
11
|
+
import { cleanLine, inferredSource } from './clean.ts';
|
|
12
12
|
import { convertCoco17, personTrack, readPoseSequence, type CameraFrame } from './pose.ts';
|
|
13
13
|
import { isObject, lineOf, parseJson, Problems, Refusal, type Problem } from './problems.ts';
|
|
14
14
|
import { boneIdentity, droppedLine, keptOutLine, mergeProblems, readFragment, readIdle, targetOf, trackLabel, type IdleReading, type RigcTrack, type Target } from './idle.ts';
|
|
15
15
|
import { bendLine, endOnLine, flipLine, holdLine, rangeLine, retarget, shortLine, turnLine, type MotionTrack, type RetargetOptions } from './retarget.ts';
|
|
16
16
|
import { attachmentBones, baseMotionOf, buildInto, rigcVersion, folds, gate, readBuild, resolveRig, restOf, slotsOf } from './rigc.ts';
|
|
17
|
-
import { artRangeLine, chainsMoving, switchLine, uncoveredFolds } from './switch.ts';
|
|
17
|
+
import { artRangeLine, chainsMoving, raisedLine, RAISED_WITHIN, reachLine, switchLine, uncoveredFolds } from './switch.ts';
|
|
18
18
|
import { landed, measureFeet, measureLanding, measurePosture, measureShear, poseFrames, postureLines } from './verify.ts';
|
|
19
19
|
import { legsOf } from './legs.ts';
|
|
20
20
|
import { reduceKeys, type ReduceBone, type Reduced } from './reduce.ts';
|
|
@@ -108,7 +108,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
|
|
|
108
108
|
resolveMap(map, rest, a.map, slots);
|
|
109
109
|
|
|
110
110
|
const r = retarget(rest, map, frames, times, a, slots);
|
|
111
|
-
say(`rig-pose retarget: ${seq.frames.length} frame(s), ${r.animation.duration.toFixed(3)}s, onto "${input.name}" as "${a.name}" — ${r.animation.tracks.length} track(s); min-score ${a.minScore}, flip ${a.flip === 'off' ? 'off' : `${a.flip}°`}, short ${a.short}, bends ${a.bends} (tolerance ${a.bendTolerance}°)${a.legScale !== undefined && a.legScale !== 'off' ? `, leg-scale ${a.legScale[0]}–${a.legScale[1]}` : ''}${a.plantFeet === true ? ', plant-feet' : ''}${a.stance !== undefined && a.stance !== 'off' ? `, stance ${a.stance}` : ''}${a.carryIdle === true ? ', carry-idle' : ''}${a.merge !== undefined ? `, merge ${a.merge}` : ''}${a.reduce !== undefined ? `, reduce ${a.reduce}°${a.reduceHip !== undefined ? ` (hip ${a.reduceHip} units)` : ''}` : ''}`);
|
|
111
|
+
say(`rig-pose retarget: ${seq.frames.length} frame(s), ${r.animation.duration.toFixed(3)}s, onto "${input.name}" as "${a.name}" — ${r.animation.tracks.length} track(s); min-score ${a.minScore}, flip ${a.flip === 'off' ? 'off' : `${a.flip}°`}, short ${a.short}, bends ${a.bends} (tolerance ${a.bendTolerance}°)${a.legScale !== undefined && a.legScale !== 'off' ? `, leg-scale ${a.legScale[0]}–${a.legScale[1]}` : ''}${a.plantFeet === true ? ', plant-feet' : ''}${a.stance !== undefined && a.stance !== 'off' ? `, stance ${a.stance}` : ''}${a.raisedByHead === true ? ', raised-by-head' : ''}${a.carryIdle === true ? ', carry-idle' : ''}${a.merge !== undefined ? `, merge ${a.merge}` : ''}${a.reduce !== undefined ? `, reduce ${a.reduce}°${a.reduceHip !== undefined ? ` (hip ${a.reduceHip} units)` : ''}` : ''}`);
|
|
112
112
|
say(` RIGC rig-c ${rigc.version} (${rigc.entry}) gates and poses this run`);
|
|
113
113
|
say(` CLEAN ${cleanLine(seq.cleaned!)}`);
|
|
114
114
|
if (a.smoothAlias !== undefined) say(` SMOOTH --smooth is carried by clean; ignored (given ${a.smoothAlias}; this sequence was cleaned with legs ${seq.cleaned!.params.legs}, rest ${seq.cleaned!.params.rest})`);
|
|
@@ -130,6 +130,9 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
|
|
|
130
130
|
for (const l of windowLines(map, r.switches, r.distances, r.times, a.switchTolerance)) say(` FRAMES ${l}`);
|
|
131
131
|
}
|
|
132
132
|
for (const e of r.range) say(` RANGE ${rangeLine(e, r.times)}`);
|
|
133
|
+
if (a.raisedByHead === true && r.raised.length === 0) say(` RAISED none — no set drawing an arm within ${RAISED_WITHIN}° of straight up showed; no key moved`);
|
|
134
|
+
for (const e of r.raised) say(` RAISED ${raisedLine(e, r.times)}`);
|
|
135
|
+
for (const e of r.raised) if (e.short.length > 0) say(` REACH ${reachLine(e, r.times)}`);
|
|
133
136
|
for (const e of r.artRange) say(` ART-RANGE ${artRangeLine(e, r.times)}`);
|
|
134
137
|
for (const l of r.legScale) say(` LEG-SCALE ${legScaleLine(l, r.times)}`);
|
|
135
138
|
if (r.pivot !== null) say(` PIVOT bone "${r.pivot.bone}"'s origin is ${r.pivot.distance.toFixed(2)} units (${r.pivot.hipWidths.toFixed(3)} hip widths) from its thigh roots' midpoint (${r.pivot.roots.join(' + ')}): dx ${r.pivot.offset[0].toFixed(2)}, dy ${r.pivot.offset[1].toFixed(2)}, at setup — the pivot every hip turn swings the legs about`);
|
|
@@ -225,7 +228,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
|
|
|
225
228
|
p.throwIfAny();
|
|
226
229
|
|
|
227
230
|
const lengths = new Map(doc.bones.map((b) => [b.name, b.length ?? 0]));
|
|
228
|
-
const posture = measurePosture(posed, lengths, map, r, frames, a.minScore);
|
|
231
|
+
const posture = measurePosture(posed, lengths, map, r, frames, a.minScore, inferredSource(seq.cleaned));
|
|
229
232
|
if (posture !== null) for (const l of postureLines(posture)) say(` POSTURE ${l}`);
|
|
230
233
|
const legBones = legBonesOf(map);
|
|
231
234
|
const feetReading = legBones === null ? null : measureFeet(posed, lengths, r, legBones);
|
|
@@ -273,7 +276,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
|
|
|
273
276
|
frames: seq.frames.length,
|
|
274
277
|
duration: r.animation.duration,
|
|
275
278
|
cleaned: seq.cleaned,
|
|
276
|
-
options: { minScore: a.minScore, flip: a.flip, short: a.short, bends: a.bends, bendTolerance: a.bendTolerance, switchTolerance: a.switchTolerance, switchMin: a.switchMin, switchAt: a.switchAt, profile: a.profile, legScale: a.legScale ?? 'off', plantFeet: a.plantFeet === true, stance: a.stance ?? 'off', idle: idleName, carryIdle: a.carryIdle === true, merge: a.merge ?? null, reduce: a.reduce ?? 'off', reduceHip: a.reduceHip ?? 'off' },
|
|
279
|
+
options: { minScore: a.minScore, flip: a.flip, short: a.short, bends: a.bends, bendTolerance: a.bendTolerance, switchTolerance: a.switchTolerance, switchMin: a.switchMin, switchAt: a.switchAt, profile: a.profile, legScale: a.legScale ?? 'off', plantFeet: a.plantFeet === true, stance: a.stance ?? 'off', raisedByHead: a.raisedByHead === true, idle: idleName, carryIdle: a.carryIdle === true, merge: a.merge ?? null, reduce: a.reduce ?? 'off', reduceHip: a.reduceHip ?? 'off' },
|
|
277
280
|
scale: r.scale,
|
|
278
281
|
holds: r.holds,
|
|
279
282
|
short: r.short,
|
|
@@ -289,6 +292,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
|
|
|
289
292
|
turns: r.turns,
|
|
290
293
|
range: r.range,
|
|
291
294
|
artRange: r.artRange,
|
|
295
|
+
raised: r.raised,
|
|
292
296
|
legScale: r.legScale.map(({ scale: _scale, ...rest }) => rest),
|
|
293
297
|
pivot: r.pivot,
|
|
294
298
|
stance: r.stance === null ? null : { tolerance: r.stance.tolerance, bounds: r.stance.bounds, runs: r.stance.runs, over: r.stance.over },
|
package/src/switch.ts
CHANGED
|
@@ -129,6 +129,8 @@ export interface ArtRange {
|
|
|
129
129
|
limits: [number, number];
|
|
130
130
|
/** Whether the bone was held at the limit (the set's `turn`) or followed the pose (its chain's `range` only). */
|
|
131
131
|
clamped: boolean;
|
|
132
|
+
/** --raised-by-head (#62): on a held upper arm, the most the re-aimed forearm left the wrist short of its placed point, eye distances; `turn` and `maxOverflow` are then the upper arm's on that frame. */
|
|
133
|
+
shortfall?: number;
|
|
132
134
|
}
|
|
133
135
|
|
|
134
136
|
export interface Uncovered {
|
|
@@ -580,6 +582,9 @@ function moveText(m: Move): string {
|
|
|
580
582
|
}
|
|
581
583
|
|
|
582
584
|
export function artRangeLine(e: ArtRange, times: readonly number[]): string {
|
|
585
|
+
if (e.shortfall !== undefined) {
|
|
586
|
+
return `set "${e.set}" chain "${e.chain}" bone "${e.bone}" held at its turn (range [${e.limits[0]}, ${e.limits[1]}]) ${spanOf(e.frames, times)}: ${e.seconds.toFixed(3)}s the re-aimed forearm leaves the wrist up to ${e.shortfall.toFixed(2)} eye distances short of its placed point (the upper arm asked ${sg(e.turn)} from drawn, ${deg(e.maxOverflow)} past) — demand for another drawing of the set`;
|
|
587
|
+
}
|
|
583
588
|
return `set "${e.set}" chain "${e.chain}" bone "${e.bone}" turned ${sg(e.turn)} from drawn (range [${e.limits[0]}, ${e.limits[1]}]) ${spanOf(e.frames, times)}: ${e.seconds.toFixed(3)}s, up to ${deg(e.maxOverflow)} past it; ${e.clamped ? 'held at the limit' : 'the bone follows the pose'} — demand for another drawing of the set`;
|
|
584
589
|
}
|
|
585
590
|
|
|
@@ -603,3 +608,162 @@ export function clampArt(switches: readonly Switch[], map: JointMap, targets: Ma
|
|
|
603
608
|
}
|
|
604
609
|
}
|
|
605
610
|
}
|
|
611
|
+
|
|
612
|
+
/**
|
|
613
|
+
* A set's arm is raised (rig-c/pose#62) when its `drawn` world angle on the
|
|
614
|
+
* chain's first bone (the upper arm) is within this many degrees of straight up.
|
|
615
|
+
*/
|
|
616
|
+
export const RAISED_WITHIN = 30;
|
|
617
|
+
|
|
618
|
+
/** Whether a drawn world angle, Spine degrees, is within `RAISED_WITHIN` of straight up (90°). */
|
|
619
|
+
export const isRaised = (drawn: number): boolean => Math.abs(wrap(drawn - 90)) <= RAISED_WITHIN;
|
|
620
|
+
|
|
621
|
+
type P2 = [number, number];
|
|
622
|
+
|
|
623
|
+
/**
|
|
624
|
+
* A two-bone solve in the world (y up): the directions, degrees, of the vector
|
|
625
|
+
* from the shoulder to the elbow (`upper`) and from the elbow to the wrist
|
|
626
|
+
* (`fore`) that put the wrist on `target`, for bones of lengths `l0` and `l1`.
|
|
627
|
+
* `side` is the elbow's bend: +1 when the forearm turns counter-clockwise from
|
|
628
|
+
* the upper arm, −1 clockwise. Out of reach (farther than l0 + l1, nearer than
|
|
629
|
+
* |l0 − l1|, beyond a relative 1e-6) both point straight at the target and
|
|
630
|
+
* `reached` is false; on the shoulder, where there is no direction, `upper` is
|
|
631
|
+
* null. `reachLimit` puts a target inside first, so the raised solve always reaches.
|
|
632
|
+
*/
|
|
633
|
+
export function twoBone(shoulder: P2, target: P2, l0: number, l1: number, side: 1 | -1): { upper: number; fore: number; reached: boolean } | { upper: null; reached: false } {
|
|
634
|
+
const dx = target[0] - shoulder[0];
|
|
635
|
+
const dy = target[1] - shoulder[1];
|
|
636
|
+
const d = Math.hypot(dx, dy);
|
|
637
|
+
if (d < 1e-9) return { upper: null, reached: false };
|
|
638
|
+
const theta = Math.atan2(dy, dx) * (180 / Math.PI);
|
|
639
|
+
const eps = 1e-6 * Math.max(1, l0 + l1);
|
|
640
|
+
if (d > l0 + l1 + eps || d < Math.abs(l0 - l1) - eps) return { upper: theta, fore: theta, reached: false };
|
|
641
|
+
const cos = Math.min(1, Math.max(-1, (l0 * l0 + d * d - l1 * l1) / (2 * l0 * d)));
|
|
642
|
+
const upper = theta - side * Math.acos(cos) * (180 / Math.PI);
|
|
643
|
+
const r = upper * (Math.PI / 180);
|
|
644
|
+
const ex = shoulder[0] + l0 * Math.cos(r);
|
|
645
|
+
const ey = shoulder[1] + l0 * Math.sin(r);
|
|
646
|
+
return { upper: wrap(upper), fore: Math.atan2(target[1] - ey, target[0] - ex) * (180 / Math.PI), reached: true };
|
|
647
|
+
}
|
|
648
|
+
|
|
649
|
+
/** Where a raised wrist is placed: the target itself, or the reach limit (rig-c/pose#62, HQ's ruling). */
|
|
650
|
+
export interface Placement {
|
|
651
|
+
point: P2;
|
|
652
|
+
/** `in`: the target is within reach. `far` / `near`: past l0 + l1, or nearer than |l0 − l1|, and placed on the ray from the head centre. `missed`: that ray never enters the reach; placed at the reach's point nearest the target. */
|
|
653
|
+
how: 'in' | 'far' | 'near' | 'missed';
|
|
654
|
+
}
|
|
655
|
+
|
|
656
|
+
/**
|
|
657
|
+
* The raised wrist's placement. The arm reaches the annulus about the shoulder
|
|
658
|
+
* between |l0 − l1| and l0 + l1. A target inside it is kept. Otherwise the wrist
|
|
659
|
+
* goes on the ray from the head centre through the target, at the point of
|
|
660
|
+
* that ray inside the annulus nearest the target — the reach limit along the
|
|
661
|
+
* head-relative direction. The ray can cross the annulus twice (out and back
|
|
662
|
+
* through the inner disc, or through the whole ring): of the reachable stretches
|
|
663
|
+
* the one nearest the target wins, a tie going to the point nearer the head. A
|
|
664
|
+
* ray that never enters it (`missed`) gives the annulus point nearest the
|
|
665
|
+
* target, on the line from the shoulder through it (toward the head centre when
|
|
666
|
+
* the target is on the shoulder).
|
|
667
|
+
*/
|
|
668
|
+
export function reachLimit(centre: P2, target: P2, shoulder: P2, l0: number, l1: number): Placement {
|
|
669
|
+
const rMin = Math.abs(l0 - l1);
|
|
670
|
+
const rMax = l0 + l1;
|
|
671
|
+
const eps = 1e-9 * Math.max(1, rMax);
|
|
672
|
+
const dT = Math.hypot(target[0] - shoulder[0], target[1] - shoulder[1]);
|
|
673
|
+
if (dT >= rMin - eps && dT <= rMax + eps) return { point: target, how: 'in' };
|
|
674
|
+
const how = dT > rMax ? 'far' : 'near';
|
|
675
|
+
const vx = target[0] - centre[0];
|
|
676
|
+
const vy = target[1] - centre[1];
|
|
677
|
+
const L = Math.hypot(vx, vy);
|
|
678
|
+
if (L > 1e-9) {
|
|
679
|
+
const u: P2 = [vx / L, vy / L];
|
|
680
|
+
// |C + s·u − S|² = s² + 2·s·b + c
|
|
681
|
+
const b = u[0] * (centre[0] - shoulder[0]) + u[1] * (centre[1] - shoulder[1]);
|
|
682
|
+
const c = (centre[0] - shoulder[0]) ** 2 + (centre[1] - shoulder[1]) ** 2;
|
|
683
|
+
const roots = (r: number): [number, number] | null => {
|
|
684
|
+
const disc = b * b - c + r * r;
|
|
685
|
+
return disc < 0 ? null : [-b - Math.sqrt(disc), -b + Math.sqrt(disc)];
|
|
686
|
+
};
|
|
687
|
+
const outer = roots(rMax);
|
|
688
|
+
const stretches: Array<[number, number]> = [];
|
|
689
|
+
if (outer !== null) {
|
|
690
|
+
const lo = Math.max(0, outer[0]);
|
|
691
|
+
const hi = outer[1];
|
|
692
|
+
const inner = rMin > 0 ? roots(rMin) : null;
|
|
693
|
+
if (hi >= lo) {
|
|
694
|
+
if (inner === null || inner[1] <= lo || inner[0] >= hi) stretches.push([lo, hi]);
|
|
695
|
+
else {
|
|
696
|
+
if (inner[0] > lo) stretches.push([lo, inner[0]]);
|
|
697
|
+
if (inner[1] < hi) stretches.push([inner[1], hi]);
|
|
698
|
+
}
|
|
699
|
+
}
|
|
700
|
+
}
|
|
701
|
+
let best: number | null = null;
|
|
702
|
+
for (const [lo, hi] of stretches) {
|
|
703
|
+
const s = Math.min(Math.max(L, lo), hi);
|
|
704
|
+
if (best === null || Math.abs(s - L) < Math.abs(best - L) - 1e-9 || (Math.abs(Math.abs(s - L) - Math.abs(best - L)) <= 1e-9 && s < best)) best = s;
|
|
705
|
+
}
|
|
706
|
+
if (best !== null) return { point: [centre[0] + best * u[0], centre[1] + best * u[1]], how };
|
|
707
|
+
}
|
|
708
|
+
const from: P2 = dT > 1e-9 ? target : centre;
|
|
709
|
+
const k = Math.hypot(from[0] - shoulder[0], from[1] - shoulder[1]) || 1;
|
|
710
|
+
const r = Math.min(Math.max(dT, rMin), rMax);
|
|
711
|
+
return { point: [shoulder[0] + (r * (from[0] - shoulder[0])) / k, shoulder[1] + (r * (from[1] - shoulder[1])) / k], how: 'missed' };
|
|
712
|
+
}
|
|
713
|
+
|
|
714
|
+
/** A run of frames whose target was out of reach, one way. */
|
|
715
|
+
export interface ReachShort {
|
|
716
|
+
frames: [number, number];
|
|
717
|
+
how: 'far' | 'near' | 'missed';
|
|
718
|
+
/** |target − placed| in the rig's eye distances: the most, and the median, over the run. */
|
|
719
|
+
shortfall: { max: number; median: number };
|
|
720
|
+
}
|
|
721
|
+
|
|
722
|
+
/** One span of a raised set on one chain: its wrist placed against the head (rig-c/pose#62). */
|
|
723
|
+
export interface Raised {
|
|
724
|
+
set: string;
|
|
725
|
+
chain: string;
|
|
726
|
+
frames: [number, number];
|
|
727
|
+
seconds: number;
|
|
728
|
+
/** The unit: the eye distance's median over the span, the rig's in world units, the reference's in picture px. */
|
|
729
|
+
eyes: { rig: number; reference: number };
|
|
730
|
+
/**
|
|
731
|
+
* Per frame of the span, the wrist's offset from the head centre in eye
|
|
732
|
+
* distances (x right, y up): the reference's; where the wrist was placed (the
|
|
733
|
+
* target, or the reach limit); the rig's on the pose's angles; the rig's as
|
|
734
|
+
* keyed.
|
|
735
|
+
*/
|
|
736
|
+
reference: P2[];
|
|
737
|
+
placed: P2[];
|
|
738
|
+
before: P2[];
|
|
739
|
+
after: P2[];
|
|
740
|
+
/** The largest |rig − reference| over the span, x and y, before and after; and the largest |after − placed|, eye distances. */
|
|
741
|
+
worst: { before: P2; after: P2; placed: number };
|
|
742
|
+
/** The runs of frames out of the arm's reach (the REACH line). */
|
|
743
|
+
short: ReachShort[];
|
|
744
|
+
/** The frames whose upper arm the set's `turn` held: the forearm re-aimed at the placed wrist. */
|
|
745
|
+
upperHeld: number[];
|
|
746
|
+
/** The arm's reach (from |l0 − l1| to l0 + l1, world units from the shoulder) and the farthest target from the shoulder over the span. */
|
|
747
|
+
reach: { min: number; max: number; farthest: number };
|
|
748
|
+
}
|
|
749
|
+
|
|
750
|
+
const xy = (v: P2) => `(${v[0].toFixed(2)}, ${v[1].toFixed(2)})`;
|
|
751
|
+
const framesText = (fs: readonly number[]): string => {
|
|
752
|
+
const flag: boolean[] = [];
|
|
753
|
+
for (const f of fs) flag[f] = true;
|
|
754
|
+
return runsOf(Array.from({ length: flag.length }, (_, i) => flag[i] === true), 0).map(([a, b]) => (a === b ? `${a}` : `${a}–${b}`)).join(', ');
|
|
755
|
+
};
|
|
756
|
+
|
|
757
|
+
export function raisedLine(e: Raised, times: readonly number[]): string {
|
|
758
|
+
const med = (vs: readonly P2[]): P2 => [median(vs.map((v) => v[0])), median(vs.map((v) => v[1]))];
|
|
759
|
+
const held = e.upperHeld.length === 0 ? 'the upper arm within its turn on every frame' : `the upper arm held by the set's turn on ${e.upperHeld.length} frame(s) (${framesText(e.upperHeld)}), the forearm re-aimed`;
|
|
760
|
+
const out = e.short.length === 0 ? 'every target in reach' : `out of reach on ${e.short.reduce((k, s) => k + s.frames[1] - s.frames[0] + 1, 0)} frame(s) (REACH)`;
|
|
761
|
+
return `set "${e.set}" chain "${e.chain}" ${spanOf(e.frames, times)}: the wrist placed against the head; eye distance (median) rig ${e.eyes.rig.toFixed(2)} units, reference ${e.eyes.reference.toFixed(2)} px; wrist − head centre in eye distances (x right, y up), median reference ${xy(med(e.reference))}, placed ${xy(med(e.placed))}, before ${xy(med(e.before))}, after ${xy(med(e.after))}; worst |rig − reference| before ${xy(e.worst.before)}, after ${xy(e.worst.after)}; worst |after − placed| ${e.worst.placed.toFixed(2)}; ${held}; ${out}`;
|
|
762
|
+
}
|
|
763
|
+
|
|
764
|
+
/** The REACH line: a raised span whose target was out of the arm's reach on some frames — the proportions, not the art's range, were the limit. */
|
|
765
|
+
export function reachLine(e: Raised, times: readonly number[]): string {
|
|
766
|
+
const why = { far: `farther than l0 + l1 = ${e.reach.max.toFixed(1)}`, near: `nearer than |l0 − l1| = ${e.reach.min.toFixed(1)}`, missed: 'and the ray from the head centre never enters the reach; placed at the reach\'s point nearest it' } as const;
|
|
767
|
+
const runs = e.short.map((s) => `${spanOf(s.frames, times)} ${s.how === 'missed' ? 'out of reach' : why[s.how]}${s.how === 'missed' ? `, ${why.missed}` : ''}: up to ${s.shortfall.max.toFixed(2)} eye distances short (median ${s.shortfall.median.toFixed(2)})`);
|
|
768
|
+
return `set "${e.set}" chain "${e.chain}": the target out of the arm's reach from the shoulder — ${runs.join('; ')}; the wrist placed at the reach limit along the head-relative direction (the farthest target ${e.reach.farthest.toFixed(1)} units from the shoulder). Not ART-RANGE: the rig's proportions, not the art's range, were the limit`;
|
|
769
|
+
}
|
package/src/verify.ts
CHANGED
|
@@ -246,8 +246,19 @@ export interface Posture {
|
|
|
246
246
|
rigShoulder: string | null;
|
|
247
247
|
/** Reference frames on which the hips (tilt) and the hips and shoulders (sway) are all good. */
|
|
248
248
|
frames: { tilt: number; sway: number; of: number };
|
|
249
|
-
/**
|
|
250
|
-
|
|
249
|
+
/**
|
|
250
|
+
* Hip-line tilt, Spine degrees (r_hip → l_hip; the rig: the pelvis chain's
|
|
251
|
+
* aim). Null when every frame with both hips good has an `inferred` hip
|
|
252
|
+
* (#63): the tilt is not read, and `tiltWhy` says why.
|
|
253
|
+
*/
|
|
254
|
+
tilt: { reference: Range; rig: Range; ratio: number } | null;
|
|
255
|
+
/** Present only when `tilt` is null: why it was not read. */
|
|
256
|
+
tiltWhy?: string;
|
|
257
|
+
/**
|
|
258
|
+
* Present only when some frame with both hips good has an `inferred` hip:
|
|
259
|
+
* how many such frames the tilt left out, and where the state came from.
|
|
260
|
+
*/
|
|
261
|
+
hipsInferred?: { frames: number; source: string };
|
|
251
262
|
/**
|
|
252
263
|
* Sway: the x-range of mid_hip − mid_shoulder, in rig units (the reference's
|
|
253
264
|
* picture px × the figure scale) and in each body's own shoulder widths.
|
|
@@ -282,8 +293,17 @@ const medianOf = (xs: number[]): number => {
|
|
|
282
293
|
* the joints it needs are good), the rig is the gated build posed by rig-c.
|
|
283
294
|
* Null when the map has no pelvis chain (joints exactly r_hip, l_hip) or no
|
|
284
295
|
* translate.
|
|
296
|
+
*
|
|
297
|
+
* Inferred hips (#63): a frame on which either hip is `inferred` is left out of
|
|
298
|
+
* the tilt — its hip line is a position the detector supplied but did not see,
|
|
299
|
+
* and a tilt read off it would be the detector's guess reported as the dancer's.
|
|
300
|
+
* When that is every frame (`clean --hips-inferred` marks them all), the tilt
|
|
301
|
+
* is null and `tiltWhy` names the reason and its source (`inferredSource`, read
|
|
302
|
+
* from the clean record). On some frames only, the tilt is read over the rest
|
|
303
|
+
* and the line counts the frames left out. Sway is read as before, inferred
|
|
304
|
+
* hips included: it is a different measure, and the card leaves it as it is.
|
|
285
305
|
*/
|
|
286
|
-
export function measurePosture(posed: Array<Map<string, RawBone>>, lengths: ReadonlyMap<string, number>, map: JointMap, r: Retarget, frames: ReadonlyArray<Partial<Record<Body18, Joint>>>, minScore: number): Posture | null {
|
|
306
|
+
export function measurePosture(posed: Array<Map<string, RawBone>>, lengths: ReadonlyMap<string, number>, map: JointMap, r: Retarget, frames: ReadonlyArray<Partial<Record<Body18, Joint>>>, minScore: number, inferredSource = 'source not stated'): Posture | null {
|
|
287
307
|
const pelvis = map.chains.find((c) => c.joints.length === 2 && c.joints[0] === 'r_hip' && c.joints[1] === 'l_hip');
|
|
288
308
|
if (pelvis === undefined || map.translate === null || r.origin === null || r.scale === null) return null;
|
|
289
309
|
const s = r.scale.factor;
|
|
@@ -294,11 +314,13 @@ export function measurePosture(posed: Array<Map<string, RawBone>>, lengths: Read
|
|
|
294
314
|
const refTilt: number[] = [];
|
|
295
315
|
const refSway: number[] = [];
|
|
296
316
|
const refSw: number[] = [];
|
|
297
|
-
|
|
317
|
+
let inferred = 0;
|
|
318
|
+
frames.forEach((f, i) => {
|
|
298
319
|
const rh = at(i, 'r_hip');
|
|
299
320
|
const lh = at(i, 'l_hip');
|
|
300
321
|
if (rh === null || lh === null) return;
|
|
301
|
-
|
|
322
|
+
if (f.r_hip?.state === 'inferred' || f.l_hip?.state === 'inferred') inferred++;
|
|
323
|
+
else refTilt.push(pictureAngle(rh, lh));
|
|
302
324
|
const rs = at(i, 'r_shoulder');
|
|
303
325
|
const ls = at(i, 'l_shoulder');
|
|
304
326
|
if (rs === null || ls === null) return;
|
|
@@ -307,9 +329,13 @@ export function measurePosture(posed: Array<Map<string, RawBone>>, lengths: Read
|
|
|
307
329
|
});
|
|
308
330
|
const bone = pelvis.bones[0]!;
|
|
309
331
|
const aim = pelvis.aim[0]!;
|
|
310
|
-
|
|
311
|
-
const
|
|
312
|
-
|
|
332
|
+
let tilt: Posture['tilt'] = null;
|
|
333
|
+
const why = `hips inferred (${inferredSource})`;
|
|
334
|
+
if (inferred === 0 || refTilt.length > 0) {
|
|
335
|
+
const rigTilt = posed.map((b) => aimAngle(b, lengths, bone, aim));
|
|
336
|
+
tilt = { reference: rangeOf(unwrapped(refTilt)), rig: rangeOf(unwrapped(rigTilt)), ratio: 0 };
|
|
337
|
+
tilt.ratio = tilt.rig.range / tilt.reference.range;
|
|
338
|
+
}
|
|
313
339
|
|
|
314
340
|
// The rig's mid-shoulder: the point a reach aims at, else the midpoint of the arms' first bones.
|
|
315
341
|
const reach = map.chains.find((c) => c.reach);
|
|
@@ -340,7 +366,16 @@ export function measurePosture(posed: Array<Map<string, RawBone>>, lengths: Read
|
|
|
340
366
|
ratioSw: rig === null || rig.rangeSw === null ? null : rig.rangeSw / reference.rangeSw,
|
|
341
367
|
};
|
|
342
368
|
}
|
|
343
|
-
return {
|
|
369
|
+
return {
|
|
370
|
+
pelvis: pelvis.name,
|
|
371
|
+
rigHip: r.origin.bone,
|
|
372
|
+
rigShoulder,
|
|
373
|
+
frames: { tilt: refTilt.length, sway: refSway.length, of: frames.length },
|
|
374
|
+
tilt,
|
|
375
|
+
...(tilt === null ? { tiltWhy: why } : {}),
|
|
376
|
+
...(inferred > 0 ? { hipsInferred: { frames: inferred, source: inferredSource } } : {}),
|
|
377
|
+
sway,
|
|
378
|
+
};
|
|
344
379
|
}
|
|
345
380
|
|
|
346
381
|
const f1 = (x: number) => x.toFixed(1);
|
|
@@ -350,8 +385,12 @@ const sg = (x: number) => `${x >= 0 ? '+' : '−'}${Math.abs(x).toFixed(1)}`;
|
|
|
350
385
|
/** The POSTURE lines. */
|
|
351
386
|
export function postureLines(q: Posture): string[] {
|
|
352
387
|
const over = (k: number) => (k === q.frames.of ? '' : ` over ${k} of ${q.frames.of} frames`);
|
|
388
|
+
const t = q.tilt;
|
|
389
|
+
const left = q.hipsInferred === undefined ? '' : `; ${q.hipsInferred.frames} frame(s) with an inferred hip not read (${q.hipsInferred.source})`;
|
|
353
390
|
const out = [
|
|
354
|
-
|
|
391
|
+
t === null
|
|
392
|
+
? `hip tilt not read: ${q.tiltWhy}`
|
|
393
|
+
: `hip tilt range (chain "${q.pelvis}", r_hip → l_hip, Spine degrees): reference ${f1(t.reference.range)}° (${sg(t.reference.min)}° … ${sg(t.reference.max)}°${over(q.frames.tilt)}), rig ${f1(t.rig.range)}° (${sg(t.rig.min)}° … ${sg(t.rig.max)}°) — rig/reference ${f3(t.ratio)}${left}`,
|
|
355
394
|
];
|
|
356
395
|
const w = q.sway;
|
|
357
396
|
if (w === null) out.push('sway: the reference has no frame with both hips and both shoulders good; not measured');
|