@rig-c/pose 0.13.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -16,12 +16,52 @@ Part of the rig-c toolchain: [core](https://github.com/rig-c/core) compiles and
16
16
  rig-pose retarget --rig examples/wave/rig --map examples/wave/map.json --pose examples/wave/pose.json --out wave.motion.json --build wave-build
17
17
  ```
18
18
 
19
- It prints the gate's verdict and where every mapped bone landed, then writes `wave.motion.json` and the gated build. `rigc preview --candidate wave-build` plays it in the Spine Web Player. A rig with no art shows only its bones.
19
+ `pose.json` is already cleaned (`make-pose.ts` passes it through `clean`). It prints the gate's verdict and where every mapped bone landed, then writes `wave.motion.json` and the gated build. `rigc preview --candidate wave-build` plays it in the Spine Web Player. A rig with no art shows only its bones.
20
20
 
21
21
  ## What the rig needs
22
22
 
23
23
  The map names **chains**: each limb as bones that descend from one another, parent to child. That means a thigh with its shin below it, and an upper arm with its forearm below it, and likewise for the trunk and the head. A rig whose arm is one bone, or that has no leg bones, cannot take a pose: the map is refused by name (`MAP_*`). [@rig-c/parts](https://github.com/rig-c/parts) builds such a rig from a painting's keypoints with `propose --keypoints`. A rig drawn by hand needs the same chains.
24
24
 
25
+ ## Clean first: `rig-pose clean`
26
+
27
+ ```sh
28
+ rig-pose clean --pose dance.scan.json --out dance.pose.json [--loop] [--legs 5] [--rest 7]
29
+ ```
30
+
31
+ `retarget` reads only a sequence that `clean` wrote ([#67](https://github.com/rig-c/pose/issues/67)). A pose model reads each frame alone, so a hanging wrist moves ±20 px a frame and a cloak-hidden leg jumps 100+ px, and every key carries that shake. `clean` takes it out once, at the sequence, where it can be inspected. A sequence without the record is refused by name (`POSE_NOT_CLEANED`), and the message is the `clean` command to run on that file. There is no flag to skip the step. Windows of 1 (`--legs 1 --rest 1`) are a legitimate run that smooths nothing, and they are recorded as such.
32
+
33
+ The steps, in this order, each printed with its figures:
34
+
35
+ 1. **Spikes.** A joint more than 0.2 trunk lengths from its 5-frame median takes the median. The trunk is neck → mid-hip, the median over the clip. Each replacement is listed by frame and joint (`SPIKE`). Without `--loop`, a clip end cuts the window of the first and last two frames. There the median is a neighbour's value, and a joint moving steadily would read as a spike. Those frames are **not tested**, and the `SPIKES` line and the record name them. Reflecting the window does not help: the median of `[x2, x1, x0, x1, x2]` is still `x1`.
36
+ 2. **Occlusion.** A joint whose state is not `observed`, or whose score is under 0.3, takes its 3-frame median (`OCCLUDED`).
37
+ 3. **Smoothing.** A quadratic Savitzky–Golay per joint: the legs (both hips, knees and ankles) over `--legs` frames (default 5), every other joint over `--rest` (default 7). A window is 1 (none) or odd and at least 5. 3 is refused because a quadratic through 3 points passes through all three. Without `--loop`, the window is cut at the ends. Where fewer than three frames have a position, the degree drops to fit the ones that do.
38
+ 4. **Floor.** Only with `--loop`: the whole figure moves in y by one constant, so that the lowest ankle over the cycle is where it was before smoothing.
39
+
40
+ A **median** is taken per coordinate (x and y apart), over the window's frames on which the joint has a position, its own frame included, whatever its state. An even count takes the mean of the middle two. **`--loop`** says the sequence is one cycle, such as a gait from strike to strike. Every window then wraps, the last frame running into the first, so nothing is untested; a cycle shorter than its widest window is refused (`CLEAN_LOOP`). **Nothing is invented.** A joint with no position on a frame gets none: `retarget` holds it and prints a `HOLD` line. States and scores are written as they were read. `--coco` reads COCO-17 (below) and converts it first. `--person` picks the person when there is more than one, and only that person is written.
41
+
42
+ The output carries the record, here the dancer's from the table below:
43
+
44
+ ```json
45
+ "cleaned": { "spec": "rig-pose-clean/1",
46
+ "params": { "loop": false, "legs": 5, "rest": 7 },
47
+ "spikes": { "found": [], "untested": [0, 1, 311, 312] },
48
+ "jitter": { "before": 7.9123, "after": 4.6225 } }
49
+ ```
50
+
51
+ `jitter` is the mean |second difference| of y, in px/f², over every joint and every three consecutive frames that have a position, wrapping with `--loop`. It is measured on the sequence as read and on the cleaned positions. `retarget` prints the record as its `CLEAN` line and copies it into `--report`.
52
+
53
+ **Measured** on HQ's references ([#67](https://github.com/rig-c/pose/issues/67)'s bars):
54
+
55
+ | Clip | Run | Jitter, px/f² | Against |
56
+ | --- | --- | --- | --- |
57
+ | The 13 s dancer | defaults | 7.91 → 4.62 | HQ's hand-cleaned file: 4.62; positions within 0.071 px |
58
+ | armor_walk, one 32-frame cycle | `--loop` | 9.43 → 3.16; 3.155 without the four arm joints | HQ's file without the arm joints: 3.156 (HQ's hand step smoothed the arms a second time) |
59
+ | armor_run, one 20-frame cycle | `--loop --rest 5` | 17.06 → 7.99 | HQ's file: 8.17 |
60
+ | school_walk, 90 frames, held out | defaults | 5.99 → 1.95 (−67 %) | the bar: −30 % |
61
+ | school_run, 90 frames, held out | defaults | 47.00 → 11.54 (−75 %) | the bar: −30 % |
62
+
63
+ A retarget of the cleaned dancer, with `--flip 90 --short 0.35 --bends clamp --leg-scale 0.8,1.05 --plant-feet`, finds the same jumps (`FEET`) as the same run on HQ's hand-cleaned file. Its hip rotate sd over frames 94–175 is 4.574, against HQ's 4.58.
64
+
25
65
  ## What it does
26
66
 
27
67
  ```sh
@@ -84,19 +124,19 @@ The demand lists:
84
124
 
85
125
  Runs in `turns`, `range` and `artRange` merge across a gap of one frame, as `endOn` does; `seconds` runs from the first frame's key time to the end of the last frame.
86
126
 
87
- `--smooth <n>` smooths each joint's positions, after its holds, with a Savitzky–Golay filter: at every frame, the least-squares cubic over the *n* frames around it (clipped at the ends), read at that frame. It is centred, so it moves no motion in time, and a cubic keeps the peaks a moving average flattens. `1` (the default) is none; otherwise *n* is odd and at least 5, since a cubic through 3 points passes through all three.
127
+ `--smooth <n>` is an alias kept since [#67](https://github.com/rig-c/pose/issues/67). The smoothing is `clean`'s (`--legs`, `--rest`), so `retarget` prints `--smooth is carried by clean; ignored` and smooths nothing: `--smooth 7` writes the same bytes as `--smooth 1`. The three paragraphs below were measured before #67, when `retarget` smoothed with a cubic Savitzky–Golay over `--smooth` frames. They are kept as the record of why the smoothing moved to the sequence.
88
128
 
89
129
  **`--smooth 7` flattens the pelvis's rebound.** Measured on HQ's dancer with `--leg-scale 0.8,1.05 --plant-feet` ([#47](https://github.com/rig-c/pose/issues/47)): the rebound — the hip's peak upward acceleration at the bottom of each bounce, rig ÷ reference, median over a window, bar ≥ 0.7 — reads 0.40–0.52 with `--smooth 7` in every configuration measured (0.461 and 0.501 in the two windows, where `--smooth 1` reads 0.648 and 0.942). So the pelvis is read at `--smooth 1`, and `--smooth 7` is not advised for it. What follows is the earlier advice, for the limbs.
90
130
 
91
131
  **`--smooth 7` at 24 fps, for the limbs.** It was measured on a 13 s dance on a @rig-c/parts rig, read back from the gated build: the motion's timing did not move (speed cross-correlation lag 0 on the wrists, ankles and hip), the frame-to-frame second difference dropped 2.5× on the wrists, 1.7–2.1× on the ankles and 1.5× on the hip, and the hip's 95th-percentile speed fell to 0.85×. The criterion set beforehand, a 2× drop on every point, was not met on the hip. That criterion was flawed: the hip's second difference is mostly real motion, while the filter cuts white noise 4.6×. A measure that isolates noise is a later card, and the default stays off until it passes.
92
132
 
93
- **The flip guard.** A pose model can put a fast sweep of a limb into one frame — under motion blur it takes the elbow or the wrist for another point — and the rig then whips the limb round between two keys. Position smoothing cannot remove a step like that. So after holds and smoothing, and before the hinge check, every segment the map aims (not a `reach`, which is aimed later from posed positions) is read frame to frame: a turn greater than `--flip <deg>` from frame k−1 to frame k is spread. For the smallest h ≥ 1 that keeps every one-frame turn at or under the threshold, frames k−h … k+h−1 are replaced by the straight line from frame k−h−1 to frame k+h, the turn's own way round. It is centred, so the step happens when the pose put it, over 2h frames, and both ends keep the pose's angles. A window that would cross the first or last frame stops there and the line says so; two windows that overlap become one. `--flip off` keeps every turn. The report lists each spread under `flips`.
133
+ **The flip guard.** A pose model can put a fast sweep of a limb into one frame — under motion blur it takes the elbow or the wrist for another point — and the rig then whips the limb round between two keys. Position smoothing cannot remove a step like that. So after holds, and before the hinge check, every segment the map aims (not a `reach`, which is aimed later from posed positions) is read frame to frame: a turn greater than `--flip <deg>` from frame k−1 to frame k is spread. For the smallest h ≥ 1 that keeps every one-frame turn at or under the threshold, frames k−h … k+h−1 are replaced by the straight line from frame k−h−1 to frame k+h, the turn's own way round. It is centred, so the step happens when the pose put it, over 2h frames, and both ends keep the pose's angles. A window that would cross the first or last frame stops there and the line says so; two windows that overlap become one. `--flip off` keeps every turn. The report lists each spread under `flips`.
94
134
 
95
135
  The threshold was fixed by measuring first ([#15](https://github.com/rig-c/pose/issues/15)). On the 13 s dance (313 frames, RTMPose), the one-frame turns of the picture angle of 10 segments — both upper arms, forearms, thighs and shins, the eye line and the hip line, 3,120 turns — run p50 1.0°, p90 5.0°, p99 23.1°, p99.5 33.4°, p99.9 78.2°, max 148.6°. 90° is the p99.9 rounded up past a margin, and marks the 3 turns above it (0.1%): both forearms between frames 181 and 182 (148.6° and 114.4°, a motion-blurred stretch) and the left forearm between 53 and 54 (99.2°, the forearm pointing at the camera). None of them is a spike that comes back: each keeps the direction the limb was already turning, and the new angle holds — a real sweep put into one frame. Holding the last angle would be wrong a frame later, so the guard spreads the step instead. With it, on the @rig-c/parts rig #11 measured, the largest one-frame turn of any guarded bone in the gated build, posed by rig-c, is 78.2° (148.6° without it), and every bone still lands.
96
136
 
97
137
  Re-measured with the guard on, against the guarded unsmoothed run, on #11's criteria (power above 6 Hz down ≥ 2× on every point, timing lag 0, p95 speed ≥ 0.85×): no window passes. `--smooth 7` comes closest — the left wrist's power drops 1.99× and the hip's p95 speed is 0.847× — so `--smooth` stays off by default, and so does the guard: the rule posted before the run made a default for either one depend on a window passing. **Use `--flip 90`**: with it, no bone of the gated build turns more than 90° in one frame (without it, 148.6°), and every bone still lands.
98
138
 
99
- **The short hold.** A segment pointing at the camera is short in the picture, and its picture direction is then detector noise: the rig turns the bone to follow it. So after holds and smoothing, and before the flip guard, every limb segment the map aims is measured on every frame (not a `reach`, and not a line across the body from a left joint to its right twin, such as the hip line or the eye line: those shorten when the body or head turns, which is a different thing, and the cutoff below was measured on limbs only) — its picture length, between the same two points its angle is read from, over its own median length across the pose. On a frame under `--short <fraction>` its angle is not read from the pose: it is interpolated, the short way round, from the last frame at or above the cutoff to the next one, and held flat at the first or last frame of the pose. The flip guard, the hinge check and the keys all see the held angle. `--short off` (the default) reads every frame. The report lists each span under `short` (chain, bone, segment, first and last frame, the lowest fraction).
139
+ **The short hold.** A segment pointing at the camera is short in the picture, and its picture direction is then detector noise: the rig turns the bone to follow it. So after holds, and before the flip guard, every limb segment the map aims is measured on every frame (not a `reach`, and not a line across the body from a left joint to its right twin, such as the hip line or the eye line: those shorten when the body or head turns, which is a different thing, and the cutoff below was measured on limbs only) — its picture length, between the same two points its angle is read from, over its own median length across the pose. On a frame under `--short <fraction>` its angle is not read from the pose: it is interpolated, the short way round, from the last frame at or above the cutoff to the next one, and held flat at the first or last frame of the pose. The flip guard, the hinge check and the keys all see the held angle. `--short off` (the default) reads every frame. The report lists each span under `short` (chain, bone, segment, first and last frame, the lowest fraction).
100
140
 
101
141
  The cutoff was fixed by measuring first ([#22](https://github.com/rig-c/pose/issues/22)), over both upper arms, forearms, thighs and shins. On the 13 s dance (24 fps, 2,504 segment-frames) the fraction's 0.5th percentile is 0.51, its 5th 0.82, and 7 segment-frames (0.28%) are under 0.35; on a 16 s second sequence (30 fps, 3,936) they are 0.27 and 0.75, and 34 (0.86%). Every frame under 0.35 was looked at, and was one of two kinds:
102
142
 
@@ -187,7 +227,7 @@ Exit codes: `0` written, `1` refused (every reason printed, nothing written), `2
187
227
  - More than one person: `--person <id>`.
188
228
  - `score` is whatever the pose model gives. RTMPose's goes above 1, so `--min-score` is a cut on that model's scale, not a probability.
189
229
 
190
- **From COCO-17** — the layout most pose models and datasets use — `rig-pose convert --coco in.json --out pose.json`, or `retarget --coco`. The input is `{ "format": "coco17-sequence/1", "width", "height", "source", "frames": [{ "t", "keypoints" }] }`, keypoints in COCO's order as 17 `[x, y, score]` or 51 numbers, or `null` for a frame with nobody. The mapping: COCO's `left_*`/`right_*` → body-18's `l_*`/`r_*` (both are the subject's sides), a score of 0 → `missing`, and body-18's `neck`, which COCO lacks, is the shoulders' midpoint scored the lower of the two (OpenPose's definition).
230
+ **From COCO-17** — the layout most pose models and datasets use — `rig-pose clean --coco in.json --out pose.json`, which converts and cleans in one step. `rig-pose convert --coco` converts only, and its output still needs `clean`. `retarget --coco` is refused (`POSE_NOT_CLEANED`), naming the `clean --coco` command. The input is `{ "format": "coco17-sequence/1", "width", "height", "source", "frames": [{ "t", "keypoints" }] }`, keypoints in COCO's order as 17 `[x, y, score]` or 51 numbers, or `null` for a frame with nobody. The mapping: COCO's `left_*`/`right_*` → body-18's `l_*`/`r_*` (both are the subject's sides), a score of 0 → `missing`, and body-18's `neck`, which COCO lacks, is the shoulders' midpoint scored the lower of the two (OpenPose's definition).
191
231
 
192
232
 
193
233
  **The camera** ([#63](https://github.com/rig-c/pose/issues/63)). A sequence may carry an optional `camera`, one `{ "s", "tx", "ty", "inliers" }` per frame: the picture's scale and translation against frame 0's, so a point p₀ of frame 0's picture appears at `s · p₀ + (tx, ty)` on that frame. Its joints are then already normalised by it, `p′ = (p − t) / s`, and `inliers` is how many background features the fit kept. `s` must be > 0, `tx` and `ty` finite, `inliers` an integer ≥ 0, and there is one entry per frame; anything else is refused by name (`POSE_CAMERA`). The retarget does not apply it again: it prints a `CAMERA` line with the range of `s` and the three frames where it changes fastest. With `--plant-feet` and no `camera`, the `CAMERA` line says the floor may step with the picture, since a zoom or a pan in the video moves the floor with it. An optional `cameraSource` string says where the camera came from: `"tracked"` or `"hand-labelled, N keys"`. It is refused without a `camera`, or when it is not a non-empty string (`POSE_CAMERA_SOURCE`), and the `CAMERA` line starts with it (`CAMERA tracked: …`; a sequence with a `camera` and no `cameraSource` prints `CAMERA (source not stated): …`). `tools/scan.py` (not shipped) reads the camera off the video's background and writes the normalised sequence with `"cameraSource": "tracked"`. A frame whose fit keeps too few background features is refused by name, and nothing is written. Where the background cannot be tracked, `tools/scan.py --camera-file` takes a hand-written camera instead, a `rig-pose-camera/1` file: `{ "spec": "rig-pose-camera/1", "keys": [ { "frame", "s", "tx", "ty" } ], "note"? }`. Its frames are integers of the pose and strictly increase, `s` is > 0, and `tx` and `ty` are finite. Values are linear between keys and held before the first key and after the last. `inliers` is 0 on every frame, since no feature was fitted, and anything malformed is refused by name (`CAMERA_FILE`). `tools/scan_selftest.py` is the synthetic check of both paths.
@@ -249,7 +289,7 @@ Bone rotation cannot carry every pose: an arm behind the head, a forearm toward
249
289
  - `offsets` (optional): a translate per bone, in its parent's axes, held while the set shows and 0 elsewhere (stepped keys), so the joints as painted are the pivots. A bone `translate` already keys (the hip) cannot take one.
250
290
  - Each name resolves against the built rig and the map, and a miss is refused by name, listing what exists: `MAP_ALT_SLOT`, `MAP_ALT_ATTACHMENT`, `MAP_ALT_HIDES`, `MAP_ALT_CHAIN`, `MAP_ALT_JOINT`, `MAP_ALT_POSE`, `MAP_ALT_DRAWN` and `MAP_ALT_OFFSET` (a bone that is not a bone of the set's chains).
251
291
 
252
- **When one shows.** Per frame and per set, the distance is the largest difference, in degrees, between a segment of its chains as the retarget asked for it (after holds, smoothing, the short hold, the flip guard and the hinge check; before any range clamp) and the same segment in its key skeleton. On each frame the nearest set within `--switch-tolerance` (default 25°) shows, ties going to the map's order; a run of one set shorter than `--switch-min` (default 0.25 s) is dropped and the base art shows there. Both figures were fixed before a run ([#6](https://github.com/rig-c/pose/issues/6)). Sets compete only with those they share a slot (shown or hidden) or an offset bone with, so a drawing of each arm can show at once; two that compete never overlap.
292
+ **When one shows.** Per frame and per set, the distance is the largest difference, in degrees, between a segment of its chains as the retarget asked for it (after holds, the short hold, the flip guard and the hinge check; before any range clamp) and the same segment in its key skeleton. On each frame the nearest set within `--switch-tolerance` (default 25°) shows, ties going to the map's order; a run of one set shorter than `--switch-min` (default 0.25 s) is dropped and the base art shows there. Both figures were fixed before a run ([#6](https://github.com/rig-c/pose/issues/6)). Sets compete only with those they share a slot (shown or hidden) or an offset bone with, so a drawing of each arm can show at once; two that compete never overlap.
253
293
 
254
294
  **Where a switch is keyed: `--switch-at`.** By default (`first`) a boundary — the frame a set starts showing on, and the frame the base art or the next set comes back on — is where the selection above puts it: the first frame within the tolerance, often a frame where the arm is slowing into the pose, so the change of drawing can be seen on a nearly still arm. `--switch-at peak` moves each boundary to the frame within ±`--switch-min`/2 (±3 frames at 24 fps with the default 0.25 s) where the chains it changes turn fastest, so the cut lands in motion ([#31](https://github.com/rig-c/pose/issues/31)). The speed on a frame is the largest |angular speed| in degrees per frame, a central difference of the target world angles (one-sided on the first and last frame), over every segment of the leaving set's and the arriving set's chains; it is read on the same targets as the distance, before the range clamp and any set's `turn`, which depend on where the spans are. The window is clipped so every span keeps `--switch-min`, a gap between two spans of one set keeps it too (the bridging holds), no boundary reaches its neighbour, and a set with `frames` shows only inside them. In it, a frame qualifies only if the set the move newly shows is within **2 × `--switch-tolerance`** of its key skeleton on every frame it newly shows it on; the boundary goes to the fastest qualifying frame, and stays when none qualifies. A tie goes to the frame nearest the original, then the earlier. A boundary between two sets moves once.
255
295
 
@@ -259,7 +299,7 @@ Either way the `SWITCH` line ends with `moved on …, off …`: each boundary's
259
299
 
260
300
  **What is written.** An `attachment` track per slot whose picture changes: the set's attachment (or `null` for a hidden slot) where a span starts, and the slot's setup attachment, as the compiled model states it, where it ends. A direct switch from one set to another puts the first set's slots back to setup on the second's first frame, so an extra slot that is empty at setup goes empty. A switch onto frame *i* is keyed halfway between frames *i*−1 and *i*: a stepped key on the frame's own time fires a frame late wherever a player's accumulated time lands just below it (rig-c's docs/AUTHORING.md §4.5), and halfway both the pose's times and a renderer's `i / fps` see it on frame *i*. Before the first key a slot shows its setup attachment, so no key is written at 0 unless a span starts there. Offsets are `translate` keys at the same times, `stepped`. rig-c's gate checks these keys like any other.
261
301
 
262
- **TURN.** The cutoff was measured on the three lines across the body before it was fixed ([#6](https://github.com/rig-c/pose/issues/6)): on the 13 s front-facing dance their picture length never drops under 0.81 of its median and none reverses; on a second sequence the hip line falls to 0.17 at frames 465–491 (side-on) and the eye line to 0.22 at 26–58 (head in profile), and 3 frames reverse. 0.5 lies between. A line is reversed when it points more than 90° from the subject's right joint to the left (front-facing, it points at 0°). The lines are read after holds and smoothing, held and smoothed as the map's points are even where the map does not use them; a line whose points are good on no frame is named as not measured.
302
+ **TURN.** The cutoff was measured on the three lines across the body before it was fixed ([#6](https://github.com/rig-c/pose/issues/6)): on the 13 s front-facing dance their picture length never drops under 0.81 of its median and none reverses; on a second sequence the hip line falls to 0.17 at frames 465–491 (side-on) and the eye line to 0.22 at 26–58 (head in profile), and 3 frames reverse. 0.5 lies between. A line is reversed when it points more than 90° from the subject's right joint to the left (front-facing, it points at 0°). The lines are read after holds, held as the map's points are even where the map does not use them; a line whose points are good on no frame is named as not measured.
263
303
 
264
304
 
265
305
  **A stopgap window: `frames`.** A set may state `"frames": [[first, last], …]`: it shows only inside those windows, and on them it is chosen over any set within tolerance that has no window. It is for two sets whose key skeletons the angles cannot tell apart. Measured on HQ's dancer, the hands clasped behind the back and the hands low in front differ by only 2–21°, and the wrist and hand-point scores do not separate them either; the cue that might is #33's depth state. It is recorded in the map as a stopgap, not a selection rule. A window **restricts** a set; it never forces one. Where the set does not show inside its window, the run says why in a `FRAMES` line: the stretch, and whether the pose there was past the tolerance (with the largest distance), or within it but the run was dropped.
package/cli.ts CHANGED
@@ -8,7 +8,8 @@
8
8
  */
9
9
  import { readFileSync, writeFileSync } from 'node:fs';
10
10
  import pkg from './package.json' with { type: 'json' };
11
- import { convertCoco17, writePoseSequence } from './src/pose.ts';
11
+ import { clean, cleanLine, untestedText, LEGS, OCCLUSION_SCORE, OCCLUSION_WINDOW, SPIKE_TRUNKS, SPIKE_WINDOW, windowProblem } from './src/clean.ts';
12
+ import { convertCoco17, readPoseSequence, writePoseSequence } from './src/pose.ts';
12
13
  import { parseWindows } from './src/compare.ts';
13
14
  import { lineOf, Refusal } from './src/problems.ts';
14
15
  import { runCompare, type CompareArgs } from './src/runcompare.ts';
@@ -17,29 +18,52 @@ import { runRetarget, type RetargetArgs } from './src/run.ts';
17
18
  const HELP = `rig-pose ${pkg.version} — motion from a pose sequence onto a rig-c rig
18
19
 
19
20
  Usage:
21
+ rig-pose clean --pose <pose.json> --out <pose.clean.json> [--loop] [--legs 5] [--rest 7] [--coco] [--person <id>]
20
22
  rig-pose retarget --rig <rig> --map <map.json> --pose <pose.json> --out <motion.json> [--name dance] [flags]
21
23
  rig-pose convert --coco <coco17.json> --out <pose.json>
22
24
  rig-pose compare --reference <pose.json> --render <pose.json> --map <map.json> [flags]
23
25
  rig-pose --version | --help
24
26
 
27
+ clean (required before retarget; rig-c/pose#67)
28
+ --pose <file> rig-pose-sequence/1 as a scan wrote it, or COCO-17 with --coco
29
+ --out <file> the cleaned rig-pose-sequence/1, carrying the record of what was done ("cleaned")
30
+ --coco read --pose as coco17-sequence/1 and convert it first (neck = the shoulders' midpoint)
31
+ --person <id> the person to clean when the sequence holds more than one; only that person is written
32
+ --loop the sequence is one cycle (a gait's strike to strike): every window wraps, last frame
33
+ to first, and the floor is re-pinned
34
+ --legs <n> Savitzky–Golay window over the legs — both hips, knees and ankles (default: 5)
35
+ --rest <n> Savitzky–Golay window over every other joint (default: 7)
36
+ n is 1 (none), or odd and ≥ 5: a quadratic through 3 points passes through all three
37
+
38
+ In order, each step printed with its figures: (1) spikes — a joint more than 0.2 trunk lengths
39
+ (neck → mid_hip, the median over the clip) from its 5-frame median takes the median, each listed by
40
+ frame and joint; without --loop the first and last two frames, whose window a clip end cuts, are not
41
+ tested and are listed as untested; (2) occlusion — a joint not "observed", or scored under 0.3, takes its 3-frame median;
42
+ (3) a quadratic Savitzky–Golay per joint, the window cut at the ends (wrapped with --loop); (4) with
43
+ --loop, the figure moved in y so the lowest ankle over the cycle is where it was before smoothing.
44
+ A median is per coordinate over the window's frames where the joint has a position. A joint with no
45
+ position on a frame is left without one (retarget holds it). States and scores are written as read.
46
+ Jitter: the mean |second difference| of y over every joint, px/f², before and after.
47
+
25
48
  retarget
26
49
  --rig <path> the rig spec rigc builds from: a rig.json, or a directory holding one
27
50
  (with its motion.json beside it — @rig-c/parts writes the pair to <out>/rig/)
28
51
  --map <map.json> rig-pose-map/1: which bones carry which joints
29
- --pose <pose.json> rig-pose-sequence/1 (body-18 per frame), or COCO-17 with --coco
52
+ --pose <pose.json> rig-pose-sequence/1 (body-18 per frame) that rig-pose clean wrote; a sequence without
53
+ its "cleaned" record is refused (POSE_NOT_CLEANED), naming the clean command to run
30
54
  --out <motion.json> where the motion spec goes: the rig's own animations plus the new one,
31
55
  ready for \`rigc build --rig <rig> --motion <motion.json>\`
32
56
  --name <name> the new animation's name (default: dance); refused if the rig has one by that name
33
- --coco read --pose as coco17-sequence/1 and convert it first
57
+ --coco read --pose as coco17-sequence/1: refused (POSE_NOT_CLEANED) — clean it with
58
+ rig-pose clean --coco, then retarget the cleaned file
34
59
  --person <id> the person to follow when the sequence holds more than one
35
60
  --base-motion <path> the rig's motion spec, when it is not motion.json beside rig.json
36
61
  --images <dir> passed to rigc build: where the rig spec's images are, when its own "images" path
37
62
  does not reach them (a rig.json copied away from its images)
38
63
  --min-score <s> a joint scored below s is not good on that frame and holds its last good
39
64
  value (default: 0.3); a joint with no score is good
40
- --smooth <n> Savitzky–Golay over n frames of joint positions: a centred least-squares cubic,
41
- read at each frame (default: 1, none); n is odd and ≥ 5 — a cubic through 3
42
- points passes through all three, so 3 would change nothing
65
+ --smooth <n> an alias kept since rig-c/pose#67: the smoothing is clean's (--legs, --rest); retarget
66
+ prints "--smooth is carried by clean; ignored" and smooths nothing
43
67
  --flip <deg>|off a segment that turns more than <deg> degrees in one frame of the pose is
44
68
  spread over the frames around the jump, centred on it, and listed as FLIP
45
69
  (default: off, every turn kept as the pose gives it; 90 is the measured setting)
@@ -179,12 +203,6 @@ function num(f: Map<string, string>, name: string, dflt: number, ok: (n: number)
179
203
  return n;
180
204
  }
181
205
 
182
- function smoothOf(f: Map<string, string>): number {
183
- const n = num(f, 'smooth', 1, (v) => Number.isInteger(v) && v >= 1 && v % 2 === 1, '1 (none) or an odd whole number ≥ 5');
184
- if (n === 3) throw new Usage('--smooth is "3"; a cubic fitted through 3 points passes through all three, so a window of 3 changes nothing — it must be 1 (none) or an odd whole number ≥ 5');
185
- return n;
186
- }
187
-
188
206
  function flipOf(f: Map<string, string>): number | 'off' {
189
207
  if (!f.has('flip') || f.get('flip') === 'off') return 'off';
190
208
  return num(f, 'flip', 90, (v) => v > 0 && v < 180, 'off, or degrees in (0, 180) — a one-frame turn is never more than 180°');
@@ -226,7 +244,7 @@ function retargetArgs(argv: string[]): RetargetArgs {
226
244
  baseMotion: f.get('base-motion'),
227
245
  images: f.get('images'),
228
246
  minScore: num(f, 'min-score', 0.3, (n) => n >= 0, 'a number ≥ 0'),
229
- smooth: smoothOf(f),
247
+ smoothAlias: f.get('smooth'),
230
248
  flip: flipOf(f),
231
249
  short: shortOf(f),
232
250
  legScale: legScaleOf(f),
@@ -299,6 +317,66 @@ function compareArgs(argv: string[]): CompareArgs {
299
317
  };
300
318
  }
301
319
 
320
+ /** A window flag of clean: 1, or odd and ≥ 5 (`windowProblem`). */
321
+ function windowOf(f: Map<string, string>, name: string, dflt: number): number {
322
+ const v = f.get(name);
323
+ if (v === undefined) return dflt;
324
+ const n = Number(v);
325
+ const why = windowProblem(`--${name}`, Number.isFinite(n) ? n : NaN);
326
+ if (!Number.isFinite(n) || why !== null) throw new Usage(why ?? `--${name} is "${v}"; it must be 1 (none) or an odd whole number ≥ 5`);
327
+ return n;
328
+ }
329
+
330
+ const fmt = (v: number, d = 2) => v.toFixed(d);
331
+ function runsOf(frames: number[]): string {
332
+ const out: string[] = [];
333
+ let a = -1;
334
+ let b = -1;
335
+ for (const f of frames) {
336
+ if (f === b + 1) b = f;
337
+ else {
338
+ if (a >= 0) out.push(a === b ? `${a}` : `${a}–${b}`);
339
+ a = f;
340
+ b = f;
341
+ }
342
+ }
343
+ if (a >= 0) out.push(a === b ? `${a}` : `${a}–${b}`);
344
+ return out.join(', ');
345
+ }
346
+
347
+ /** `rig-pose clean` (#67): read, clean, print each step's figures, write. */
348
+ function runClean(argv: string[], say: (l: string) => void): void {
349
+ const f = parseFlags(argv, ['pose', 'out', 'legs', 'rest', 'person'], ['loop', 'coco']);
350
+ const input = need(f, 'pose');
351
+ const out = need(f, 'out');
352
+ const params = { loop: f.has('loop'), legs: windowOf(f, 'legs', 5), rest: windowOf(f, 'rest', 7) };
353
+ const text = readFileSync(input, 'utf8');
354
+ const seq = f.has('coco') ? convertCoco17(text, input) : readPoseSequence(text, input);
355
+ const c = clean(seq, params, f.get('person'), input);
356
+ const n = seq.frames.length;
357
+ const tOf = (i: number) => `t=${seq.frames[i]!.t.toFixed(3)}s`;
358
+ say(`rig-pose clean: ${n} frame(s) of person "${c.person}" from ${input}${f.has('coco') ? ' (COCO-17)' : ''}; legs ${params.legs}, rest ${params.rest}, ${params.loop ? 'loop (every window wraps)' : 'no loop (windows cut at the ends)'}`);
359
+ say(` TRUNK neck → mid_hip: median ${fmt(c.trunk.length)} px over ${c.trunk.frames} frame(s); a spike is > ${fmt(SPIKE_TRUNKS * c.trunk.length)} px (${SPIKE_TRUNKS} trunk) from its ${SPIKE_WINDOW}-frame median`);
360
+ for (const s of c.spikes) say(` SPIKE joint "${s.joint}" frame ${s.frame} (${tOf(s.frame)}): ${fmt(s.off)} px from its ${SPIKE_WINDOW}-frame median, over ${fmt(SPIKE_TRUNKS * c.trunk.length)} — took the median`);
361
+ say(` SPIKES ${c.spikes.length} replaced; ${untestedText(c.untested)}`);
362
+ const byJoint = new Map<string, typeof c.occluded>();
363
+ for (const o of c.occluded) byJoint.set(o.joint, [...(byJoint.get(o.joint) ?? []), o]);
364
+ for (const [joint, os] of byJoint) say(` OCCLUDED joint "${joint}" on ${os.length} frame(s) (${runsOf(os.map((o) => o.frame))}): not observed or scored under ${OCCLUSION_SCORE}; took its ${OCCLUSION_WINDOW}-frame median, moved up to ${fmt(Math.max(...os.map((o) => o.moved)))} px`);
365
+ say(` OCCLUSION ${c.occluded.length} joint-frame(s) took the median`);
366
+ for (const m of c.missing) say(` MISSING joint "${m.joint}" frame${m.from === m.to ? ` ${m.from}` : `s ${m.from}–${m.to}`}: no position; left without one (retarget holds it)`);
367
+ const sg = (w: number) => (w === 1 ? '1 frame (none)' : `${w} frames`);
368
+ say(` SMOOTH quadratic Savitzky–Golay per joint: legs (${LEGS.join(', ')}) over ${sg(params.legs)}, the rest over ${sg(params.rest)}; ${params.loop ? 'circular' : 'the window cut at the ends'}`);
369
+ if (params.loop) {
370
+ if (c.floor === null) say(' FLOOR no ankle has a position on any frame; not re-pinned');
371
+ else say(` FLOOR the lowest ankle over the cycle: y ${fmt(c.floor.before)} before smoothing, ${fmt(c.floor.after)} after; every joint moved ${c.floor.shift >= 0 ? '+' : ''}${fmt(c.floor.shift, 3)} px in y`);
372
+ }
373
+ const jit = (v: number | null) => (v === null ? 'not measured' : `${fmt(v, 3)}`);
374
+ say(` JITTER mean |second difference| of y over ${c.triples} joint-frame triple(s): ${jit(c.record.jitter.before)} → ${jit(c.record.jitter.after)} px/f²${c.record.jitter.before !== null && c.record.jitter.after !== null && c.record.jitter.before > 0 ? ` (${fmt(100 * (1 - c.record.jitter.after / c.record.jitter.before), 1)}% lower)` : ''}`);
375
+ writeFileSync(out, writePoseSequence(c.seq));
376
+ say(` CLEAN ${cleanLine(c.record)}`);
377
+ say(` wrote ${out}`);
378
+ }
379
+
302
380
  export function main(argv: string[]): number {
303
381
  const [cmd, ...rest] = argv;
304
382
  if (cmd === undefined || cmd === '--help' || cmd === '-h' || cmd === 'help') {
@@ -322,6 +400,10 @@ export function main(argv: string[]): number {
322
400
  runCompare(compareArgs(rest), (l) => process.stdout.write(`${l}\n`));
323
401
  return 0;
324
402
  }
403
+ if (cmd === 'clean') {
404
+ runClean(rest, (l) => process.stdout.write(`${l}\n`));
405
+ return 0;
406
+ }
325
407
  if (cmd === 'convert') {
326
408
  const f = parseFlags(rest, ['coco', 'out'], []);
327
409
  const input = need(f, 'coco');
@@ -331,7 +413,7 @@ export function main(argv: string[]): number {
331
413
  process.stdout.write(`rig-pose convert: ${seq.frames.length} frame(s) of COCO-17 → body-18; wrote ${out}\n`);
332
414
  return 0;
333
415
  }
334
- process.stderr.write(`rig-pose: unknown command "${cmd}" — the commands are: retarget, convert, compare\n`);
416
+ process.stderr.write(`rig-pose: unknown command "${cmd}" — the commands are: clean, retarget, convert, compare\n`);
335
417
  return 2;
336
418
  } catch (e) {
337
419
  if (e instanceof Usage) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@rig-c/pose",
3
- "version": "0.13.0",
3
+ "version": "0.14.0",
4
4
  "description": "Puts motion from a body-18 pose sequence onto a rig built with rig-c: bone keys in rig-c's motion format, checked through rig-c's gate before anything is written. A CLI for agents that cannot see the rig.",
5
5
  "type": "module",
6
6
  "bin": {
package/src/clean.ts ADDED
@@ -0,0 +1,413 @@
1
+ /**
2
+ * `rig-pose clean` (rig-c/pose#67): the sequence-cleaning step every sequence
3
+ * passes through before `retarget`. A pose model reads each frame alone, so a
4
+ * hanging wrist moves ±20 px a frame and a cloak-hidden leg jumps 100+ px; this
5
+ * removes that once, at the sequence, where others can inspect the result.
6
+ *
7
+ * Four steps, in this order, each on one person's joints:
8
+ *
9
+ * 1. **Spikes.** A joint more than `SPIKE_TRUNKS` (0.2) trunk lengths from its
10
+ * `SPIKE_WINDOW`-frame (5) median takes the median. The trunk is neck →
11
+ * mid_hip (the hips' midpoint), the median of its picture length over the
12
+ * frames where all three have a position. Every replacement is listed by
13
+ * frame and joint. Without `loop`, a frame whose window a clip end cuts
14
+ * (the first and last two) is not tested, and is listed as untested: on a
15
+ * cut window the median is a neighbour's value, so a joint moving steadily
16
+ * would read as a spike there. Reflecting the window at the end does not
17
+ * help — the median of [x2, x1, x0, x1, x2] is still x1.
18
+ * 2. **Occlusion.** A joint whose state is not `observed`, or whose score is
19
+ * under `OCCLUSION_SCORE` (0.3), takes its `OCCLUSION_WINDOW`-frame (3)
20
+ * median.
21
+ * 3. **Smoothing.** A quadratic Savitzky–Golay per joint: the legs (`LEGS`:
22
+ * both hips, knees and ankles) over `legs` frames, every other joint over
23
+ * `rest`. A window of 1 is none.
24
+ * 4. **Floor.** With `loop` only: the whole figure is moved in y by one
25
+ * constant, so the lowest ankle over the cycle (the largest y of either
26
+ * ankle on any frame) is at its height before smoothing.
27
+ *
28
+ * What a median is here: per coordinate (x and y apart), over the frames of the
29
+ * window on which the joint has a position — its own frame included, any state.
30
+ * An even count takes the mean of the middle two. The windows are centred; with
31
+ * `loop` they wrap (the sequence is one cycle, its last frame followed by its
32
+ * first), without it they are cut at the ends. The medians read the step's
33
+ * input, never a value the same step already replaced.
34
+ *
35
+ * What is never invented: a joint with no position on a frame (`missing`, or a
36
+ * frame the person is absent from) gets none here. It has nothing to take a
37
+ * median of on its own frame, and a value made from its neighbours would be a
38
+ * guess; `retarget` holds it, and says so with a HOLD line. A joint's state and
39
+ * score are the detector's and are written as they were read: this step moves
40
+ * positions only.
41
+ *
42
+ * The smoothing fits the frames of the window where the joint has a position;
43
+ * where fewer than three are there, the polynomial's degree drops to fit the
44
+ * ones that are (two give a line, one returns the frame's own value).
45
+ *
46
+ * Jitter, the step's figure, is the mean |second difference| of y, in px/f²,
47
+ * over every joint and every three consecutive frames on which it has a
48
+ * position (wrapping with `loop`), before (the sequence as read) and after
49
+ * (the cleaned positions, before the writer rounds them to 0.01 px).
50
+ *
51
+ * Pure: no clock, no randomness, no disk.
52
+ */
53
+ import { BODY18, type Body18 } from './joints.ts';
54
+ import type { CleanRecord, CleanSpike, Joint, PoseSequence } from './pose.ts';
55
+ import { CLEAN_SPEC, personTrack, sortedJoints } from './pose.ts';
56
+ import { Problems } from './problems.ts';
57
+
58
+ /** A spike is a joint this many trunk lengths or more from its median. */
59
+ export const SPIKE_TRUNKS = 0.2;
60
+ /** The spike median's window, in frames. */
61
+ export const SPIKE_WINDOW = 5;
62
+ /** The occlusion median's window, in frames. */
63
+ export const OCCLUSION_WINDOW = 3;
64
+ /** A joint scored under this is treated as occluded. */
65
+ export const OCCLUSION_SCORE = 0.3;
66
+ /** The joints smoothed over `legs`; every other joint is smoothed over `rest`. */
67
+ export const LEGS: readonly Body18[] = ['r_hip', 'r_knee', 'r_ankle', 'l_hip', 'l_knee', 'l_ankle'];
68
+ /** The polynomial's degree. */
69
+ const ORDER = 2;
70
+
71
+ export interface CleanParams {
72
+ /** The sequence is one cycle: every window wraps, and the floor is re-pinned. */
73
+ loop: boolean;
74
+ /** Savitzky–Golay window over the legs, frames: 1 (none), or odd and ≥ 5. */
75
+ legs: number;
76
+ /** Savitzky–Golay window over every other joint, frames: 1 (none), or odd and ≥ 5. */
77
+ rest: number;
78
+ }
79
+
80
+ export type Pt = [number, number];
81
+ /** One joint's positions over the frames; null where it has none. */
82
+ export type Series = Array<Pt | null>;
83
+
84
+ export interface SpikeFound extends CleanSpike {
85
+ /** Its distance from the median, px. */
86
+ off: number;
87
+ }
88
+ export interface Occluded {
89
+ joint: Body18;
90
+ frame: number;
91
+ /** How far the median moved it, px. */
92
+ moved: number;
93
+ why: string;
94
+ }
95
+ export interface Floor {
96
+ /** The lowest ankle's y before smoothing, and after, px. */
97
+ before: number;
98
+ after: number;
99
+ /** The y every joint was moved by, px. */
100
+ shift: number;
101
+ }
102
+
103
+ export interface Cleaned {
104
+ seq: PoseSequence;
105
+ record: CleanRecord;
106
+ person: string;
107
+ /** Neck → mid_hip, the median picture length, px, and over how many frames. */
108
+ trunk: { length: number; frames: number };
109
+ spikes: SpikeFound[];
110
+ occluded: Occluded[];
111
+ /** Joints without a position on some frames, as runs: left without one. */
112
+ missing: Array<{ joint: Body18; from: number; to: number }>;
113
+ /** The frames the spike step did not test (a clip end cuts their window). */
114
+ untested: number[];
115
+ /** With loop, where an ankle exists; else null. */
116
+ floor: Floor | null;
117
+ /** How many (joint, three-frame) runs the jitter was read over: before, after. */
118
+ triples: number;
119
+ }
120
+
121
+ /**
122
+ * The windows `clean` accepts: 1, or odd and ≥ 5. A window of 3 is refused:
123
+ * a quadratic fitted through 3 points passes through all three, so it would
124
+ * change nothing (and at the ends the window cut to 2 frames is a line through
125
+ * both). An even window has no centre frame.
126
+ */
127
+ export function windowProblem(flag: string, n: number): string | null {
128
+ if (!Number.isInteger(n) || n < 1) return `${flag} is "${n}"; it must be 1 (none) or an odd whole number ≥ 5`;
129
+ if (n === 3) return `${flag} is "3"; a quadratic fitted through 3 points passes through all three, so a window of 3 changes nothing — it must be 1 (none) or an odd whole number ≥ 5`;
130
+ if (n % 2 === 0) return `${flag} is "${n}"; an even window has no centre frame — it must be 1 (none) or an odd whole number ≥ 5`;
131
+ return null;
132
+ }
133
+
134
+ /** The frame offsets of a centred window at frame i over n frames: wrapped with loop, cut at the ends without. */
135
+ function windowAt(i: number, n: number, size: number, loop: boolean): number[] {
136
+ const h = size >> 1;
137
+ const out: number[] = [];
138
+ for (let u = -h; u <= h; u++) {
139
+ const j = i + u;
140
+ if (loop) out.push(((j % n) + n) % n);
141
+ else if (j >= 0 && j < n) out.push(j);
142
+ }
143
+ return out;
144
+ }
145
+
146
+ function median1(xs: number[]): number {
147
+ const s = [...xs].sort((a, b) => a - b);
148
+ const m = s.length >> 1;
149
+ return s.length % 2 === 1 ? s[m]! : (s[m - 1]! + s[m]!) / 2;
150
+ }
151
+
152
+ /** The per-coordinate median of a joint's positions over the window around i; null when it has none there. */
153
+ export function medianAt(xs: Series, i: number, size: number, loop: boolean): Pt | null {
154
+ const pts = windowAt(i, xs.length, size, loop).map((j) => xs[j]).filter((p): p is Pt => p !== null && p !== undefined);
155
+ if (pts.length === 0) return null;
156
+ return [median1(pts.map((p) => p[0])), median1(pts.map((p) => p[1]))];
157
+ }
158
+
159
+ /** Whether the spike step tests frame i: under loop always; without it, only where no clip end cuts the window. */
160
+ export function testedAt(i: number, n: number, loop: boolean, size = SPIKE_WINDOW): boolean {
161
+ const h = size >> 1;
162
+ return loop ? n >= size : i - h >= 0 && i + h < n;
163
+ }
164
+
165
+ /**
166
+ * Spikes on one joint: every frame where it lies more than `limit` px from its
167
+ * `size`-frame median (read from `xs`, not from a replaced value). Returns the
168
+ * frames, each with its median and distance; `xs` is not changed.
169
+ */
170
+ export function spikesOf(xs: Series, limit: number, loop: boolean, size = SPIKE_WINDOW): Array<{ frame: number; at: Pt; off: number }> {
171
+ const out: Array<{ frame: number; at: Pt; off: number }> = [];
172
+ xs.forEach((p, i) => {
173
+ if (p === null || !testedAt(i, xs.length, loop, size)) return;
174
+ const m = medianAt(xs, i, size, loop)!;
175
+ const off = Math.hypot(p[0] - m[0], p[1] - m[1]);
176
+ if (off > limit) out.push({ frame: i, at: m, off });
177
+ });
178
+ return out;
179
+ }
180
+
181
+ /**
182
+ * Savitzky–Golay, centred: at each frame with a position, the least-squares
183
+ * polynomial of degree `ORDER` (2) through the window's frames that have one,
184
+ * read at that frame. With `loop` the window wraps; without it, it is cut at
185
+ * the ends. Fewer than three points drop the degree to fit them exactly. A
186
+ * polynomial of degree ≤ 2 passes through unchanged; a window of 1 is the input.
187
+ */
188
+ export function savitzkyGolay(xs: Series, window: number, loop: boolean): Series {
189
+ if (window <= 1) return xs.map((p) => (p === null ? null : [p[0], p[1]]));
190
+ const n = xs.length;
191
+ const h = window >> 1;
192
+ const cache = new Map<string, number[]>();
193
+ return xs.map((p, i) => {
194
+ if (p === null) return null;
195
+ const us: number[] = [];
196
+ const at: Pt[] = [];
197
+ for (let u = -h; u <= h; u++) {
198
+ let j = i + u;
199
+ if (loop) j = ((j % n) + n) % n;
200
+ else if (j < 0 || j >= n) continue;
201
+ const q = xs[j];
202
+ if (q === null || q === undefined) continue;
203
+ us.push(u);
204
+ at.push(q);
205
+ }
206
+ const key = us.join(',');
207
+ let w = cache.get(key);
208
+ if (w === undefined) {
209
+ w = sgWeights(us, Math.min(ORDER, us.length - 1));
210
+ cache.set(key, w);
211
+ }
212
+ let x = 0;
213
+ let y = 0;
214
+ w.forEach((c, k) => {
215
+ x += c * at[k]![0];
216
+ y += c * at[k]![1];
217
+ });
218
+ return [x, y];
219
+ });
220
+ }
221
+
222
+ /** The weights that read a least-squares polynomial of degree q over the offsets `us` at offset 0. */
223
+ function sgWeights(us: readonly number[], q: number): number[] {
224
+ const m = q + 1;
225
+ // Normal equations: A[j][l] = Σ u^(j+l); solve A z = e0, then w(u) = Σ z_j u^j.
226
+ const A = Array.from({ length: m }, (_, j) => Array.from({ length: m + 1 }, (_, l) => {
227
+ if (l === m) return j === 0 ? 1 : 0;
228
+ let s = 0;
229
+ for (const u of us) s += u ** (j + l);
230
+ return s;
231
+ }));
232
+ for (let c = 0; c < m; c++) {
233
+ let piv = c;
234
+ for (let r = c + 1; r < m; r++) if (Math.abs(A[r]![c]!) > Math.abs(A[piv]![c]!)) piv = r;
235
+ [A[c], A[piv]] = [A[piv]!, A[c]!];
236
+ for (let r = 0; r < m; r++) {
237
+ if (r === c) continue;
238
+ const f = A[r]![c]! / A[c]![c]!;
239
+ for (let l = c; l <= m; l++) A[r]![l]! -= f * A[c]![l]!;
240
+ }
241
+ }
242
+ const z = A.map((row, j) => row[m]! / row[j]!);
243
+ return us.map((u) => z.reduce((s, zj, j) => s + zj * u ** j, 0));
244
+ }
245
+
246
+ /** Mean |second difference| of y, px/f², over every series and every three consecutive frames with a position (wrapping with loop). */
247
+ export function jitterOf(all: readonly Series[], loop: boolean): { value: number | null; triples: number } {
248
+ let sum = 0;
249
+ let count = 0;
250
+ for (const xs of all) {
251
+ const n = xs.length;
252
+ for (let i = 0; i < n; i++) {
253
+ if (!loop && (i === 0 || i === n - 1)) continue;
254
+ if (loop && n < 3) continue;
255
+ const a = xs[(i - 1 + n) % n];
256
+ const b = xs[i];
257
+ const c = xs[(i + 1) % n];
258
+ if (a === null || b === null || c === null || a === undefined || b === undefined || c === undefined) continue;
259
+ sum += Math.abs(c[1] - 2 * b[1] + a[1]);
260
+ count++;
261
+ }
262
+ }
263
+ return { value: count === 0 ? null : sum / count, triples: count };
264
+ }
265
+
266
+ /**
267
+ * Clean one person's track. Refuses (every reason at once) a window that is
268
+ * not 1 or odd ≥ 5, a loop shorter than the widest window it reads, and a
269
+ * sequence whose trunk is measured on no frame.
270
+ */
271
+ export function clean(seq: PoseSequence, params: CleanParams, wanted: string | undefined, where: string): Cleaned {
272
+ const p = new Problems();
273
+ for (const [flag, v] of [['--legs', params.legs], ['--rest', params.rest]] as const) {
274
+ const why = windowProblem(flag, v);
275
+ if (why !== null) p.add('CLEAN_WINDOW', why);
276
+ }
277
+ const track = personTrack(seq, wanted, where);
278
+ const n = track.length;
279
+ const person = wanted ?? [...new Set(seq.frames.flatMap((f) => f.people.map((q) => q.id)))].sort()[0]!;
280
+ if (params.loop) {
281
+ const widest = Math.max(SPIKE_WINDOW, params.legs, params.rest);
282
+ if (n < widest) p.add('CLEAN_LOOP', `--loop over ${n} frame(s) of ${where}: a circular window of ${widest} frames would read a frame twice; a cycle needs ≥ ${widest} frames (the spike median reads ${SPIKE_WINDOW}, --legs ${params.legs}, --rest ${params.rest})`);
283
+ }
284
+ const at = (j: Joint | undefined): Pt | null => (j === undefined || j.state === 'missing' || j.at === undefined ? null : [j.at[0], j.at[1]]);
285
+ const names = BODY18.filter((name) => track.some((f) => f[name] !== undefined));
286
+ const raw = new Map<Body18, Series>(names.map((name) => [name, track.map((f) => at(f[name]))]));
287
+
288
+ // The trunk: neck → mid_hip, its median picture length.
289
+ const lengths: number[] = [];
290
+ for (let i = 0; i < n; i++) {
291
+ const neck = raw.get('neck')?.[i] ?? null;
292
+ const r = raw.get('r_hip')?.[i] ?? null;
293
+ const l = raw.get('l_hip')?.[i] ?? null;
294
+ if (neck === null || r === null || l === null) continue;
295
+ lengths.push(Math.hypot(neck[0] - (r[0] + l[0]) / 2, neck[1] - (r[1] + l[1]) / 2));
296
+ }
297
+ if (lengths.length === 0) p.add('CLEAN_TRUNK', `${where} person "${person}": neck, r_hip and l_hip have a position together on none of its ${n} frame(s); the spike limit is ${SPIKE_TRUNKS} trunk lengths (neck → mid_hip), so there is no limit to read`);
298
+ p.throwIfAny();
299
+ const trunk = median1(lengths);
300
+ const limit = SPIKE_TRUNKS * trunk;
301
+
302
+ // 1. Spikes.
303
+ const spikes: SpikeFound[] = [];
304
+ const despiked = new Map<Body18, Series>();
305
+ for (const name of names) {
306
+ const xs = raw.get(name)!;
307
+ const out = [...xs];
308
+ for (const s of spikesOf(xs, limit, params.loop)) {
309
+ out[s.frame] = s.at;
310
+ spikes.push({ frame: s.frame, joint: name, off: s.off });
311
+ }
312
+ despiked.set(name, out);
313
+ }
314
+ spikes.sort((a, b) => a.frame - b.frame || BODY18.indexOf(a.joint) - BODY18.indexOf(b.joint));
315
+
316
+ // 2. Occlusion.
317
+ const occluded: Occluded[] = [];
318
+ const unhidden = new Map<Body18, Series>();
319
+ for (const name of names) {
320
+ const xs = despiked.get(name)!;
321
+ const out = [...xs];
322
+ xs.forEach((q, i) => {
323
+ if (q === null) return;
324
+ const j = track[i]![name]!;
325
+ const low = j.score !== undefined && j.score < OCCLUSION_SCORE;
326
+ if (j.state === 'observed' && !low) return;
327
+ const m = medianAt(xs, i, OCCLUSION_WINDOW, params.loop)!;
328
+ out[i] = m;
329
+ occluded.push({ joint: name, frame: i, moved: Math.hypot(m[0] - q[0], m[1] - q[1]), why: j.state !== 'observed' ? j.state : `score ${j.score!.toFixed(3)} < ${OCCLUSION_SCORE}` });
330
+ });
331
+ unhidden.set(name, out);
332
+ }
333
+
334
+ // 3. Smoothing.
335
+ const smoothed = new Map<Body18, Series>();
336
+ for (const name of names) smoothed.set(name, savitzkyGolay(unhidden.get(name)!, LEGS.includes(name) ? params.legs : params.rest, params.loop));
337
+
338
+ // 4. Floor (loop only).
339
+ let floor: Floor | null = null;
340
+ if (params.loop) {
341
+ const lowest = (m: Map<Body18, Series>): number | null => {
342
+ const ys = (['r_ankle', 'l_ankle'] as const).flatMap((a) => (m.get(a) ?? []).filter((q): q is Pt => q !== null).map((q) => q[1]));
343
+ return ys.length === 0 ? null : Math.max(...ys);
344
+ };
345
+ const before = lowest(unhidden);
346
+ const after = lowest(smoothed);
347
+ if (before !== null && after !== null) {
348
+ const shift = before - after;
349
+ floor = { before, after, shift };
350
+ for (const name of names) smoothed.set(name, smoothed.get(name)!.map((q) => (q === null ? null : [q[0], q[1] + shift])));
351
+ }
352
+ }
353
+
354
+ const missing: Cleaned['missing'] = [];
355
+ for (const name of names) {
356
+ raw.get(name)!.forEach((q, i) => {
357
+ if (q !== null) return;
358
+ const last = missing[missing.length - 1];
359
+ if (last !== undefined && last.joint === name && last.to === i - 1) last.to = i;
360
+ else missing.push({ joint: name, from: i, to: i });
361
+ });
362
+ }
363
+
364
+ const untested = Array.from({ length: n }, (_, i) => i).filter((i) => !testedAt(i, n, params.loop));
365
+ const before = jitterOf([...raw.values()], params.loop);
366
+ const after = jitterOf([...smoothed.values()], params.loop);
367
+ const record: CleanRecord = {
368
+ spec: CLEAN_SPEC,
369
+ params: { loop: params.loop, legs: params.legs, rest: params.rest },
370
+ spikes: { found: spikes.map((s) => ({ frame: s.frame, joint: s.joint })), untested },
371
+ jitter: { before: before.value, after: after.value },
372
+ };
373
+ const frames = seq.frames.map((f, i) => {
374
+ const q = f.people.find((x) => x.id === person);
375
+ if (q === undefined) return { t: f.t, people: [] };
376
+ const joints: Partial<Record<Body18, Joint>> = {};
377
+ for (const [name, j] of Object.entries(q.joints) as Array<[Body18, Joint]>) {
378
+ const moved = smoothed.get(name)?.[i] ?? null;
379
+ joints[name] = moved === null ? { ...j } : { ...j, at: [moved[0], moved[1]] };
380
+ }
381
+ return { t: f.t, people: [{ id: person, joints: sortedJoints(joints) }] };
382
+ });
383
+ const { cleaned: _was, ...rest } = seq;
384
+ return {
385
+ seq: { ...rest, cleaned: record, frames },
386
+ record,
387
+ person,
388
+ trunk: { length: trunk, frames: lengths.length },
389
+ spikes,
390
+ occluded,
391
+ missing,
392
+ untested,
393
+ floor,
394
+ triples: before.triples,
395
+ };
396
+ }
397
+
398
+ /** The untested frames as text: "frames 0, 1, 30, 31 untested (a clip end cuts their window)", or "every frame tested". */
399
+ export function untestedText(frames: readonly number[]): string {
400
+ if (frames.length === 0) return 'every frame tested';
401
+ return `frame${frames.length === 1 ? '' : 's'} ${frames.join(', ')} untested (a clip end cuts the ${SPIKE_WINDOW}-frame window)`;
402
+ }
403
+
404
+ /** The record as one line: retarget's CLEAN line, and clean's own summary. */
405
+ export function cleanLine(r: CleanRecord): string {
406
+ const { loop, legs, rest } = r.params;
407
+ const windows = legs === 1 && rest === 1 ? 'legs 1, rest 1 — no smoothing' : `legs ${legs}, rest ${rest}`;
408
+ const found = r.spikes.found;
409
+ const shown = found.slice(0, 8).map((s) => `frame ${s.frame} ${s.joint}`).join(', ');
410
+ const spikes = `${found.length === 0 ? 'no spike' : `${found.length} spike(s) replaced (${shown}${found.length > 8 ? `, … ${found.length - 8} more` : ''})`}, ${untestedText(r.spikes.untested)}`;
411
+ const jit = (v: number | null) => (v === null ? 'not measured' : v.toFixed(2));
412
+ return `${r.spec}: ${windows}, ${loop ? 'loop' : 'no loop'}; ${spikes}; jitter (mean |second difference| of y) ${jit(r.jitter.before)} → ${jit(r.jitter.after)} px/f²`;
413
+ }
package/src/pose.ts CHANGED
@@ -52,6 +52,28 @@ export interface CameraFrame {
52
52
  ty: number;
53
53
  inliers: number;
54
54
  }
55
+ /**
56
+ * The record `rig-pose clean` writes into the sequence it cleaned (#67), and
57
+ * the one thing `retarget` requires of a sequence: the step is the record, not
58
+ * a fixed amount of smoothing (windows of 1 are a cleaned sequence too).
59
+ */
60
+ export const CLEAN_SPEC = 'rig-pose-clean/1';
61
+ export interface CleanSpike {
62
+ frame: number;
63
+ joint: Body18;
64
+ }
65
+ export interface CleanRecord {
66
+ spec: typeof CLEAN_SPEC;
67
+ params: { loop: boolean; legs: number; rest: number };
68
+ /**
69
+ * `found`: every joint the spike step replaced, by frame (index into `frames`)
70
+ * and joint. `untested`: the frames the spike step did not test, because a
71
+ * clip end cuts their window (none under `loop`).
72
+ */
73
+ spikes: { found: CleanSpike[]; untested: number[] };
74
+ /** Mean |second difference| of y, px/f², before and after; null where no joint has three consecutive positions. */
75
+ jitter: { before: number | null; after: number | null };
76
+ }
55
77
  export interface PoseSequence {
56
78
  spec: typeof POSE_SPEC;
57
79
  space: typeof POSE_SPACE;
@@ -62,11 +84,13 @@ export interface PoseSequence {
62
84
  cameraSource?: string;
63
85
  /** Optional: one per frame, when a scan read the camera (#63). */
64
86
  camera?: CameraFrame[];
87
+ /** Present once `rig-pose clean` has run on the sequence (#67). */
88
+ cleaned?: CleanRecord;
65
89
  frames: PoseFrame[];
66
90
  }
67
91
 
68
92
  const TOP_KEYS = ['spec', 'space', 'width', 'height', 'source', 'frames'];
69
- const OPTIONAL_KEYS = ['cameraSource', 'camera'];
93
+ const OPTIONAL_KEYS = ['cameraSource', 'camera', 'cleaned'];
70
94
  const CAMERA_KEYS = ['s', 'tx', 'ty', 'inliers'];
71
95
 
72
96
  /**
@@ -151,9 +175,10 @@ export function readPoseSequence(text: string, where: string): PoseSequence {
151
175
  if (typeof doc.cameraSource !== 'string' || doc.cameraSource.trim() === '') p.add('POSE_CAMERA_SOURCE', `${where} "cameraSource" is ${typeName(doc.cameraSource)}; it must be a non-empty string saying where the camera came from ("tracked", "hand-labelled, N keys")`);
152
176
  else if (!('camera' in doc)) p.add('POSE_CAMERA_SOURCE', `${where} has "cameraSource" ${JSON.stringify(doc.cameraSource)} and no "camera"; it states where a camera came from, so it needs one`);
153
177
  }
178
+ const cleaned = 'cleaned' in doc ? readCleaned(p, where, doc.cleaned, Array.isArray(doc.frames) ? doc.frames.length : null) : undefined;
154
179
  p.throwIfAny();
155
180
  const cameraSource = typeof doc.cameraSource === 'string' ? doc.cameraSource : undefined;
156
- return { spec: POSE_SPEC, space: POSE_SPACE, width: doc.width as number, height: doc.height as number, source: doc.source as string, ...(cameraSource !== undefined ? { cameraSource } : {}), ...(camera !== undefined ? { camera } : {}), frames };
181
+ return { spec: POSE_SPEC, space: POSE_SPACE, width: doc.width as number, height: doc.height as number, source: doc.source as string, ...(cameraSource !== undefined ? { cameraSource } : {}), ...(camera !== undefined ? { camera } : {}), ...(cleaned !== undefined ? { cleaned } : {}), frames };
157
182
  }
158
183
 
159
184
  /** `camera`: one `{ s > 0, tx, ty, inliers ≥ 0 }` per frame, as many as `frames`. */
@@ -194,6 +219,77 @@ function readCamera(p: Problems, where: string, camera: unknown, frames: number
194
219
  return out;
195
220
  }
196
221
 
222
+ /** `cleaned`: { spec, params: { loop, legs, rest }, spikes: { found: [{ frame, joint }], untested: [frame] }, jitter: { before, after } }. */
223
+ function readCleaned(p: Problems, where: string, c: unknown, frames: number | null): CleanRecord | undefined {
224
+ const cw = `${where} "cleaned"`;
225
+ const shape = '{ spec, params: { loop, legs, rest }, spikes: { found: [{ frame, joint }], untested: [frame] }, jitter: { before, after } }';
226
+ if (!isObject(c)) {
227
+ p.add('POSE_CLEANED', `${cw} is ${typeName(c)}; it is the record rig-pose clean writes, ${shape}`);
228
+ return undefined;
229
+ }
230
+ const before = p.list.length;
231
+ refuseUnknownKeys(p, 'POSE_CLEANED', cw, c, ['spec', 'params', 'spikes', 'jitter']);
232
+ for (const k of ['spec', 'params', 'spikes', 'jitter']) if (!(k in c)) p.add('POSE_CLEANED', `${cw} has no "${k}"; the record is ${shape}`);
233
+ if ('spec' in c && c.spec !== CLEAN_SPEC) p.add('POSE_CLEANED', `${cw} "spec" is ${typeName(c.spec)}; this reader reads "${CLEAN_SPEC}"`);
234
+ const window = (v: unknown) => Number.isInteger(v) && (v === 1 || ((v as number) >= 5 && (v as number) % 2 === 1));
235
+ if ('params' in c) {
236
+ const q = c.params;
237
+ if (!isObject(q)) p.add('POSE_CLEANED', `${cw} "params" is ${typeName(q)}; it must be { loop, legs, rest }`);
238
+ else {
239
+ refuseUnknownKeys(p, 'POSE_CLEANED', `${cw} "params"`, q, ['loop', 'legs', 'rest']);
240
+ if (typeof q.loop !== 'boolean') p.add('POSE_CLEANED', `${cw} "params" "loop" is ${typeName(q.loop)}; it must be true or false`);
241
+ for (const k of ['legs', 'rest'] as const) if (!window(q[k])) p.add('POSE_CLEANED', `${cw} "params" "${k}" is ${typeName(q[k])}; it must be 1 (none) or an odd whole number ≥ 5`);
242
+ }
243
+ }
244
+ const frameOk = (v: unknown) => Number.isInteger(v) && (v as number) >= 0 && (frames === null || (v as number) < frames);
245
+ const frameRange = `[0, ${frames === null ? 'the frame count' : frames - 1}]`;
246
+ if ('spikes' in c) {
247
+ const k = c.spikes;
248
+ if (!isObject(k)) p.add('POSE_CLEANED', `${cw} "spikes" is ${typeName(k)}; it must be { found: [{ frame, joint }], untested: [frame] }`);
249
+ else {
250
+ refuseUnknownKeys(p, 'POSE_CLEANED', `${cw} "spikes"`, k, ['found', 'untested']);
251
+ if (!Array.isArray(k.found)) p.add('POSE_CLEANED', `${cw} "spikes" "found" is ${typeName(k.found)}; it must be an array of { frame, joint }`);
252
+ else k.found.forEach((s: unknown, i: number) => {
253
+ const sw = `${cw} spikes.found[${i}]`;
254
+ if (!isObject(s)) {
255
+ p.add('POSE_CLEANED', `${sw} is ${typeName(s)}; a spike is { frame, joint }`);
256
+ return;
257
+ }
258
+ refuseUnknownKeys(p, 'POSE_CLEANED', sw, s, ['frame', 'joint']);
259
+ if (!frameOk(s.frame)) p.add('POSE_CLEANED', `${sw} "frame" is ${typeName(s.frame)}; it must be a frame index in ${frameRange}`);
260
+ if (typeof s.joint !== 'string' || !isBody18(s.joint)) p.add('POSE_CLEANED', `${sw} "joint" is ${typeName(s.joint)}; it must be a body-18 joint: ${BODY18.join(', ')}`);
261
+ });
262
+ if (!Array.isArray(k.untested)) p.add('POSE_CLEANED', `${cw} "spikes" "untested" is ${typeName(k.untested)}; it must be an array of frame indices`);
263
+ else k.untested.forEach((f: unknown, i: number) => {
264
+ if (!frameOk(f)) p.add('POSE_CLEANED', `${cw} spikes.untested[${i}] is ${typeName(f)}; it must be a frame index in ${frameRange}`);
265
+ });
266
+ }
267
+ }
268
+ if ('jitter' in c) {
269
+ const j = c.jitter;
270
+ if (!isObject(j)) p.add('POSE_CLEANED', `${cw} "jitter" is ${typeName(j)}; it must be { before, after }`);
271
+ else {
272
+ refuseUnknownKeys(p, 'POSE_CLEANED', `${cw} "jitter"`, j, ['before', 'after']);
273
+ for (const k of ['before', 'after'] as const) {
274
+ const v = j[k];
275
+ if (!(v === null || (typeof v === 'number' && Number.isFinite(v) && v >= 0))) p.add('POSE_CLEANED', `${cw} "jitter" "${k}" is ${typeName(v)}; it must be px/f² ≥ 0, or null (not measured)`);
276
+ }
277
+ }
278
+ }
279
+ if (p.list.length > before) return undefined;
280
+ const q = c.params as { loop: boolean; legs: number; rest: number };
281
+ const j = c.jitter as { before: number | null; after: number | null };
282
+ return {
283
+ spec: CLEAN_SPEC,
284
+ params: { loop: q.loop, legs: q.legs, rest: q.rest },
285
+ spikes: {
286
+ found: ((c.spikes as { found: Array<{ frame: number; joint: Body18 }> }).found).map((s) => ({ frame: s.frame, joint: s.joint })),
287
+ untested: [...(c.spikes as { untested: number[] }).untested],
288
+ },
289
+ jitter: { before: j.before, after: j.after },
290
+ };
291
+ }
292
+
197
293
  function readJoints(p: Problems, pw: string, joints: unknown): Partial<Record<Body18, Joint>> {
198
294
  const out: Partial<Record<Body18, Joint>> = {};
199
295
  if (!isObject(joints)) {
@@ -376,5 +472,8 @@ export function writePoseSequence(seq: PoseSequence): string {
376
472
  })),
377
473
  }));
378
474
  const camera = seq.camera?.map((c) => ({ s: round(c.s, 6), tx: round(c.tx, 3), ty: round(c.ty, 3), inliers: c.inliers }));
379
- return `${JSON.stringify({ spec: seq.spec, space: seq.space, width: seq.width, height: seq.height, source: seq.source, ...(seq.cameraSource !== undefined ? { cameraSource: seq.cameraSource } : {}), ...(camera !== undefined ? { camera } : {}), frames }, null, 2)}\n`;
475
+ const c = seq.cleaned;
476
+ const jit = (v: number | null) => (v === null ? null : round(v, 4));
477
+ const cleaned = c === undefined ? undefined : { spec: c.spec, params: { loop: c.params.loop, legs: c.params.legs, rest: c.params.rest }, spikes: { found: c.spikes.found.map((s) => ({ frame: s.frame, joint: s.joint })), untested: [...c.spikes.untested] }, jitter: { before: jit(c.jitter.before), after: jit(c.jitter.after) } };
478
+ return `${JSON.stringify({ spec: seq.spec, space: seq.space, width: seq.width, height: seq.height, source: seq.source, ...(seq.cameraSource !== undefined ? { cameraSource: seq.cameraSource } : {}), ...(camera !== undefined ? { camera } : {}), ...(cleaned !== undefined ? { cleaned } : {}), frames }, null, 2)}\n`;
380
479
  }
package/src/retarget.ts CHANGED
@@ -19,9 +19,8 @@
19
19
  * pose puts the segment's end, relative to the translated joint and scaled. So
20
20
  * the hip and its travel are computed before any bone's target.
21
21
  *
22
- * Smoothing is Savitzky–Golay: a centred least-squares cubic over the window,
23
- * evaluated at the frame, on each joint's positions after its holds. Centred,
24
- * it moves no motion in time; a cubic keeps a peak a moving average flattens.
22
+ * Smoothing is not done here: the sequence comes cleaned (`clean.ts`, #67 —
23
+ * spikes, occlusion, a per-joint Savitzky–Golay), and its record says how.
25
24
  *
26
25
  * The short hold (rig-c/pose#22) comes next: a segment whose picture length on
27
26
  * a frame is under `short` of its own median length over the sequence is seen
@@ -35,7 +34,7 @@
35
34
  * The flip guard (rig-c/pose#15) then spreads a segment's turn greater than
36
35
  * `flip` degrees in one frame over the frames around it, linearly and centred on
37
36
  * the jump (`spreadFlips`). It reads the mapped segments' target angles after
38
- * holds, smoothing and the short hold, and before the hinge check. A reach's
37
+ * holds and the short hold, and before the hinge check. A reach's
39
38
  * target is aimed later, from posed positions, and is neither held short nor
40
39
  * guarded.
41
40
  *
@@ -52,7 +51,7 @@
52
51
  * against it (`artTurn`, and `artRange` where it leaves its range).
53
52
  *
54
53
  * The lines across the body — hips, shoulders, eyes — are read from the pose
55
- * after holds and smoothing, whether the map uses them or not: a line under
54
+ * after holds, whether the map uses them or not: a line under
56
55
  * `TURN_FRACTION` of its median picture length, or reversed (pointing more
57
56
  * than 90° from the subject's right to left), for ≥ `TURN_SECONDS`, is a `TURN`:
58
57
  * the body or head turned, a pose a flat rig cannot draw. It changes no key.
@@ -77,8 +76,6 @@ const DEG = 180 / Math.PI;
77
76
  export interface RetargetOptions extends SwitchOptions {
78
77
  /** A joint with a score below this is not good on that frame (a joint with no score is). */
79
78
  minScore: number;
80
- /** Savitzky–Golay window over joint positions, in frames: 1 = none, else odd and ≥ 5. */
81
- smooth: number;
82
79
  /** What a hinge bent the wrong way does: refuse the run, or straighten that hinge on that frame. */
83
80
  bends: 'refuse' | 'clamp';
84
81
  /** Degrees past straight a hinge may bend the wrong way before it counts. */
@@ -331,64 +328,6 @@ function median(xs: number[]): number {
331
328
  return s.length % 2 === 1 ? s[m]! : (s[m - 1]! + s[m]!) / 2;
332
329
  }
333
330
 
334
- /**
335
- * Savitzky–Golay, centred: at each frame, the least-squares polynomial of
336
- * `order` over the frames of the window that exist (clipped at the ends), read
337
- * at that frame. With fewer points than order + 1 the order drops to fit them
338
- * exactly, so a window clipped to 4 frames or fewer returns the input there.
339
- * A polynomial of degree ≤ `order` passes through unchanged.
340
- */
341
- export function savitzkyGolay(xs: readonly Pt[], window: number, order = 3): Pt[] {
342
- if (window <= 1) return xs.map((x) => [x[0], x[1]]);
343
- const h = window >> 1;
344
- const n = xs.length;
345
- const cache = new Map<string, number[]>();
346
- return xs.map((_, i) => {
347
- const lo = Math.max(0, i - h) - i;
348
- const hi = Math.min(n - 1, i + h) - i;
349
- const key = `${lo},${hi}`;
350
- let w = cache.get(key);
351
- if (w === undefined) {
352
- w = sgWeights(lo, hi, Math.min(order, hi - lo));
353
- cache.set(key, w);
354
- }
355
- let x = 0;
356
- let y = 0;
357
- for (let u = lo; u <= hi; u++) {
358
- const c = w[u - lo]!;
359
- x += c * xs[i + u]![0];
360
- y += c * xs[i + u]![1];
361
- }
362
- return [x, y];
363
- });
364
- }
365
-
366
- /** The weights that read a least-squares polynomial of degree q over u = lo..hi at u = 0. */
367
- function sgWeights(lo: number, hi: number, q: number): number[] {
368
- const m = q + 1;
369
- // Normal equations: A[j][l] = Σ u^(j+l); solve A z = e0, then w(u) = Σ z_j u^j.
370
- const A = Array.from({ length: m }, (_, j) => Array.from({ length: m + 1 }, (_, l) => {
371
- if (l === m) return j === 0 ? 1 : 0;
372
- let s = 0;
373
- for (let u = lo; u <= hi; u++) s += u ** (j + l);
374
- return s;
375
- }));
376
- for (let c = 0; c < m; c++) {
377
- let piv = c;
378
- for (let r = c + 1; r < m; r++) if (Math.abs(A[r]![c]!) > Math.abs(A[piv]![c]!)) piv = r;
379
- [A[c], A[piv]] = [A[piv]!, A[c]!];
380
- for (let r = 0; r < m; r++) {
381
- if (r === c) continue;
382
- const f = A[r]![c]! / A[c]![c]!;
383
- for (let l = c; l <= m; l++) A[r]![l]! -= f * A[c]![l]!;
384
- }
385
- }
386
- const z = A.map((row, j) => row[m]! / row[j]!);
387
- const w: number[] = [];
388
- for (let u = lo; u <= hi; u++) w.push(z.reduce((s, zj, j) => s + zj * u ** j, 0));
389
- return w;
390
- }
391
-
392
331
  const r3 = (v: number) => {
393
332
  const r = Math.round(v * 1000) / 1000;
394
333
  return Object.is(r, -0) ? 0 : r;
@@ -439,7 +378,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
439
378
  const series = new Map<string, Pt[]>();
440
379
  const good = new Map<string, boolean[]>();
441
380
  const holds: Hold[] = [];
442
- /** A point held where it is not good, then smoothed; or why it is good on no frame. */
381
+ /** A point held where it is not good; or why it is good on no frame. */
443
382
  const heldOf = (name: string): { at: Pt[]; good: boolean[]; holds: Hold[] } | { never: string } => {
444
383
  const raw = frames.map((f) => pointOn(f, name, opts.minScore));
445
384
  const firstGood = raw.findIndex((v) => typeof v !== 'string');
@@ -463,7 +402,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
463
402
  spans.push(span);
464
403
  }
465
404
  }
466
- return { at: savitzkyGolay(out, opts.smooth), good: raw.map((v) => typeof v !== 'string'), holds: spans };
405
+ return { at: out, good: raw.map((v) => typeof v !== 'string'), holds: spans };
467
406
  };
468
407
  for (const name of [...needed].sort()) {
469
408
  const h = heldOf(name);
@@ -477,7 +416,7 @@ export function retarget(rest: Rest, map: JointMap, frames: ReadonlyArray<Partia
477
416
  }
478
417
  p.throwIfAny();
479
418
 
480
- // TURN: the lines across the body, read from the pose (held and smoothed as the
419
+ // TURN: the lines across the body, read from the pose (held as the
481
420
  // map's points are, whether or not the map uses them). A reading; no key moves.
482
421
  const turns: Turn[] = [];
483
422
  const turnsUnread: Retarget['turnsUnread'] = [];
package/src/run.ts CHANGED
@@ -8,6 +8,7 @@ import { cpSync, existsSync, mkdtempSync, readFileSync, rmSync, statSync, writeF
8
8
  import { tmpdir } from 'node:os';
9
9
  import { dirname, join, resolve } from 'node:path';
10
10
  import { readMap, resolveMap, type JointMap, type Rest } from './map.ts';
11
+ import { cleanLine } from './clean.ts';
11
12
  import { convertCoco17, personTrack, readPoseSequence, type CameraFrame } from './pose.ts';
12
13
  import { isObject, lineOf, parseJson, Problems, Refusal, type Problem } from './problems.ts';
13
14
  import { boneIdentity, droppedLine, keptOutLine, mergeProblems, readFragment, readIdle, targetOf, trackLabel, type IdleReading, type RigcTrack, type Target } from './idle.ts';
@@ -26,6 +27,8 @@ export interface RetargetArgs extends RetargetOptions {
26
27
  out: string;
27
28
  name: string;
28
29
  coco: boolean;
30
+ /** `--smooth`'s value when given: an alias since #67 — the cleaning carries the smoothing; it prints so and changes nothing. */
31
+ smoothAlias?: string;
29
32
  person?: string;
30
33
  baseMotion?: string;
31
34
  /** Passed to rigc build as --images: the directory the rig spec's images resolve in, when rig.json's own `images` does not point there (a copied rig). */
@@ -87,6 +90,7 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
87
90
  const map = readMap(readFileSync(a.map, 'utf8'), a.map);
88
91
  const poseText = readFileSync(a.pose, 'utf8');
89
92
  const seq = a.coco ? convertCoco17(poseText, a.pose) : readPoseSequence(poseText, a.pose);
93
+ if (seq.cleaned === undefined) refuse([{ code: 'POSE_NOT_CLEANED', text: notCleaned(a.pose, a.coco) }]);
90
94
  const frames = personTrack(seq, a.person, a.pose);
91
95
  const times = seq.frames.map((f) => f.t);
92
96
 
@@ -104,8 +108,10 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
104
108
  resolveMap(map, rest, a.map, slots);
105
109
 
106
110
  const r = retarget(rest, map, frames, times, a, slots);
107
- say(`rig-pose retarget: ${seq.frames.length} frame(s), ${r.animation.duration.toFixed(3)}s, onto "${input.name}" as "${a.name}" — ${r.animation.tracks.length} track(s); min-score ${a.minScore}, smooth ${a.smooth}, flip ${a.flip === 'off' ? 'off' : `${a.flip}°`}, short ${a.short}, bends ${a.bends} (tolerance ${a.bendTolerance}°)${a.legScale !== undefined && a.legScale !== 'off' ? `, leg-scale ${a.legScale[0]}–${a.legScale[1]}` : ''}${a.plantFeet === true ? ', plant-feet' : ''}${a.stance !== undefined && a.stance !== 'off' ? `, stance ${a.stance}` : ''}${a.carryIdle === true ? ', carry-idle' : ''}${a.merge !== undefined ? `, merge ${a.merge}` : ''}${a.reduce !== undefined ? `, reduce ${a.reduce}°${a.reduceHip !== undefined ? ` (hip ${a.reduceHip} units)` : ''}` : ''}`);
111
+ say(`rig-pose retarget: ${seq.frames.length} frame(s), ${r.animation.duration.toFixed(3)}s, onto "${input.name}" as "${a.name}" — ${r.animation.tracks.length} track(s); min-score ${a.minScore}, flip ${a.flip === 'off' ? 'off' : `${a.flip}°`}, short ${a.short}, bends ${a.bends} (tolerance ${a.bendTolerance}°)${a.legScale !== undefined && a.legScale !== 'off' ? `, leg-scale ${a.legScale[0]}–${a.legScale[1]}` : ''}${a.plantFeet === true ? ', plant-feet' : ''}${a.stance !== undefined && a.stance !== 'off' ? `, stance ${a.stance}` : ''}${a.carryIdle === true ? ', carry-idle' : ''}${a.merge !== undefined ? `, merge ${a.merge}` : ''}${a.reduce !== undefined ? `, reduce ${a.reduce}°${a.reduceHip !== undefined ? ` (hip ${a.reduceHip} units)` : ''}` : ''}`);
108
112
  say(` RIGC rig-c ${rigc.version} (${rigc.entry}) gates and poses this run`);
113
+ say(` CLEAN ${cleanLine(seq.cleaned!)}`);
114
+ if (a.smoothAlias !== undefined) say(` SMOOTH --smooth is carried by clean; ignored (given ${a.smoothAlias}; this sequence was cleaned with legs ${seq.cleaned!.params.legs}, rest ${seq.cleaned!.params.rest})`);
109
115
  if (seq.camera !== undefined) say(` CAMERA ${cameraLine(seq.camera, times, seq.cameraSource)}`);
110
116
  else if (a.plantFeet === true) say(' CAMERA none in the sequence: --plant-feet reads the picture as a fixed camera, so a zoom or a pan in the video moves the floor with it and the floor may step with the picture (tools/scan.py reads the camera, rig-c/pose#63)');
111
117
  if (r.scale !== null) say(` SCALE ${r.scale.factor.toFixed(5)} rig units per picture px (rig ${r.scale.rig.toFixed(2)} / pose median ${r.scale.pose.toFixed(2)})`);
@@ -266,7 +272,8 @@ export function runRetarget(a: RetargetArgs, log: (line: string) => void): RunRe
266
272
  animation: a.name,
267
273
  frames: seq.frames.length,
268
274
  duration: r.animation.duration,
269
- options: { minScore: a.minScore, smooth: a.smooth, flip: a.flip, short: a.short, bends: a.bends, bendTolerance: a.bendTolerance, switchTolerance: a.switchTolerance, switchMin: a.switchMin, switchAt: a.switchAt, profile: a.profile, legScale: a.legScale ?? 'off', plantFeet: a.plantFeet === true, stance: a.stance ?? 'off', idle: idleName, carryIdle: a.carryIdle === true, merge: a.merge ?? null, reduce: a.reduce ?? 'off', reduceHip: a.reduceHip ?? 'off' },
275
+ cleaned: seq.cleaned,
276
+ options: { minScore: a.minScore, flip: a.flip, short: a.short, bends: a.bends, bendTolerance: a.bendTolerance, switchTolerance: a.switchTolerance, switchMin: a.switchMin, switchAt: a.switchAt, profile: a.profile, legScale: a.legScale ?? 'off', plantFeet: a.plantFeet === true, stance: a.stance ?? 'off', idle: idleName, carryIdle: a.carryIdle === true, merge: a.merge ?? null, reduce: a.reduce ?? 'off', reduceHip: a.reduceHip ?? 'off' },
270
277
  scale: r.scale,
271
278
  holds: r.holds,
272
279
  short: r.short,
@@ -436,3 +443,14 @@ function legScaleLine(l: LegScale, times: readonly number[]): string {
436
443
  const clamp = (rs: Array<[number, number]>, at: number, how: string) => (rs.length === 0 ? `none ${how} ${at}` : `${count(rs)} frame(s) ${how} ${at}, clamped (frames ${rs.map(span).join(', ')})`);
437
444
  return `chain "${l.chain}" bone "${l.bone}" (${l.joints[0]} → ${l.joints[1]}): straight length ${l.straight.toFixed(1)}px (95th percentile of ${times.length} frames); picture length ÷ straight ${l.ratio.min.toFixed(3)} … ${l.ratio.max.toFixed(3)} (median ${l.ratio.median.toFixed(3)}); scalex keyed in [${l.limits[0]}, ${l.limits[1]}]: ${clamp(l.low, l.limits[0], 'under')}; ${clamp(l.high, l.limits[1], 'over')}`;
438
445
  }
446
+
447
+ /**
448
+ * POSE_NOT_CLEANED (#67): retarget reads only a sequence `rig-pose clean` wrote.
449
+ * The message is the command to run on this file; the output name is a
450
+ * suggestion beside it (`<name>.clean.json`).
451
+ */
452
+ export function notCleaned(pose: string, coco: boolean): string {
453
+ const out = pose.endsWith('.json') ? `${pose.slice(0, -'.json'.length)}.clean.json` : `${pose}.clean.json`;
454
+ const what = coco ? `${pose} is COCO-17 (--coco), which carries no "cleaned" record` : `${pose} has no "cleaned" record`;
455
+ return `${what}; retarget reads a sequence rig-pose clean wrote (spikes, occlusion, smoothing; there is no flag to skip it). Run \`rig-pose clean ${coco ? '--coco ' : ''}--pose ${pose} --out ${out}\` (add --loop for one cycle of a gait) and retarget --pose ${out}`;
456
+ }
package/src/switch.ts CHANGED
@@ -4,7 +4,7 @@
4
4
  *
5
5
  * The metric, per frame and per set: the largest |wrap(pose − key)| over every
6
6
  * segment of the set's chains, where "pose" is the segment's angle exactly as
7
- * the retarget asked the bone for it (after holds, smoothing, the short hold,
7
+ * the retarget asked the bone for it (after holds, the short hold,
8
8
  * the flip guard and the hinge check; before any range clamp) and "key" is the
9
9
  * same segment's angle in the set's key skeleton, both by `pictureAngle`. Per
10
10
  * frame the candidate is the set with the smallest distance within the