cozyclay 1.9.0 → 2.0.0

package/tools/bench/EXP3.md

@@ -0,0 +1,61 @@
+# Cube-contact validation (#454)
+
+Run from this checkout. Requires Node, ffmpeg/ffprobe, Chrome, and installed npm dependencies. Live extraction additionally requires the existing GVHMR installation over SSH. Never run two GPU extractions concurrently.
+
+```sh
+E=/Users/yun/ccFalToMocap/evidence/exp3
+node tools/bench/exp3.mjs --root "$E" --stage gt
+node tools/bench/exp3.mjs --root "$E" --stage gt-path --host yun@ubuntu-baremetal
+node tools/bench/exp3.mjs --root "$E" --stage fal
+node tools/bench/exp3.mjs --root "$E" --stage summary
+```
+
+Defaults are Vite 5194, CDP 9234, and `--variants shaded,skin`. `--motions sit` selects a scenario; `--variants skin` selects only the grey/YOLO path. `--stage all` runs all stages serially. Inputs: `gt-motions/{bump,handon,sit,stepup}.npz` and `gen-log*.json` containing the action prompts. A failed stage exits nonzero; it never substitutes a guessed motion or passes a failed extraction into fitting. Completed extraction/scoring artifacts can be reused; use a fresh evidence directory after changing inputs or derivation code (the driver is not a content-addressed cache).
+
+## Renderer
+
+```sh
+node tools/gt-render/render.mjs --out /tmp/example --port 5194 --cdp-port 9234 \
+  --azimuth 30 --elevation 5 --f-mm 35 --keep-frames \
+  --scene-box '{"x":0,"z":1,"rot":0,"sx":0.5,"sy":0.4,"sz":0.5}' motion.npz
+```
+
+`scaleX/scaleY/scaleZ` aliases are accepted. Studio scale/position bounds are validated at the CLI boundary (sizes 0.1..100 m), and yaw is normalized. Sizes are metres; the library cube is 1 m on each axis. Base is on Y=0, footprint centred on X/Z, rotation is yaw degrees about +Y. The single QA-hook line exposes only the existing Studio place and update operations. A scene prop's committed scale is checked before capture; clamped unsupported values fail instead of recording inaccurate dimensions. `scene.json` is written at the motion root and beside each `camera.json`. Rotated boxes record centre/half-extents/yaw and corners; axis-aligned boxes also record min/max. The existing `--box` contact scorer and F5 accept **axis-aligned boxes only**; this experiment derives yaw-zero boxes.
+
+The auto-camera includes both character support over the full clip and all box corners. Masks redraw only the character magenta: the neutral cube remains an occluder but is not included in the character mask. `score.mjs` now reads `scene.json` and places the same visible box in prediction/GT mask renders, including calibrated placements. Without a scene file its old behavior is unchanged.
+
+`--export-vertices --no-video` exports `vertices.f32` (little-endian float32, frame/vertex/XYZ order) and `vertices.json` plus the usual joints/masks. Vertices are world-space CPU-skinned points, not native NPZ joint estimates. `<variant>/contact-sheet.png` shows up to four sampled poses with the eight cube corners/twelve edges projected using the saved camera. `contact-sheet.json` identifies frames.
+
+## Derivation and interpretation
+
+Every span is 0.5 s at the Studio's 24 fps. Sit chooses the lowest sustained hips after the first third; hand-on chooses the lowest sustained mean wrist height in the latter half that permits a skin-safe palm-height table in front of the body (the absolute lowest span can have hands beside the thighs); step-up uses the terminal raised foot; bump uses maximal skin reach along the dominant horizontal travel axis (not necessarily a torso impact). A finite grid of footprint translations around the intended contact is considered, with 0.5 m seat, 0.9 m table, 0.4 m step. For horizontal supports, the top equals the **lowest actual skin vertex within that footprint across the span**. Thus a witness touches exactly and no sampled skin penetrates in that span; vertices lower than the intended contact are never silently discarded. Anatomically poor candidates are reported as warnings, not asserted to be valid motion. A static box cannot force every frame of an imperfect motion to touch. `contact-baseline.json` reports maximum gap as well as minimum gap and penetration; its pass criterion checks every declared contact frame (gap <=3 cm, penetration <=2 cm).
+
+`scenario.json` preserves witness, target height, span, box, prompt, method and uncertainties. Inspect the contact sheet; a small distance alone does not certify sitting, stepping, or palm support. Thin triangle intersections without an enclosed vertex are outside the existing skinned-vertex metric.
+
+GT path runs T2 prod/prod+fmm on shaded and native yolo-vitpose/yolo-vitpose+fmm on skin, fits F0..F5 with the scenario box and matching `--base-condition`, and runs `score.mjs --box` for each step plus GT-as-pred. `gt-path/summary.md` has a variant column (48 rows). Fits, scores and GT-self checks are stored under `gt-path/{fit,score,gt-self}/<variant>/<scenario>/`. It uses RAW box contact (scene safety must not benefit from evaluation-only alignment), aligned root-relative MPJPE, aligned last-frame mean joint error, and aligned IoU. F4 uses only native NPZ endpoint poses, not intermediate GT; rig retargeting/anchoring means these pins are not exact rendered-rig endpoints. F5 is skeletal collision resolution, not skinned-contact optimization.
+
+## fal
+
+```sh
+node tools/bench/fal-generate.mjs --scenario "$E/gt/sit" --out "$E/fal/clips" --variant skin --runs 3 --dry-run
+```
+
+Live key defaults to `~/ccFalToMocap/.fal-key`, or `--key <file>`. Missing key automatically produces dry-run request records and A/B stills; it never waits for the key. The key is not saved or logged. A and B are first/last decoded GT frames of `--variant shaded|skin` (default shaded), cube included. Clip names include scenario, variant and independent run number. The experiment driver plans 4 scenarios x 2 variants x 3 runs = 24 clips. Requests import the repo's `buildH3LockedPrompt` contract and use `minimax/h3-max-turbo/image-to-video`, 480P, 5 s, disabled prompt expansion. No undocumented seed field is sent: three independent requests, explicitly recorded. The queue polling has no overall deadline, logs every status, and bounds individual HTTP requests. Each clip saves request ID, request metadata, all queue responses, MP4, and provider-reported cost fields. Absent cost is unknown, not zero. Mock tests validate the client contract, not live provider availability.
+
+`--stage fal` generates then extracts and fits available clips. It runs prod/prod+fmm for shaded and yolo-vitpose/yolo-vitpose+fmm for skin, supplying true F0/F1 baselines and detector-matched in-camera evidence. Video is normalized to the A-still's 832x480 and 24 fps without temporal stretching. A changed aspect ratio (>1%) fails; unchanged framing is an explicit unverified H3 assumption. F4 pins the original A/B GT poses to the fal clip endpoints; F5 receives the box. **No intermediate fal motion GT exists**, so no MPJPE/trajectory-GT claims are made. Each F step is evaluated for first/last world joint error against A/B, whole-clip skin distance/penetration (contact timing unknown), and same-camera unaligned mask IoU.
+
+Palette segmentation is a coloured-pixel proxy: max(R,G,B)-min(R,G,B) >=35; HSV saturation >=0.25; value >=0.18, on decoded RGB bytes. Neutral cube/floor/shadows are excluded; saturated hallucinations are included and desaturated body pixels can be lost. It is not semantic segmentation. Threshold boundary tests are deterministic.
+
+For skin clips, segmentation is **background difference**, not colour: compare each decoded RGB frame against the GT's `plate.png` (same fixed A-still camera, cube present, character hidden). A pixel is foreground if any channel changes by >=30/255. Moving shadows, lighting drift, compression and generated set changes can contaminate this proxy; no semantic or camera-drift correction is claimed. Shaded and skin IoUs use different proxies and should not be compared as identical measures.
+
+`qa-pack/<scenario>-<variant>-<run>.mp4`: fal | F5 Studio render with cube | 50% blend, same camera/time. `qa-pack/ratings.csv` has empty human rating columns (`prompt_adherence_0_2,contact_natural_0_2,usable_without_fix_YN,notes`), and resumed runs preserve existing ratings. No pack video is fabricated when generation is pending.
+
+## Validation
+
+```sh
+node test/verify-cube-contact.mjs
+npm run build
+node tools/run-tests.mjs
+```
+
+Pure tests cover support witness/nonpenetration, sustained spans, all scenario derivations, rotated box coordinates, independent Three camera projection, segmentation thresholds, request fields, and mocked queue completion/errors. No fixed sleeps or live network calls occur in these tests.

package/tools/bench/cclay_bench_extract_incam.py

@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+"""Issue #432: add camera evidence around the UNMODIFIED production runner.
+
+python cclay_bench_extract_incam.py <runner.py> <video> <out.npz>
+    --static-cam --f-mm 35 --out-root /tmp/cclay-fit-.../cache
+    --detector palette --keypoints hybrid --smooth-sigma 3
+
+Import-only reuse: runner.main, FastRuntime and trajectory_job. The two repo
+worker modules are deployed alongside this file, never into the GVHMR tree.
+No GT poses/joints are inputs. The output contains the runner's members plus:
+  incam_raw_{global_orient,transl,body_pose,betas}: SMPL-X model parameters
+  incam_global_orient / incam_pelvis: production-smoothed camera-space root
+  ayfz_to_camera[T,4,4]: per-frame pelvis/orientation relation, column vectors
+  ay_to_ayfz[4,4]: exact runner normalization including floor/heading offset
+  K_fullimg[T,3,3], ay_global_orient, ay_transl: raw prediction diagnostics
+
+SMPL model transl is NOT its regressed pelvis. We evaluate the same SMPL-X ->
+SMPL mesh regressor as production to get camera pelvis, then smooth with the
+runner's own function. ayfz_to_camera is framewise, not a claim that noisy
+monocular camera and integrated world trajectories agree on one static SE(3).
+The JS fit uses one initial registration to preserve F1's trajectory.
+"""
+import importlib.util
+from pathlib import Path
+import sys
+
+
+def main():
+    runner_path = Path(sys.argv[1]).resolve()
+    arguments = sys.argv[2:]
+    direct = "--bench-direct" in arguments
+    if direct:
+        arguments.remove("--bench-direct")
+    if "--static-cam" not in arguments or "--f-mm" not in arguments:
+        raise ValueError("bench camera evidence requires --static-cam and --f-mm")
+    sys.path.insert(0, str(runner_path.parent))
+    spec = importlib.util.spec_from_file_location("cozyclay_fit_runner", runner_path)
+    runner = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(runner)
+    import numpy as np
+    import torch
+
+    capture = {}
+    detach = runner.detach_to_cpu
+    normalize = runner.compute_T_ayfz2ay
+
+    def save_prediction(value):
+        result = detach(value)
+        capture["prediction"] = result
+        return result
+
+    def save_normalization(joints, *args, **kwargs):
+        result = normalize(joints, *args, **kwargs)
+        capture["heading"] = result.detach().cpu()
+        return result
+
+    runner.detach_to_cpu = save_prediction
+    runner.compute_T_ayfz2ay = save_normalization
+    sys.argv = [str(runner_path), *arguments]
+    video, output = Path(arguments[0]), Path(arguments[1])
+    root = Path(arguments[arguments.index("--out-root") + 1])
+    sigma = float(arguments[arguments.index("--smooth-sigma") + 1]) if "--smooth-sigma" in arguments else 1.2
+    # Match the T2 condition's execution path as well as its detector flags.
+    # YOLO/direct must NOT gain production's trajectory correction or palette
+    # fast path merely because this launcher also captures camera evidence.
+    if direct:
+        runner.main()
+    else:
+        from gvhmr_fastpath import FastRuntime
+        from gvhmr_trajectory import trajectory_job
+        runtime = FastRuntime(runner, enabled=True)
+        with runtime.job(), trajectory_job(runner, video, root):
+            runner.main()
+    pred = capture["prediction"]
+    cp, gp = pred["smpl_params_incam"], pred["smpl_params_global"]
+    with np.load(output) as archive:
+        arrays = {key: archive[key] for key in archive.files}
+
+    # Re-evaluate regressed pelvis in BOTH frames. The SMPL parameter transl
+    # omits the shaped rest-pelvis offset; using it directly shifts placement.
+    with torch.no_grad():
+        model = runner.make_smplx("supermotion").cuda()
+        mapping = torch.load("hmr4d/utils/body_model/smplx2smpl_sparse.pt").cuda()
+        regressor = torch.load("hmr4d/utils/body_model/smpl_neutral_J_regressor.pt").cuda()
+
+        def pelvis(params):
+            mesh = model(**runner.to_cuda(params)).vertices
+            verts = torch.stack([torch.matmul(mapping, v) for v in mesh])
+            return torch.einsum("v,tvc->tc", regressor[0], verts).cpu()
+
+        camera_pelvis = pelvis(cp)
+        world_pelvis = pelvis(gp)
+        del model, mapping, regressor
+        runner._release_gpu()
+        # The prediction packs 21 axis-angle vectors into 63 scalars; the
+        # runner smoother requires a final axis of 3, as in its own main().
+        orient, _, camera_pelvis = runner.smooth_motion_params(
+            cp["global_orient"], cp["body_pose"].reshape(-1, 21, 3), camera_pelvis, sigma=sigma)
+        # Reconstruct normalization translation from production's unsmoothed
+        # mesh-regressed pelvis. Its first root is saved in legacy positions.
+        heading = capture["heading"][0]
+        rotation = heading[:3, :3]
+        norm = heading.clone()
+        norm[:3, 3] = torch.from_numpy(arrays["positions"][0, 0]) - rotation @ world_pelvis[0]
+        camera_r = runner.axis_angle_to_matrix(orient)
+        ayfz_r = runner.axis_angle_to_matrix(torch.from_numpy(arrays["smpl_global_orient"]))
+        r = camera_r @ ayfz_r.transpose(-1, -2)
+        relation = torch.eye(4).repeat(len(orient), 1, 1)
+        relation[:, :3, :3] = r
+        relation[:, :3, 3] = camera_pelvis - (r @ torch.from_numpy(arrays["smpl_transl"])[..., None])[..., 0]
+
+    c32 = lambda value: np.ascontiguousarray(np.asarray(value, dtype=np.float32))
+    arrays.update({"incam_raw_" + key: c32(value) for key, value in cp.items()})
+    arrays.update(incam_global_orient=c32(orient), incam_pelvis=c32(camera_pelvis),
+                  ayfz_to_camera=c32(relation), ay_to_ayfz=c32(norm),
+                  ay_global_orient=c32(gp["global_orient"]), ay_transl=c32(gp["transl"]),
+                  K_fullimg=c32(pred["K_fullimg"]))
+    if not all(np.isfinite(value).all() for value in arrays.values()):
+        raise ValueError("nonfinite camera evidence")
+    np.savez(output, **arrays)
+    print(f"[fit] wrote camera parameters and transforms: {output}", flush=True)
+
+
+if __name__ == "__main__":
+    main()

package/tools/bench/cclay_bench_extract_obs.py

@@ -0,0 +1,204 @@
+#!/usr/bin/env python3
+"""Extract the OBS NPZ contract from one GVHMR forward pass.
+
+Units and frames: image coordinates are pixels; translations and joints are
+metres. ``incam_*`` uses OpenCV camera coordinates (+X right, +Y down, +Z
+forward). ``global_*`` uses GVHMR gravity-view ``ay`` coordinates (+Y up).
+Joints are the first 22 SMPL joints obtained by SMPL-X FK/regression using the
+runner's ``make_smplx('supermotion')`` body model. K is float64 and is copied
+exactly from --K-json.
+"""
+import argparse
+import importlib.util
+import json
+import sys
+from pathlib import Path
+
+import numpy as np
+
+
+def load_runner(path):
+    path = Path(path).resolve()
+    sys.path.insert(0, str(path.parent))
+    spec = importlib.util.spec_from_file_location("cclay_obs_runner", path)
+    runner = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(runner)
+    return runner
+
+
+def fk22(runner, params, model, sparse, regressor):
+    import torch
+    # Pipeline params are batched (1, T, ...); SMPL-X expects (T, ...), the
+    # same per-frame layout the runner feeds it via pred["smpl_params_global"].
+    flat = {key: (value[0] if value.dim() >= 2 and value.shape[0] == 1 else value) for key, value in params.items()}
+    with torch.no_grad():
+        body = model(**runner.to_cuda(flat))
+        verts = torch.stack([torch.matmul(sparse, v) for v in body.vertices])
+        joints = torch.einsum("jv,tvi->tji", regressor, verts)
+        return joints[:, :22]
+
+
+def parameterized_pp(outputs, endecoder, cp_thr, clamp):
+    """Exact pp_static_joint_cam, exposing its two correction limits."""
+    import torch
+    from hmr4d.utils.geo_transform import apply_T_on_points, transform_mat
+    from hmr4d.utils.net_utils import gaussian_smooth
+    from pytorch3d.transforms import axis_angle_to_matrix
+
+    incam = outputs["pred_smpl_params_incam"].copy()
+    global_params = outputs["pred_smpl_params_global"]
+    logits = outputs["static_conf_logits"].clone()[:, :-1]
+    joint_ids = [7, 10, 8, 11, 20, 21]
+    batch, length = incam["transl"].shape[:2]
+    assert batch == 1
+
+    pred_w = endecoder.fk_v2(**global_params)
+    incam["transl"] = gaussian_smooth(incam["transl"], sigma=5, dim=-2)
+    pred_c = endecoder.fk_v2(**incam)
+    r_gv = axis_angle_to_matrix(global_params["global_orient"][:, 0])
+    r_c = axis_angle_to_matrix(incam["global_orient"][:, 0])
+    r_c2w = r_gv @ r_c.mT
+    t_c2w = pred_w[:, 0, 0] - torch.einsum("bij,bj->bi", r_c2w, pred_c[:, 0, 0])
+    t_c2w = transform_mat(r_c2w, t_c2w)
+    pred_c_in_w = apply_T_on_points(pred_c, t_c2w[:, None])
+
+    post_transl = global_params["transl"].clone()
+    post_joints = pred_w.clone()
+    threshold = torch.as_tensor([cp_thr] * 3, device=post_joints.device, dtype=post_joints.dtype)
+    for i in range(1, length):
+        diff = post_joints[:, i, 0] - pred_c_in_w[:, i, 0]
+        diff = diff * ~((diff > -threshold) * (diff < threshold))
+        diff = torch.clamp(diff, -clamp, clamp)
+        post_transl[:, i:] -= diff
+        post_joints[:, i:] -= diff[:, None, None]
+
+    static = logits.sigmoid() > 0.8
+    count = static.sum(-1, keepdim=True).clamp_min(1)
+    disp = (post_joints[:, 1:, joint_ids] - post_joints[:, :-1, joint_ids])
+    disp = (disp * static[..., None]).sum(-2) / count
+    disp[:, :, 1] = 0
+    for i in range(1, length):
+        post_transl[:, i:] -= disp[:, [i - 1]]
+        post_joints[:, i:] -= disp[:, [i - 1], None]
+    ground_y = post_joints[..., 1].flatten(-2).min(dim=-1)[0]
+    post_transl[..., 1] -= ground_y
+    return post_transl
+
+
+def as_np(value, dtype=np.float32):
+    if hasattr(value, "detach"):
+        value = value.detach().cpu().numpy()
+    return np.ascontiguousarray(value, dtype=dtype)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("runner")
+    ap.add_argument("video")
+    ap.add_argument("output")
+    ap.add_argument("--out-root", default="outputs/cclay-obs")
+    ap.add_argument("--K-json", required=True)
+    ap.add_argument("--detector", choices=("palette", "yolo"), default="palette")
+    ap.add_argument("--keypoints", choices=("vitpose", "hybrid"), default="vitpose")
+    ap.add_argument("--betas-json")
+    ap.add_argument("--pp-thr", type=float, default=0.0)
+    ap.add_argument("--pp-clamp", type=float, default=1.0)
+    ap.add_argument("--check-pp", action="store_true")
+    args = ap.parse_args()
+
+    runner = load_runner(args.runner)
+    import torch
+    from hmr4d.model.gvhmr.utils.postprocess import pp_static_joint_cam, process_ik
+    from hmr4d.utils.geo.hmr_cam import normalize_kp2d
+
+    with open(args.K_json, encoding="utf-8") as stream:
+        K = np.asarray(json.load(stream), dtype=np.float64)
+    if K.shape != (3, 3):
+        raise ValueError("--K-json must contain a 3x3 array")
+
+    video = Path(args.video)
+    length, width, height = runner.get_video_lwh(str(video))
+    cfg = runner.build_cfg(video, True, None, Path(args.out_root))
+    data = runner.preprocess(cfg, video, args.detector, args.keypoints)
+    data["K_fullimg"] = torch.as_tensor(K, dtype=data["K_fullimg"].dtype,
+                                         device=data["K_fullimg"].device).repeat(length, 1, 1)
+    model = runner.hydra.utils.instantiate(cfg.model, _recursive_=False)
+    model.load_pretrained_model(cfg.ckpt_path)
+    model = model.eval().cuda()
+    batch = {
+        "length": data["length"][None],
+        "obs": normalize_kp2d(data["kp2d"], data["bbx_xys"])[None],
+        "bbx_xys": data["bbx_xys"][None], "K_fullimg": data["K_fullimg"][None],
+        "cam_angvel": data["cam_angvel"][None], "f_imgseq": data["f_imgseq"][None],
+    }
+    batch = {key: value.cuda() for key, value in batch.items()}
+    with torch.no_grad():
+        raw = model.pipeline.forward(batch, train=False, postproc=False, static_cam=True)
+
+    pred = {key: {name: value.clone() for name, value in raw[key].items()}
+            for key in ("pred_smpl_params_incam", "pred_smpl_params_global")}
+    pred["static_conf_logits"] = raw["static_conf_logits"].clone()
+    if args.betas_json:
+        with open(args.betas_json, encoding="utf-8") as stream:
+            loaded = json.load(stream)
+        # Accept a bare list or tools/bench/obs/mannequin-betas.json ({"betas": [...], ...}).
+        betas_used = np.asarray(loaded["betas"] if isinstance(loaded, dict) else loaded, dtype=np.float32)
+        if betas_used.shape != (10,):
+            raise ValueError("--betas-json must contain 10 floats")
+        betas = torch.as_tensor(betas_used, device=pred["pred_smpl_params_incam"]["betas"].device,
+                                dtype=pred["pred_smpl_params_incam"]["betas"].dtype)
+        for params in (pred["pred_smpl_params_incam"], pred["pred_smpl_params_global"]):
+            params["betas"] = betas[None, None].expand_as(params["betas"])
+    else:
+        betas_used = as_np(pred["pred_smpl_params_incam"]["betas"][0].mean(0))
+
+    default_transl = pp_static_joint_cam(pred, model.pipeline.endecoder)
+    relaxed_transl = parameterized_pp(pred, model.pipeline.endecoder, args.pp_thr, args.pp_clamp)
+    if args.check_pp:
+        exact = parameterized_pp(pred, model.pipeline.endecoder, 0.25, 0.02)
+        pp_diff = float((exact - default_transl).abs().max().item())
+        print(f"pp_default vs GVHMR postproc max abs diff: {pp_diff:.9g}", flush=True)
+        if pp_diff > 1e-5:
+            raise AssertionError(f"pp_default mismatch: {pp_diff}")
+
+    variants = {}
+    for name, transl in (("pp_default", default_transl), ("pp_relaxed", relaxed_transl)):
+        params = {key: value.clone() for key, value in pred["pred_smpl_params_global"].items()}
+        params["transl"] = transl
+        ik_input = {"pred_smpl_params_global": params,
+                    "pred_smpl_params_incam": pred["pred_smpl_params_incam"],
+                    "static_conf_logits": pred["static_conf_logits"]}
+        params["body_pose"] = process_ik(ik_input, model.pipeline.endecoder)
+        variants[name] = params
+
+    smplx = runner.make_smplx("supermotion").cuda()
+    sparse = torch.load("hmr4d/utils/body_model/smplx2smpl_sparse.pt").cuda()
+    regressor = torch.load("hmr4d/utils/body_model/smpl_neutral_J_regressor.pt").cuda()
+    incam = pred["pred_smpl_params_incam"]
+    incam_joints = fk22(runner, incam, smplx, sparse, regressor)
+    output = {
+        "fps": np.int32(runner.video_fps(video)), "K": K,
+        "bbx_xys": as_np(data["bbx_xys"]), "kp2d": as_np(data["kp2d"]),
+        "static_conf_logits": as_np(pred["static_conf_logits"][0]),
+        "betas_pred": as_np(raw["pred_smpl_params_incam"]["betas"][0]),
+        "betas_used": as_np(betas_used),
+        "incam_global_orient": as_np(incam["global_orient"][0]),
+        "incam_body_pose": as_np(incam["body_pose"][0].reshape(length, 21, 3)),
+        "incam_transl": as_np(incam["transl"][0]), "incam_joints": as_np(incam_joints),
+        "incam_pelvis": as_np(incam_joints[:, 0]),
+    }
+    for name, params in variants.items():
+        joints = fk22(runner, params, smplx, sparse, regressor)
+        output.update({f"global_{name}_global_orient": as_np(params["global_orient"][0]),
+                       f"global_{name}_body_pose": as_np(params["body_pose"][0].reshape(length, 21, 3)),
+                       f"global_{name}_transl": as_np(params["transl"][0]),
+                       f"global_{name}_joints": as_np(joints)})
+    if not all(np.isfinite(value).all() for value in output.values()):
+        raise ValueError("nonfinite OBS output")
+    Path(args.output).parent.mkdir(parents=True, exist_ok=True)
+    np.savez(args.output, **output)
+    print(f"wrote OBS NPZ: {args.output} fields={len(output)}", flush=True)
+
+
+if __name__ == "__main__":
+    main()

package/tools/bench/cclay_bench_runner.py

@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+"""Bench launcher for cclay_gvhmr_extract.py (issue #430), YOLO runs only.
+
+    python cclay_bench_runner.py <runner.py> <runner args...>
+
+Runs the runner's own main() with exactly the given argv, in the same way
+tools/ardy/cclay_gvhmr_worker.py loads it, and adds one observation: the
+runner logs how many frames its YOLO track detected only on the raw-detection
+fallback, so the tracker path gets a "[bench] yolo track: N/M frames" line
+here. The wrapped method's inputs and return value are passed through
+untouched; the runner's decisions do not change.
+"""
+import importlib.util
+from pathlib import Path
+import sys
+
+
+def main():
+    runner_path = Path(sys.argv[1]).resolve()
+    sys.path.insert(0, str(runner_path.parent))
+    from hmr4d.utils.preproc.tracker import Tracker
+
+    original = Tracker.sort_track_length
+
+    def counted(track_history, video_path):
+        result = original(track_history, video_path)
+        id_to_frame_ids, _, id_sorted = result
+        if id_sorted:
+            print(f"[bench] yolo track: {len(id_to_frame_ids[id_sorted[0]])}/{len(track_history)} frames",
+                  file=sys.stderr, flush=True)
+        return result
+
+    Tracker.sort_track_length = staticmethod(counted)
+    spec = importlib.util.spec_from_file_location("cozyclay_gvhmr_bench", runner_path)
+    runner = importlib.util.module_from_spec(spec)
+    sys.argv = [str(runner_path), *sys.argv[2:]]
+    spec.loader.exec_module(runner)
+    runner.main()
+
+
+if __name__ == "__main__":
+    main()

package/tools/bench/cube-contact.mjs

@@ -0,0 +1,162 @@
+import { sceneBoxRecord, projectBox, BOX_EDGES } from "../gt-render/scene-box.mjs";
+
+/** Lowest sustained window, never a one-frame outlier. Ties choose the later span. */
+export function lowestWindow(values, length, start = 0) {
+	if (!values.length || !values.every(Number.isFinite) || length < 1 || length > values.length - start) throw new Error("invalid sustained window");
+	let best = start, score = Infinity;
+	for (let i = start; i <= values.length - length; i++) {
+		const mean = values.slice(i, i + length).reduce((a, b) => a + b, 0) / length;
+		if (mean <= score) { score = mean; best = i; }
+	}
+	return Array.from({ length }, (_, i) => best + i);
+}
+const mean = a => a.reduce((s, v) => s + v, 0) / a.length;
+const xyz = (v, i) => [v[i], v[i + 1], v[i + 2]];
+
+/** Axis-aligned horizontal support: the highest nonpenetrating top beneath
+ * ALL sampled skin in this footprint. No joint radius or guessed skin offset.
+ * Returns the witness vertex, making exact contact independently checkable. */
+export function supportTop(frames, vertices, { x, z, sx, sz }) {
+	let height = Infinity, witness = null;
+	for (const frame of frames) {
+		const v = vertices(frame);
+		for (let i = 0; i < v.length; i += 3) {
+			if (Math.abs(v[i] - x) <= sx / 2 && Math.abs(v[i + 2] - z) <= sz / 2 && v[i + 1] < height) {
+				height = v[i + 1]; witness = { frame, vertex: i / 3, point: xyz(v, i) };
+			}
+		}
+	}
+	return { height, witness };
+}
+
+export function deriveBox(scenario, joints, vertices) {
+	const index = name => { const i = joints.joints.findIndex(j => j.name === name); if (i < 0) throw new Error(`missing joint ${name}`); return i; };
+	const at = (f, name) => joints.world[f][index(name)];
+	const n = joints.frames, length = Math.min(n, Math.round(joints.fps / 2));
+	const heights = name => joints.world.map((_, f) => at(f, name)[1]);
+	const warnings = [];
+	let frames, box, witness, targetHeight;
+	if (scenario === "bump") {
+		const first = at(0, "Hips"), travel = joints.world.map((_, f) => at(f, "Hips").map((v, a) => v - first[a]));
+		const extent = a => Math.max(...travel.map(v => Math.abs(v[a])));
+		const axis = extent(0) > extent(2) ? 0 : 2;
+		const far = travel.reduce((a, b) => Math.abs(b[axis]) > Math.abs(a[axis]) ? b : a), sign = Math.sign(far[axis]) || 1;
+		let reach = -Infinity, peak = 0;
+		for (let f = 0; f < n; f++) {
+			const v = vertices(f);
+			for (let i = 0; i < v.length; i += 3) if (v[i + 1] > 0.15 && v[i + 1] < 1.5 && sign * v[i + axis] > reach) {
+				reach = sign * v[i + axis]; peak = f; witness = { frame: f, vertex: i / 3, point: xyz(v, i) };
+			}
+		}
+		frames = Array.from({ length: Math.min(length, n) }, (_, i) => Math.max(0, Math.min(n - length, peak - Math.floor(length / 2))) + i);
+		const centre = at(peak, "Hips").slice(), side = axis === 0 ? 2 : 0;
+		centre[axis] = sign * (reach + 0.2);
+		box = { x: centre[0], z: centre[2], rot: 0, sx: axis === 0 ? 0.4 : 1.5, sy: 1.5, sz: axis === 2 ? 0.4 : 1.5 };
+		// Centre width on the actual contact witness, not a drifting pelvis.
+		box[side === 0 ? "x" : "z"] = witness.point[side];
+		warnings.push("Bump frame is maximal skin reach along the dominant travel axis; this may be an extended hand rather than torso impact.");
+	} else {
+		let centres;
+		if (scenario === "sit") {
+			frames = lowestWindow(heights("Hips"), length, Math.floor(n / 3));
+			centres = frames.map(f => at(f, "Hips"));
+			targetHeight = mean(centres.map(p => p[1])) - 0.12;
+			if (Math.max(...heights("Hips")) - mean(centres.map(p => p[1])) < 0.15) warnings.push("No clear seated pelvis drop; using the lowest available sustained span.");
+		} else if (scenario === "handon") {
+			const travel = [at(n - 1, "Hips")[0] - at(0, "Hips")[0], at(n - 1, "Hips")[2] - at(0, "Hips")[2]];
+			const norm = Math.hypot(...travel), forward = norm > 1e-6 ? travel.map(v => v / norm) : [0, 1];
+			const windows = Array.from({ length: n - length - Math.floor(n / 2) + 1 }, (_, i) => {
+				const span = Array.from({ length }, (_, k) => Math.floor(n / 2) + i + k);
+				const palms = span.flatMap(f => [at(f, "LeftHand"), at(f, "RightHand")]);
+				return { span, palms, height: mean(palms.map(p => p[1])) };
+			}).sort((a, b) => a.height - b.height);
+			// The lowest hand span can still be beside the thighs. Select the
+			// lowest span that permits a palm-height table IN FRONT, not a box
+			// shifted sideways to touch a leg. Keep the failed-motion caveat.
+			for (const { span, palms } of windows) {
+				let best = Infinity;
+				const target = Math.min(...palms.map(p => p[1])) - 0.025;
+				for (let step = 14; step <= 36; step++) {
+					const distance = step * 0.025;
+					const candidate = { x: mean(palms.map(p => p[0])) + forward[0] * distance, z: mean(palms.map(p => p[2])) + forward[1] * distance, sx: 0.9, sz: 0.9 };
+					const support = supportTop(span, vertices, candidate);
+					if (support.height < Math.max(0.1, target - 0.1) || support.height > target + 0.1) continue;
+					const loss = Math.abs(support.height - target) + 0.02 * distance;
+					if (loss < best) { best = loss; box = { ...candidate, sy: support.height, rot: 0 }; witness = support.witness; frames = span; centres = palms; targetHeight = target; }
+				}
+				if (box) break;
+			}
+			if (!box) throw new Error("handon: no skin-safe palm-height table in front of the body in any sustained latter-half span");
+			warnings.push("The absolute lowest hands are beside the thighs, not in a table-support pose. Selected the lowest sustained span permitting a skin-safe palm-height table in front. Both-palms natural contact is not established.");
+		} else if (scenario === "stepup") {
+			frames = Array.from({ length }, (_, i) => n - length + i);
+			const foot = mean(frames.map(f => at(f, "LeftFoot")[1])) > mean(frames.map(f => at(f, "RightFoot")[1])) ? "LeftFoot" : "RightFoot";
+			centres = frames.map(f => at(f, foot));
+			targetHeight = Math.min(...centres.map(p => p[1])) - 0.04;
+			if (targetHeight < 0.12) warnings.push("No clearly raised terminal stance foot; best available terminal span used, not a fabricated 0.3 m step.");
+		} else throw new Error(`unknown scenario ${scenario}`);
+		if (!box) {
+			const x = mean(centres.map(p => p[0])), z = mean(centres.map(p => p[2]));
+			const size = scenario === "sit" ? 0.5 : 0.4;
+			let best = Infinity;
+			// A footprint enclosing the feet is rejected by its near-floor
+			// support top, not repaired by ignoring those vertices.
+			const offsets = [-0.3, -0.2, -0.1, 0, 0.1, 0.2, 0.3];
+			for (const dx of offsets) for (const dz of offsets) {
+				const candidate = { x: x + dx, z: z + dz, sx: size, sz: size };
+				const support = supportTop(frames, vertices, candidate);
+				if (!Number.isFinite(support.height) || support.height < 0.1 || support.height > targetHeight + 0.1) continue;
+				const loss = Math.abs(support.height - targetHeight) + 0.3 * Math.hypot(dx, dz);
+				if (loss < best) { best = loss; box = { ...candidate, sy: support.height, rot: 0 }; witness = support.witness; }
+			}
+			if (!box) throw new Error(`${scenario}: no Studio-supported (>=0.1 m) support footprint near the intended contact`);
+		}
+		if (scenario === "stepup" && box.sy > 0.45) warnings.push(`Kimodo's terminal sole is ${box.sy.toFixed(3)} m high; retained the motion-derived height rather than inventing a 0.3-0.4 m step.`);
+		if (Math.abs(box.sy - targetHeight) > 0.1) warnings.push(`Skin-safe top ${box.sy.toFixed(3)} m differs from anatomical target ${targetHeight.toFixed(3)} m: motion/contact is not a clean instance of the requested action.`);
+	}
+	return { scenario, box, scene: sceneBoxRecord(box), contactFrames: frames, contactSeconds: [frames[0] / joints.fps, frames.at(-1) / joints.fps], witness, targetHeight, warnings, method: "Static box from rendered skin in a declared 0.5 s contact span; no intermediate GT is used in fitting. A support top equals the lowest skin vertex within its footprint across that span." };
+}
+
+/** Chroma-only proxy: RGB range >= 35, HSV S >= .25, V >= .18;
+ * includes all saturated palette colours, excludes neutral cube/floor/shadows.
+ * It is NOT semantic segmentation and includes similarly coloured hallucinations. */
+export const PALETTE_THRESHOLDS = Object.freeze({ minRange: 35, minSaturation: 0.25, minValue: 0.18 });
+export function paletteMask(rgb, thresholds = PALETTE_THRESHOLDS) {
+	if (rgb.length % 3) throw new Error("RGB byte count must be divisible by three");
+	const mask = Buffer.alloc(rgb.length / 3);
+	for (let p = 0; p < mask.length; p++) {
+		const r = rgb[p * 3], g = rgb[p * 3 + 1], b = rgb[p * 3 + 2], hi = Math.max(r, g, b), lo = Math.min(r, g, b);
+		mask[p] = hi - lo >= thresholds.minRange && (hi - lo) / hi >= thresholds.minSaturation && hi / 255 >= thresholds.minValue ? 255 : 0;
+	}
+	return mask;
+}
+
+/** Grey-body proxy against the same camera's empty Studio plate (cube stays).
+ * Any RGB channel changing by >=30 counts as foreground. Lighting drift,
+ * moving shadows and generated background changes are intentionally NOT
+ * corrected: they are uncertainties of this proxy, not motion ground truth. */
+export function backgroundDifferenceMask(rgb, plate, threshold = 30) {
+	if (!plate.length || plate.length % 3 || rgb.length % plate.length || !Number.isFinite(threshold) || threshold <= 0 || threshold > 255) throw new Error("invalid RGB background-difference inputs");
+	const mask = Buffer.alloc(rgb.length / 3);
+	for (let p = 0; p < mask.length; p++) {
+		const i = p * 3, q = i % plate.length;
+		mask[p] = Math.max(Math.abs(rgb[i] - plate[q]), Math.abs(rgb[i + 1] - plate[q + 1]), Math.abs(rgb[i + 2] - plate[q + 2])) >= threshold ? 255 : 0;
+	}
+	return mask;
+}
+
+/** Draw all 8 projected corners and 12 edges directly into RGB (no image dependency). */
+export function drawProjectedBox(rgb, camera, box) {
+	const out = Buffer.from(rgb), points = projectBox(box, camera), { width, height } = camera;
+	const dot = (x, y, radius) => { for (let dx = -radius; dx <= radius; dx++) for (let dy = -radius; dy <= radius; dy++) {
+		const u = Math.round(x) + dx, v = Math.round(y) + dy;
+		if (u >= 0 && u < width && v >= 0 && v < height) { const p = (v * width + u) * 3; out[p] = 255; out[p + 1] = 25; out[p + 2] = 25; }
+	} };
+	for (const [a, b] of BOX_EDGES) {
+		const p = points[a], q = points[b]; if (p[2] <= 0 || q[2] <= 0) continue;
+		const steps = Math.ceil(Math.max(Math.abs(p[0] - q[0]), Math.abs(p[1] - q[1])));
+		for (let i = 0; i <= steps; i++) { const t = steps ? i / steps : 0; dot(p[0] + (q[0] - p[0]) * t, p[1] + (q[1] - p[1]) * t, 0); }
+	}
+	for (const [u, v, d] of points) if (d > 0) dot(u, v, 3);
+	return out;
+}