lecodes-assets 0.1.2 → 0.2.1

package/README.md

@@ -24,6 +24,7 @@ lecodes assets doctor hero.glb [--fix] [--ktx2] [--dump]         # the old tools
 lecodes assets doctor hero.glb --fix-scale                       # bake a unit fix (see Scale below)
 lecodes assets doctor props/*.glb --lightmap-uv -o props/         # + TEXCOORD_1 lightmap UVs (xatlas) for `lecodes lightmap bake`
 lecodes assets retarget fox.glb --from wolf.fbx --map auto       # clips from a DIFFERENT skeleton (map from skeleton shape)
+lecodes assets avatar girl.glb --meta SK_Girl.fbx.meta           # a humanoid's bones in the Unity/VRM canon (vs Unity's own map)
 lecodes assets unpack pack.unitypackage -o out [--only re …]      # stream a .unitypackage: unity-index.txt + assets (+ .meta)
 lecodes assets convert Barrel_v1.FBX --list-nodes                # multi-variant prop file: what is in it
 lecodes assets convert Barrel_v1.FBX --keep Barrel_v1_LD1 -o barrel.glb   # one variant, re-origined on the ground
@@ -31,6 +32,9 @@ lecodes assets convert Barrel_v1.FBX --split props/              # every top-lev
 lecodes assets convert Body.FBX --merge-skins Shirt.FBX Gloves.FBX --clips … -o hero.glb   # modular character → ONE skeleton
 lecodes assets probe hero.glb --filmstrip                        # a CPU-skinned PNG strip per clip (no GPU)
 lecodes assets scene Map_v8.unity -o level.json                  # Unity scene → placed prefabs with world TRS
+lecodes assets unreal ls "Stairs Set/StairsSet"                  # what an UNREAL pack holds (see below)
+lecodes assets unreal tex T_Manny_01_D.uasset -o manny_d.png     # a texture's SOURCE image (needs Oodle, see below)
+lecodes assets convert SKM_Manny.uasset --clips Anims/**/*.uasset -o manny.glb   # Unreal mesh + clips → GLB
 ```
 
 ## Props, modular characters, scenes — the Unity-pack workflow
@@ -176,10 +180,111 @@ that decides quality is what the two REFERENCE poses are and what happens where
   against node space (Maya "actor" exports, ×100) is rescaled; a donor facing the other way is turned
   (facing is read from the skeleton shape, so `joint1…` rigs work too); non-uniform scales are refused.
 
-What it does not do yet: foot locking on the target beyond what the donor's feet do (an in-place
-donor still treadmills), stride warping when a leg is far too short for the donor's reach (reported
-as "out of reach"), torso pitch to compensate for mismatched front/hind leg ratios, bipeds/arms (the
-auto-map is quadruped-shaped; a humanoid map is best written by hand or by name).
+- **Second pass on proportions** (humanoid plan phase 3, 2026-08-27; the quadruped path shares the
+  planting and the root-motion switch):
+  - **A humanoid foot = contact + orientation** (2026-08-27). The leg's IK effector is the ankle,
+    but its goal comes from the BALL of the foot (the canonical `Toes` bone): the ball goes where
+    the donor's ball goes (leg and foot each scaled by their own ratio, stance-corrected), the foot
+    keeps the donor's rotation on the target's own flat bind, and the ankle follows from the two.
+    Rigs put the ankle at different places over the ball — a boot's foot bone dives 56° to the ball,
+    X-bot's 27°, both soles flat: driving the ball with the foot as an IK link let the solver pick
+    the foot's pitch (50° off the donor in the swing), transferring the ankle's position AND the
+    foot's rotation lifted the boot 10 cm off the ground at toe-off. Now the stance foot is flat,
+    toe-off has the heel up and the toes on the ground, and the report says `contact <ball>` when
+    it differs from the effector. What the two feet's shapes cost at full extension (a few cm) is
+    silent; `reach` reports what the LEG cannot do.
+  - **Foot planting** (default; `--no-plant`). While a donor foot stands still on its ground (under
+    15 % of a leg length per second, within 6 % of a leg length of its stance height, ≥ 3 frames),
+    the target's foot goal is held at ONE world point — the mean of where the FK would have put it
+    over the span, which splits the slide between landing and lift-off; a 4-frame ease at each end
+    hides both halves, a span across a cycle's seam is one plant held modulo the clip's travel. A
+    span whose mean sits more than 5 % of a leg length from some frame's FK goal is a shuffle, not a
+    plant, and is left alone. Per-leg ratios and the hips' own sway otherwise slide a planted foot a
+    few cm per step. The planted point is the contact (the ball rolls under a rising heel; the ankle
+    moves). Reported per leg (`planted N frames`) and per clip as spans.
+  - **Where the travel goes** (`--root-motion auto|hips|root|none`). A rig with a root ABOVE the hips
+    gets the donor's horizontal travel on the root and the vertical bob on the hips (`root`, the
+    default when such a root exists — a character controller reads the root's travel and must not
+    bob with it); `hips` puts everything on the hips as the donor had it; `none` strips the travel
+    for an in-place clip (feet still planted in the travelling frame, so they slide back under the
+    body like a treadmill). A donor whose hips bob relative to their own root (MoCap Central:
+    `root` travels, `pelvis` bobs) has that bob re-applied on the target's hips.
+  - **Hands as effectors** (`--ik hands`): the same ratio-scaled goal as a foot, measured from the
+    upper arm, chain capped at upper + lower arm, for clips where the hands meet or hold.
+  - **Twist helpers** (`--twist N`, default 0.5; `off` = ride rigidly). A helper at fraction `t`
+    along its limb bone takes `f·(t·twistOut − (1 − t)·twistIn)` about the bone's axis on top of
+    riding it — near the joint it counter-rotates the bone's own axial turn (the shoulder skin stays
+    with the clavicle), near the far end it follows the child's (the wrist follows the hand's
+    pronation). Helpers come from the avatar's `twist` map (shape classifier or `.avatar.json`).
+  - The stance correction inside a foot goal (where THIS rig rests its foot under its hip, against
+    where the donor does) is shrunk — never the donor's vector — when it would put the goal past the
+    leg's length (a running push-off on a rig whose hips sit higher above its feet); that removed
+    the "out of reach" residue on Humanoid_F.
+
+What it does not do yet: stride warping when a leg is far too short for the donor's reach (reported
+as "out of reach"), torso pitch to compensate for mismatched front/hind leg ratios; for humanoids
+the knee/elbow bend-plane roll of the UPPER arm and the legs —
+[docs/humanoid-retarget-plan.md](../../docs/humanoid-retarget-plan.md).
+
+### Humanoids — avatars
+
+A humanoid rig is described by its **avatar**: its bones named in the Unity/VRM humanoid canon
+(`Hips`, `Spine`/`Chest`/`UpperChest`, `Neck`, `Head`, `Jaw`, `LeftEye`…, `LeftShoulder`/`LeftUpperArm`/
+`LeftLowerArm`/`LeftHand`, `LeftUpperLeg`/`LeftLowerLeg`/`LeftFoot`/`LeftToes`, `Left Thumb Proximal`…
+— 55 bones). An avatar belongs to ONE rig; a humanoid map is two avatars joined on the canon, so
+each rig is mapped once and any pair retargets.
+
+```
+lecodes assets avatar girl.glb                                  # read the avatar off the skeleton's shape
+lecodes assets avatar girl.glb --meta SK_Girl.fbx.meta -o girl.avatar.json   # check vs Unity's map, write Unity's
+lecodes assets retarget girl.glb --from Swat.fbx --auto-map      # both rigs' avatars from shape → map.json
+lecodes assets retarget girl.glb --from Swat.fbx --avatar girl.avatar.json --map auto --clips Walking
+```
+
+`avatar` reads the structure off the bind pose (`src/retarget/humanoid.ts`): the hips are the joint
+two chains hang from that reach the ground on opposite sides; the hands are the lateral joints fanning
+out into short chains (fingers) — a weapon socket or IK target on the hand is shed; the spine is the
+centre path from the hips to where the arms meet; the head is the last branch above the arms (hair
+rigs branch below it, so a `head` name breaks that tie); the toes say which way is forward, and so
+which side is left. Spine bones: 1 → `Spine`, 2 → `+Chest`, 3 → `+UpperChest`, more → first /
+arc-middle / arm-attach with the rest **bridged**; a second neck bone and finger metacarpals bridge
+too. **Twist/roll helpers** (leaf children on their parent's axis — `upperarm_twist_01_l`,
+`calf_twist_02_r`) are recorded as `twist: { bone: canonical }`, never mapped. Names only break ties
+(thumb vs index when both are named; a `jaw` child) and the report says when they did. Everything
+else (hair, cloth, `ik_*`, bag, breasts) is listed as "not part of the body" and keeps its bind.
+
+**A person's word rides in the avatar** (`.avatar.json` `transfer`, written by the character studio,
+read by `retarget --avatar`): `align: { <canonical bone>: "keep" | "align" }` — this bone is BUILT
+that way (keep the bind, whatever the angle) or POSED (align to the donor, the 8° threshold
+notwithstanding; a standing leg too); `roll: { Left?, Right? }` — degrees added to the measured palm
+roll per arm, for a finger that lands almost right; and the transfer settings (`rootMotion`,
+`plant`, `hands`, `twist`) the CLI takes as defaults — flags still win. The report lists
+`under 8° kept` bones with their angle so the decision can be checked, and `avatar transfer: …`
+says what the file asked for.
+
+**The humanoid transfer** (`retarget` with both avatars — `--map auto`, `--avatar`, or a map.json
+that carries `avatars`) keeps the world-delta machinery above and decides the reference by canonical
+ROLE: hips, spine, neck, head, face and (standing) legs keep their bind; shoulder → upper arm →
+forearm → hand and every phalanx align to the donor's bind directions with explicit references (the
+hand to its finger bases), but only past an **8° threshold** — under it a mismatch is joint placement
+and stays, over it is posture (an A-pose against a T-pose, a bent elbow) and turns. Direction fixes
+two of three degrees of freedom; the **palm roll** — where the palm faces, the thing no direction can
+see — is read off the knuckle line (index base → little base) across the forearm, and the forearm
+chain turns until the target's line lies where the donor's does. Metacarpals ride the hand rigidly.
+The donor is referenced to its bind (its clips are authored on it); a target `root` above the pelvis
+that shares the donor's hips takes the hips' travel only. An identity retarget is exact
+(`tests/humanoidRetarget.test.ts`, key reduction off); Mixamo Walking on the zoo lands arm directions
+within 3° and palms within 10° on every frame.
+
+A Unity model importer's `.meta` carries Unity's own humanoid map: `--meta` diffs the shape result
+against it — the zoo in the plan agrees on every bone but the Chest/UpperChest picks on a 5-bone spine
+— and `-o` with `--meta` writes Unity's map as the `.avatar.json`. `retarget --avatar` / `--donor-avatar`
+accept either file; without them both rigs classify from shape (a biped↔quadruped pair is refused).
+
+Bind space: IBMs live in the MESH's geometry space. When an exporter parks the Z-up→Y-up turn on a
+geometry helper node (Female Runner's `Humanoid_F`) or ×100 on the mesh node (Maya "actor"), `rig.ts`
+folds that node's world transform in front of `inv(IBM)` — kept only when it brings the bind closer to
+the node rest pose.
 
 ### Donors whose clips are not in the FBX
 
@@ -196,6 +301,24 @@ lecodes assets retarget fox.glb --from Cat.fbx --map fox-from-cat.map.json   --a
 Pass the donor's locomotion clips even when you do not want them in the output: `--donor-ref auto`
 takes the donor's neutral posture from a walk/idle, and without one it falls back to the bind pose.
 
+**Skeleton-only clip files** — the other pack convention (Female Runner, MoCap Central, Mixamo
+"without skin"): each clip is an FBX of nodes + curves and NO mesh, so it has no skin to classify
+and no bind to measure from. `--donor-rig <model.fbx|.glb>` names the rig it was animated on; the
+clips bind to it by bone name (exact, then without the `namespace:` prefix — `rebindClips` in
+`rig.ts`) and the model's bind, skin and avatar carry the motion:
+
+```
+lecodes assets retarget vex.glb --from MCU_af_Loco_Walk_Fwd.FBX --donor-rig rig_MCUnity.fbx --map auto -o vex-mc-walk.glb
+```
+
+Whether a clip file IS the character's own rig (copy the tracks by name, the Mixamo flow) or only
+shares its naming convention (two Unreal-style rigs — retarget) is decided by `sameSkeleton`:
+≥ 80 % of the names must match AND every matched bone ≥ 5 cm must keep its local offset's
+DIRECTION (the child in its parent's frame — fixed by the rig, unlike the node rotations, which
+a clip file carries posed at its first frame). A Mixamo character against another's clip stays
+within ~20°; MoCap Central's rig against IdaFaber's turns the pelvis by 88° — a copied clip would
+lay the character down.
+
 ### What a short spine cannot borrow
 
 Rotations transfer; positions follow from the TARGET's own bones. Leg IK covers that for feet, but
@@ -221,6 +344,104 @@ CELL=440 bun tools/filmstrip.ts fox-anim.glb strip.png 4 wolf_walk side "fox.glb
 
 For the real thing on the real engine, `packages/viewer` → `node anim-check/model-check.mjs`.
 
+## Unreal packs — `.uasset` skeletons, clips, meshes
+
+An Unreal asset-store pack is a Content folder of **uncooked editor packages**: one `.uasset` per
+asset with everything inline. `convert`, `probe` and `retarget` read them directly — no Unreal
+install, no FBX export step:
+
+```
+lecodes assets unreal ls "Stairs Set/StairsSet"                       # triage: classes, rigs, what is unreadable and why
+lecodes assets probe .../A_Stairs_WalkFwd_Up_Loop.uasset              # engine, data model, skeleton, keys, root travel
+lecodes assets convert .../SKM_Manny.uasset --clips .../Animations/Mannequin/**/*.uasset -o manny.glb
+lecodes assets convert .../A_Stairs_WalkFwd_Up_Loop.uasset -o walk.glb            # clip-only GLB (its skeleton + the clip)
+lecodes assets convert hero.glb --clips .../A_Stairs_WalkFwd_Up_Loop.uasset       # merge one onto YOUR character by bone name
+lecodes assets retarget vex.glb --from .../SKM_Manny.uasset --anim .../Walk/*.uasset --map auto --root-motion hips -o vex-stairs.glb
+```
+
+**Both data models are read.** An `AnimSequence` keeps its raw motion in one of two places, and a
+pack's choice is not yours: `AnimDataModel` is bone tracks (what an FBX import writes), while a clip
+authored or edited in **Sequencer** (UE 5.2+, through a Control Rig) keeps the same motion as
+MovieScene channels — one FK control per bone, nine `FMovieSceneFloatChannel`s each (3 translation,
+3 rotation as an `FRotator` in degrees, 3 scale). `sequencer.ts` evaluates those onto the clip's own
+frame grid, so callers never learn which model a file used. The grid lands exactly on the authored
+keys (checked across a pack's 48 Sequencer clips: 0 ms offset, 0 value error), and the channels'
+constant values reproduce the skeleton's reference pose to 1e-4 cm and 0.001° — which is what pins
+the channel order and the `FRotator` convention.
+
+**What is read.** `SkeletalMesh` (LOD geometry, UVs, skin weights, bone list — from the editor
+import data the FBX importer left behind), `Skeleton` (bones + reference pose), `AnimSequence`
+(the raw uncompressed bone tracks of its `AnimDataModel`, plus fps, length, root-motion flag,
+additive type, sync markers / notifies), and `Texture2D` — the SOURCE image the artist imported,
+not the compressed mips a cooked build ships (see **Textures** below).
+
+**Where a clip's skeleton comes from.** An AnimSequence names it only as an object path
+(`/Game/StairsSet/Demo/Mannequins/Meshes/SK_Mannequin`), where `/Game/` is the pack's Content
+folder — which nothing on disk marks. The path is turned into a relative file name and looked for
+walking up from the clip; the first directory that answers IS the Content root. `--skeleton
+<SK_*.uasset>` names it outright, `--content-root <dir>` names the folder. A `SKM_*.uasset` works
+as a skeleton too (its own bone list is the bind reference).
+
+**Coordinates.** UE is centimetres, X-forward/Y-right/Z-up and left-handed; glTF is metres, Y-up
+and right-handed. A `.uasset` GLB has to be interchangeable with one made from the same pack's FBX
+(both feed `--clips` and the retargeter), so the conversion reproduces what UE's FBX exporter plus
+ufbx do: mirror Y on every local transform, and turn the ROOT bone −90° about X. Checked against
+the Stairs pack's own `SourceFiles.zip`: all 267 channels of a clip match by bone name to
+**1e-5 m / 0.02°** (`tests/unreal.test.ts`), and the recipe's composition is pinned by a unit test —
+the per-bone rules must add up to the global `(x, z, y)·0.01` map on world points.
+
+**Limits** (each one says so at the point it bites, rather than producing a broken GLB):
+
+| Not supported | What you get |
+|---|---|
+| UE 5.4+ / UE 5.5+ packages, UE4 packages, cooked assets | refused by version, with the engine that saved the file |
+| meshes whose LODs were generated in-engine (no import data — the Stairs pack's UE4 mannequin) | refused; only the skeleton of such a mesh can be read |
+| morph targets, curves, sockets, physics assets, Oodle-compressed MESH bulk | reported, not guessed at |
+
+Twelve bones per vertex become four (top by weight, renormalised) — the trim is reported as a
+percentage of redistributed weight. Triangle winding is decided from the data (the mirror reverses
+handedness) rather than hidden behind a double-sided material.
+
+**Textures.** A texture asset keeps its source image (usually the original PNG) inside an
+`FCompressedBuffer`, compressed with **Oodle** — proprietary, so nothing is shipped here. The
+decoder is loaded from a library the machine already has:
+
+```sh
+export LECODES_OODLE=".../Engine/Binaries/ThirdParty/Oodle/Win64/oo2core_9_win64.dll"   # any UE install
+# or drop the file into ~/.lecodes/oodle/ — Unreal installs are also found automatically
+lecodes assets unreal tex T_Manny_01_D.uasset -o manny_d.png   # one texture, out
+lecodes assets convert SKM_Manny.uasset -o manny.glb           # textures wired to the materials
+```
+
+Without it the conversion still runs and says so once; only the materials stay grey. With it, each
+slot's `MaterialInstanceConstant` is read and its `TextureParameterValues` are wired to glTF:
+base colour, normal, and ORM (UE packs occlusion/roughness/metallic into R/G/B, which is exactly
+what glTF reads). Two fixes are applied on the way, both required and both verified against the
+render: a `TSF_BGRA8` source has its red and blue channels the other way round (without the swap the
+mannequin comes out blue), and UE authors normal maps DirectX-style, so the green channel is flipped
+for glTF (`--no-normal-flip` keeps them as they were). Textures are capped at `--max-size` (2048)
+and re-encoded — JPEG unless the alpha channel is actually used.
+
+What is NOT wired: everything the material graph would have to be evaluated to understand. The UE5
+mannequin's material names seven textures — a bent normal, an aniso/AO/paint mask, a clear-coat
+mask, an MSR pack, a tangent map — and only the base colour and the normal map have an honest glTF
+slot. The rest are listed in the report (`not in glTF: …`) rather than guessed into one. Scalar and
+vector parameters (colour tints, roughness powers) are read but not applied for the same reason:
+gear comes out in its neutral colours.
+
+**Retargeting from a UE rig.** `--from x.uasset` builds the rig directly. The UE5 mannequin weights
+its limbs to the twist helpers (`thigh_twist_01_l`) and leaves `thigh_l` itself without a single
+vertex, so "part of the body" is the skinned subtree's INTERIOR, not just the bones carrying
+weights — otherwise the humanoid classifier finds no legs and refuses the rig. Unskinned ends
+(`ik_foot_root`, `ik_hand_gun`, `weapon_l`, `interaction`, `center_of_mass`, `root`) stay out of the
+body, which is what they should be.
+
+Stair clips carry vertical root travel (0.40 m per loop on the Stairs pack's walk-up). `--root-motion
+hips` keeps the climb on the character; `auto`/`root` treat vertical motion as hips bob and would
+swallow it. Feet below the origin plane on a walk-DOWN clip are reported as "toe under the ground" —
+expected for stair content, where the ground is not flat.
+
+
 ## Scale
 
 `probe` and `doctor` always **report** the model's rendered world size; nothing is changed without a
@@ -301,6 +522,16 @@ GLB on the Filament wasm engine, binds every clip through the native Animator, s
 pose and a playing clip (the mesh must draw and deform). `bun test` here covers the reader/converter
 against the Mixamo sample set when `LECODES_FBX_SAMPLES` (or the default sample folder) exists.
 
+The Unreal reader was verified against two real packs — *Stairs Set UE 5.0-5.4* (UE 5.0: 159
+packages, 84 AnimSequences on two mannequins, a 26 MB zlib-chunked mesh bulk) and a UE 5.3 modular
+character pack (796 packages, raw bulk, all 48 clips Sequencer-authored) — every package parsed or refused
+with a reason, and the clips compared channel-by-channel against the pack's own FBX sources.
+`tests/unreal.test.ts` runs those fixtures when `LECODES_UE_PACK` (or the default pack folder)
+exists, plus format/math unit tests that always run. The texture path was checked against the
+predecessor Python extractor with the same Oodle library: the images come out BYTE-IDENTICAL on
+both packs (67 MB, 21 MB and 1 MB payloads), and the converted mannequin renders with its own
+materials through `packages/viewer`'s Filament check.
+
 Retargeting was tuned on the polyperfect *Low Poly Animated Animals* zoo (different authoring
 conventions within one pack: AdvancedSkeleton `Root_M/Hip_L`, Maya `actor:skin_*` with ×100 bind
 space and −Z facing, unnamed `joint1…joint42`): wolf → Meshy fox, lion → cat, wolf → horse / golden
@@ -317,7 +548,11 @@ src/wasm/fbxReader.js    generated single-file Emscripten module (tracked; rebui
 src/fbx/                 read.ts (wasm boundary + IR) · toDocument.ts · convert.ts (merge, write)
 src/textures/            resolveImage.ts (sniff, TGA/BMP decode, PNG encode, external lookup)
 src/doctor/              doctor.ts (analyze/apply/report) · reduceGlbBones · transcodeTextures · transcodeKtx2 · normalizeScale · lightmapUv (xatlas wasm) · validateGlb (mirror of viewer's)
-src/retarget/            rig.ts (hierarchy + bind pose + clip sampling) · retarget.ts (reference poses, world-delta transfer, leg IK, bridges) · automap.ts (skeleton classification + chain pairing) · unityAnim.ts (Unity .anim YAML reader)
+src/retarget/            rig.ts (hierarchy + bind pose + clip sampling) · retarget.ts (reference poses, world-delta transfer, leg IK, bridges) · automap.ts (quadruped classification + chain pairing) · humanoid.ts (humanoid canon, biped classifier, Unity .meta avatars, avatar↔avatar map) · unityAnim.ts (Unity .anim YAML reader)
+tools/humanoid-survey.ts bind-pose survey of the humanoid zoo (arm drop / elbow / facing / proportions per rig)
+tools/humanoid-phase3.ts every zoo clip onto every zoo target: planted frames, reach, twist, root motion (--strips for filmstrips)
+src/retarget/ghost.ts    a rig's skeleton alone with one clip — the character studio's donor "ghost"
+src/unreal/              package.ts (uncooked UE5 .uasset: summary/names/imports/exports/tagged properties/bulk data) · skeleton.ts · anim.ts (both data models) · sequencer.ts (MovieScene ControlRig channels) · mesh.ts (FSkeletalMeshImportData) · convert.ts (the UE→glTF coordinate recipe) · toDocument.ts · clipSource.ts (--clips merging) · rig.ts (retargeter) · resolve.ts (object path → file) · texture.ts (FCompressedBuffer + FTextureSource) · oodle.ts (the machine's own oo2core, via bun:ffi / koffi) · materials.ts (material instances → glTF slots) · probe.ts · list.ts (unreal ls) · convertAsset.ts · index.ts
 src/unity/               unpack.ts (.unitypackage streaming) · materials.ts (.mat → glTF materials, ORM packing) · png.ts · scene.ts (.unity → level JSON)
 src/props.ts             --list-nodes / --keep / --split (subtree carving + re-origin)
 src/mergeSkins.ts        --merge-skins (skinned meshes onto one skeleton) · duplicateNodeNames