@woosh/meep-engine 3.14.3 → 3.14.5

package/src/shade/playground/skinned_mesh_soup/README.md

@@ -0,0 +1,143 @@
+# skinned mesh soup — repro
+
+The character from `skinned_blas_refit`, loaded the way the game loads one, with the pieces that
+could be making soup on switches.
+
+```bash
+npm run dev
+```
+
+then open `/src/shade/playground/skinned_mesh_soup/index.html`.
+
+`?count=N` for N characters, `?skin=off|bind|animated`, `?trace=1` for the path tracer. Every switch
+is in the url so a state worth showing someone can be sent as a link.
+
+## What it turned out to be
+
+**`chunk_skin_blend_mesh_local` requires `inverse(mesh.global) × joint.global × inverse_bind` to be
+rigid. For `Michelle.glb` it carries a uniform scale of exactly 100, and a dual quaternion cannot
+carry a scale.**
+
+The chunk's own doc comment states the precondition — *"the four products must be rigid, which holds
+whenever the joints and the mesh node share the scaling ancestors"* — and names the shape it was
+written for: an FBX-derived glTF that puts its unit conversion on a root node and authors the
+inverse-bind matrices in **pre-scale** mesh space, so `joint.global × inverse_bind` carries the same
+scale `mesh.global` does and the fold cancels it.
+
+Michelle is the other legal shape. Her POSITION data and her inverse-bind matrices are authored in
+the **already-scaled** space — 1.66 m tall, Y-up, as they stand — so
+
+```
+joint.global × inverse_bind = I          for every one of her 65 joints
+mesh.global                 = 0.01 scale × the up-axis rotation, from her "Character" root
+```
+
+and the fold is `inverse(mesh.global) × I`, a bare **100×**. Measured, at bind and under the dance:
+every folded matrix's column length is `100.0000`, where rigid is 1.
+
+`dual_quat_from_m4` builds its quaternion from `sqrt(max(0, 1 ± m00 ± m11 ± m22))`. Those terms are
+`1 + 100·(…)`, so the `1` is swamped and every component whose sum goes negative is **clamped to
+zero** by the `max`. The rotation it extracts is not the matrix's rotation, and how wrong it is
+depends on the matrix — so it is wrong differently for every joint. That is the soup: not one
+transform applied badly, but sixty-five of them disagreeing.
+
+The two modes made the mechanism visible:
+
+| `?skin=` | what it did | why |
+| --- | --- | --- |
+| `bind` | the right character, exactly 1/100 the size | every joint folds to the *same* matrix, `100·Rx(90°)`. Its `w` and `x` terms are both `1 + 100`, its `y` and `z` both clamp to zero, and the normalize recovers `Rx(90°)` — by luck. Only the scale is lost. Sampled output was `expected / 100` to the digit. |
+| `animated` | soup | the joints' matrices differ, each loses a different amount, and the mesh comes apart. Mean error 121 in a mesh-local space where the body is 224 across; worst 165, on vertices bound to **a single joint at weight 1** — so the blend was not even involved. |
+
+`__soup_blend_probe.spec.js` is that measurement: the WGSL of `dual_quat_from_m4`,
+`dual_quat_blend4` and `dual_quat_transform_point` ported to JS, run over the character's own
+matrices at bind and at `t = 0.5 s` of `SambaDance`, against the position the glTF formula puts each
+vertex at. No device.
+
+### What it is not
+
+`__soup_probe.spec.js` is the ruling-out, and every one of these passes on the parsed asset before a
+GPU is involved:
+
+- the geometry is a surface, not a soup — 28 106 triangles, no degenerates, longest edge 0.093
+  against a 2.24 diagonal, every meshlet index in range;
+- the skinning attributes are sound — all 19 106 vertices land back on their rest position under
+  `joint.global × inverse_bind`, no zero-weight vertices, no out-of-range joint;
+- the source BLAS is exact — 56 211 nodes all reachable, 28 106 leaves each naming a distinct
+  triangle whose box matches to the bit, all 28 105 internal nodes exactly their children's union.
+
+So it is not the loader, not the meshlet build, not the tree, and not the refit. `skinned_blas_refit`'s
+`verify()` cannot see this at all, and that is worth saying plainly: it holds the tree's root box
+against the bounds chain's box, and **both are derived from the clone's vertex region**. Vertices
+that are wrong in the same way agree with each other perfectly. A green `verify()` and a soup
+picture are consistent.
+
+## What was done about it
+
+`renderer/animation/SKINNING_FRAME_PLAN.md`. The short version: glTF says a skinned mesh's node
+transform is ignored, so the frame is not the asset's to choose — it is the engine's.
+`register_skin` now sets it to the skin's own bind transform (`skin_normalize_mesh_frame`), which
+makes the fold the identity at bind and a rigid displacement under a clip, for **both** shapes a
+glTF file arrives in. One matrix per skin, once, at registration. No shader change and no
+per-vertex cost, and a file that was already right is not touched.
+
+Same measurement, after:
+
+| | folded scale | mean error | worst | worst on a single-joint vertex |
+| --- | --- | --- | --- | --- |
+| bind, mesh node as the file gave it | 100.0000 | 121.2470 | 165.4281 | 165.4281 |
+| posed at 0.5 s, the same | 100.0000 | 121.6197 | 165.5332 | — |
+| bind, on the skin frame | 1.0000 | 0.0002 | 0.0016 | 0.0014 |
+| posed at 0.5 s, on the skin frame | 1.0000 | 0.0001 | 0.0062 | 0.000014 |
+| bind, skin frame and §9 | 1.0000 | 0.0000 | 0.0000 | **0** |
+| posed at 0.5 s, skin frame and §9 | 1.0000 | 0.0001 | 0.0062 | **0.000001** |
+
+Neither residual was the frame. The blended ones are dual-quaternion skinning declining to be a
+linear blend, which is what it is for. The `0.0014` at bind was `dual_quat_from_m4`'s square-root
+conditioning near the identity — pre-existing, and maximally exposed by a fold that now lands every
+joint *on* the identity. Plan §9 moved the conversion onto `quat_from_m3`'s branch-picking form,
+which was already in the tree and used by nothing, and it went to zero.
+
+Confirmed on an NVIDIA RTX 4090: the frame correction fires and logs, `?skin=bind` draws a full-size
+character, `?skin=animated` dances, and §9's WGSL — which nothing offline type-checks — compiles.
+
+So the modes now read the other way round:
+
+| `?skin=` | what it should be |
+| --- | --- |
+| `off` | a 1.7 cm figure on her side — nothing deforms the clone, so the mesh node's own transform is all there is. Unchanged by the fix, and still the control for "is the geometry upload sound". |
+| `bind` | a full-size character standing in bind pose, and `await diagnose()` clean |
+| `animated` | dancing, with the traced and rasterized silhouettes the same silhouette |
+
+## The switches
+
+- **`?skin=off`** — no skin registered. A `SkinnedMesh` still gets a vertex clone
+  (`GPUMeshSkinningContext#obtain_geometry_index` allocates one for every skinned mesh), but nothing
+  deforms it, so what is drawn is the memcpy of the source: the rest pose under the mesh node's own
+  transform. For Michelle a *correct* result is a 1.7 cm figure on her side. Soup here would be the
+  geometry upload — the meshlet copy or its address patch — and nothing to do with skinning.
+- **`?skin=bind`** — the skin registered, no clip. `await diagnose()` in the console reads the
+  clone's vertices back off the GPU and holds them against `inverse(mesh.global) × rest`, which is
+  what the bind pose must produce. It reports which vertices are wrong and in what shape: a
+  vertex-slot range says the workgroup stride loop, whole meshlets say addressing, and a uniform
+  factor says this bug. It checks the clone's meshlet records address a region of their own first —
+  a clone still pointing at the source's vertex data would be deforming the shared rest pose in
+  place. Since the frame is now the skin's, `inverse(mesh.global)` is the identity here and the
+  clone should come back holding the source's own positions.
+- **`?skin=animated`** — the file's dance, as `skinned_blas_refit` runs it.
+- **Path tracer** — off by default. The rasterizer reads the deformed vertices directly, with no
+  tree in the way, so it is the reading to establish the answer on: soup in *both* is the vertices,
+  soup in the tracer alone is the tree.
+
+## Why through `GLTFSceneBundleAssetLoader`
+
+`skinned_blas_refit` calls `load_gltf` directly and parses the file once per character. The asset
+loader is what the game and the editor register, and it hands out **the same bundle** to every
+caller — so a second character is not a second parse, it is `instantiate_scene_bundle`: its own node
+tree, its own skins bound to that tree's joints, its own clips retargeted onto it, sharing the
+geometry and the inverse-bind matrices. Running the repro on that path is what makes a fix found
+here a fix for the engine rather than for one prototype page.
+
+## Assets
+
+`/test_assets/Michelle.glb`, gitignored and unpublished — see `test_assets/README.md`. The two
+probes skip themselves when it is absent.

package/src/shade/playground/skinned_mesh_soup/index.html

@@ -0,0 +1,202 @@
+<!doctype html>
+<html lang="en">
+<head>
+    <meta charset="utf-8">
+    <title>meep — skinned mesh soup</title>
+    <style>
+        :root {
+            color-scheme: dark;
+            --panel: rgba(20, 22, 26, 0.88);
+            --line: #2b3038;
+            --text: #d6dae0;
+            --muted: #8b939f;
+            --pass: #55c46a;
+            --fail: #ff6b6b;
+            --warn: #e5b54a;
+        }
+
+        * { box-sizing: border-box; }
+
+        html, body { margin: 0; height: 100%; overflow: hidden; background: #0d0f12; }
+
+        /* body carries tabindex so the camera controller's keyboard device has something focusable
+           to listen on; the focus ring around the whole viewport is not wanted */
+        body:focus { outline: none; }
+
+        #view { display: block; width: 100vw; height: 100vh; }
+
+        #panel {
+            position: fixed;
+            top: 12px;
+            left: 12px;
+            width: 340px;
+            padding: 12px 14px;
+            background: var(--panel);
+            border: 1px solid var(--line);
+            border-radius: 8px;
+            backdrop-filter: blur(6px);
+            color: var(--text);
+            font: 13px/1.5 system-ui, -apple-system, Segoe UI, sans-serif;
+        }
+
+        #panel h1 { font-size: 13px; font-weight: 600; margin: 0 0 8px; }
+
+        label { display: flex; align-items: center; gap: 8px; padding: 3px 0; cursor: pointer; }
+        label input { margin: 0; }
+        label .hint { color: var(--muted); font-size: 11.5px; }
+
+        hr { border: 0; border-top: 1px solid var(--line); margin: 10px 0; }
+
+        button, select, input[type=number] {
+            font: inherit;
+            padding: 5px 9px;
+            border-radius: 6px;
+            border: 1px solid var(--line);
+            background: #1c1f25;
+            color: var(--text);
+        }
+
+        button { cursor: pointer; }
+        button:hover:not(:disabled) { border-color: #3d4552; }
+        button:disabled { opacity: 0.45; cursor: default; }
+
+        select { width: 100%; cursor: pointer; }
+
+        .row { display: flex; gap: 6px; align-items: center; margin-top: 6px; }
+        .row button { flex: 0 0 auto; }
+
+        #character_count {
+            flex: 1 1 auto;
+            min-width: 0;
+            font: 12px ui-monospace, SFMono-Regular, Consolas, monospace;
+            background: #14161a;
+        }
+
+        #status { font-size: 12px; color: var(--muted); word-break: break-word; }
+        #status.pass { color: var(--pass); }
+        #status.fail { color: var(--fail); }
+        #status.warn { color: var(--warn); }
+
+        #stats {
+            margin-top: 6px;
+            font: 12px ui-monospace, SFMono-Regular, Consolas, monospace;
+            color: var(--text);
+        }
+
+        p.note { margin: 8px 0 0; font-size: 11.5px; color: var(--muted); }
+
+        code { font: 11.5px ui-monospace, SFMono-Regular, Consolas, monospace; color: var(--text); }
+
+        dl { margin: 6px 0 0; font-size: 11.5px; color: var(--muted); }
+        dt { color: var(--text); font-weight: 600; margin-top: 6px; }
+        dd { margin: 0; }
+    </style>
+</head>
+<body tabindex="0">
+
+<canvas id="view"></canvas>
+
+<div id="panel">
+    <h1>Skinned mesh soup</h1>
+
+    <p class="note">
+        The same character <code>skinned_blas_refit</code> draws, loaded through
+        <code>GLTFSceneBundleAssetLoader</code> — one bundle, one instance per placement — with the
+        pieces that could be making soup on switches.
+    </p>
+    <p class="note">
+        <strong>The answer, for the record.</strong> <code>chunk_skin_blend_mesh_local</code> needs
+        <code>inverse(mesh.global) × joint.global × inverse_bind</code> to be rigid; for this file
+        it carried a uniform <strong>100×</strong>, and a dual quaternion cannot carry a scale.
+        <code>register_skin</code> now puts the mesh on the skin's own bind transform, so the fold
+        is the identity at bind and a rigid displacement under a clip. The README and
+        <code>SKINNING_FRAME_PLAN.md</code> have the measurement; this page is where it is confirmed
+        on a device.
+    </p>
+
+    <hr>
+
+    <label for="skin_mode"><span>How the skin is driven</span></label>
+    <select id="skin_mode">
+        <option value="off">off — no skin registered, nothing deforms the clone</option>
+        <option value="bind">bind — skin registered, no clip</option>
+        <option value="animated">animated — the file's dance, playing</option>
+    </select>
+
+    <dl>
+        <dt>off</dt>
+        <dd>
+            What is drawn is the clone's memcpy of the source: the rest pose under the mesh node's
+            own transform. Michelle's carries the file's centimetre scale and its up-axis rotation,
+            so <em>correct</em> here is a 1.7 cm figure on her side. Soup here is the geometry
+            upload, not skinning.
+        </dd>
+        <dt>bind</dt>
+        <dd>
+            The skinning pass runs every frame over a pose that never moves — and that pose has an
+            answer the CPU knows exactly. <code>await diagnose()</code> reads the clone's vertices
+            back and says which are wrong, and whether the wrong ones are a vertex-slot range (the
+            workgroup stride loop) or whole meshlets (addressing). Should be a full-size character
+            standing still, and a clean report.
+        </dd>
+        <dt>animated</dt>
+        <dd>
+            Soup only in this mode is the pose: the joint matrices, their range, or the order the
+            skin's joints resolved in. Should be dancing, with the traced and rasterized silhouettes
+            the same silhouette.
+        </dd>
+    </dl>
+
+    <hr>
+
+    <label>
+        <input type="checkbox" id="path_tracing">
+        <span>Path tracer <span class="hint">— traverses the BLAS</span></span>
+    </label>
+    <p class="note">
+        Off by default. The rasterizer reads the deformed vertices directly, with no tree in the
+        way, so it is the reading to establish the answer on first: soup in <em>both</em> is the
+        vertices, soup in the tracer alone is the tree.
+    </p>
+
+    <div class="row">
+        <button id="animation">Freeze pose</button>
+    </div>
+
+    <hr>
+
+    <div class="row">
+        <input type="number" id="character_count" value="1" min="1" max="64" step="1">
+        <button id="respawn">Characters</button>
+    </div>
+    <p class="note">
+        One parse whatever the count — the asset hands out the model itself, and each placement gets
+        its own node tree, skins and clips from <code>instantiate_scene_bundle</code>.
+    </p>
+
+    <hr>
+
+    <div id="status"></div>
+    <div id="stats"></div>
+
+    <p class="note">
+        Every switch is in the url — <code>?count=1&amp;skin=bind&amp;trace=1</code> — so a state
+        worth showing someone can be sent as a link.
+    </p>
+</div>
+
+<script type="module">
+    import { start } from './main.js';
+
+    start().catch(error => {
+        const status = document.getElementById('status');
+
+        status.className = 'fail';
+        status.textContent = String(error?.message ?? error);
+
+        console.error(error);
+    });
+</script>
+
+</body>
+</html>

package/src/shade/playground/skinned_mesh_soup/main.d.ts

@@ -0,0 +1,7 @@
+/**
+ * Stand the page up.
+ *
+ * @returns {Promise<void>}
+ */
+export function start(): Promise<void>;
+//# sourceMappingURL=main.d.ts.map

package/src/shade/playground/skinned_mesh_soup/main.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"main.d.ts","sourceRoot":"","sources":["../../../../../src/shade/playground/skinned_mesh_soup/main.js"],"names":[],"mappings":"AAiQA;;;;GAIG;AACH,yBAFa,QAAQ,IAAI,CAAC,CAkTzB"}