@woosh/meep-engine 3.49.0 → 3.50.0

package/src/shade/renderer/gpu_primitive/bvh/GPU_BVH_BUILD_PLAN.md

@@ -893,8 +893,121 @@ reproduced it:
 - 1,000 forced rebuilds per tree at 20,000 leaves over reused node buffers, alternating two
   far-apart leaf sets, 200 of them one per submission and 800 eight per submission.
 
-So the rate is low and unattributed. The climb's visibility assumption is the open question, and
-R1's per-level-dispatch fallback is its designed answer.
+It was reproduced and attributed the same day, and the climb now handles it: §4.8.
+
+### 4.8 R1 measured, and the refit's repair — landed 2026-10-16
+
+**What happens.** The climb's second arrival at a node sometimes reads its sibling's box before the
+sibling's stores have reached it. It is not rare enough to ignore. A refit-only probe, which builds a
+tree once and then re-runs only the refit, 64 a submission, alternating two leaf sets of one
+topology, measured it. Each refit's internal bounds were compared word for word with the CPU's exact
+tree:
+
+| configuration, RTX 4090, 20,000 leaves | refits | stale |
+|---|---|---|
+| refit only, nothing else on the GPU | 32,000 | 14 |
+| refit only, a second device hammering the GPU | 16,000 | 9 |
+| harness-shaped full builds, nothing else | 1,000 builds | 0 |
+| harness-shaped full builds, second device | 1,000 builds | 2 |
+
+That is about one refit in 2,000. It accounts for the harness's single failure, and for the 1,000
+rebuilds that showed nothing. The node pair the harness reported, 8195 and 8192, recurs in the
+refit-only probe at the same seed.
+
+**What it is, and what it is not.** The pooled arrival counters were read back after every build and
+every refit, and every internal node read exactly 2, so the protocol ran as designed and no counter
+was stale. The stale node is always the union of its fresh child and its other child's previous
+value, with 1 to 4 of the 6 words stale, and the stale child is always internal: leaves come from
+emission, a dispatch earlier. A copy of the climb that records the thread of each arrival found the
+same in 17 of 17 events: the stale child's writer was the parent's first arrival, which then exits,
+and the reader, the second arrival, was always in a different workgroup, always an adjacent one. It
+is cross-workgroup visibility of the bound stores after the counter, exactly as AR3 named it.
+
+**Why no ordering fix.** WGSL has no device-scope fence. Its atomics are relaxed, and
+`storageBarrier` is workgroup-scoped and needs uniform control flow. Inside one dispatch the only
+formally coherent channel between workgroups is a single word's read-modify-write, and a box is six
+words. So the fix is the CSDLDF one: make the read that went wrong detectable, and finish the job
+where a dispatch boundary guarantees visibility.
+
+**The fix.**
+
+1. Emission seeds every internal node's six bound words with `BVH_UNFITTED`, a quiet NaN no union
+   produces (`BVH_REFIT_DEFERRED_LAYOUT.js`). Emission's dispatch ends before the refit's begins, so
+   a refit read returns the store or the seed, never the previous build's box.
+2. The climb's second arrival reads both children through a check (`chunk_bvh_refit_climb`). A seed
+   word counts as the empty box's word (±inf, made at runtime from the seed, because WGSL will not
+   fold an infinity), the node is fitted from what can be seen and the climb goes on, and the node is
+   appended to a deferred list with its tree's node and parents offsets. The list's first three words
+   are the repair's dispatch command, folded as the engine's builders fold one, and raised with
+   `atomicMax` as entries are taken, so no prepare pass is needed.
+3. `shader_bvh_refit_repair`, the build's 20th dispatch, is indirect over that command, so it is
+   empty unless something was deferred. For each listed node it takes the union of the children,
+   now visible, and grows the node and every ancestor with `atomic_min/max_f32_storage`
+   (`chunk_bvh_refit_repair`).
+
+Why that is exact. Everything the climb fitted holds no more than the true box, because an
+unarrived word reads as empty. Every repair contribution holds no more than the true box either.
+For any leaf under a node, the lowest node on the path between them that fails to hold it must
+have read a seed for its path child, so it was listed, and its repair grows the whole path with a
+box that holds the leaf. Growth is a per-word min or max, which commute, so repairs that meet need no
+order.
+
+The single-workgroup build takes the same climb with the same seed. It repairs in-workgroup after one
+more barrier, which gives one workgroup what a dispatch boundary gives the multi-pass build. The
+per-frame skinned BLAS refit (`GPUBLASRefitBatch`) takes it too, with a seeding dispatch of its own
+(`shader_blas_refit_batch_seed`, one lane per node slot) and the same repair over every tree in the
+batch. The batch writes its leaf boxes in the same dispatch too, so the seed covers leaves as
+well. On the device every stale read classified so far was of an internal child, 6 of 6
+(`BLAS_REFIT_BATCH_PLAN.md` §3.7).
+
+The refit-only update of a built tree (`graph_bvh_refit_lbvh`, the GPU fluid's forcing tree above a
+sort tile) has no emission ahead of its climb, so its prepare pass (`shader_bvh_refit_prepare`), the
+dispatch before the climb, seeds each internal node as it rebuilds that node's children's parents. It
+binds a deferred list and records the same indirect repair: three dispatches. On the RTX 4090 at
+20,000 leaves, refits alternating with a pre-R1 twin (no seed, no repair) in every submission: 0
+stale in 96,000 against the twin's 92 in 96,000, with the repair firing in 129. The seed moved the
+prepare from 2.99 to 3.62 µs (mean, 1 µs timer), the empty repair costs 2.64 µs, and the sum of
+passes went from 19.5 to 22.7 µs.
+
+R1's per-level-dispatch fallback was not built. It needs a dispatch per tree level. The 100,000-leaf
+tree is 22 levels deep, and the recorder would have to record the worst case, about 64, because the
+depth is only known on the device. At the measured 3.1 µs for an empty indirect dispatch that is
+70–200 µs, against the ~8 µs below.
+
+**Measured, RTX 4090, on the device:**
+
+- **Stale rate, refit only, the real kernels before and after:** 23 stale refits in 48,000
+  before, 0 in 48,000 after. The fallback fired 62 times in those 48,000, and every refit came out
+  exact.
+- **The skinned batch, before and after:** eight UV spheres, about 45,000 leaves, two poses
+  alternated over the meshlet data. Before, 11 stale refits in 48,000, at least 9 of them real
+  holes, a child escaping its parent; one genuine stale node per bad tree. In the 32,000 that were
+  classified, all 6 stale children were internal and none a leaf. After, 0 in 48,000. At that
+  size the batch's own refit pass is unchanged (p10 about 38 µs either way); the seeding pass adds
+  10.2 µs at p10 (16–19 µs median) and the empty repair 3.1 µs.
+- **Equivalence:** the fix's trees equal the old recorder's, word for word, at 1, 2, 16, 1,000,
+  1,024, 1,025, 4,096, 20,000 and 100,000 leaves.
+- **Harness:** `bvh_repro` "Run all", three times: 118 PASS, 0 FAIL.
+- **Cost:** per-pass timestamps, medians of 320 builds per arm, builds alternating between the old
+  recorder and the new in one page, with other sessions holding the GPU at about 50%:
+
+  | | before | after |
+  |---|---|---|
+  | 100,000 leaves, sum of passes | 140.3 µs | 148.5 µs (+8.2, +5.8%) |
+  | … emission | 25.6 µs | 27.7 µs |
+  | … refit | 41.0 µs | 43.0 µs |
+  | … repair, empty | — | 3.1 µs |
+  | 1,024 leaves, one workgroup | 50.2 µs | 54.3 µs (+4.1) |
+  | 128 leaves, one workgroup | 12.3 µs | 13.3 µs (+1.0) |
+
+  The gaps between passes are not inside these spans; the extra pass adds one.
+
+**Specs.** `bvh_refit_late_stores.spec.js` runs emission, the refit and the repair on the emulator
+with node cells whose stores reach other invocations only when the dispatch ends — a visibility the
+memory model allows, and the one the device showed. It holds the tree exact for one late store, the
+harness's case, and for none landing before the end. `shader_blas_refit_batch.spec.js` does the same
+for a batch frame. On the old kernels the harness's case leaves the parent exactly 3.0 short, and the
+batch leaves children outside their parents.
 
 ---
 
@@ -1087,7 +1200,10 @@ through `array<atomic<u32>>` accesses only, never plain loads/stores gated by an
 strictest formulation available in WGSL; (d) the browser verify harness asserts structural
 validity per platform, and a formally-clean fallback (iterative per-level dispatches —
 inter-dispatch ordering is guaranteed by WebGPU) is sketched and cheap to add if a platform ever
-misbehaves. Not hand-waved: named, bounded, fallback designed.
+misbehaves. Not hand-waved: named, bounded, fallback designed. **Update, 2026-10-16:** the RTX 4090
+does misbehave, about one refit in 2,000, and (c) alone does not prevent it. The landed answer is
+not the per-level fallback: emission seeds the bounds, the climb defers what it cannot see, and a
+repair dispatch grows it (§4.8).
 
 **AR4 — "The 32-slot traversal stack is shared TLAS+BLAS and your LBVH can exceed the depth the
 current balanced trees keep. Silent geometry drop-out is the failure mode."** The sharpest finding
@@ -1162,7 +1278,7 @@ toward AR2's alternative rather than against it.
 
 | # | risk | mitigation |
 |---|---|---|
-| R1 | WGSL relaxed-atomic cross-workgroup visibility (refit/treelet climbs) is beyond formal guarantees | All-atomic access discipline; no unbounded waiting anywhere (arrival-counter losers exit); per-platform structural verification in the harness; per-level-dispatch fallback designed (inter-dispatch ordering is spec-guaranteed); precedent: csdldf ships on the same visibility assumption. See AR3. **Observed once, 2026-10-16**: one stale-sibling fit in the multi-pass refit, in one harness run on the RTX 4090; not reproduced since (§4.7). |
+| R1 | WGSL relaxed-atomic cross-workgroup visibility (refit/treelet climbs) is beyond formal guarantees | **Realised and closed for the refit, 2026-10-16** (§4.8): about one refit in 2,000 at 20,000 leaves on the RTX 4090 read a sibling's box before it arrived, always across workgroups. Emission seeds the bounds with `BVH_UNFITTED`, the climb defers a node it cannot see whole, and an indirect repair grows it after the dispatch boundary. This is exact, with no waits, and costs +8 µs at 100,000 leaves. The same climb serves the single-workgroup build and the skinned BLAS batch, which seed and repair too. Treelet climbs, when they come, inherit the discipline: a read across an in-dispatch handoff must be able to see that it read too early. |
 | R2 | The sort is new engineering on phase 1's critical path (the fallback-less port was removed; nothing shippable exists today) | v1 deliberately assembled from proven parts — `graph_prefix_scan_csdldf` reused as-is for the spine, ranking/scatter machinery adapted from the deleted port in history, interface kept; ships and verifies before the builder consumes it; single-pass v2 deferred behind a measurement. See AR7, §4.2.1. |
 | R3 | ~~Phase-1 quality dip (raw LBVH vs treelet-optimised dynamic tree)~~ | **Retired.** Measured (§4.6): a dip at ~100 instances that does not reach the frame, and an improvement on every axis at 20k. See AR1. |
 | R4 | Tree depth vs the shared 32-slot traversal stack; occlusion path truncates silently | Depth histogram in verify harness + phase gates; treelets shorten chains; `BVH_TRAVERSAL_STACK_SIZE` bump is a measured last resort. Pre-existing cliff, now instrumented — and **measured**: 22/32 at 100k scattered instances, 20/32 at 20k (§4.6). Not a hypothetical any more. See AR4. |

package/src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_climb.d.ts

@@ -5,27 +5,39 @@
  * each internal node on the way. An internal node has exactly two children, so exactly two climbs
  * pass through it. The first arrival exits — its sibling's subtree is still being fitted, and it
  * has nothing to wait ON: no spin, no assumption about anyone else's forward progress. The second
- * arrival finds both children written and owns the node from here up. That is the whole
- * synchronisation, and it assumes atomic-word visibility rather than any scheduling fairness —
- * which is why it sits on the accepted side of the engine's line on cross-workgroup communication.
+ * arrival owns the node from here up.
  *
- * Every word this climb communicates between workgroups — bounds and counters — goes through
- * atomic accesses only, never a plain load or store gated by an atomic: the strictest formulation
- * WGSL offers, and the discipline the builder plan's R1 rests on.
+ * The second arrival cannot assume both children's boxes have reached it. The counter it took is one
+ * atomic word and the boxes are six others, and WGSL atomics are relaxed: arriving second orders
+ * nothing about the sibling's bound stores. On an RTX 4090 the second arrival read its sibling's
+ * box before the stores landed about once in 2,000 refits at 20,000 leaves, always across
+ * workgroups (R1 in GPU_BVH_BUILD_PLAN.md). So every bound word the climb reads must have been
+ * seeded with {@link BVH_UNFITTED} by an earlier dispatch. A read then returns either the store or
+ * the seed, and a seed word is a box the second arrival can see has not arrived. It reads as the
+ * empty box's word, the node is fitted from what can be seen and climbed on, and the node goes on
+ * the deferred list. The repair ({@link chunk_bvh_refit_repair}), a dispatch later, grows it and
+ * every node above it by what was missing. Nothing waits on anyone, and nothing assumes a store
+ * reached anyone before its dispatch ended.
+ *
+ * Every word this climb communicates between invocations — bounds, counters, the deferred list —
+ * goes through atomic accesses only.
  *
  * Three offsets, so one climb serves a tree that owns its arrays and a tree that is a tenant in
  * somebody else's. `node_word_offset` is where node 0 starts in the bound node buffer, in words;
  * `parents_offset` is where this tree's parents start in the bound parents array;
  * `counters_offset` is where this tree's arrival counters start in the bound counters array. A
  * build passes 0 for all three. A refit of a tree living in a shared arena passes its position in
- * each — including the counters, when one buffer holds the counters of many trees at once.
+ * each — including the counters, when one buffer holds the counters of many trees at once. A
+ * deferred node is listed with its tree's node and parents offsets, so one list serves many trees.
  *
  * The counters must be zero when the climb starts, one per node of the tree: a leftover count lets
- * a first arrival believe it is a second and fit a node from a sibling nothing has written yet.
+ * a first arrival believe it is a second and fit a node from a sibling nothing has written yet. The
+ * deferred list's header must be zero too ({@link BVH_REFIT_DEFERRED_COUNT_WORD}).
  *
- * Assumes the module-scope bindings `parents` (`array<u32>`), `counters` (`array<atomic<u32>>`)
- * and `nodes` (`array<atomic<u32>>`, the node buffer bound flat) — by name, the same way
- * `read_meshlet_triangle_bounds` assumes `meshlet_metadata`.
+ * Assumes the module-scope bindings `parents` (`array<u32>`), `counters` (`array<atomic<u32>>`),
+ * `nodes` (`array<atomic<u32>>`, the node buffer bound flat) and `refit_deferred`
+ * (`array<atomic<u32>>`) — by name, the same way `read_meshlet_triangle_bounds` assumes
+ * `meshlet_metadata`.
  *
  * @type {CodeChunk}
  */

package/src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_climb.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"chunk_bvh_refit_climb.d.ts","sourceRoot":"","sources":["../../../../../../src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_climb.js"],"names":[],"mappings":"AAIA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8BG;AACH,oCAFU,SAAS,CAoEhB;0BArGuB,oCAAoC"}
+{"version":3,"file":"chunk_bvh_refit_climb.d.ts","sourceRoot":"","sources":["../../../../../../src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_climb.js"],"names":[],"mappings":"AAQA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA0CG;AACH,oCAFU,SAAS,CAyHhB;0BA1KuB,oCAAoC"}

package/src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_climb.js

@@ -1,102 +1,171 @@
-import { CodeChunk } from "../../shader/compiler/CodeChunk.js";
-import { BVH_NULL_NODE_DEFINITION } from "../../shader/chunk/bvh/BVH_NULL_NODE_DEFINITION.js";
-import { BVH_NODE_LAYOUT_DEFINITION } from "./BVH_NODE_LAYOUT_DEFINITION.js";
-
-/**
- * The bottom-up AABB refit climb, as a function a kernel calls once per leaf.
- *
- * Starting above `leaf_node`, the caller climbs the `parents` chain, taking an arrival counter at
- * each internal node on the way. An internal node has exactly two children, so exactly two climbs
- * pass through it. The first arrival exits — its sibling's subtree is still being fitted, and it
- * has nothing to wait ON: no spin, no assumption about anyone else's forward progress. The second
- * arrival finds both children written and owns the node from here up. That is the whole
- * synchronisation, and it assumes atomic-word visibility rather than any scheduling fairness —
- * which is why it sits on the accepted side of the engine's line on cross-workgroup communication.
- *
- * Every word this climb communicates between workgroups — bounds and counters — goes through
- * atomic accesses only, never a plain load or store gated by an atomic: the strictest formulation
- * WGSL offers, and the discipline the builder plan's R1 rests on.
- *
- * Three offsets, so one climb serves a tree that owns its arrays and a tree that is a tenant in
- * somebody else's. `node_word_offset` is where node 0 starts in the bound node buffer, in words;
- * `parents_offset` is where this tree's parents start in the bound parents array;
- * `counters_offset` is where this tree's arrival counters start in the bound counters array. A
- * build passes 0 for all three. A refit of a tree living in a shared arena passes its position in
- * each — including the counters, when one buffer holds the counters of many trees at once.
- *
- * The counters must be zero when the climb starts, one per node of the tree: a leftover count lets
- * a first arrival believe it is a second and fit a node from a sibling nothing has written yet.
- *
- * Assumes the module-scope bindings `parents` (`array<u32>`), `counters` (`array<atomic<u32>>`)
- * and `nodes` (`array<atomic<u32>>`, the node buffer bound flat) — by name, the same way
- * `read_meshlet_triangle_bounds` assumes `meshlet_metadata`.
- *
- * @type {CodeChunk}
- */
-export const chunk_bvh_refit_climb = CodeChunk.from(
-    //language=WGSL
-    `
-fn bvh_node_read_bounds_low(node_word: u32) -> vec3<f32> {
-    return vec3<f32>(
-        bitcast<f32>(atomicLoad(&nodes[node_word + BVH_NODE_BOUNDS])),
-        bitcast<f32>(atomicLoad(&nodes[node_word + BVH_NODE_BOUNDS + 1u])),
-        bitcast<f32>(atomicLoad(&nodes[node_word + BVH_NODE_BOUNDS + 2u])),
-    );
-}
-
-fn bvh_node_read_bounds_high(node_word: u32) -> vec3<f32> {
-    return vec3<f32>(
-        bitcast<f32>(atomicLoad(&nodes[node_word + BVH_NODE_BOUNDS + 3u])),
-        bitcast<f32>(atomicLoad(&nodes[node_word + BVH_NODE_BOUNDS + 4u])),
-        bitcast<f32>(atomicLoad(&nodes[node_word + BVH_NODE_BOUNDS + 5u])),
-    );
-}
-
-fn bvh_node_store_bounds(node_word: u32, low: vec3<f32>, high: vec3<f32>) {
-    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS], bitcast<u32>(low.x));
-    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 1u], bitcast<u32>(low.y));
-    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 2u], bitcast<u32>(low.z));
-    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 3u], bitcast<u32>(high.x));
-    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 4u], bitcast<u32>(high.y));
-    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 5u], bitcast<u32>(high.z));
-}
-
-fn bvh_refit_climb(node_word_offset: u32, parents_offset: u32, counters_offset: u32, leaf_node: u32) {
-    // Start above the leaf and climb. Which node that leaf is comes from the caller rather than
-    // from a layout this function assumes, so a tree that was built and a tree that was copied
-    // refit alike.
-    var node = parents[parents_offset + leaf_node];
-
-    loop {
-        if (node == BVH_NULL_NODE) {
-            // climbed past the root
-            break;
-        }
-
-        let arrivals = atomicAdd(&counters[counters_offset + node], 1u);
-
-        if (arrivals == 0u) {
-            // first arrival: the sibling subtree is still being fitted, and this is not a wait
-            break;
-        }
-
-        let node_word = node_word_offset + node * BVH_NODE_STRIDE;
-
-        let child_1 = atomicLoad(&nodes[node_word + BVH_NODE_CHILD_1]);
-        let child_2 = atomicLoad(&nodes[node_word + BVH_NODE_CHILD_2]);
-
-        let word_1 = node_word_offset + child_1 * BVH_NODE_STRIDE;
-        let word_2 = node_word_offset + child_2 * BVH_NODE_STRIDE;
-
-        let low = min(bvh_node_read_bounds_low(word_1), bvh_node_read_bounds_low(word_2));
-        let high = max(bvh_node_read_bounds_high(word_1), bvh_node_read_bounds_high(word_2));
-
-        bvh_node_store_bounds(node_word, low, high);
-
-        node = parents[parents_offset + node];
-    }
-}
-`, [
-    BVH_NULL_NODE_DEFINITION,
-    BVH_NODE_LAYOUT_DEFINITION,
-]);
+import { CodeChunk } from "../../shader/compiler/CodeChunk.js";
+import { BVH_NULL_NODE_DEFINITION } from "../../shader/chunk/bvh/BVH_NULL_NODE_DEFINITION.js";
+import {
+    WEBGPU_GUARANTEED_MAX_COMPUTE_WORKGROUPS_PER_DIMENSION
+} from "../indirect/WEBGPU_GUARANTEED_MAX_COMPUTE_WORKGROUPS_PER_DIMENSION.js";
+import { BVH_NODE_LAYOUT_DEFINITION } from "./BVH_NODE_LAYOUT_DEFINITION.js";
+import { BVH_REFIT_DEFERRED_DEFINITION } from "./BVH_REFIT_DEFERRED_DEFINITION.js";
+
+/**
+ * The bottom-up AABB refit climb, as a function a kernel calls once per leaf.
+ *
+ * Starting above `leaf_node`, the caller climbs the `parents` chain, taking an arrival counter at
+ * each internal node on the way. An internal node has exactly two children, so exactly two climbs
+ * pass through it. The first arrival exits — its sibling's subtree is still being fitted, and it
+ * has nothing to wait ON: no spin, no assumption about anyone else's forward progress. The second
+ * arrival owns the node from here up.
+ *
+ * The second arrival cannot assume both children's boxes have reached it. The counter it took is one
+ * atomic word and the boxes are six others, and WGSL atomics are relaxed: arriving second orders
+ * nothing about the sibling's bound stores. On an RTX 4090 the second arrival read its sibling's
+ * box before the stores landed about once in 2,000 refits at 20,000 leaves, always across
+ * workgroups (R1 in GPU_BVH_BUILD_PLAN.md). So every bound word the climb reads must have been
+ * seeded with {@link BVH_UNFITTED} by an earlier dispatch. A read then returns either the store or
+ * the seed, and a seed word is a box the second arrival can see has not arrived. It reads as the
+ * empty box's word, the node is fitted from what can be seen and climbed on, and the node goes on
+ * the deferred list. The repair ({@link chunk_bvh_refit_repair}), a dispatch later, grows it and
+ * every node above it by what was missing. Nothing waits on anyone, and nothing assumes a store
+ * reached anyone before its dispatch ended.
+ *
+ * Every word this climb communicates between invocations — bounds, counters, the deferred list —
+ * goes through atomic accesses only.
+ *
+ * Three offsets, so one climb serves a tree that owns its arrays and a tree that is a tenant in
+ * somebody else's. `node_word_offset` is where node 0 starts in the bound node buffer, in words;
+ * `parents_offset` is where this tree's parents start in the bound parents array;
+ * `counters_offset` is where this tree's arrival counters start in the bound counters array. A
+ * build passes 0 for all three. A refit of a tree living in a shared arena passes its position in
+ * each — including the counters, when one buffer holds the counters of many trees at once. A
+ * deferred node is listed with its tree's node and parents offsets, so one list serves many trees.
+ *
+ * The counters must be zero when the climb starts, one per node of the tree: a leftover count lets
+ * a first arrival believe it is a second and fit a node from a sibling nothing has written yet. The
+ * deferred list's header must be zero too ({@link BVH_REFIT_DEFERRED_COUNT_WORD}).
+ *
+ * Assumes the module-scope bindings `parents` (`array<u32>`), `counters` (`array<atomic<u32>>`),
+ * `nodes` (`array<atomic<u32>>`, the node buffer bound flat) and `refit_deferred`
+ * (`array<atomic<u32>>`) — by name, the same way `read_meshlet_triangle_bounds` assumes
+ * `meshlet_metadata`.
+ *
+ * @type {CodeChunk}
+ */
+export const chunk_bvh_refit_climb = CodeChunk.from(
+    //language=WGSL
+    `
+fn bvh_node_store_bounds(node_word: u32, low: vec3<f32>, high: vec3<f32>) {
+    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS], bitcast<u32>(low.x));
+    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 1u], bitcast<u32>(low.y));
+    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 2u], bitcast<u32>(low.z));
+    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 3u], bitcast<u32>(high.x));
+    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 4u], bitcast<u32>(high.y));
+    atomicStore(&nodes[node_word + BVH_NODE_BOUNDS + 5u], bitcast<u32>(high.z));
+}
+
+// A child's box as the second arrival sees it, and whether all of it had arrived.
+struct BVHRefitSeenBox {
+    low: vec3<f32>,
+    high: vec3<f32>,
+    complete: bool,
+}
+
+fn bvh_refit_seen_word(word: u32, high: bool, complete: ptr<function, bool>) -> f32 {
+    let raw = atomicLoad(&nodes[word]);
+
+    if (raw != BVH_UNFITTED) {
+        return bitcast<f32>(raw);
+    }
+
+    *complete = false;
+
+    // The empty box's word, +inf for a minimum and -inf for a maximum, so the union ignores it. Made
+    // from the seed just read, whose exponent bits are all set, because WGSL will not fold an
+    // infinity out of a constant.
+    return bitcast<f32>((raw & 0x7F800000u) | select(0u, 0x80000000u, high));
+}
+
+fn bvh_refit_seen_box(node_word: u32) -> BVHRefitSeenBox {
+    var complete = true;
+
+    let low = vec3<f32>(
+        bvh_refit_seen_word(node_word + BVH_NODE_BOUNDS, false, &complete),
+        bvh_refit_seen_word(node_word + BVH_NODE_BOUNDS + 1u, false, &complete),
+        bvh_refit_seen_word(node_word + BVH_NODE_BOUNDS + 2u, false, &complete),
+    );
+
+    let high = vec3<f32>(
+        bvh_refit_seen_word(node_word + BVH_NODE_BOUNDS + 3u, true, &complete),
+        bvh_refit_seen_word(node_word + BVH_NODE_BOUNDS + 4u, true, &complete),
+        bvh_refit_seen_word(node_word + BVH_NODE_BOUNDS + 5u, true, &complete),
+    );
+
+    return BVHRefitSeenBox(low, high, complete);
+}
+
+// The node goes on the deferred list for the repair, which runs when every store of this dispatch
+// has landed.
+fn bvh_refit_defer(node_word_offset: u32, parents_offset: u32, node: u32) {
+    let k = atomicAdd(&refit_deferred[BVH_REFIT_DEFERRED_COUNT], 1u);
+
+    let entry = BVH_REFIT_DEFERRED_ENTRIES + k * BVH_REFIT_DEFERRED_ENTRY_WORDS;
+
+    atomicStore(&refit_deferred[entry], node_word_offset);
+    atomicStore(&refit_deferred[entry + 1u], parents_offset);
+    atomicStore(&refit_deferred[entry + 2u], node);
+
+    // The repair's dispatch command, folded the way the engine's command builders fold one: the
+    // entry that opens a workgroup's worth raises x, and y once x is full. Raised rather than
+    // counted, so it comes out the same whatever order the entries were taken in.
+    if (k % BVH_REFIT_REPAIR_WORKGROUP_SIZE == 0u) {
+        const WIDTH = ${WEBGPU_GUARANTEED_MAX_COMPUTE_WORKGROUPS_PER_DIMENSION}u;
+
+        let group = k / BVH_REFIT_REPAIR_WORKGROUP_SIZE;
+
+        atomicMax(&refit_deferred[0], min(group + 1u, WIDTH));
+        atomicMax(&refit_deferred[1], min(group / WIDTH + 1u, WIDTH));
+        atomicMax(&refit_deferred[2], group / (WIDTH * WIDTH) + 1u);
+    }
+}
+
+fn bvh_refit_climb(node_word_offset: u32, parents_offset: u32, counters_offset: u32, leaf_node: u32) {
+    // Start above the leaf and climb. Which node that leaf is comes from the caller rather than
+    // from a layout this function assumes, so a tree that was built and a tree that was copied
+    // refit alike.
+    var node = parents[parents_offset + leaf_node];
+
+    loop {
+        if (node == BVH_NULL_NODE) {
+            // climbed past the root
+            break;
+        }
+
+        let arrivals = atomicAdd(&counters[counters_offset + node], 1u);
+
+        if (arrivals == 0u) {
+            // first arrival: the sibling subtree is still being fitted, and this is not a wait
+            break;
+        }
+
+        let node_word = node_word_offset + node * BVH_NODE_STRIDE;
+
+        let child_1 = atomicLoad(&nodes[node_word + BVH_NODE_CHILD_1]);
+        let child_2 = atomicLoad(&nodes[node_word + BVH_NODE_CHILD_2]);
+
+        let seen_1 = bvh_refit_seen_box(node_word_offset + child_1 * BVH_NODE_STRIDE);
+        let seen_2 = bvh_refit_seen_box(node_word_offset + child_2 * BVH_NODE_STRIDE);
+
+        if (!(seen_1.complete && seen_2.complete)) {
+            // What did arrive is still part of this node's box, so the climb fits it and goes on;
+            // the repair adds the rest, here and above.
+            bvh_refit_defer(node_word_offset, parents_offset, node);
+        }
+
+        bvh_node_store_bounds(node_word, min(seen_1.low, seen_2.low), max(seen_1.high, seen_2.high));
+
+        node = parents[parents_offset + node];
+    }
+}
+`, [
+    BVH_NULL_NODE_DEFINITION,
+    BVH_NODE_LAYOUT_DEFINITION,
+    BVH_REFIT_DEFERRED_DEFINITION,
+]);

package/src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_repair.d.ts

@@ -0,0 +1,23 @@
+/**
+ * Repair of one node the refit climb ({@link chunk_bvh_refit_climb}) deferred: the union of its
+ * children as they stand now, grown into the node and into every node above it.
+ *
+ * Call it where every store of the climb is visible — in a later dispatch, or in the same
+ * workgroup after a storage barrier. Every child has been fitted by then, if only from part of its
+ * own children, so no word reads as {@link BVH_UNFITTED}.
+ *
+ * Why growing is enough, and exact. Whatever the climb fitted holds no more than the true box,
+ * because a word that had not arrived read as the empty box's. The union grown here holds no more
+ * than the true box either, for the same reason. And a leaf's box is missing from a node only if,
+ * somewhere between the two, a node read the child on that path before it arrived — and that node
+ * was deferred, so its repair grows the whole path by a box that holds the leaf. Growth is a min
+ * and a max per word, which commute, so repairs that meet on the way up need no order between them.
+ *
+ * Assumes the module-scope bindings `parents` (`array<u32>`) and `nodes` (`array<atomic<u32>>`),
+ * as the climb does.
+ *
+ * @type {CodeChunk}
+ */
+export const chunk_bvh_refit_repair: CodeChunk;
+import { CodeChunk } from "../../shader/compiler/CodeChunk.js";
+//# sourceMappingURL=chunk_bvh_refit_repair.d.ts.map

package/src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_repair.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"chunk_bvh_refit_repair.d.ts","sourceRoot":"","sources":["../../../../../../src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_repair.js"],"names":[],"mappings":"AAMA;;;;;;;;;;;;;;;;;;;GAmBG;AACH,qCAFU,SAAS,CAuDhB;0BA/EuB,oCAAoC"}

package/src/shade/renderer/gpu_primitive/bvh/chunk_bvh_refit_repair.js

@@ -0,0 +1,80 @@
+import { CodeChunk } from "../../shader/compiler/CodeChunk.js";
+import { BVH_NULL_NODE_DEFINITION } from "../../shader/chunk/bvh/BVH_NULL_NODE_DEFINITION.js";
+import { chunk_atomic_max_f32_storage } from "../../shader/chunk/atomic/chunk_atomic_max_f32_storage.js";
+import { chunk_atomic_min_f32_storage } from "../../shader/chunk/atomic/chunk_atomic_min_f32_storage.js";
+import { BVH_NODE_LAYOUT_DEFINITION } from "./BVH_NODE_LAYOUT_DEFINITION.js";
+
+/**
+ * Repair of one node the refit climb ({@link chunk_bvh_refit_climb}) deferred: the union of its
+ * children as they stand now, grown into the node and into every node above it.
+ *
+ * Call it where every store of the climb is visible — in a later dispatch, or in the same
+ * workgroup after a storage barrier. Every child has been fitted by then, if only from part of its
+ * own children, so no word reads as {@link BVH_UNFITTED}.
+ *
+ * Why growing is enough, and exact. Whatever the climb fitted holds no more than the true box,
+ * because a word that had not arrived read as the empty box's. The union grown here holds no more
+ * than the true box either, for the same reason. And a leaf's box is missing from a node only if,
+ * somewhere between the two, a node read the child on that path before it arrived — and that node
+ * was deferred, so its repair grows the whole path by a box that holds the leaf. Growth is a min
+ * and a max per word, which commute, so repairs that meet on the way up need no order between them.
+ *
+ * Assumes the module-scope bindings `parents` (`array<u32>`) and `nodes` (`array<atomic<u32>>`),
+ * as the climb does.
+ *
+ * @type {CodeChunk}
+ */
+export const chunk_bvh_refit_repair = CodeChunk.from(
+    //language=WGSL
+    `
+fn bvh_refit_repair_read_low(bounds_word: u32) -> vec3<f32> {
+    return vec3<f32>(
+        bitcast<f32>(atomicLoad(&nodes[bounds_word])),
+        bitcast<f32>(atomicLoad(&nodes[bounds_word + 1u])),
+        bitcast<f32>(atomicLoad(&nodes[bounds_word + 2u])),
+    );
+}
+
+fn bvh_refit_repair_read_high(bounds_word: u32) -> vec3<f32> {
+    return vec3<f32>(
+        bitcast<f32>(atomicLoad(&nodes[bounds_word + 3u])),
+        bitcast<f32>(atomicLoad(&nodes[bounds_word + 4u])),
+        bitcast<f32>(atomicLoad(&nodes[bounds_word + 5u])),
+    );
+}
+
+fn bvh_refit_repair(node_word_offset: u32, parents_offset: u32, node: u32) {
+    let node_word = node_word_offset + node * BVH_NODE_STRIDE;
+
+    let word_1 = node_word_offset + atomicLoad(&nodes[node_word + BVH_NODE_CHILD_1]) * BVH_NODE_STRIDE + BVH_NODE_BOUNDS;
+    let word_2 = node_word_offset + atomicLoad(&nodes[node_word + BVH_NODE_CHILD_2]) * BVH_NODE_STRIDE + BVH_NODE_BOUNDS;
+
+    let low = min(bvh_refit_repair_read_low(word_1), bvh_refit_repair_read_low(word_2));
+    let high = max(bvh_refit_repair_read_high(word_1), bvh_refit_repair_read_high(word_2));
+
+    var current = node;
+
+    loop {
+        if (current == BVH_NULL_NODE) {
+            // grown past the root
+            break;
+        }
+
+        let bounds_word = node_word_offset + current * BVH_NODE_STRIDE + BVH_NODE_BOUNDS;
+
+        atomic_min_f32_storage(&nodes[bounds_word], low.x);
+        atomic_min_f32_storage(&nodes[bounds_word + 1u], low.y);
+        atomic_min_f32_storage(&nodes[bounds_word + 2u], low.z);
+        atomic_max_f32_storage(&nodes[bounds_word + 3u], high.x);
+        atomic_max_f32_storage(&nodes[bounds_word + 4u], high.y);
+        atomic_max_f32_storage(&nodes[bounds_word + 5u], high.z);
+
+        current = parents[parents_offset + current];
+    }
+}
+`, [
+    BVH_NULL_NODE_DEFINITION,
+    BVH_NODE_LAYOUT_DEFINITION,
+    chunk_atomic_min_f32_storage,
+    chunk_atomic_max_f32_storage,
+]);

package/src/shade/renderer/gpu_primitive/bvh/graph_bvh_build_lbvh.d.ts

@@ -15,9 +15,11 @@ export function enquire_bvh_build_lbvh_pipelines(enquiry: PipelineEnquiry): void
  *
  * Two recordings, chosen by `element_count` and nothing else. Up to one sort tile of leaves
  * ({@link SORT_TILE_SIZE}), the whole build is one dispatch of one workgroup,
- * {@link shader_bvh_build_single_workgroup}. Above it the build is 19 dispatches: Morton, the four
- * digit passes of {@link graph_radix_sort}, emission and refit. Over a few leaves each of those 19 is
- * nearly empty and costs its fixed overhead, and a rebuild per spawn or despawn pays it every time.
+ * {@link shader_bvh_build_single_workgroup}. Above it the build is 20 dispatches: Morton, the four
+ * digit passes of {@link graph_radix_sort}, emission, refit, and the refit's repair
+ * ({@link shader_bvh_refit_repair}), which is indirect and empty unless a refit climb read a box
+ * before it arrived. Over a few leaves each of those is nearly empty and costs its fixed overhead,
+ * and a rebuild per spawn or despawn pays it every time.
  * Both write the same tree and the same Morton order, word for word. The single-workgroup path sorts
  * by code and then leaf index, which is the stable radix sort's order.
  *

package/src/shade/renderer/gpu_primitive/bvh/graph_bvh_build_lbvh.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"graph_bvh_build_lbvh.d.ts","sourceRoot":"","sources":["../../../../../../src/shade/renderer/gpu_primitive/bvh/graph_bvh_build_lbvh.js"],"names":[],"mappings":"AAYA;;GAEG;AAEH;;;;GAIG;AACH,0DAFW,eAAe,QAQzB;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6CG;AACH;IAhB+B,KAAK;IACT,WAAW,EAA3B,MAAM;IACU,YAAY,EAA5B,MAAM;IACU,YAAY,EAA5B,MAAM;IACU,aAAa,EAA7B,MAAM;IACU,KAAK,EAArB,MAAM;IACW,gBAAgB,GAAjC,MAAM;IAGW,gBAAgB,GAAjC,MAAM;IAEJ;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAC,CA6LzD;8BAtPY,OAAO,6CAA6C,EAAE,eAAe"}
+{"version":3,"file":"graph_bvh_build_lbvh.d.ts","sourceRoot":"","sources":["../../../../../../src/shade/renderer/gpu_primitive/bvh/graph_bvh_build_lbvh.js"],"names":[],"mappings":"AAcA;;GAEG;AAEH;;;;GAIG;AACH,0DAFW,eAAe,QASzB;AAED;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA+CG;AACH;IAhB+B,KAAK;IACT,WAAW,EAA3B,MAAM;IACU,YAAY,EAA5B,MAAM;IACU,YAAY,EAA5B,MAAM;IACU,aAAa,EAA7B,MAAM;IACU,KAAK,EAArB,MAAM;IACW,gBAAgB,GAAjC,MAAM;IAGW,gBAAgB,GAAjC,MAAM;IAEJ;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAC,CAsOzD;8BAlSY,OAAO,6CAA6C,EAAE,eAAe"}