@woosh/meep-engine 3.60.4 → 3.60.6

package/src/engine/graphics/render/frame_graph/FRAME_GRAPH_RECOMMENDATIONS.md

@@ -1,235 +0,0 @@
-# FrameGraph — recommendations
-
-Derived from `shade/renderer/extension/RENDER_EXTENSION_DESIGN.md`. Only items where the design either
-asks for an engine change (§13), or is forced to *route around* something the graph gets wrong (§4).
-
-Ordered by value. Everything here is additive or a bug fix; none of it changes how a pass records.
-
-**Status.** Items 1, 2, 3, 4, 6 and 7 have landed. **Item 5 is withdrawn** — the requirement behind
-it was wrong, not merely unbuilt. Nothing on this list is open, and the extension design asks
-`FrameGraph` for nothing further. The "noted, not now" list at the bottom is performance and naming,
-untouched on purpose — functionality and design first.
-
-A later deep audit (2026-09-04) found defects this round did not cover, and its correctness and
-invariant fixes are recorded in `CORRECTNESS_FIXES_2026_09_04.md`. Four of them change the standing
-picture here and are noted in place below.
-
----
-
-## 1. Make the lifecycle one-shot and assert it — **DONE**
-
-`compile()` was **not idempotent**: it reset `pass.ref_count` but accumulated
-`resource_node.ref_count`. Compiling twice inflated every read count, so nothing culled. And
-`execute()` did not know whether `compile()` had run — skip it and every `ref_count` is 0, so the
-whole frame culls itself and **nothing says so**.
-
-Landed as `FrameGraphState`: `Recording` → `Compiled` → `Executed`, monotonic, guarded by
-`assert.equal` at every entry point that mutates the graph (`add`, `create_resource`,
-`import_resource`, `clone_resource`, all four `FramePassBuilder` declarations) plus `compile()` and
-`execute()`. `FrameGraph.state` is public so `FramePassBuilder` can assert against it.
-
-Every caller was already one-shot (`ShadeGPUCommandContext.encodeGraph` is the only production
-path), so this only names a contract that was already being kept. Asserts are stripped in
-production, so the checks cost nothing in a shipped build; the two state assignments survive and are
-the whole runtime price.
-
-This also deletes the §4 caveat that the extension layer "must never compile the graph itself" — the
-graph says so now.
-
-## 2. Implement `validate()` — read-before-write — **DONE**
-
-`validate()` was a stub returning `true`, carrying `// TODO check for "read before write"`, called
-from nowhere. That TODO was the one check the graph was missing.
-
-`execute()` iterates in **insertion order** and never topologically sorts (design §4), so two reads
-can silently return the wrong contents. `validate()` now reports both, and `compile()` asserts on it —
-the honest answer to §4 is not to sort but to **prove the insertion order is a valid topological
-order**:
-
-- **read before write** — a pass reads a resource node written by a pass recorded after it.
-- **superseded read** — a pass reads a resource node an earlier pass has already replaced. A
-  `ResourceEntry` owns one concrete resource, so an older node is not a snapshot of the old
-  contents; it is the same resource holding the new ones.
-
-One shape is deliberately excluded, and the reason sits on `validate()` itself so that nobody
-"completes" it later and breaks the dev build: reading a transient no pass writes — a resource can
-be created by a pass that only reserves it, and the reader is then the pass about to fill it.
-
-A second exclusion — re-attaching a superseded resource the same pass then writes — shipped with
-this check and has since been **removed**, because the one thing it existed for was the renderer
-defect in item 3. Accumulating into a shared target is not an exemption: each pass has to re-attach
-the handle its predecessor produced, and once the rasterizer did that the exemption had nothing left
-to excuse.
-
-Verified against the real 61-pass `Shading` graph through `probeShadePassInjection`, with injected
-passes at every phase that accepts one. Device-free tests cover all four outcomes.
-
-## 3. Renderer defect found by item 2: colour targets are not threaded — **DONE**
-
-`graph_rasterize_scene.js` threaded the depth handle through the two-phase opaque rasterization —
-`gr_texture_depth = gr_depth_opaque_partial`, then `gr_texture_depth = gr_depth_opaque_full` — but
-handed the same `raster_color_targets` array, still at version 0, to every phase.
-`graph_rasterize_partial_opaque` and `graph_rasterize_remaining_opaque` returned the new depth
-handle and never the new colour handles, and the two alpha-tested calls below them discarded their
-results entirely. `graph_rasterize_scene_fast.js` had the same shape: its opaque loop threaded both
-handles, and the alpha-tested calls after it dropped theirs, so the function returned the
-pre-alpha-tested handles.
-
-Observed in the live `Shading` graph: pass 14 `rasterize` read `out_color/0` v0 and wrote v1; pass
-28 `rasterize` read `viz/depth` **v1** (threaded) but `out_color/0` **v0** (not).
-
-Harmless as it stood — each of those passes writes an imported resource, so it is a side-effect pass
-that always executes, and insertion order gave the right picture. What it cost is that the colour
-dependency between the rasterization passes was not a graph fact: `out_color/* v1` was a dead node
-nobody read, and anything later reading `raster_color_targets` got the state from before any
-rasterization happened. The moment one of those passes stopped writing an imported resource, culling
-would remove work the picture needs.
-
-Landed by completing the pattern depth already had. The two opaque helpers now return
-`color_targets` alongside `depth`, and `graph_rasterize_meshes_alpha_tested` returns
-`{executed, depth, color_targets}` instead of a bare boolean — handing back the handles it was given
-when it records no pass, so the call sites thread the same way whether or not there is anything
-alpha-tested to draw. Both scene builders thread every return.
-
-In the live `Shading` graph, colour and depth now advance together: pass 14 v0→v1, pass 28 v1→v2,
-and with alpha-tested materials in the scene, `rasterize alpha tested meshes` at v2→v3 (front) and
-v3→v4 (double). The fast path's cube-shadow faces chain the same way. All four graphs a frame
-records — `Shading` plus three `shadows` — validate clean **with the accumulation exemption removed**,
-which is what item 2 above records.
-
-**The threading stopped at the entry point, and the 2026-09-04 audit found where.**
-`graph_rasterize_scene` returned only the bucket buffer, so `Renderer` re-imported the same
-`GPUTextureContext` objects to read what the rasterizer had written — one physical texture, two
-unrelated entry chains, no edge. Fixed in `CORRECTNESS_FIXES_2026_09_04.md` §22: the entry point now
-returns `{buckets, depth, color_targets}`. The same defect was still unfixed in `GPUProbeAtlas` (§19)
-and `GPUSky` (§18).
-
-## 4. Recording scopes — **DONE** (supersedes §13.1's `pass_count`)
-
-§13.1 asks for a `pass_count` getter so the cull diagnostic can recover "the node range extension X
-recorded". A scope is strictly better and cost one field on `FrameGraphNode`:
-
-```js
-graph.push_scope("FogOfWarExtension");
-extension.record(frame);
-graph.pop_scope();
-```
-
-Every pass and resource node created while a scope is open is stamped with it, nested scopes
-included — a `FrameGraphScope` carries its `parent`, so the graph keeps one field rather than a
-stack, and `is_within()` answers "did scope X record this, directly or through a nested one".
-`toString()` gives the path, `shadows/light 0`.
-
-What that buys:
-
-- **§1.2 attribution.** `exportToDot` showed whatever names the `graph_*` helpers happened to give a
-  node, unrelated to who recorded it. Scoped nodes now come out inside `subgraph cluster_N` boxes,
-  nested as the scopes were, and unscoped nodes keep exactly the flat layout they had — a graph
-  that opens no scope exports byte-identically to before.
-- **§5's "everything culled" check.** "Did anything scope X recorded survive" is a stamp query, so
-  it survives interleaved recording, which the index range `pass_count` implies cannot.
-- `compile()` asserts the scope stack came back empty, so an unbalanced `push_scope` is caught at
-  the point it can still be attributed.
-
-`exportToJson` was left alone: it is broken independently of scopes (item 7), and labelling a
-broken exporter is not worth doing before fixing it.
-
-**Nothing opens a scope yet.** The intended caller is the extension registry the design describes,
-which does not exist; the renderer could scope its own sections too, but that is Shade internals.
-The mechanism, the DOT clustering and the tests are here so that whoever builds either one does not
-have to.
-
-## 5. Report culling instead of only encoding it — **WITHDRAWN**
-
-It asked for `pass_count` plus per-pass `culled` state, so that the extension layer could assert on
-"an extension whose entire recorded work was dead". **The requirement was wrong, so the exposure it
-needed is not wanted.** Withdrawn by the sponsor, 2026-08-24; the design doc's §5.1 carries the
-argument.
-
-In short: culling is what a frame graph is for, and a diagnostic that calls it a failure when the
-recorder happens to be an extension invents a distinction that does not exist. It also outlaws the
-speculative producer — a pass publishing a resource for a consumer that may not be registered, which
-the graph correctly elides when the consumer is absent. That is the pattern an open resource
-vocabulary exists to allow.
-
-Culling is already reported where reporting belongs: `exportToJson` carries `culled` per pass and
-`exportToDot` greys culled passes out, so with item 4's scopes a fully-culled caller is a grey
-labelled cluster in a dump. Diagnosis, not enforcement.
-
-**Nothing further is asked of `FrameGraph` by the extension design.** With this withdrawn, its
-engine-change list is empty.
-
-## 6. Assign `ResourceNode.producer` at record time — **DONE**
-
-It was set in `compile()`, which is why §4 had to state that "this handle was produced by the
-extension publishing it" cannot be checked at publication time. It was known earlier: every written
-node comes from `FramePassBuilder.create` or from `clone_resource` inside `write`, both of which
-have the pass in hand. The builder now assigns it, and `compile()` keeps only ref-counting.
-
-Strictly more information, strictly earlier, and the same set of nodes: measured on the live
-124-node `Shading` graph before `compile()` ran, all 89 written nodes carry a producer and no
-unwritten node carries one.
-
-It paid for itself immediately — `validate()` was rebuilding this fact into a `writer_of` typed
-array on every call, and now reads `node.producer` instead, so the array and the loop that filled it
-are gone.
-
-The publication-time check §3.4 wants is now possible, and item 5 becomes cheaper: a culled pass can
-be traced back to the node that lost its last reference.
-
-## 7. `exportToJson()` was broken — **DONE**, fixed
-
-It walked `__resource_registry` and fed `entry.resource_id` — an **entry** index — to
-`getResourceNode()`, `pass.reads()` and `pass.writes()`, which all take **node** indices. The two
-spaces are equal until the first write and diverge from there, because `write` clones a node without
-adding an entry.
-
-Confirmed by measurement before anything was changed, and it was worse than the diagnosis above. A
-graph that imports `alpha`, writes it, then imports `beta` reported two resources for three nodes,
-with `beta` carrying `alpha`'s name and `alpha`'s readers. Beyond the mislabelling: **no resource
-ever reported a writer at all** in a graph containing a write — a pass writes the clone and the
-export only ever asked about the original — and `passes[].reads`/`writes` cite node IDs, so every
-version past the first was a dangling reference into an array that did not contain it.
-
-Fixed rather than deleted, which is the right call: the IDs the passes already record are node IDs,
-so a node-keyed export is the only internally consistent one. Each record now carries its version,
-and `createdBy` comes from `ResourceNode.producer` — per version, and populated for imports and for
-anything a culled pass created, none of which the entry's producer covers. That field only reads
-correctly because of item 6.
-
-Re-checked independently on the divergent case: three resources for three nodes, `beta` named
-`beta`, the written clone reporting `createdBy` and `writers`, and every ID a pass cites resolving.
-And on the live 124-node `Shading` graph: 124 resources for 124 nodes, no dangling reference, and
-`createdBy`/`writers` on exactly the 89 written nodes item 6 measured — the two items agreeing on
-the same number from opposite ends.
-
-`ResourceEntry.resource_id` no longer initialises from a module-global counter — it was overwritten
-with the registry index on the only path that constructs an entry, so the counter was dead, and a
-field that looks like a unique ID but is really an array index is the trap this whole confusion fell
-into.
-
----
-
-## Already satisfied — do not build these
-
-- §3.2 `frame.describe(handle)` → `FrameGraph.getDescriptor(handle)` exists and returns the entry's
-  descriptor. Transient and imported resources both carry a real descriptor with absolute pixels, so
-  a derived `frame.resolution` works today. (`FrameContext.describe` skipped the `is_open` guard that
-  `read` and `publish` enforce until `CORRECTNESS_FIXES_2026_09_04.md` §12.)
-- §3.4 publication check → `is_valid_resource(handle)` exists and does exactly the version compare
-  the design describes. It answers about one `ResourceEntry`, so it is blind to a resource imported
-  twice; `FrameGraph.report_resource_aliasing` names that condition
-  (`CORRECTNESS_FIXES_2026_09_04.md` §15).
-- §5 post-execute hook → `onExecuted` exists.
-
-## Noted, not now
-
-- `execute()` rescans **every** resource entry after **every** pass to find lifetime ends —
-  O(executed passes × registry entries) per frame. `compile()` already knows `entry.last`; it should
-  build a per-pass destroy list, in the shape `FramePassNode.resource_allocates` already has for the
-  other direction. Write it against the survivor set `compile()` now computes
-  (`CORRECTNESS_FIXES_2026_09_04.md` §2), not against `ref_count`.
-- `resource_creates` / `reads` / `writes` are arrays searched with `includes()`, and
-  `FramePassResources.get` hits them per resource per pass.
-- The class mixes conventions: `create_resource` / `is_valid_resource` beside `getResourceEntry` /
-  `exportToDot`. Pick one.