@carbonenginejs/runtime-resource 0.12.2 → 0.13.0

package/docs/concepts/shader-resource-model.md

@@ -1,114 +1,111 @@
-# Shader resource model
-
-Status: Stable
-Scope: `@carbonenginejs/runtime-resource`, with notes on `@carbonenginejs/runtime-trinity`
-Audience: Anyone touching `Tr2EffectRes`, `Tr2Shader`, `Tr2Effect`, or shader effect formats
-Summary: Explains how one effect file, its permutations, and the objects that resolve them relate.
-
-## Three levels
-
-The distinction between resource, resolved shader, and effect instance is
-load-bearing:
-
-| Level | Represents | Owns |
-| --- | --- | --- |
-| `Tr2EffectRes` | One effect file | Bytes, permutation axes, offset table, and a cache of resolved shaders |
-| `Tr2Shader` | One permutation | Techniques and passes for one option set |
-| `Tr2Effect` | One instance | Authored options, a resource reference, and the currently resolved shader |
-
-One file yields many shaders. One shader represents one permutation. Many
-effect instances may share the same resource and cached shader.
-
-## Carbon model
-
-Carbon's `Tr2EffectRes` retains the whole compiled file, its permutation
-records, and a map from permutation index to `Tr2Shader`. Its
-`GetShader(options, count)` resolves an option tuple to an index and reuses the
-cached shader for that index.
-
-`Tr2Effect` inherits the shader pointer from `Tr2Material`. Rebuilding an
-effect clears that pointer and resolves it again through the resource. This
-creates two deliberate caches:
-
-- the resource caches one hydrated shader per permutation index; and
-- each effect instance caches its currently resolved pointer.
-
-Changing effect options therefore selects another shader from the same loaded
-file rather than loading another file.
-
-## Runtime-resource model
-
-`runtime-resource/src/resource/shader/Tr2EffectRes.js` follows the same shape:
-
-| Carbon | Runtime-resource |
-| --- | --- |
-| shader map keyed by permutation index | private `#shaders` map |
-| `GetShader(options, count)` | option resolution followed by `GetShaderByIndex` |
-| permutation records | `permutationGraph.axes` and `variants` |
-| offset-table body lookup | portable reflection lookup; Carbon-record adapter pending |
-| retained file bytes | `GetPayload()` |
-
-`runtime-trinity`'s `Tr2Effect.RebuildCachedDataInternal` clears and
-re-resolves its shader through the effect resource. Renderer-owned pipelines,
-bind groups, and GPU handles are not part of this device-free graph.
-
-## Package coverage
-
-Carbon effect files carry every permutation and select through a dense offset
-table. Representative source files demonstrate why body count and permutation
-count are different:
-
-| File | Permutations | Distinct bodies |
-| --- | ---: | ---: |
-| `effect.dx11/.../unpacked_quadv5.sm_hi` | 480 | 144 |
-| `effect.gles2/.../geometryviewer.sm_hi` | 80 | 27 |
-| `effect.gles2/.../textureviewer.sm_hi` | 18 | 3 |
-
-Current `.cewgpu` bytes use Carbon's version-15 record layout and retain every
-permutation row and representable non-program description fields. Non-dynamic
-sampler names are unrecoverable and stage order is canonicalized. Emitted body
-dedupe follows exact emitted bytes, so it need not preserve the original source
-alias partition. `mode: "selected"` narrows which body receives translated
-WGSL; it does not discard source permutations. `mode: "all"` attempts every
-distinct body after the resolved selection passes the initial translation gate.
-
-`.cewg` remains its own CEWG chunk format. Its current package contract also
-preserves complete source permutation topology and supports selected versus
-all backend coverage.
-
-The selected/all distinction is therefore backend translation scope, not
-source cardinality. A resource can still reason about every option tuple even
-when some bodies have no translated backend program.
-
-## Current integration boundaries
-
-The wire topology is complete, but two adapters still require focused
-regression proof:
-
-- the raw CEWGPU container's Carbon description must be converted to the
-  portable envelope expected by `Tr2Shader.fromPortable`; and
-- the engine's derived body-program view must agree on stable pass-unit
-  identity.
-
-Until those adapters are reconciled, the presence of every permutation proves
-source preservation, not successful runtime hydration or execution.
-
-## Reading the model without inventing gaps
-
-Three recurring mistakes explain most false conclusions in this area:
-
-1. **Searching only one package.** `Tr2EffectRes` is in runtime-resource while
-   `Tr2Effect` is in runtime-trinity.
-2. **Searching only the derived class.** The effect's shader pointer is
-   declared on its `Tr2Material` base.
-3. **Confusing permutation rows with stored bodies.** Several rows may alias
-   one description body while remaining distinct option selections.
-
-When an expected mechanism appears absent, check the owner package, base
-classes, and record indirection before treating the absence as a design gap.
-
-## Related documentation
-
-- [CEWGPU effect container](../formats/webgpu/formats/cewgpu.md)
-- [Carbon compiled-effect container](../formats/carbon-effect-container.md)
-- [WebGPU effect packaging](../formats/webgpu/guides/effect-packaging.md)
+# Shader resource model
+
+Status: Stable
+Scope: `@carbonenginejs/runtime-resource`, with notes on `@carbonenginejs/runtime-trinity`
+Audience: Anyone touching `Tr2EffectRes`, `Tr2Shader`, `Tr2Effect`, or shader effect formats
+Summary: Explains how one effect file, its permutations, and the objects that resolve them relate.
+
+## Three levels
+
+The distinction between resource, resolved shader, and effect instance is
+load-bearing:
+
+| Level | Represents | Owns |
+| --- | --- | --- |
+| `Tr2EffectRes` | One effect file | Bytes, permutation axes, offset table, and a cache of resolved shaders |
+| `Tr2Shader` | One permutation | Techniques and passes for one option set |
+| `Tr2Effect` | One instance | Authored options, a resource reference, and the currently resolved shader |
+
+One file yields many shaders. One shader represents one permutation. Many
+effect instances may share the same resource and cached shader.
+
+## Carbon model
+
+Carbon's `Tr2EffectRes` retains the whole compiled file, its permutation
+records, and a map from permutation index to `Tr2Shader`. Its
+`GetShader(options, count)` resolves an option tuple to an index and reuses the
+cached shader for that index.
+
+`Tr2Effect` inherits the shader pointer from `Tr2Material`. Rebuilding an
+effect clears that pointer and resolves it again through the resource. This
+creates two deliberate caches:
+
+- the resource caches one hydrated shader per permutation index; and
+- each effect instance caches its currently resolved pointer.
+
+Changing effect options therefore selects another shader from the same loaded
+file rather than loading another file.
+
+## Runtime-resource model
+
+`runtime-resource/src/resource/shader/Tr2EffectRes.js` follows the same shape:
+
+| Carbon | Runtime-resource |
+| --- | --- |
+| shader map keyed by permutation index | private `#shaders` map |
+| `GetShader(options, count)` | option resolution followed by `GetShaderByIndex` |
+| permutation records | `permutationGraph.axes` and `variants` |
+| offset-table body lookup | `CjsCarbonEffectReader` retained by `DoLoad`, read per index |
+| retained file bytes | `GetPayload()` |
+
+`runtime-trinity`'s `Tr2Effect.RebuildCachedDataInternal` clears and
+re-resolves its shader through the effect resource. Renderer-owned pipelines,
+bind groups, and GPU handles are not part of this device-free graph.
+
+## Package coverage
+
+Carbon effect files carry every permutation and select through a dense offset
+table. Representative source files demonstrate why body count and permutation
+count are different:
+
+| File | Permutations | Distinct bodies |
+| --- | ---: | ---: |
+| `effect.dx11/.../unpacked_quadv5.sm_hi` | 480 | 144 |
+| `effect.gles2/.../geometryviewer.sm_hi` | 80 | 27 |
+| `effect.gles2/.../textureviewer.sm_hi` | 18 | 3 |
+
+Current `.carbonwebgpu` bytes use Carbon's version-15 record layout and retain every
+permutation row and representable non-program description fields, including
+non-dynamic sampler names and the file's authored pass-stage order — Carbon's
+runtime discards both, the file does not. Emitted body dedupe follows exact
+emitted bytes, so it need not preserve the original source alias partition. `mode: "selected"` narrows which body receives translated
+WGSL; it does not discard source permutations. `mode: "all"` attempts every
+distinct body after the resolved selection passes the initial translation gate.
+
+`.carbonwebgl` remains its own Carbon WebGL chunk format. Its current package contract also
+preserves complete source permutation topology and supports selected versus
+all backend coverage.
+
+The selected/all distinction is therefore backend translation scope, not
+source cardinality. A resource can still reason about every option tuple even
+when some bodies have no translated backend program.
+
+## Current integration boundaries
+
+The read path is direct: `Tr2EffectRes.DoLoad` retains a
+`CjsCarbonEffectReader` over the container bytes, and
+`Tr2Shader.fromCarbonBinary(reader, index)` builds the device-free graph from
+one description record. No intermediate document sits between them.
+
+What remains unproven is execution, not construction. The presence of every
+permutation proves source preservation; it does not prove a rendered result.
+
+## Reading the model without inventing gaps
+
+Three recurring mistakes explain most false conclusions in this area:
+
+1. **Searching only one package.** `Tr2EffectRes` is in runtime-resource while
+   `Tr2Effect` is in runtime-trinity.
+2. **Searching only the derived class.** The effect's shader pointer is
+   declared on its `Tr2Material` base.
+3. **Confusing permutation rows with stored bodies.** Several rows may alias
+   one description body while remaining distinct option selections.
+
+When an expected mechanism appears absent, check the owner package, base
+classes, and record indirection before treating the absence as a design gap.
+
+## Related documentation
+
+- [Carbon WebGPU effect container](../formats/webgpu/formats/carbon-webgpu.md)
+- [Carbon compiled-effect container](../formats/carbon-effect-container.md)
+- [WebGPU effect packaging](../formats/webgpu/guides/effect-packaging.md)

package/docs/concepts/writing-an-engine-adapter.md

@@ -1,115 +1,115 @@
-# Writing an engine adapter
-
-Status: Stable
-Scope: `@carbonenginejs/runtime-resource`, addressed to engine packages
-Audience: Anyone building `engine-webgl`, a second WebGPU engine, or any package that realizes CPU payloads into backend objects
-Summary: The coupling rules, the reflection/topology seam, and the two mistakes `engine-webgpu` already made that a second engine must not repeat.
-
-## Read this before writing a second engine
-
-`engine-webgpu` is currently the only engine package. Most of its shape is right
-and worth copying. Two things are not, and both are easier to avoid than to undo.
-This document exists because the second engine is being written after the first
-one's mistakes were diagnosed but before they were fixed.
-
-## The coupling rule
-
-**Engine packages take functions and duck-typed objects. They do not import the
-layers above or beside them.**
-
-`engine-webgpu` declares this as a non-goal and holds it: no dependency on
-`runtime-core`, `runtime-resource`, or `runtime-trinity`. Three existing seams
-show the pattern, and a new engine should reach for one of them rather than
-inventing a fourth:
-
-| seam | shape |
-|---|---|
-| `CjsWebGPUPackage.fromBytes(bytes, { read })` | the format reader arrives as a **function** |
-| `CjsWebGPUTrinityBatchDispatcher(hooks)` | `ResolveMaterial` / `ResolveBindings` arrive as **hooks**; the batch is duck-typed on the `Tr2RenderBatch` shape |
-| `CjsTextureArrayRes` | the engine calls `ConsumeUpdateRequest()`, prepares a candidate, then `CommitPreparedAdapterRevision()` — **the resource layer owns the state machine and never holds engine code** |
-
-The third is the most instructive. Realization is not a callback the resource
-layer fires into the engine; it is a request the engine consumes and a commit it
-returns. The engine drives its own frame, which is what an engine must do, while
-the resource layer keeps the queue, the revisions, and the failure handling.
-
-## What the resource layer owns
-
-Resource identity, the cache, CPU payload lifecycle, format selection, the
-load/publication queues, and **permutation selection**. It stops at a published
-CPU payload; it hands out objects and accepts commits.
-
-It is GPU-free and stays that way. It does not define GPU-shaped interfaces for
-engines to implement, which is precisely why the consume/commit shape is used
-instead of an injected realizer.
-
-## The seam: reflection from the shader, topology from the package
-
-This is the distinction the first engine got wrong.
-
-**Carbon reflection belongs to `Tr2Shader`.** One file yields many shaders, one
-shader is one permutation, and many effects share them — see
-[shader-resource-model.md](shader-resource-model.md). The surface is
-`GetConstant(name)`, `GetResource(name)`, `GetParameterAnnotations(parameterName)`,
-`GetEffectDescription()` and `iterateStages()`, reachable through
-`GetPortableEffectReflection` -> `Tr2Shader.fromPortable`.
-
-**Backend binding topology belongs to the package**, because it has no Carbon
-counterpart — it comes from the lowered IR, not from Carbon's D3D-era reflection.
-
-| ask the shader | ask the package |
-|---|---|
-| `constants[].{name, offset, size}` | `group`, `binding`, `visibility` |
-| resource `type`, `isSRGB` | `generatedSymbol`, `resourceKind` |
-| annotations | `registerIndex`, `registerSpace` |
-| the parameter's name | `viewDimension` |
-
-If you find yourself wanting *richer package reflection*, you are on the wrong
-side of this table. The data you want is on the shader, and it is there because
-Carbon put it there.
-
-## The two mistakes not to copy
-
-### 1. Do not read format-package records for Carbon reflection
-
-`engine-webgpu/src/core/packageHelpers.js` and
-`src/core/spaceObjectMainBindings.js` read `metadataName`, `heapView`,
-`carbon.type`, `carbon.isSRGB` and `carbon.constants[].{name, offset, size}`
-directly out of the format package, to pack real material uniform bytes. Twelve
-lines, two files, and a known layering defect: engine code must consume the
-resource-owned `Tr2Shader` reflection graph rather than make format-package
-records its material API.
-
-It is deferred rather than fixed because nothing can use that path until the
-shader work lands, so the break is theoretical. That is a reason not to rush it,
-not a reason to reproduce it.
-
-### 2. Do not reimplement permutation selection
-
-`Tr2EffectRes` already keeps a permutation-index-keyed shader cache and resolves
-options through it — Carbon's own mechanism, and ours matches. An engine that
-grows its own index-keyed resolution is writing a second copy of a tested thing.
-
-Building pipelines, bind groups and GPU objects is **not** duplication; that is
-the engine's whole job. The line falls exactly where the table above falls.
-
-## runtime-core is optional
-
-`runtime-core` wires named services by default — `RegisterResourceBehavior`
-registers "a structural request policy without importing its owner", and the
-resource manager, space-object factory and audio manager register the same way.
-It is a convenience wrapper, not a dependency.
-
-Every hook it registers can be passed by hand. An engine that only works when
-`runtime-core` is present has taken a dependency through the back door.
-
-## Checklist for a new engine package
-
-- no imports of `runtime-core`, `runtime-resource`, or `runtime-trinity`
-- readers, material resolvers and batch shapes arrive injected or duck-typed
-- Carbon reflection is read from a shader object handed in, never from format
-  records
-- permutation selection is asked for, never reimplemented
-- long-lived state machines and queues stay in the resource layer; the engine
-  consumes requests and commits results
-- the package works with `runtime-core` absent
+# Writing an engine adapter
+
+Status: Stable
+Scope: `@carbonenginejs/runtime-resource`, addressed to engine packages
+Audience: Anyone building `engine-webgl`, a second WebGPU engine, or any package that realizes CPU payloads into backend objects
+Summary: The coupling rules, the reflection/topology seam, and the two mistakes `engine-webgpu` already made that a second engine must not repeat.
+
+## Read this before writing a second engine
+
+`engine-webgpu` is currently the only engine package. Most of its shape is right
+and worth copying. Two things are not, and both are easier to avoid than to undo.
+This document exists because the second engine is being written after the first
+one's mistakes were diagnosed but before they were fixed.
+
+## The coupling rule
+
+**Engine packages take functions and duck-typed objects. They do not import the
+layers above or beside them.**
+
+`engine-webgpu` declares this as a non-goal and holds it: no dependency on
+`runtime-core`, `runtime-resource`, or `runtime-trinity`. Three existing seams
+show the pattern, and a new engine should reach for one of them rather than
+inventing a fourth:
+
+| seam | shape |
+|---|---|
+| `CjsWebGPUPackage.fromBytes(bytes, { read })` | the format reader arrives as a **function** |
+| `CjsWebGPUTrinityBatchDispatcher(hooks)` | `ResolveMaterial` / `ResolveBindings` arrive as **hooks**; the batch is duck-typed on the `Tr2RenderBatch` shape |
+| `CjsTextureArrayRes` | the engine calls `ConsumeUpdateRequest()`, prepares a candidate, then `CommitPreparedAdapterRevision()` — **the resource layer owns the state machine and never holds engine code** |
+
+The third is the most instructive. Realization is not a callback the resource
+layer fires into the engine; it is a request the engine consumes and a commit it
+returns. The engine drives its own frame, which is what an engine must do, while
+the resource layer keeps the queue, the revisions, and the failure handling.
+
+## What the resource layer owns
+
+Resource identity, the cache, CPU payload lifecycle, format selection, the
+load/publication queues, and **permutation selection**. It stops at a published
+CPU payload; it hands out objects and accepts commits.
+
+It is GPU-free and stays that way. It does not define GPU-shaped interfaces for
+engines to implement, which is precisely why the consume/commit shape is used
+instead of an injected realizer.
+
+## The seam: reflection from the shader, topology from the package
+
+This is the distinction the first engine got wrong.
+
+**Carbon reflection belongs to `Tr2Shader`.** One file yields many shaders, one
+shader is one permutation, and many effects share them — see
+[shader-resource-model.md](shader-resource-model.md). The surface is
+`GetConstant(name)`, `GetResource(name)`, `GetParameterAnnotations(parameterName)`,
+`GetEffectDescription()` and `iterateStages()`, reachable through
+`Tr2EffectRes.DoLoad(bytes)` -> `Tr2Shader.fromCarbonBinary(reader, index)`.
+
+**Backend binding topology belongs to the package**, because it has no Carbon
+counterpart — it comes from the lowered IR, not from Carbon's D3D-era reflection.
+
+| ask the shader | ask the package |
+|---|---|
+| `constants[].{name, offset, size}` | `group`, `binding`, `visibility` |
+| resource `type`, `isSRGB` | `generatedSymbol`, `resourceKind` |
+| annotations | `registerIndex`, `registerSpace` |
+| the parameter's name | `viewDimension` |
+
+If you find yourself wanting *richer package reflection*, you are on the wrong
+side of this table. The data you want is on the shader, and it is there because
+Carbon put it there.
+
+## The two mistakes not to copy
+
+### 1. Do not read format-package records for Carbon reflection
+
+`engine-webgpu/src/core/packageHelpers.js` and
+`src/core/spaceObjectMainBindings.js` read `metadataName`, `heapView`,
+`carbon.type`, `carbon.isSRGB` and `carbon.constants[].{name, offset, size}`
+directly out of the format package, to pack real material uniform bytes. Twelve
+lines, two files, and a known layering defect: engine code must consume the
+resource-owned `Tr2Shader` reflection graph rather than make format-package
+records its material API.
+
+It is deferred rather than fixed because nothing can use that path until the
+shader work lands, so the break is theoretical. That is a reason not to rush it,
+not a reason to reproduce it.
+
+### 2. Do not reimplement permutation selection
+
+`Tr2EffectRes` already keeps a permutation-index-keyed shader cache and resolves
+options through it — Carbon's own mechanism, and ours matches. An engine that
+grows its own index-keyed resolution is writing a second copy of a tested thing.
+
+Building pipelines, bind groups and GPU objects is **not** duplication; that is
+the engine's whole job. The line falls exactly where the table above falls.
+
+## runtime-core is optional
+
+`runtime-core` wires named services by default — `RegisterResourceBehavior`
+registers "a structural request policy without importing its owner", and the
+resource manager, space-object factory and audio manager register the same way.
+It is a convenience wrapper, not a dependency.
+
+Every hook it registers can be passed by hand. An engine that only works when
+`runtime-core` is present has taken a dependency through the back door.
+
+## Checklist for a new engine package
+
+- no imports of `runtime-core`, `runtime-resource`, or `runtime-trinity`
+- readers, material resolvers and batch shapes arrive injected or duck-typed
+- Carbon reflection is read from a shader object handed in, never from format
+  records
+- permutation selection is asked for, never reimplemented
+- long-lived state machines and queues stay in the resource layer; the engine
+  consumes requests and commits results
+- the package works with `runtime-core` absent