@carbonenginejs/runtime-resource 0.11.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/formats/bnk/CjsBnkFormat.js +12 -2
- package/dist/formats/bnk/CjsBnkFormat.js.map +1 -1
- package/dist/formats/bnk/core/graph.js +3 -6
- package/dist/formats/bnk/core/graph.js.map +1 -1
- package/dist/formats/bnk/core/helpers.js +53 -11
- package/dist/formats/bnk/core/helpers.js.map +1 -1
- package/dist/formats/bnk/core/sfxNodes.js +532 -0
- package/dist/formats/bnk/core/sfxNodes.js.map +1 -0
- package/dist/formats/bnk/core/soundbanksInfo.js +59 -51
- package/dist/formats/bnk/core/soundbanksInfo.js.map +1 -1
- package/dist/formats/gr2/CjsGr2Format.js +256 -13
- package/dist/formats/gr2/CjsGr2Format.js.map +1 -1
- package/dist/formats/gr2/core/helpers.js +15 -15
- package/dist/formats/gr2/core/helpers.js.map +1 -1
- package/dist/formats/gr2/core/json.js +1 -1
- package/dist/formats/gr2/core/json.js.map +1 -1
- package/dist/formats/gr2/core/targets.js +1 -1
- package/dist/formats/gr2/core/targets.js.map +1 -1
- package/dist/formats/gr2/index.js +0 -1
- package/dist/formats/gr2/index.js.map +1 -1
- package/dist/formats/hlsl/core/analysis.js +1 -1
- package/dist/formats/hlsl/index.js +1 -0
- package/dist/formats/hlsl/index.js.map +1 -1
- package/dist/formats/index.js +3 -6
- package/dist/formats/index.js.map +1 -1
- package/dist/formats/webgl/core/cewg/binary.js +3 -3
- package/dist/formats/webgl/core/cewg/binary.js.map +1 -1
- package/dist/formats/webgl/core/effectPackage.js +6 -3
- package/dist/formats/webgl/core/effectPackage.js.map +1 -1
- package/dist/formats/webgl/core/effectPackageValidation.js +1 -1
- package/dist/formats/webgl/core/effectPackageValidation.js.map +1 -1
- package/dist/formats/webgl/core/errors.js +3 -3
- package/dist/formats/webgl/core/errors.js.map +1 -1
- package/dist/formats/webgpu/core/packageMetadata.js +2 -2
- package/dist/formats/webgpu/core/packageMetadata.js.map +1 -1
- package/docs/formats/dxbc/README.md +68 -0
- package/docs/formats/dxbc/architecture.md +80 -0
- package/docs/formats/dxbc/reference/api.md +77 -0
- package/docs/formats/dxbc/reference/classes/README.md +76 -0
- package/docs/formats/dxbc/reference/decoded-output.md +122 -0
- package/docs/formats/gr2.md +3 -4
- package/docs/formats/hlsl/README.md +54 -0
- package/docs/formats/hlsl/architecture.md +67 -0
- package/docs/formats/hlsl/guides/hydrating-json-output.md +62 -0
- package/docs/formats/hlsl/guides/reading-effects.md +64 -0
- package/docs/formats/hlsl/reference/advanced-analysis.md +66 -0
- package/docs/formats/hlsl/reference/api.md +97 -0
- package/docs/formats/hlsl/reference/classes/README.md +16 -0
- package/docs/formats/hlsl/reference/classes/carbon-compatibility.md +66 -0
- package/docs/formats/hlsl/reference/classes/public-api.md +26 -0
- package/docs/formats/hlsl/reference/classes/tr2-effect-model.md +150 -0
- package/docs/formats/hlsl/reference/json-graph.md +100 -0
- package/docs/formats/hlsl/reference/portable-reflection.md +141 -0
- package/docs/formats/provenance.md +9 -4
- package/docs/formats/webgl/README.md +57 -0
- package/docs/formats/webgl/architecture.md +70 -0
- package/docs/formats/webgl/carbon-constant-layouts.md +326 -0
- package/docs/formats/webgl/decl-io.md +1234 -0
- package/docs/formats/webgl/effect-reflection.md +127 -0
- package/docs/formats/webgl/memory-structured.md +871 -0
- package/docs/formats/webgl/reference/classes/README.md +96 -0
- package/docs/formats/webgl/texture-sample.md +964 -0
- package/docs/formats/webgpu/README.md +81 -0
- package/docs/formats/webgpu/architecture.md +93 -0
- package/docs/formats/webgpu/formats/cewgpu.md +479 -0
- package/docs/formats/webgpu/guides/effect-packaging.md +199 -0
- package/docs/formats/webgpu/reference/api.md +197 -0
- package/docs/formats/webgpu/reference/classes/README.md +66 -0
- package/docs/formats/webgpu/reference/wgsl-compatibility.md +1442 -0
- package/docs/formats/wwise.md +16 -1
- package/docs/reference/classes/formats.md +10 -10
- package/format-notices/bnk/NOTICE +5 -4
- package/format-notices/webgl/NOTICE +1 -1
- package/package.json +1 -1
- package/dist/formats/gr2/core/CjsFormatGr2.js +0 -273
- package/dist/formats/gr2/core/CjsFormatGr2.js.map +0 -1
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# WebGPU format documentation
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource/formats/webgpu`
|
|
5
|
+
Audience: Shader-tool authors, engine integrators, and maintainers
|
|
6
|
+
Summary: Explains the CEWGPU package format, compiled-effect conversion API, and bounded DXBC-to-WGSL compiler.
|
|
7
|
+
|
|
8
|
+
## Purpose
|
|
9
|
+
|
|
10
|
+
`@carbonenginejs/runtime-resource/formats/webgpu` reads and builds CEWGPU shader packages and
|
|
11
|
+
converts supported compiled effect stages into portable WGSL package data. It
|
|
12
|
+
owns effect analysis, DXBC-to-intermediate-representation lowering, WGSL
|
|
13
|
+
emission, pass-global binding planning, and CEWGPU assembly.
|
|
14
|
+
|
|
15
|
+
Unsupported requested shader semantics fail explicitly instead of producing a
|
|
16
|
+
partially translated selected pass. `BuildEffect` preserves complete source
|
|
17
|
+
permutation topology in `PGRF` and, for version-15 effects, complete portable
|
|
18
|
+
reflection for every unique body in RFLX v2/RBLB. ANLS/WGSL remain selected
|
|
19
|
+
backend-body data, so backend and runtime completeness are still open.
|
|
20
|
+
|
|
21
|
+
## Use this package when
|
|
22
|
+
|
|
23
|
+
Use `format-webgpu` when you need to:
|
|
24
|
+
|
|
25
|
+
- inspect or build a `.cewgpu` package;
|
|
26
|
+
- analyze caller-supplied compiled effect bytes;
|
|
27
|
+
- lower supported DXBC vertex and fragment programs to WGSL;
|
|
28
|
+
- build one collision-free WebGPU binding layout across a complete pass; or
|
|
29
|
+
- convert one selected compiled-effect body/pass scope into a CEWGPU package while
|
|
30
|
+
preserving its complete version-15 source permutation graph and all-unique
|
|
31
|
+
portable reflection.
|
|
32
|
+
|
|
33
|
+
Use `@carbonenginejs/runtime-resource/formats/hlsl` directly for effect metadata without WGSL
|
|
34
|
+
conversion, and `@carbonenginejs/runtime-resource/formats/dxbc` directly for standalone DXBC
|
|
35
|
+
inspection. GPU device, shader-module, bind-group, and pipeline realization
|
|
36
|
+
belong in `@carbonenginejs/engine-webgpu`.
|
|
37
|
+
|
|
38
|
+
## Where it fits
|
|
39
|
+
|
|
40
|
+
```text
|
|
41
|
+
compiled effect bytes
|
|
42
|
+
|
|
|
43
|
+
+---- format-hlsl ---- effect and binding metadata
|
|
44
|
+
|
|
|
45
|
+
+---- format-dxbc ---- decoded shader programs
|
|
46
|
+
| |
|
|
47
|
+
+------------------------------+
|
|
48
|
+
|
|
|
49
|
+
v
|
|
50
|
+
format-webgpu
|
|
51
|
+
analysis + WGSL + CEWGPU
|
|
52
|
+
|
|
|
53
|
+
v
|
|
54
|
+
engine-webgpu
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
The package is browser-safe at its public source boundary. Repository-only
|
|
58
|
+
commands may adapt filesystem input for development, but the core conversion
|
|
59
|
+
path accepts bytes and does not depend on Node filesystem APIs or native
|
|
60
|
+
executables.
|
|
61
|
+
|
|
62
|
+
## Start here
|
|
63
|
+
|
|
64
|
+
```js
|
|
65
|
+
import { CjsWebgpuFormat } from "@carbonenginejs/runtime-resource/formats/webgpu";
|
|
66
|
+
|
|
67
|
+
const summary = CjsWebgpuFormat.inspect(packageBytes);
|
|
68
|
+
const packageData = CjsWebgpuFormat.read(packageBytes);
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
For compiled-effect conversion, continue with the
|
|
72
|
+
[effect packaging guide](guides/effect-packaging.md).
|
|
73
|
+
|
|
74
|
+
## Documentation map
|
|
75
|
+
|
|
76
|
+
- [Architecture and boundaries](architecture.md)
|
|
77
|
+
- [Effect packaging guide](guides/effect-packaging.md)
|
|
78
|
+
- [Public API reference](reference/api.md)
|
|
79
|
+
- [CEWGPU package format](formats/cewgpu.md)
|
|
80
|
+
- [WGSL compatibility](reference/wgsl-compatibility.md)
|
|
81
|
+
- [Class-purpose catalog](reference/classes/README.md)
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# Architecture and boundaries
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource/formats/webgpu`
|
|
5
|
+
Audience: Shader-tool authors, engine integrators, and maintainers
|
|
6
|
+
Summary: Defines the package's compiler, container, dependency, and engine boundaries.
|
|
7
|
+
|
|
8
|
+
## Purpose
|
|
9
|
+
|
|
10
|
+
`format-webgpu` is the transformation and package layer between compiled
|
|
11
|
+
Carbon-style effects and WebGPU engine realization. It converts supported
|
|
12
|
+
shader programs into portable WGSL descriptors without creating live GPU
|
|
13
|
+
objects.
|
|
14
|
+
|
|
15
|
+
## Dependency direction
|
|
16
|
+
|
|
17
|
+
```text
|
|
18
|
+
@carbonenginejs/runtime-resource/formats/hlsl
|
|
19
|
+
|
|
|
20
|
+
+---- effect selection and binding metadata
|
|
21
|
+
|
|
|
22
|
+
v
|
|
23
|
+
@carbonenginejs/runtime-resource/formats/webgpu <---- @carbonenginejs/runtime-resource/formats/dxbc
|
|
24
|
+
| decoded DXBC
|
|
25
|
+
|
|
|
26
|
+
+---- CEWGPU bytes, analysis, WGSL, canonical layouts
|
|
27
|
+
|
|
|
28
|
+
v
|
|
29
|
+
@carbonenginejs/engine-webgpu
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Node build tools may call the public byte-oriented API, but the format package
|
|
33
|
+
does not import the toolchain. Dependency direction remains tools to formats,
|
|
34
|
+
then package data to the engine.
|
|
35
|
+
|
|
36
|
+
## Owned responsibilities
|
|
37
|
+
|
|
38
|
+
- Reading, inspecting, and building the CEWGPU v1 chunk container.
|
|
39
|
+
- Resolving one compiled-effect permutation and complete pass.
|
|
40
|
+
- Validating and preserving every source permutation index and raw-body alias
|
|
41
|
+
in a backend-neutral identity-only graph.
|
|
42
|
+
- Packing every unique version-15 body's complete shared portable reflection,
|
|
43
|
+
with exact immutable byte vectors in one validated blob arena.
|
|
44
|
+
- Normalizing effect, stage, binding, and decoded DXBC analysis.
|
|
45
|
+
- Building validated shader intermediate representation and structured control
|
|
46
|
+
flow.
|
|
47
|
+
- Lowering the supported vertex and fragment profiles to WGSL.
|
|
48
|
+
- Allocating a pass-global binding plan with explicit stage scope.
|
|
49
|
+
- Assembling emitted shaders and layouts into a portable WGSL set.
|
|
50
|
+
- Rejecting unsupported or ambiguous semantics with explicit diagnostics.
|
|
51
|
+
|
|
52
|
+
## Ownership elsewhere
|
|
53
|
+
|
|
54
|
+
- `format-hlsl` owns compiled-effect parsing, permutation resolution,
|
|
55
|
+
unique-body enumeration, binding-manifest interpretation, and the shared
|
|
56
|
+
body-local portable reflection schema/validator.
|
|
57
|
+
- `format-dxbc` owns DXBC container and instruction decoding.
|
|
58
|
+
- `engine-webgpu` owns `GPUDevice`, shader-module compilation, bind groups,
|
|
59
|
+
pipelines, resource realization, device loss, and draw execution.
|
|
60
|
+
- Node tooling owns indexed input acquisition, filesystem adapters, caching,
|
|
61
|
+
build reports, and optional native comparison.
|
|
62
|
+
- `runtime-resource` owns effect-resource lifecycle, permutation selection,
|
|
63
|
+
canonical device-free `Tr2Shader` hydration, and shader caching.
|
|
64
|
+
- `runtime-trinity` owns authored effect/material facades, parameters, options,
|
|
65
|
+
sampler overrides, and scene references.
|
|
66
|
+
|
|
67
|
+
## Browser and Node boundary
|
|
68
|
+
|
|
69
|
+
The public class accepts `ArrayBuffer`, typed-array, and compatible byte views.
|
|
70
|
+
Its source does not read files, inspect processes, or invoke native programs.
|
|
71
|
+
This allows applications to fetch or select bytes and build CEWGPU data in a
|
|
72
|
+
browser.
|
|
73
|
+
|
|
74
|
+
The repository's command-line scripts are development adapters over the same
|
|
75
|
+
public operations. They are not a second compiler contract.
|
|
76
|
+
|
|
77
|
+
## Shader target
|
|
78
|
+
|
|
79
|
+
The general translation target is DX11 SM5.0 vertex and fragment bytecode.
|
|
80
|
+
The compiler also admits a finite, frozen set of exact bounded compute
|
|
81
|
+
profiles. DX12 SM5.1 input is useful for differential analysis where
|
|
82
|
+
supported, but DX12-only bindless resource ranges are not part of the current
|
|
83
|
+
translation target.
|
|
84
|
+
|
|
85
|
+
Geometry, hull, and domain stages, plus compute programs outside the admitted
|
|
86
|
+
profiles, are not emitted. The exact supported and adapted boundaries are
|
|
87
|
+
listed in [WGSL compatibility](reference/wgsl-compatibility.md).
|
|
88
|
+
|
|
89
|
+
## Related documentation
|
|
90
|
+
|
|
91
|
+
- [Package documentation](README.md)
|
|
92
|
+
- [CEWGPU package format](formats/cewgpu.md)
|
|
93
|
+
- [Public API reference](reference/api.md)
|
|
@@ -0,0 +1,479 @@
|
|
|
1
|
+
# CEWGPU package format
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource/formats/webgpu` CEWGPU version 1
|
|
5
|
+
Audience: Shader-tool authors and engine integrators
|
|
6
|
+
Summary: Defines the flat CEWGPU v1 container, common chunks, and structured WGSL package records.
|
|
7
|
+
|
|
8
|
+
## Purpose
|
|
9
|
+
|
|
10
|
+
CEWGPU is a CarbonEngineJS-defined container for WebGPU shader analysis,
|
|
11
|
+
emitted WGSL, and pass layout metadata. It is designed for deterministic
|
|
12
|
+
offline construction and simple browser-side reading.
|
|
13
|
+
|
|
14
|
+
## Binary layout
|
|
15
|
+
|
|
16
|
+
All integers are unsigned 32-bit little-endian values.
|
|
17
|
+
|
|
18
|
+
| Field | Size | Meaning |
|
|
19
|
+
| --- | ---: | --- |
|
|
20
|
+
| Magic | 4 bytes | ASCII `CWGP`. |
|
|
21
|
+
| Version | 4 bytes | Container version; the current reader accepts `1`. |
|
|
22
|
+
| Chunk count | 4 bytes | Number of following chunks. |
|
|
23
|
+
| Chunk tag | 4 bytes | ASCII four-character code. |
|
|
24
|
+
| Chunk size | 4 bytes | Payload byte length. |
|
|
25
|
+
| Chunk payload | Variable | Raw bytes, UTF-8 text, or UTF-8 JSON by chunk contract. |
|
|
26
|
+
|
|
27
|
+
The tag, size, and payload fields repeat in declaration order. A reader rejects
|
|
28
|
+
an unsupported version, truncated chunk, invalid magic, or trailing bytes.
|
|
29
|
+
|
|
30
|
+
## Common chunks
|
|
31
|
+
|
|
32
|
+
| Tag | Payload | Purpose |
|
|
33
|
+
| --- | --- | --- |
|
|
34
|
+
| `INFO` | JSON | Format and translator information. |
|
|
35
|
+
| `META` | JSON | Caller provenance and effect-selection metadata. |
|
|
36
|
+
| `PGRF` | JSON | Complete source permutation topology and identity-only body records. |
|
|
37
|
+
| `RFLX` | JSON | Complete portable reflection for every unique version-15 source body. |
|
|
38
|
+
| `RBLB` | Raw bytes | Exact immutable byte payloads referenced by `RFLX`. |
|
|
39
|
+
| `ANLS` | JSON or text | Compact selected-body diagnostic stage/binding data; not lossless effect reflection. |
|
|
40
|
+
| `WGSL` | WGSL text or JSON | One raw module or a structured shader set with layouts. |
|
|
41
|
+
|
|
42
|
+
Unknown four-character chunks remain readable as raw bytes. The package
|
|
43
|
+
builder preserves the caller's chunk order. Chunk tags must be four printable
|
|
44
|
+
ASCII characters, and duplicate tags are rejected by both builder and reader.
|
|
45
|
+
|
|
46
|
+
## Selected-effect envelope
|
|
47
|
+
|
|
48
|
+
Generic CEWGPU containers may omit common chunks and may retain raw WGSL text.
|
|
49
|
+
A package declaring `INFO.packageKind: "tr2-effect-webgpu"` has a stricter
|
|
50
|
+
contract. The reader requires JSON `INFO`, `META`, `ANLS`, and `WGSL` chunks.
|
|
51
|
+
Current producers also declare and emit `PGRF`; legacy INFO v1/v2 packages may
|
|
52
|
+
omit it. Version-15 sources additionally declare and emit `RFLX` plus `RBLB`.
|
|
53
|
+
INFO v3 makes the graph and reflection chunks one mandatory, indivisible unit.
|
|
54
|
+
Legacy INFO v2 may omit reflection or carry selected-body RFLX v1. The reader
|
|
55
|
+
validates current
|
|
56
|
+
schema versions and reconciles
|
|
57
|
+
translator/source/body identity, the complete source permutation topology,
|
|
58
|
+
selected options, counts, pass/stage metadata, emitted shader and layout
|
|
59
|
+
descriptors, explicit selection coverage, WGSL-set version features, and
|
|
60
|
+
source/backend completeness flags. Declared effect layouts use unique bind
|
|
61
|
+
groups contiguous from group zero and unique binding slots and physical
|
|
62
|
+
identities.
|
|
63
|
+
|
|
64
|
+
The binary container version and the `INFO` document version are independent.
|
|
65
|
+
`BuildEffect` emits `INFO.formatVersion: 3` for version-15 input and version 2
|
|
66
|
+
for versions 8-14. The reader retains legacy selected-effect INFO versions 1
|
|
67
|
+
and 2 and rejects unknown INFO versions; generic packages remain outside this
|
|
68
|
+
marker-gated schema.
|
|
69
|
+
|
|
70
|
+
INFO versions 2 and 3 record `targetBackend: "webgpu"`, the producing backend package
|
|
71
|
+
name/version, and the translator name/version. Its `sourceIdentity` contains
|
|
72
|
+
the exact source byte length and a required lower-case SHA-256 digest. The
|
|
73
|
+
builder computes that digest synchronously over the exact input byte view and
|
|
74
|
+
rejects a conflicting caller-supplied digest. Optional MD5 is retained only as
|
|
75
|
+
source-system provenance.
|
|
76
|
+
|
|
77
|
+
The INFO v3 provenance subset uses the following keys. Package and translator
|
|
78
|
+
versions use semantic-version syntax; the complete INFO document also includes
|
|
79
|
+
source/output paths, selected backend-body mode, source/backend body coverage,
|
|
80
|
+
completeness flags, and stage/layout counts.
|
|
81
|
+
|
|
82
|
+
```json
|
|
83
|
+
{
|
|
84
|
+
"format": "CEWGPU",
|
|
85
|
+
"formatVersion": 3,
|
|
86
|
+
"packageKind": "tr2-effect-webgpu",
|
|
87
|
+
"targetBackend": "webgpu",
|
|
88
|
+
"backendPackage": "@carbonenginejs/runtime-resource/formats/webgpu",
|
|
89
|
+
"backendPackageVersion": "0.6.0",
|
|
90
|
+
"translator": "dxbc-js-wgsl",
|
|
91
|
+
"translatorVersion": "0.6.0",
|
|
92
|
+
"permutationGraph": {
|
|
93
|
+
"chunk": "PGRF",
|
|
94
|
+
"format": "CJS_EFFECT_PERMUTATION_GRAPH",
|
|
95
|
+
"formatVersion": 1,
|
|
96
|
+
"sha256": "0000000000000000000000000000000000000000000000000000000000000000",
|
|
97
|
+
"permutationCount": 1,
|
|
98
|
+
"uniqueBodyCount": 1
|
|
99
|
+
},
|
|
100
|
+
"effectReflection": {
|
|
101
|
+
"chunk": "RFLX",
|
|
102
|
+
"format": "CJS_CEWGPU_EFFECT_REFLECTION",
|
|
103
|
+
"formatVersion": 2,
|
|
104
|
+
"blobChunk": "RBLB",
|
|
105
|
+
"sha256": "0000000000000000000000000000000000000000000000000000000000000000",
|
|
106
|
+
"coverage": "all-unique",
|
|
107
|
+
"permutationCount": 6,
|
|
108
|
+
"bodyCount": 2,
|
|
109
|
+
"sourceProgramCount": 4,
|
|
110
|
+
"blobCount": 6,
|
|
111
|
+
"blobByteLength": 4096
|
|
112
|
+
},
|
|
113
|
+
"sourceBodyCoverage": "all-unique",
|
|
114
|
+
"backendBodyCoverage": "selected",
|
|
115
|
+
"bodyMode": "selected",
|
|
116
|
+
"completeness": {
|
|
117
|
+
"packageValid": true,
|
|
118
|
+
"sourceComplete": true,
|
|
119
|
+
"backendComplete": false,
|
|
120
|
+
"runtimeComplete": false
|
|
121
|
+
},
|
|
122
|
+
"sourceIdentity": {
|
|
123
|
+
"logicalPath": "res:/graphics/effect.dx11/example.sm_hi",
|
|
124
|
+
"game": "Eve",
|
|
125
|
+
"client": "tranquility",
|
|
126
|
+
"build": "0000000",
|
|
127
|
+
"byteLength": 1024,
|
|
128
|
+
"md5": null,
|
|
129
|
+
"sha256": "0000000000000000000000000000000000000000000000000000000000000000"
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
The `INFO`, `META`, and `ANLS` source labels must agree.
|
|
135
|
+
`INFO.sourceIdentity.logicalPath` is a separate canonical resource identity and
|
|
136
|
+
may differ from that diagnostic label.
|
|
137
|
+
|
|
138
|
+
The selected-effect validator also requires compact `ANLS` stages to omit raw
|
|
139
|
+
byte arrays and retain null DXBC/IR fields. `BuildEffect` runs the same reader
|
|
140
|
+
validation before returning `qualification.ok: true`.
|
|
141
|
+
|
|
142
|
+
## Source permutation graph
|
|
143
|
+
|
|
144
|
+
`BuildEffect` emits `CJS_EFFECT_PERMUTATION_GRAPH` version 1 in `PGRF`.
|
|
145
|
+
INFO v2/v3 points to the chunk and records its exact permutation and
|
|
146
|
+
unique-body counts. INFO v3 also binds the exact PGRF chunk bytes with
|
|
147
|
+
lower-case SHA-256. A pointer without the chunk, the chunk without a pointer,
|
|
148
|
+
or disagreeing counts/digest fail closed. Older selected-effect INFO v1/v2
|
|
149
|
+
packages without either remain readable.
|
|
150
|
+
|
|
151
|
+
PGRF preserves:
|
|
152
|
+
|
|
153
|
+
- ordered axes with index, exact name/options/default, description, and type;
|
|
154
|
+
- one variant for every first-axis-least-significant mixed-radix permutation
|
|
155
|
+
index;
|
|
156
|
+
- the exact option-index tuple and source body record for every variant;
|
|
157
|
+
- deterministic package-local body keys; and
|
|
158
|
+
- the byte length and lower-case SHA-256 digest of every unique raw source body
|
|
159
|
+
record.
|
|
160
|
+
|
|
161
|
+
The complete version-1 document shape is:
|
|
162
|
+
|
|
163
|
+
```json
|
|
164
|
+
{
|
|
165
|
+
"format": "CJS_EFFECT_PERMUTATION_GRAPH",
|
|
166
|
+
"formatVersion": 1,
|
|
167
|
+
"coverage": {
|
|
168
|
+
"permutations": "complete",
|
|
169
|
+
"bodies": "identity-only",
|
|
170
|
+
"reflection": "absent"
|
|
171
|
+
},
|
|
172
|
+
"axes": [
|
|
173
|
+
{
|
|
174
|
+
"index": 0,
|
|
175
|
+
"name": "QUALITY",
|
|
176
|
+
"options": [ "LOW", "HIGH" ],
|
|
177
|
+
"defaultOption": 1,
|
|
178
|
+
"description": "quality tier",
|
|
179
|
+
"type": 0
|
|
180
|
+
}
|
|
181
|
+
],
|
|
182
|
+
"variants": [
|
|
183
|
+
{
|
|
184
|
+
"permutationIndex": 0,
|
|
185
|
+
"optionIndices": [ 0 ],
|
|
186
|
+
"bodyKey": "body0",
|
|
187
|
+
"sourceRecord": { "offset": 256, "byteLength": 64 }
|
|
188
|
+
},
|
|
189
|
+
{
|
|
190
|
+
"permutationIndex": 1,
|
|
191
|
+
"optionIndices": [ 1 ],
|
|
192
|
+
"bodyKey": "body0",
|
|
193
|
+
"sourceRecord": { "offset": 256, "byteLength": 64 }
|
|
194
|
+
}
|
|
195
|
+
],
|
|
196
|
+
"bodies": [
|
|
197
|
+
{
|
|
198
|
+
"key": "body0",
|
|
199
|
+
"byteLength": 64,
|
|
200
|
+
"sha256": "0000000000000000000000000000000000000000000000000000000000000000"
|
|
201
|
+
}
|
|
202
|
+
]
|
|
203
|
+
}
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Axis and option counts plus `axes[].type` use the source format's unsigned
|
|
207
|
+
8-bit limits. Permutation indices, `bodies[].byteLength`, and source-record
|
|
208
|
+
offset/length fields use unsigned 32-bit limits. A `sourceRecord.offset` is an
|
|
209
|
+
absolute byte offset from the beginning of the exact compiled-effect input,
|
|
210
|
+
and `offset + byteLength` must not exceed `INFO.sourceIdentity.byteLength`.
|
|
211
|
+
Source ranges may be exact aliases or disjoint; partial overlaps fail closed.
|
|
212
|
+
The synchronous browser-safe producer and reader add an implementation
|
|
213
|
+
resource limit of 65,536 Cartesian permutations per effect; larger graphs fail
|
|
214
|
+
explicitly before variant materialization.
|
|
215
|
+
|
|
216
|
+
Raw body bytes are deduplicated with SHA-256 as a lookup accelerator and exact
|
|
217
|
+
byte equality as the final identity check. A body entry therefore provides one
|
|
218
|
+
package-local identity for a unique raw source-body byte sequence; the bytes
|
|
219
|
+
themselves are not embedded, and duplicate body digests are invalid. Body keys
|
|
220
|
+
are package-local because compiled body records refer to the enclosing
|
|
221
|
+
effect's shared string table; they are not cross-package content identities.
|
|
222
|
+
|
|
223
|
+
The graph's `coverage` declares `permutations: "complete"`,
|
|
224
|
+
`bodies: "identity-only"`, and `reflection: "absent"`. It records
|
|
225
|
+
builder-derived source topology that can be verified against the original
|
|
226
|
+
compiled effect; a package reader can validate only the graph's schema and
|
|
227
|
+
internal relationships because raw bodies are not embedded. It does not claim
|
|
228
|
+
reflection or backend translation by itself; INFO v3 joins separate RFLX v2
|
|
229
|
+
for complete source reflection. `META.bodyIndex` remains the
|
|
230
|
+
selected permutation index; the matching PGRF variant supplies its body key.
|
|
231
|
+
Stage filtering remains selected-body-local and does not change the graph.
|
|
232
|
+
|
|
233
|
+
The package-kind marker is the opt-in discriminator: without it, a container is
|
|
234
|
+
generic even when it happens to use the standard chunk tags. A consumer that
|
|
235
|
+
expects an effect package must therefore require the marker as well as calling
|
|
236
|
+
the reader.
|
|
237
|
+
|
|
238
|
+
## Complete source reflection
|
|
239
|
+
|
|
240
|
+
For compiled-effect version 15, `BuildEffect` emits
|
|
241
|
+
`CJS_CEWGPU_EFFECT_REFLECTION` version 2 in `RFLX` and one shared raw `RBLB`
|
|
242
|
+
byte arena. INFO v3 points to both, binds the exact RFLX chunk with SHA-256,
|
|
243
|
+
and records all-unique coverage plus permutation, body, source-program, blob,
|
|
244
|
+
and byte counts. A missing pointer/chunk, a digest or count disagreement, or
|
|
245
|
+
partial coverage fails closed. Versions 8-14 remain INFO v2 without RFLX/RBLB.
|
|
246
|
+
|
|
247
|
+
RFLX v2 stores common source identity and a `bodies` array in exact PGRF body
|
|
248
|
+
order. Each body records its PGRF `bodyKey`, first representative permutation
|
|
249
|
+
index, byte length, SHA-256, and complete body-local
|
|
250
|
+
`CJS_EFFECT_BODY_REFLECTION` version-1 effect graph. It preserves:
|
|
251
|
+
|
|
252
|
+
- complete technique, pass, stage, and library topology;
|
|
253
|
+
- render states, authored constants/resources/UAVs/samplers and annotations;
|
|
254
|
+
- exact stage and library source programs;
|
|
255
|
+
- exact immutable constant-default byte vectors;
|
|
256
|
+
- signatures, registers, static samplers, pipeline inputs, and thread groups;
|
|
257
|
+
- the opaque version-15 native source hash; and
|
|
258
|
+
- source/body identities joined to INFO and PGRF.
|
|
259
|
+
|
|
260
|
+
Every portable byte array is replaced by an exact reference containing
|
|
261
|
+
`blobKey`, `offset`, `byteLength`, and lower-case SHA-256. One
|
|
262
|
+
`RFLX.blobStore` lists canonical contiguous `blobN` records covering RBLB
|
|
263
|
+
exactly. Identical payloads are deduplicated across every reflected body by
|
|
264
|
+
digest plus exact byte equality; dangling, overlapping, corrupt, or
|
|
265
|
+
unreferenced payloads are rejected.
|
|
266
|
+
|
|
267
|
+
The reader reconstructs and validates one portable document for every unique
|
|
268
|
+
body. It requires every PGRF body exactly once in PGRF order, with the first
|
|
269
|
+
matching variant as representative. For selected-backend reconciliation it
|
|
270
|
+
joins `META.bodyIndex` to `PGRF.variants[index].bodyKey`, then to the matching
|
|
271
|
+
RFLX body and ANLS pass/stage identities.
|
|
272
|
+
|
|
273
|
+
JSON reads expose RFLX references plus `reflectionBlobByteLength`. To consume
|
|
274
|
+
the exact bytes, read with `emit: "raw"` and use
|
|
275
|
+
`CewgpuPackage.GetReflectionBlob(referenceOrKey)`, which returns an owned copy.
|
|
276
|
+
An object reference must exactly equal its inventory record; a string performs
|
|
277
|
+
a package-local blob-key lookup. To consume one complete portable body, use
|
|
278
|
+
`CewgpuPackage.GetPortableEffectReflection(permutationIndex)`. It selects the
|
|
279
|
+
PGRF variant (defaulting to `META.bodyIndex`), joins its RFLX body, expands all
|
|
280
|
+
references to fresh owned `Uint8Array` payloads, and reruns the
|
|
281
|
+
`@carbonenginejs/runtime-resource/formats/hlsl/portable` validator.
|
|
282
|
+
|
|
283
|
+
INFO v3 declares `sourceBodyCoverage: "all-unique"` and
|
|
284
|
+
`completeness.sourceComplete: true`. This means complete portable source-effect
|
|
285
|
+
semantics for that exact compiled input. It does not embed raw body records and
|
|
286
|
+
is not an archival reconstruction of the original `.sm_*` bytes. The accessor
|
|
287
|
+
returns a fresh, owned plain portable document. `runtime-resource`
|
|
288
|
+
`Tr2EffectRes` consumes it to select and cache a canonical device-free
|
|
289
|
+
`Tr2Shader`; renderer handles, resource-set descriptions, derived dynamic
|
|
290
|
+
classifications, layouts, and programs remain engine-owned.
|
|
291
|
+
PGRF
|
|
292
|
+
correctly continues to describe its own body table as `identity-only` with
|
|
293
|
+
`reflection: "absent"` because RFLX is a separate document.
|
|
294
|
+
|
|
295
|
+
`bodyMode: "selected"` and `backendBodyCoverage: "selected"` describe ANLS and
|
|
296
|
+
WGSL scope. `backendComplete` and `runtimeComplete` remain false.
|
|
297
|
+
|
|
298
|
+
## All-body backend graph (`WGSB`)
|
|
299
|
+
|
|
300
|
+
`mode: "all"` (or the `allPermutations: true` compatibility request) additionally
|
|
301
|
+
translates every unique source body and stores the result in a `WGSB`
|
|
302
|
+
`CJS_WGSL_BODY_SET` chunk. It requires complete version-15 source reflection,
|
|
303
|
+
because the unique-body inventory comes from RFLX/PGRF.
|
|
304
|
+
|
|
305
|
+
The translation unit is one pass of one body: the binding plan, the
|
|
306
|
+
resource-transform plan, and every stage's WGSL are derived together, so bodies
|
|
307
|
+
whose pass carries byte-identical stage bytecode, semantic bindings, and render
|
|
308
|
+
states share exactly one unit. `bodies[]` maps each `bodyKey` to
|
|
309
|
+
`{passKey, unitKey}` references, and `passUnits[]` holds the shared translated
|
|
310
|
+
programs, layouts, and transforms. Real Quad ship families collapse several
|
|
311
|
+
hundred passes into a small fraction of that many units.
|
|
312
|
+
|
|
313
|
+
A body that cannot be lowered is retained as
|
|
314
|
+
`status: "unsupported"` with an explicit reason and no passes. Its complete
|
|
315
|
+
source reflection remains in RFLX, so a partial backend never removes source
|
|
316
|
+
truth; `coverage.bodies` and `INFO.backendBodyCoverage` then report `partial`
|
|
317
|
+
instead of `all-unique`.
|
|
318
|
+
|
|
319
|
+
`INFO.backendBodySet` binds the exact chunk digest and counts. Selected-mode
|
|
320
|
+
packages must not carry the chunk, and an all-body package must contain one
|
|
321
|
+
record for every unique permutation-graph body. All-body packages still emit
|
|
322
|
+
`WGSL` for the selected body, and its programs are byte-identical to the
|
|
323
|
+
corresponding shared translation units.
|
|
324
|
+
|
|
325
|
+
Translating every body still does not make the package backend- or
|
|
326
|
+
runtime-complete: `backendComplete` and `runtimeComplete` stay false until the
|
|
327
|
+
engine parses these records, realizes their layouts and resource transforms,
|
|
328
|
+
and passes an exact draw gate.
|
|
329
|
+
|
|
330
|
+
Legacy INFO v2 may omit reflection or carry selected-body RFLX v1/RBLB. The
|
|
331
|
+
current reader validates both forms. INFO v3 requires PGRF plus RFLX v2/RBLB;
|
|
332
|
+
older readers do not accept this new schema.
|
|
333
|
+
|
|
334
|
+
## Analysis document
|
|
335
|
+
|
|
336
|
+
The current analysis document records normalized data for one selected effect
|
|
337
|
+
body:
|
|
338
|
+
|
|
339
|
+
- selected permutation and effect body;
|
|
340
|
+
- techniques, passes, and stage topology;
|
|
341
|
+
- Carbon binding-manifest data;
|
|
342
|
+
- per-stage bytecode summaries without raw byte arrays; and
|
|
343
|
+
- null DXBC/IR fields reserved for return-only `AnalyzeEffect` diagnostics.
|
|
344
|
+
|
|
345
|
+
Analysis is retained as provenance even when `BuildEffect` emits WGSL for only
|
|
346
|
+
some complete selected passes. `ANLS` is not lossless source reflection. It
|
|
347
|
+
omits exact constant-default bytes, complete nested reflection/libraries, and
|
|
348
|
+
some typed annotations needed to hydrate a complete source effect resource.
|
|
349
|
+
Those values live in RFLX/RBLB for version-15 input. Ordered axes and the total
|
|
350
|
+
permutation-index-to-body mapping live in PGRF rather than ANLS.
|
|
351
|
+
|
|
352
|
+
`AnalyzeEffect` uses transient selected-body bytecode to return DXBC and,
|
|
353
|
+
when requested, shader-IR diagnostics. `BuildEffect` uses the same transient
|
|
354
|
+
byte index for WGSL compilation but does not persist raw bytecode, decoded
|
|
355
|
+
instructions, or compiler IR in `ANLS`.
|
|
356
|
+
|
|
357
|
+
`BuildEffect` records `bodyMode` in `INFO` and `META`; for INFO v3 this is
|
|
358
|
+
explicitly backend scope, and it stays `selected` unless all-body packaging was
|
|
359
|
+
requested. Its returned qualification record uses
|
|
360
|
+
`validator: "cewgpu-structural"` and reports `packageValid: true`. Version-15
|
|
361
|
+
packages report `sourceComplete: true`, `backendComplete: false`, and
|
|
362
|
+
`runtimeComplete: false`; versions 8-14 report all three completeness flags
|
|
363
|
+
false. These flags prevent source preservation from being mistaken for
|
|
364
|
+
all-body translation or runtime validation. The same booleans are embedded
|
|
365
|
+
under `INFO.completeness`.
|
|
366
|
+
|
|
367
|
+
`@carbonenginejs/runtime-resource/formats/hlsl` owns source parsing, selected-body resolution,
|
|
368
|
+
unique-body enumeration, and the shared browser-safe portable reflection
|
|
369
|
+
contract. `format-webgpu` validates the parsed header into PGRF, aggregates
|
|
370
|
+
every unique version-15 body into RFLX/RBLB, owns transient byte indexing for
|
|
371
|
+
diagnostics/translation, and owns selected backend programs, layouts, and
|
|
372
|
+
transforms. Lossless reflection remains separate from compact ANLS
|
|
373
|
+
diagnostics.
|
|
374
|
+
|
|
375
|
+
## Structured WGSL set
|
|
376
|
+
|
|
377
|
+
`CJS_WGSL_SET` version 2 records contain emitted shader descriptors and
|
|
378
|
+
optional pass-level `layouts`. A layout records the exact numeric bind group
|
|
379
|
+
and binding slots already present in the WGSL source. A set remains version 2
|
|
380
|
+
when its source resources map one-to-one to physical WebGPU bindings.
|
|
381
|
+
|
|
382
|
+
Each binding keeps:
|
|
383
|
+
|
|
384
|
+
- a D3D-derived base `identity`;
|
|
385
|
+
- a resource-resolution `scopeIdentity`;
|
|
386
|
+
- stage visibility;
|
|
387
|
+
- the buffer, texture, or sampler layout; and
|
|
388
|
+
- its numeric group and binding.
|
|
389
|
+
|
|
390
|
+
Version 2 treats resource tuples as stage-scoped unless the caller explicitly
|
|
391
|
+
confirms one compatible shared identity. The builder rejects duplicate scopes,
|
|
392
|
+
duplicate numeric slots, mixed shared and stage-scoped forms, incomplete
|
|
393
|
+
visibility, and stage/layout conflicts. It never renumbers slots during WGSL
|
|
394
|
+
set assembly.
|
|
395
|
+
|
|
396
|
+
Version 1 binding plans remain accepted as legacy input. Ordinary new plans
|
|
397
|
+
and WGSL sets use version 2.
|
|
398
|
+
|
|
399
|
+
### Version 3 resource transforms
|
|
400
|
+
|
|
401
|
+
A set becomes version 3 when the compiler proves that several logical source
|
|
402
|
+
resources can be represented by one physical WebGPU resource. The top-level
|
|
403
|
+
`resourceTransforms` array records the realization recipe; the matching
|
|
404
|
+
physical layout binding carries its `transformId` and `arrayLayerCount`.
|
|
405
|
+
|
|
406
|
+
The currently defined version-1 recipe has this shape:
|
|
407
|
+
|
|
408
|
+
```json
|
|
409
|
+
{
|
|
410
|
+
"id": "Main.pass0:detail-map-array:sampled-resource:0:16",
|
|
411
|
+
"version": 1,
|
|
412
|
+
"kind": "texture-2d-array",
|
|
413
|
+
"layoutKey": "Main.pass0",
|
|
414
|
+
"stage": "fragment",
|
|
415
|
+
"inputs": [
|
|
416
|
+
{
|
|
417
|
+
"parameter": "Detail1Map",
|
|
418
|
+
"layer": 0,
|
|
419
|
+
"identity": "sampled-resource:0:16",
|
|
420
|
+
"scopeIdentity": "sampled-resource:0:16@fragment"
|
|
421
|
+
},
|
|
422
|
+
{
|
|
423
|
+
"parameter": "Detail2Map",
|
|
424
|
+
"layer": 1,
|
|
425
|
+
"identity": "sampled-resource:0:17",
|
|
426
|
+
"scopeIdentity": "sampled-resource:0:17@fragment"
|
|
427
|
+
}
|
|
428
|
+
],
|
|
429
|
+
"output": {
|
|
430
|
+
"name": "DetailMapArray",
|
|
431
|
+
"identity": "sampled-resource:0:16",
|
|
432
|
+
"scopeIdentity": "sampled-resource:0:16@fragment",
|
|
433
|
+
"viewDimension": "2d-array",
|
|
434
|
+
"layerCount": 2
|
|
435
|
+
},
|
|
436
|
+
"representation": "native-or-rgba8",
|
|
437
|
+
"missingLayer": "reject"
|
|
438
|
+
}
|
|
439
|
+
```
|
|
440
|
+
|
|
441
|
+
Inputs are ordered by their exact fixed array layer. The output reuses layer
|
|
442
|
+
zero's D3D identity; later logical inputs do not remain as physical bindings.
|
|
443
|
+
The compiler emits every affected sample with that fixed integer layer.
|
|
444
|
+
|
|
445
|
+
`native-or-rgba8` requires the consumer to realize one compatible
|
|
446
|
+
`texture_2d_array` from the named source textures, either in a shared native
|
|
447
|
+
representation or after decoding every layer to RGBA8. Dimensions, mip
|
|
448
|
+
coverage, sample type, and texture format must be compatible with one WebGPU
|
|
449
|
+
array view. `missingLayer: "reject"` forbids substituting a fallback layer.
|
|
450
|
+
|
|
451
|
+
The set builder fails closed unless every recipe:
|
|
452
|
+
|
|
453
|
+
- targets an emitted fragment stage in its own pass;
|
|
454
|
+
- links exactly one `texture_2d_array<f32>` physical binding;
|
|
455
|
+
- numbers distinct inputs contiguously from layer zero;
|
|
456
|
+
- matches the binding's identity, view dimension, and layer count; and
|
|
457
|
+
- removes only the later input scopes from that recipe's owning pass.
|
|
458
|
+
|
|
459
|
+
WGSL-set version 3 is currently a compiler/module contract. The committed
|
|
460
|
+
`engine-webgpu` package reader accepts versions 1 and 2 and rejects version 3,
|
|
461
|
+
so a runtime must add explicit recipe realization before it can consume these
|
|
462
|
+
packages. Raw emitted modules may still be validated independently.
|
|
463
|
+
|
|
464
|
+
## Encoding values
|
|
465
|
+
|
|
466
|
+
`Build` accepts chunk payloads as strings, plain objects, typed bytes,
|
|
467
|
+
`ArrayBuffer`, or other array-buffer views. Plain objects are serialized as
|
|
468
|
+
UTF-8 JSON. Byte values are preserved without interpretation.
|
|
469
|
+
|
|
470
|
+
`Read(..., { emit: "raw" })` exposes the internal package object and chunk byte
|
|
471
|
+
views for zero-copy tooling. Treat those chunk views as immutable. Mutating
|
|
472
|
+
them after JSON or reflection lookup is outside the reader contract; rebuild
|
|
473
|
+
or reread a package instead.
|
|
474
|
+
|
|
475
|
+
## Related documentation
|
|
476
|
+
|
|
477
|
+
- [Effect packaging guide](../guides/effect-packaging.md)
|
|
478
|
+
- [Public API reference](../reference/api.md)
|
|
479
|
+
- [WGSL compatibility](../reference/wgsl-compatibility.md)
|