@carbonenginejs/runtime-resource 0.11.1 → 0.12.1

package/docs/formats/hlsl/reference/portable-reflection.md

@@ -0,0 +1,141 @@
+# Portable body reflection
+
+Status: Evolving
+Scope: `@carbonenginejs/runtime-resource/formats/hlsl/portable`
+Audience: Backend effect-packager authors
+Summary: Defines exact, handle-free reflection for one compiled effect body.
+
+## Purpose
+
+The ordinary JSON and metadata modes describe one option-selected effect for
+inspection. Backend package builders need a stronger boundary: exact
+permutation-table selection, authored constant defaults, source programs, and
+all reflection fields without renderer handles.
+
+The portable subpath supplies that boundary:
+
+```js
+import {
+    buildEffectBodyReflection,
+    enumerateUniqueEffectBodies,
+    readEffectBodyReflection,
+    validateEffectBodyReflection
+} from "@carbonenginejs/runtime-resource/formats/hlsl/portable";
+
+const document = readEffectBodyReflection(bytes, {
+    source: "effect.sm_depth",
+    permutationIndex: 4
+});
+
+validateEffectBodyReflection(document);
+
+const groups = enumerateUniqueEffectBodies(effectRes);
+const uniqueReflections = groups.map((group) =>
+    buildEffectBodyReflection(effectRes, group.permutationIndex));
+```
+
+`buildEffectBodyReflection(effectRes, permutationIndex)` accepts an already
+loaded raw `Tr2EffectRes`. This avoids reparsing when a backend builder already
+uses `readEffectAnalysis`.
+
+`enumerateUniqueEffectBodies(effectRes)` inspects version-15 source records and
+bytes without decoding. It returns first-occurrence-ordered groups containing
+one canonical `permutationIndex`/`sourceRecord` plus every byte-identical
+variant alias. Exact range aliases are a fast path; distinct ranges are
+fingerprinted and then compared byte-for-byte. The inventory caps the
+Cartesian body table at 65,536 records, rejects partial overlaps, and leaves
+the effect cache and state-manager registries unchanged.
+
+## Contract
+
+Version 1 accepts compiled effect version 15 only. Earlier container versions
+synthesize or normalize several signature fields while reading; rejecting them
+keeps every portable field source-exact.
+
+The root identifies `CJS_EFFECT_BODY_REFLECTION` version 1,
+`mode: "single-body"`, `keyScope: "body-local"`,
+`coverage.bodies: "single"`, the diagnostic source label and
+compiled-effect version/compiler/shared-table/source envelope in `source`, the exact
+`permutationIndex`, and its
+`sourceRecord.offset` / `sourceRecord.byteLength` span. Its effect graph
+preserves:
+
+- ordered parameter annotation groups;
+- ordered techniques, passes, raw render-state pairs, and libraries;
+- every present vertex, pixel, compute, geometry, hull, or domain stage;
+- constant, resource, UAV, dynamic sampler, and stage-annotation metadata;
+- complete pipeline-input, register, static-sampler, and thread-group
+  signatures;
+- exact authored constant-default bytes and declared length; and
+- owned copies of every stage and library source-program payload.
+
+Every source-program record retains its exact shared-table offset. The
+validator requires its complete byte range to remain within
+`source.stringTableByteLength`.
+Stage programs are explicitly `kind: "stage"` and carry their authored stage
+identity. DXR library programs are `kind: "library"` and deliberately carry no
+synthetic compute-stage identity.
+
+Numeric BOOL, INT, and FLOAT annotations retain their exact serialized
+`rawValue`. Sampler floats retain raw IEEE-754 bits. Static sampler border
+colors remain their source enum byte; dynamic sampler border colors remain
+four float bit patterns.
+
+`source.nativeHash` is the compiler-provided v15 hash field, not a package
+content digest. The backend package must independently hash its whole source
+and body payloads.
+
+Version-15 resource and signature counts retain Carbon's authored zero value.
+For heap-view/bindless SRV, UAV, and sampler bindings, `arrayElements: 0` and
+the matching signature `arrayCount: 0` / `registerCount: 0` represent an
+unbounded descriptor range; they are valid source reflection, not an empty or
+malformed binding. Every resource and UAV map entry must reconcile with
+exactly one signature register; signature-only records remain valid.
+
+The compiled body has no authored effect-name field. The caller-supplied
+diagnostic name or path is retained only as `source.label`.
+
+The serializer rejects defaults or constant extents beyond Carbon's 4,096-byte
+stage constant-buffer limit.
+
+## Source truth and realization
+
+The portable document excludes shader, program, render-state, sampler, and
+library handles. It also excludes resource-set descriptions, heap-view arrays,
+backend layouts, masks, sort values, and caches. `runtime-resource` consumes
+the document to hydrate the canonical device-free shader/reflection graph,
+select permutations, and cache shaders. Engines own handles, layouts,
+resource sets, programs, pipelines, and other GPU realization.
+
+Register dynamic classification is not persisted. It depends on per-frame
+reader/engine policy and is not an authored binary field.
+
+Authored constant defaults are separate from the raw model's mutable
+`constantValues`. Carbon-compatible sampler-heap realization may zero-extend
+the latter; it cannot change the portable source prefix.
+
+## Selection and completeness
+
+The parser-internal `GetShaderByIndex(index)` and the portable serializer
+bypass global and local option selection. This is distinct from
+`runtime-resource` `Tr2EffectRes.GetShaderByIndex`, which hydrates and caches a
+canonical shader. The portable serializer performs a fresh non-caching decode
+with a temporary state manager, so mutation of a previously cached raw shader
+cannot change reflection rebuilt from the same owned source bytes.
+This makes a body-table index stable even when an application has global
+effect options.
+
+Version 1 describes one complete listed body. Body-local technique, pass,
+stage, and library keys must be namespaced by a backend package before they are
+flattened across bodies. This document does not claim that a backend package
+contains every permutation body or a program for every body. A backend
+container must reconcile all three of `permutationIndex`,
+`sourceRecord.offset`, and `sourceRecord.byteLength` with its own body identity
+and permutation graph, then reconcile the source byte length and native hash
+before claiming completeness.
+
+## Related documentation
+
+- [JSON and metadata graphs](json-graph.md)
+- [Advanced analysis exports](advanced-analysis.md)
+- [Architecture](../architecture.md)

package/docs/formats/provenance.md

@@ -6,8 +6,10 @@ Audience: Users, integrators, and maintainers
 Summary: Records where each format implementation came from, retained snapshots and digests, and what was deliberately not copied.
 
 On 2026-07-13, the non-shader runtime format implementations below were copied
-once into `runtime-resource`. Their standalone repositories remain frozen with
-their existing APIs and names; they are not upstreams for the runtime copies.
+once into `runtime-resource`. The table records the donor revisions or
+working-tree states used for that migration; it does not assert that a
+standalone repository still exists. Any surviving standalone repository is
+historical provenance, not an upstream for the runtime copy.
 
 Copied paths were `src/` and the behavioral `test/` corpus. Package publishing
 scripts and CLIs were not copied. Exact donor license and notice files are kept
@@ -53,26 +55,32 @@ GR2/GSF reader joined the runtime copies:
 |---|---|---|---|
 | `format-gr2` | `fa64607de7a3a96ed3b1aec5288bf71057642043` (v0.2.0, MIT) | `CjsGr2Format` | `@carbonenginejs/runtime-resource/formats/gr2` |
 
-The copied engine keeps its donor class name (`CjsFormatGr2`, re-exported)
-under `formats/gr2/core/`; `CjsGr2Format` is the runtime-authored contract
-wrapper. Donor license and notice files are kept under
-`format-notices/gr2/`.
+The donor engine was folded into the single public `CjsGr2Format` class, so
+the copied `CjsFormatGr2` name is gone and `formats/gr2/core/` holds only
+role-named helper modules, matching every other format. Donor license and
+notice files are kept under `format-notices/gr2/`.
 
 ## Black definition snapshot
 
 The Black reader uses the package-owned generated definition snapshot at
-`src/formats/black/core/black-schema-v1-2026-07-11.json`. The Red format exposes
+`src/formats/black/core/black-schema-v1-2026-07-23.json`. The Red format exposes
 the same catalog for discovery, but its YAML reader currently accepts named
-fields without registry enforcement. The snapshot was copied from
-`format-carbon` revision `d2a3c67cf3d46e8ba78ca19e66558d868178ec24`
-with SHA-256
-`008ECB29E670EFC678B471A6EFF099600A29C2907912FC42B854995904604691`.
+fields without registry enforcement. The refreshed consumer snapshot landed
+in runtime-resource revision `6beda70` with SHA-256
+`9234DD86DBB5EB3676EF060E383ADBCE8F073941E76A32865D3528DD5DA7B737`.
+Its exact tools-core generator revision was not recorded and remains unknown;
+this page does not infer one from the consumer commit.
 
 This retained generated artifact keeps the published readers deterministic and
 browser-safe without a runtime dependency on a sibling checkout or an
-unpublished `format-carbon` export. `format-carbon` remains the build-time
-authority for future schema regeneration; an updated snapshot must record its
-new source revision and digest here.
+unpublished generator export. Future schema generation is owned by
+`tools-core/schema`. A consumer of generated output is responsible for
+reviewing and copying the result into its own worktree; an updated snapshot
+must record the generator revision and new digest here.
+
+This generator boundary is transitional. As runtime classes gain embedded
+schema, the shared snapshot and its copy-in step can be reduced and eventually
+removed once every required class is self-describing.
 
 ## Native additions
 
@@ -85,6 +93,11 @@ format attribution rather than fork provenance.
 | Wwise soundbank (`.bnk`) | `CjsBnkFormat` | `@carbonenginejs/runtime-resource/formats/bnk` | Original code; chunk layout from public community documentation (ww2ogg, vgmstream, wwiser), no code copied. Also carries the SoundbanksInfo JSON helpers (`parseSoundbanksInfo`, `buildSoundbanksCatalog`, `joinSoundbanksInfo`) and `wwiseIdFromName` (FNV-1 32 of the lowercased name, verified against EVE bank/language ids). HIRC entries additionally decode version-stable typed fields (event action lists, action type/target, sound and music-track source ids), pinned by hexdump against bank generator version 150, and the `wwise` static namespace groups the domain toolkit (SoundbanksInfo helpers, id hash, `wwise.eventMediaFromBanks` event → media resolution over inspected banks — graph interpretation for consumers; never used by the resource lifecycle). |
 | Wwise media (`.wem`) | `CjsWemFormat` | `@carbonenginejs/runtime-resource/formats/wem` | Original code; container/codec-tag behavior from public community documentation (ww2ogg, vgmstream, wwiser), no code copied. Includes a Wwise-Vorbis→Ogg repacker (`emit: "ogg"`), an original reimplementation of the ww2ogg algorithm with inline granule computation (no revorb pass needed), and a PTADPCM/16-bit-PCM decoder (`emit: "pcm"` / `toPcm()`, AudioBuffer-ready float32; PTADPCM algorithm from community documentation, verified against EVE media). |
 
+The BNK toolkit also exposes `wwise.sfxNodesFromBanks`: typed, exact-end
+version-150 Random/Sequence, Switch/State, and Layer tails plus event/action
+records and explicit parser diagnostics. It preserves resource facts for
+consumer-owned lowering and is never invoked by the resource lifecycle.
+
 ## Post-fork additions inside copied formats
 
 - `formats/dds` gained original, dependency-free **BC6H and BC7 CPU decoders**
@@ -157,9 +170,11 @@ snapshot must record its new source and digest here.
 - `format-gr2` migrated into `formats/gr2` on 2026-07-24 (see the dated
   table above) after its EUPL constraint was resolved; its standalone
   repository is now a frozen legacy distribution like the other donors.
-- `format-carbon` remains the schema emitter/generator and build-time schema
-  authority. Black consumes its published definitions; Red exposes the copied
-  catalog but does not yet enforce it while reading YAML fields.
+- `format-carbon` is historical provenance for the current copied snapshot,
+  not the authority for future regeneration. `tools-core/schema` owns schema
+  generation, while each consuming package reviews and copies generated output
+  into its own worktree. Black consumes the retained snapshot; Red exposes the
+  copied catalog but does not yet enforce it while reading YAML fields.
 - The shader formats were excluded from the original migration while their
   implementations were still being finished. That sequencing guard was lifted
   on 2026-07-29 and they moved in as ordinary formats. All four landed:

package/docs/formats/webgl/README.md

@@ -0,0 +1,57 @@
+# WebGL format documentation
+
+Status: Evolving
+Scope: `@carbonenginejs/runtime-resource/formats/webgl`
+Audience: Shader-tool authors, runtime integrators, and maintainers
+Summary: Explains CEWG packages, portable reflection, and DXBC-to-GLSL conversion.
+
+## Purpose
+
+`@carbonenginejs/runtime-resource/formats/webgl` reads and builds CEWG packages and converts
+supported compiled Carbon effect stages into GLSL ES 3.00 package data. It
+preserves complete source permutation topology and, for version-15 effects,
+portable reflection for every unique body without creating live shader or GPU
+objects.
+
+## Use this package when
+
+Use `format-webgl` to inspect or build CEWG bytes, translate supported DXBC
+stages, or convert one compiled effect while preserving all source
+permutations and portable reflection. Use runtime and engine packages for live
+shader objects, resource selection, bindings, and draws.
+
+## Where it fits
+
+```text
+compiled effect bytes
+        |
+        +---- format-hlsl ---- portable source reflection
+        +---- format-dxbc ---- decoded shader programs
+        |
+        v
+     format-webgl
+  CEWG + GLSL ES 3.00
+        |
+        +---- runtime-resource ---- selection, cache, Tr2Shader hydration
+        +---- runtime-trinity ----- effect/material facade and parameters
+        `---- WebGL engine -------- programs, bindings, and draws
+```
+
+## Start here
+
+```js
+import { CjsWebglFormat } from "@carbonenginejs/runtime-resource/formats/webgl";
+
+const summary = CjsWebglFormat.inspect(packageBytes);
+const packageData = CjsWebglFormat.read(packageBytes);
+```
+
+## Documentation map
+
+- [Architecture and ownership](architecture.md)
+- [Effect reflection contract](effect-reflection.md)
+- [Constant-buffer layouts](carbon-constant-layouts.md)
+- [Declaration and I/O lowering](decl-io.md)
+- [Structured-memory lowering](memory-structured.md)
+- [Texture sampling](texture-sample.md)
+- [Class-purpose catalog](reference/classes/README.md)

package/docs/formats/webgl/architecture.md

@@ -0,0 +1,70 @@
+# Architecture and ownership
+
+Status: Evolving
+Scope: `@carbonenginejs/runtime-resource/formats/webgl`
+Audience: Shader-tool authors, runtime integrators, and maintainers
+Summary: Defines the package's compiler, container, runtime, and engine boundaries.
+
+## Purpose
+
+`format-webgl` is the transformation and package layer between compiled
+Carbon-style effects and WebGL engine realization. It emits portable CEWG data
+and GLSL ES 3.00 without constructing mutable runtime shader classes or WebGL
+objects.
+
+## Dependency direction
+
+```text
+@carbonenginejs/runtime-resource/formats/hlsl
+          |
+          +---- effect parsing and portable reflection
+          |
+          v
+@carbonenginejs/runtime-resource/formats/webgl <---- @carbonenginejs/runtime-resource/formats/dxbc
+          |                              decoded DXBC
+          |
+          +---- CEWG bytes, metadata, reflection, GLSL
+          |
+          +---- runtime-resource ---- package selection + Tr2Shader hydration
+          +---- runtime-trinity ----- effect/material facade + parameters
+          `---- WebGL engine -------- GPU realization
+```
+
+Tools may call the public byte-oriented API, but the format package does not
+import the toolchain. Runtime packages consume immutable package data through
+an explicit adapter boundary.
+
+## Owned responsibilities
+
+- Reading, inspecting, validating, and building CEWG v1 containers.
+- Preserving every source permutation and unique body identity.
+- Packing complete version-15 portable reflection and exact shared bytes.
+- Translating supported DXBC vertex, pixel, and bounded map-style compute
+  stages into GLSL ES 3.00.
+- Preserving backend bodies, stages, programs, manifests, and render states.
+- Rejecting incomplete or inconsistent canonical effect packages.
+
+## Ownership elsewhere
+
+- `format-hlsl` owns compiled-effect parsing and the portable reflection
+  schema.
+- `format-dxbc` owns DXBC decoding.
+- `runtime-resource` owns `Tr2EffectRes`, package bytes, option selection,
+  per-permutation cache identity, canonical `Tr2Shader`, reflection records,
+  sampler setup records, and portable hydration.
+- `runtime-trinity` owns the mutable `Tr2Effect`/`Tr2Material` facade,
+  parameters, authored options, and sampler overrides.
+- The WebGL engine owns program compilation, locations, layouts, resource
+  binding, uploads, draws, and context recovery.
+
+## Completeness boundary
+
+Version-15 CEWG can be source-complete while remaining backend- and
+runtime-incomplete. Successful GLSL translation and compile/link do not prove
+the engine's physical binding/layout contract or a rendered result.
+
+## Related documentation
+
+- [Package documentation](README.md)
+- [Effect reflection contract](effect-reflection.md)
+- [Class-purpose catalog](reference/classes/README.md)