@carbonenginejs/runtime-resource 0.7.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/NOTICE +1 -1
- package/README.md +67 -537
- package/dist/formats/dds/core/bc6h.js +288 -0
- package/dist/formats/dds/core/bc6h.js.map +1 -0
- package/dist/formats/dds/core/bc7.js +251 -0
- package/dist/formats/dds/core/bc7.js.map +1 -0
- package/dist/formats/dds/core/helpers.js +6 -3
- package/dist/formats/dds/core/helpers.js.map +1 -1
- package/dist/formats/gr2/CjsGr2Format.js +46 -0
- package/dist/formats/gr2/CjsGr2Format.js.map +1 -0
- package/dist/formats/gr2/core/CjsFormatGr2.js +273 -0
- package/dist/formats/gr2/core/CjsFormatGr2.js.map +1 -0
- package/dist/formats/gr2/core/bitknit2.js +280 -0
- package/dist/formats/gr2/core/bitknit2.js.map +1 -0
- package/dist/formats/gr2/core/curves.js +1047 -0
- package/dist/formats/gr2/core/curves.js.map +1 -0
- package/dist/formats/gr2/core/gsf.js +72 -0
- package/dist/formats/gr2/core/gsf.js.map +1 -0
- package/dist/formats/gr2/core/helpers.js +332 -0
- package/dist/formats/gr2/core/helpers.js.map +1 -0
- package/dist/formats/gr2/core/json.js +622 -0
- package/dist/formats/gr2/core/json.js.map +1 -0
- package/dist/formats/gr2/core/oodle1.js +388 -0
- package/dist/formats/gr2/core/oodle1.js.map +1 -0
- package/dist/formats/gr2/core/reader.js +617 -0
- package/dist/formats/gr2/core/reader.js.map +1 -0
- package/dist/formats/gr2/core/tangents.js +48 -0
- package/dist/formats/gr2/core/tangents.js.map +1 -0
- package/dist/formats/gr2/core/targets.js +351 -0
- package/dist/formats/gr2/core/targets.js.map +1 -0
- package/dist/formats/gr2/index.js +3 -0
- package/dist/formats/gr2/index.js.map +1 -0
- package/dist/formats/index.js +7 -0
- package/dist/formats/index.js.map +1 -1
- package/dist/index.js +1 -0
- package/dist/index.js.map +1 -1
- package/dist/resources/Tr2TexturePipelineStepGenerateMips.js +22 -0
- package/dist/resources/Tr2TexturePipelineStepGenerateMips.js.map +1 -0
- package/dist/resources/TriGeometryRes.js +24 -1
- package/dist/resources/TriGeometryRes.js.map +1 -1
- package/docs/README.md +69 -0
- package/docs/architecture.md +86 -0
- package/docs/concepts/resource-lifecycle.md +217 -0
- package/docs/formats/README.md +104 -0
- package/{FORMAT-PROVENANCE.md → docs/formats/provenance.md} +34 -5
- package/docs/formats/stl.md +37 -0
- package/docs/formats/wwise.md +44 -0
- package/docs/reference/events.md +92 -0
- package/docs/reference/motherlode-cache.md +244 -0
- package/docs/reference/queues.md +102 -0
- package/docs/reference/reload.md +107 -0
- package/docs/reference/texture-arrays.md +113 -0
- package/docs/reference/texture-pipeline.md +53 -0
- package/docs/roadmap.md +104 -0
- package/format-notices/gr2/LICENSE +21 -0
- package/format-notices/gr2/NOTICE +60 -0
- package/format-notices/gr2/THIRD-PARTY-NOTICES.md +93 -0
- package/package.json +51 -50
- package/resource-lifecycle.md +0 -679
|
@@ -1,5 +1,10 @@
|
|
|
1
1
|
# Format ownership and fork provenance
|
|
2
2
|
|
|
3
|
+
Status: Stable
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource/formats`
|
|
5
|
+
Audience: Users, integrators, and maintainers
|
|
6
|
+
Summary: Records where each format implementation came from, retained snapshots and digests, and what was deliberately not copied.
|
|
7
|
+
|
|
3
8
|
On 2026-07-13, the non-shader runtime format implementations below were copied
|
|
4
9
|
once into `runtime-resource`. Their standalone repositories remain frozen with
|
|
5
10
|
their existing APIs and names; they are not upstreams for the runtime copies.
|
|
@@ -35,6 +40,20 @@ The unborn donors had no commit-addressable `HEAD`; this document deliberately
|
|
|
35
40
|
records them as working-tree snapshots rather than inventing a revision. Their
|
|
36
41
|
copied runtime files are the deterministic retained snapshot.
|
|
37
42
|
|
|
43
|
+
On 2026-07-24, following `format-gr2`'s MIT relicense (its EUPL-derived
|
|
44
|
+
BitKnit decoder was replaced by a clean-room implementation written from the
|
|
45
|
+
published specification in that package's `docs/formats/bitknit2.md`), the
|
|
46
|
+
GR2/GSF reader joined the runtime copies:
|
|
47
|
+
|
|
48
|
+
| Legacy package | Source revision/state | Runtime class | Runtime import |
|
|
49
|
+
|---|---|---|---|
|
|
50
|
+
| `format-gr2` | `fa64607de7a3a96ed3b1aec5288bf71057642043` (v0.2.0, MIT) | `CjsGr2Format` | `@carbonenginejs/runtime-resource/formats/gr2` |
|
|
51
|
+
|
|
52
|
+
The copied engine keeps its donor class name (`CjsFormatGr2`, re-exported)
|
|
53
|
+
under `formats/gr2/core/`; `CjsGr2Format` is the runtime-authored contract
|
|
54
|
+
wrapper. Donor license and notice files are kept under
|
|
55
|
+
`format-notices/gr2/`.
|
|
56
|
+
|
|
38
57
|
## Black definition snapshot
|
|
39
58
|
|
|
40
59
|
The Black reader uses the package-owned generated definition snapshot at
|
|
@@ -64,6 +83,17 @@ format attribution rather than fork provenance.
|
|
|
64
83
|
|
|
65
84
|
## Post-fork additions inside copied formats
|
|
66
85
|
|
|
86
|
+
- `formats/dds` gained original, dependency-free **BC6H and BC7 CPU decoders**
|
|
87
|
+
in `runtime-resource` 0.8.0 (2026-07-21). BC6H covers all fourteen modes,
|
|
88
|
+
signed and unsigned HDR, transformed endpoints, partition/anchor fixups,
|
|
89
|
+
interpolation, reserved opaque-black modes, and float RGBA output. BC7 covers
|
|
90
|
+
all eight modes,
|
|
91
|
+
two- and three-subset partitions, anchor fixups, P-bits, dual index streams,
|
|
92
|
+
channel rotation, edge blocks, and the reserved transparent mode. Fixed bit
|
|
93
|
+
layouts and tables follow the Khronos Data Format Specification and Microsoft
|
|
94
|
+
BC6H/BC7 documentation. Tests cover every mode and signed/unsigned fixtures;
|
|
95
|
+
BC7 was also checked against randomized valid-mode blocks and both decoders
|
|
96
|
+
were exercised on real EVE textures acquired through `tools-core`.
|
|
67
97
|
- `formats/stl` received a writer hardening pass (2026-07-18) without changing
|
|
68
98
|
its donor origin: binary provenance headers now round-trip the caller's solid
|
|
69
99
|
name, shared triangle indices are validated as in-range safe integers, scaled
|
|
@@ -78,7 +108,7 @@ format attribution rather than fork provenance.
|
|
|
78
108
|
meshoptimizer vertex/index compression (index compression canonicalizes
|
|
79
109
|
triangle rotation, matching the engine's own writer test expectations), and
|
|
80
110
|
the post-crc32 file checksum. Verified by write→read roundtrips against the
|
|
81
|
-
runtime reader; `
|
|
111
|
+
runtime reader; CarbonEngine's `mesh` CMF sources were the behavioral
|
|
82
112
|
reference, no code copied. `writeShared`/`writeSharedAsync` plus
|
|
83
113
|
`core/pack.js` (channel interleaving, index packing, unique buffer-index
|
|
84
114
|
assignment) serialize shared geometry directly, enabling GR2/OBJ/glTF→CMF —
|
|
@@ -120,10 +150,9 @@ snapshot must record its new source and digest here.
|
|
|
120
150
|
|
|
121
151
|
## Deliberately not copied
|
|
122
152
|
|
|
123
|
-
- `format-gr2`
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
made. It has not been deprecated or relabeled as MIT.
|
|
153
|
+
- `format-gr2` migrated into `formats/gr2` on 2026-07-24 (see the dated
|
|
154
|
+
table above) after its EUPL constraint was resolved; its standalone
|
|
155
|
+
repository is now a frozen legacy distribution like the other donors.
|
|
127
156
|
- `format-carbon` remains the schema emitter/generator and build-time schema
|
|
128
157
|
authority. Black consumes its published definitions; Red exposes the copied
|
|
129
158
|
catalog but does not yet enforce it while reading YAML fields.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# STL export
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource/formats/stl`
|
|
5
|
+
Audience: Users and integrators
|
|
6
|
+
Summary: Defines the STL writer contract for shared geometry, including validation and watertight checking.
|
|
7
|
+
|
|
8
|
+
## Contract
|
|
9
|
+
|
|
10
|
+
`CjsStlFormat` writes shared geometry directly to binary or ASCII STL. The
|
|
11
|
+
writer consumes `mesh.vertex.position` and triangular `mesh.indices[].faces`;
|
|
12
|
+
multiple meshes and index groups are flattened in encounter order because STL
|
|
13
|
+
does not carry portable scene, material, skin, or animation structure.
|
|
14
|
+
|
|
15
|
+
```js
|
|
16
|
+
import { CjsStlFormat } from "@carbonenginejs/runtime-resource/formats/stl";
|
|
17
|
+
|
|
18
|
+
const bytes = CjsStlFormat.write(sharedGeometry, {
|
|
19
|
+
binary: true,
|
|
20
|
+
solidName: "ship_hull",
|
|
21
|
+
scale: 1000,
|
|
22
|
+
requireWatertight: true
|
|
23
|
+
});
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Writes do not mutate the shared input. Facet normals are recalculated from
|
|
27
|
+
winding by default; set `recalculateNormals: false` to average valid vertex
|
|
28
|
+
normals. Degenerate triangles are skipped by default. Index values must be
|
|
29
|
+
safe integers within the position channel, and binary output rejects
|
|
30
|
+
coordinates outside float32 range instead of silently emitting infinities.
|
|
31
|
+
The `requireWatertight` option rejects open, non-manifold, inconsistently
|
|
32
|
+
wound, or degenerate output.
|
|
33
|
+
|
|
34
|
+
## Related documentation
|
|
35
|
+
|
|
36
|
+
- [Format subpaths](README.md)
|
|
37
|
+
- [Format ownership and fork provenance](provenance.md)
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Wwise soundbanks and media
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource/formats/bnk`, `@carbonenginejs/runtime-resource/formats/wem`
|
|
5
|
+
Audience: Users and integrators
|
|
6
|
+
Summary: Defines the Wwise container inspection, event/media graph extraction, and WEM conversion contracts.
|
|
7
|
+
|
|
8
|
+
## Scope
|
|
9
|
+
|
|
10
|
+
`formats/bnk` and `formats/wem` cover Wwise container inspection, event/media
|
|
11
|
+
graph extraction, embedded-member access, and the currently supported WEM
|
|
12
|
+
conversion routes. They do not own AudioBuffer construction, playback, audio
|
|
13
|
+
manager behavior, or decoded-backend retention.
|
|
14
|
+
|
|
15
|
+
## Soundbank inspection
|
|
16
|
+
|
|
17
|
+
`CjsBnkFormat.inspect()` decodes the chunk map, embedded media index, bank
|
|
18
|
+
names, and the HIRC listing with version-stable typed fields (event action
|
|
19
|
+
lists, action type/target, sound and music-track source ids; pinned against
|
|
20
|
+
bank generator version 150). The Wwise-domain toolkit is grouped under the
|
|
21
|
+
`CjsBnkFormat.wwise` static: the SoundbanksInfo catalog helpers, the FNV-1
|
|
22
|
+
id hash, and event to media resolution:
|
|
23
|
+
|
|
24
|
+
```js
|
|
25
|
+
import { CjsBnkFormat } from "@carbonenginejs/runtime-resource/formats/bnk";
|
|
26
|
+
import { CjsWemFormat } from "@carbonenginejs/runtime-resource/formats/wem";
|
|
27
|
+
|
|
28
|
+
const inspections = bankByteArrays.map(bytes => CjsBnkFormat.inspect(bytes));
|
|
29
|
+
const { eventMedia } = CjsBnkFormat.wwise.eventMediaFromBanks(inspections);
|
|
30
|
+
// eventMedia: Map<eventObjectId, Set<wemId>> - banks may split events from
|
|
31
|
+
// their target sounds, so pass every related bank to one call.
|
|
32
|
+
|
|
33
|
+
const ogg = CjsWemFormat.toOgg(wemBytes); // Wwise Vorbis -> Ogg (lossless)
|
|
34
|
+
const pcm = CjsWemFormat.toPcm(wemBytes); // PTADPCM / 16-bit PCM -> float32
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The read/inspect path stays a pure container reader;
|
|
38
|
+
`wwise.eventMediaFromBanks` is graph interpretation offered for consumers
|
|
39
|
+
with their own engines — the resource lifecycle never calls it.
|
|
40
|
+
|
|
41
|
+
## Related documentation
|
|
42
|
+
|
|
43
|
+
- [Format subpaths](README.md)
|
|
44
|
+
- [Format ownership and fork provenance](provenance.md)
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Event emitter contract
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource`
|
|
5
|
+
Audience: Users and integrators
|
|
6
|
+
Summary: Defines the `CjsEventEmitter` API surface used by resource managers and the memory rules that keep listeners from leaking graphs.
|
|
7
|
+
|
|
8
|
+
## Purpose
|
|
9
|
+
|
|
10
|
+
`CjsEventEmitter` (from `core-types/model`) is a separate base class so
|
|
11
|
+
non-model runtime services can emit events without extending `CjsModel`;
|
|
12
|
+
`CjsResMan` uses that path. CarbonEngineJS exposes one small event-emitter API
|
|
13
|
+
and avoids short generic names such as `On`, `Once`, `Off`, and `Emit` on
|
|
14
|
+
Carbon-shaped classes. There is no separate resource notification/callback
|
|
15
|
+
compatibility layer beside events, and no listener scopes, subscription
|
|
16
|
+
handles, or owner-side `ListenTo()` helpers.
|
|
17
|
+
|
|
18
|
+
## API
|
|
19
|
+
|
|
20
|
+
- `AddEvents(events)`
|
|
21
|
+
- `OnEvent(eventName, listener, source?)`
|
|
22
|
+
- `OnceEvent(eventName, listener, source?)`
|
|
23
|
+
- `OffEvent(eventName = "*", listener?, source?)`
|
|
24
|
+
- `EmitEvent(eventName, ...args)`
|
|
25
|
+
- `HasEvent(eventName = "*", listener?, source?)`
|
|
26
|
+
- `ClearEvent(eventName = "*")`
|
|
27
|
+
- `GetEventNames()` and `GetEventListenerCount(eventName = "*")`
|
|
28
|
+
|
|
29
|
+
The optional `source` is the callback's `this` value and an explicit matching
|
|
30
|
+
identity for removal. Mutating event methods return the emitter for chaining.
|
|
31
|
+
Event names are normalized to lowercase and dispatched by exact match.
|
|
32
|
+
Wildcard names are accepted only by lookup and cleanup methods; there is no
|
|
33
|
+
wildcard listener dispatch, no `family.event` or ancestor routing, and no
|
|
34
|
+
event payload history. Resource classes may emit their own state or domain
|
|
35
|
+
events, but the emitter does not invent a resource lifecycle contract.
|
|
36
|
+
|
|
37
|
+
## Memory rules
|
|
38
|
+
|
|
39
|
+
Event storage is the optional `events` member of the emitter's non-enumerable
|
|
40
|
+
`__state` object:
|
|
41
|
+
|
|
42
|
+
```text
|
|
43
|
+
emitter.__state (non-enumerable, allocated only when some subsystem needs it)
|
|
44
|
+
events -> eventName -> Set<listenerRecord>
|
|
45
|
+
|
|
46
|
+
listenerRecord
|
|
47
|
+
emitter
|
|
48
|
+
eventName
|
|
49
|
+
listener
|
|
50
|
+
source
|
|
51
|
+
once
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
The event map is created only when the first listener is registered and
|
|
55
|
+
deleted when the last record is removed. That does not make listeners weak:
|
|
56
|
+
as long as an emitter is reachable, its event map strongly references
|
|
57
|
+
listener functions and sources, and those listeners can keep whole
|
|
58
|
+
scene/resource graphs alive. The contract is therefore:
|
|
59
|
+
|
|
60
|
+
- `OnceEvent()` removes the listener on first dispatch, even if the callback
|
|
61
|
+
throws.
|
|
62
|
+
- `OffEvent(eventName, listener, source)` removes the exact listener/source
|
|
63
|
+
entry; `target.OffEvent("*", null, source)` removes all of a source's
|
|
64
|
+
records. An external party that no longer observes a target must call
|
|
65
|
+
`OffEvent()`.
|
|
66
|
+
- `Unload()` and `Purge()` are resource state/cache operations, not an
|
|
67
|
+
implied listener-destruction lifecycle.
|
|
68
|
+
- `OnEvent()` returns the emitter; it does not return unsubscribe closures,
|
|
69
|
+
because those closures create another reference path.
|
|
70
|
+
- Deterministic cleanup is the contract; `WeakRef`/`FinalizationRegistry` may
|
|
71
|
+
help diagnostics but are not a lifecycle mechanism.
|
|
72
|
+
|
|
73
|
+
Multiple listeners on the same event are allowed because each event bucket is
|
|
74
|
+
a set of records. A raw `CjsEventEmitter` does not gain model-owned `dirty`
|
|
75
|
+
or `rebuild` state. The target is "easy to debug, hard to leak": clear
|
|
76
|
+
ownership of who subscribed, who unsubscribes, and which cleanup phase clears
|
|
77
|
+
remaining listeners.
|
|
78
|
+
|
|
79
|
+
## Relationship to model dirty state
|
|
80
|
+
|
|
81
|
+
`CjsModel` has dirty-state helpers (`MarkDirty`, `ClearDirty`, `ConsumeDirty`,
|
|
82
|
+
`GetDirtyNotifications`) for model invalidation. `SetValues()` compares
|
|
83
|
+
incoming values with the current field values and only marks dirty when a
|
|
84
|
+
value actually changes. A plain `MarkDirty()` means broad dirty invalidation;
|
|
85
|
+
it does not request a rebuild, and deferred rebuild reasons belong to the
|
|
86
|
+
independent `model.__state.rebuild` set. That model machinery is not a
|
|
87
|
+
resource lifecycle event system; resource lifecycle events remain a
|
|
88
|
+
resource/resman concern.
|
|
89
|
+
|
|
90
|
+
## Related documentation
|
|
91
|
+
|
|
92
|
+
- [MotherLode identity, cache, and retention](motherlode-cache.md)
|
|
@@ -0,0 +1,244 @@
|
|
|
1
|
+
# MotherLode identity, cache, and retention
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource`
|
|
5
|
+
Audience: Users and integrators
|
|
6
|
+
Summary: Defines canonical resource identity, ownership and replacement, the recorded-byte cache, payload retention, read-cache provenance, and purge contracts.
|
|
7
|
+
|
|
8
|
+
## Canonical identity
|
|
9
|
+
|
|
10
|
+
Canonical resource identity is the normalized source path plus its promised
|
|
11
|
+
output tag. `variant` is the explicit tag; otherwise `emit`, `requirement`, or
|
|
12
|
+
`payload` supplies it. Human-readable identities are written as
|
|
13
|
+
`res:/ship.gr2@cmf`, although MotherLode uses an internal delimiter. Reader,
|
|
14
|
+
constructor, and format-option implementations never enter the key. CjsLibrary
|
|
15
|
+
chooses the promised output and ResMan executes the current setup-time format
|
|
16
|
+
registration for it.
|
|
17
|
+
|
|
18
|
+
Output selection is case-insensitive for identity and matching, but format
|
|
19
|
+
readers receive the canonical declared spelling (for example `cmfJson`). A
|
|
20
|
+
legacy direct object loader exposes only its unforced default; named output
|
|
21
|
+
variants belong on a format class. Unsupported `@output` requests fail before
|
|
22
|
+
cache lookup, so a resident handle cannot bypass the declaration.
|
|
23
|
+
|
|
24
|
+
The selected constructor, reader, and format defaults/options are setup-time
|
|
25
|
+
execution details, not MotherLode identity. A changed registration does not
|
|
26
|
+
create a hidden second resource; reset the affected identity
|
|
27
|
+
(`Delete`/`Clear`) or create a new manager. A changed output contract must use
|
|
28
|
+
a new tag such as `@cmf2`.
|
|
29
|
+
|
|
30
|
+
## Ownership and replacement
|
|
31
|
+
|
|
32
|
+
`CjsResMan` resolves each normalized path and promised output to one canonical
|
|
33
|
+
MotherLode key. `Insert(key, resource, options)` reports `{ inserted,
|
|
34
|
+
replaced, displaced }`; replacement, deletion, clearing, and shutdown destroy
|
|
35
|
+
attached adapter allocations and release the complete CPU payload by default.
|
|
36
|
+
Callers that deliberately retain ownership may pass `{ cleanup: false }` and
|
|
37
|
+
keep the returned displaced resource. If replacement cleanup fails, insertion
|
|
38
|
+
throws a contextual error and leaves the existing owner registered. These
|
|
39
|
+
ordinary ownership removals preserve the handle's last resource state;
|
|
40
|
+
`PURGED` is reserved for successful policy eviction through inactivity or
|
|
41
|
+
byte pressure.
|
|
42
|
+
|
|
43
|
+
`Startup()` and `Shutdown()` are idempotent. `HasKey`, `Lookup`, `Delete`,
|
|
44
|
+
`GetKeys`, `GetValues`, `GetSize`, `SetCacheSize`, `GetCacheSize`, `GetStats`,
|
|
45
|
+
`TrimCache`, `ReplaceExpected`, `Clear`, and `ClearCached` provide the
|
|
46
|
+
Carbon-shaped cache vocabulary plus the exact-owner compare-and-swap required
|
|
47
|
+
by staged JavaScript reload. The old `Has`, `GetCount`, and `DeleteAll` names
|
|
48
|
+
remain temporary compatibility aliases.
|
|
49
|
+
|
|
50
|
+
## Recorded-byte cache
|
|
51
|
+
|
|
52
|
+
Carbon can infer when only its cache retains a resource through
|
|
53
|
+
weak-reference and refcount transitions. JavaScript cannot reproduce that
|
|
54
|
+
ownership test reliably, so `CjsMotherLode` budgets only entries that a caller
|
|
55
|
+
explicitly classifies with `{ cached: true, bytes }`; JavaScript reachability
|
|
56
|
+
is never inferred. The byte value is an exact caller-supplied safe-integer
|
|
57
|
+
eviction weight, not a heuristic walk of the resource graph; runtime-resource
|
|
58
|
+
does not walk arbitrary cyclic/shared object graphs or invoke payload getters
|
|
59
|
+
to guess size.
|
|
60
|
+
|
|
61
|
+
Explicit cached entries receive a monotonic admission sequence. With default
|
|
62
|
+
cleanup, `TrimCache(options)` destroys adapters, releases payloads, detaches
|
|
63
|
+
lifecycle callbacks, marks compatible handles `PURGED`, and removes
|
|
64
|
+
positive-byte cached identities in oldest-admission order until
|
|
65
|
+
`cacheBytes <= cacheSize`. Live, locked, `cacheable: false`, and zero-byte
|
|
66
|
+
entries do not create pressure. `KeepAlive()` and `Lock()` promote a cached
|
|
67
|
+
record to live; `Unlock()` does not silently re-admit it.
|
|
68
|
+
|
|
69
|
+
`SetCacheSize(bytes, options)` installs and immediately enforces the new
|
|
70
|
+
budget. `CjsResMan.Update()` and `Tick()` retry cache housekeeping after
|
|
71
|
+
pumping queues; `{ cache: false }` skips it for one update without changing
|
|
72
|
+
policy. Cleanup is transactional per identity: a failed candidate remains
|
|
73
|
+
canonical, later candidates are still attempted, and the aggregate
|
|
74
|
+
`CJS_MOTHERLODE_CACHE_TRIM_FAILED` error carries successful evictions and any
|
|
75
|
+
remaining over-budget state. Trimming never reads, prepares, or reloads data.
|
|
76
|
+
|
|
77
|
+
## Payload retention
|
|
78
|
+
|
|
79
|
+
Reader and converter outputs are plain transient payload objects, not resource
|
|
80
|
+
classes or DTO models. A payload may contain more decoded data than a
|
|
81
|
+
particular resource or engine adapter needs. Each concrete resource validates
|
|
82
|
+
the fields it requires before publishing the payload and retains the scalars
|
|
83
|
+
and references it needs. An adapter may retain additional references in
|
|
84
|
+
adapter-owned state. Referencing payload-owned typed arrays is valid and
|
|
85
|
+
preferable to copying them merely to change ownership.
|
|
86
|
+
|
|
87
|
+
The lifecycle treats resource residency and payload residency independently:
|
|
88
|
+
|
|
89
|
+
```text
|
|
90
|
+
resource.KeepAlive()
|
|
91
|
+
-> renew resource/cache residency
|
|
92
|
+
|
|
93
|
+
resource.KeepPayloadAlive()
|
|
94
|
+
-> renew the attached payload lease
|
|
95
|
+
|
|
96
|
+
resource.ReleasePayload()
|
|
97
|
+
-> explicitly release the full payload reference
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
`CjsResMan` binds resource-facing `KeepAlive()`, `KeepPayloadAlive()`,
|
|
101
|
+
`Lock()`, and `Unlock()` to the resource's canonical MotherLode key.
|
|
102
|
+
`SetPayload()` renews both identity and payload activity when it publishes a
|
|
103
|
+
non-null payload. `GetPayload()`, `HasPayload()`, `IsGood()`, and other
|
|
104
|
+
queries are pure; reading the payload does not implicitly renew its lease.
|
|
105
|
+
Detached and purged handles retain deterministic no-op liveness methods rather
|
|
106
|
+
than silently starting work.
|
|
107
|
+
|
|
108
|
+
A released CPU payload retains only the small request needed to reconstruct
|
|
109
|
+
that same path/output from its source and `sourceRevision`. The retained
|
|
110
|
+
promised-output fields and source provenance win over later
|
|
111
|
+
`Ready()`/`GetObject()` overrides, while cache/reload policy remains per-call.
|
|
112
|
+
Payload leases protect active consumers; an engine may release its own backend
|
|
113
|
+
adapter without destroying shared CPU data.
|
|
114
|
+
|
|
115
|
+
Both semantic and generic/base resource results use the payload slot: the
|
|
116
|
+
manager stores the complete result through `SetPayload()` and mirrors it on
|
|
117
|
+
the compatibility `object` property for base resources. Payload release clears
|
|
118
|
+
that alias only while it still identifies the released value. Semantic
|
|
119
|
+
resources continue to expose `object === resource` while holding their
|
|
120
|
+
validated plain payload privately.
|
|
121
|
+
|
|
122
|
+
Object-operation promises are retained only while in flight. Concurrent
|
|
123
|
+
`GetObject()`/`Ready()` calls share one operation; a resident result is
|
|
124
|
+
returned without source work and renews the explicit payload lease.
|
|
125
|
+
Settlement removes the operation record so its result graph can be reclaimed
|
|
126
|
+
and a failure can be retried. After payload release, only a new explicit
|
|
127
|
+
object/readiness call reconstructs it; queries, lease calls, and purge sweeps
|
|
128
|
+
do not.
|
|
129
|
+
|
|
130
|
+
The processor preparing a resource decides when the full payload can be
|
|
131
|
+
released:
|
|
132
|
+
|
|
133
|
+
```text
|
|
134
|
+
format reader -> plain payload -> resource validation + adapter prepare
|
|
135
|
+
|
|
|
136
|
+
+-> resource retains required values/references
|
|
137
|
+
+-> adapter retains adapter-specific state/references
|
|
138
|
+
+-> release payload after successful preparation
|
|
139
|
+
`-> or renew its lease for deferred/further work
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
A time- or frame-based lease is a fallback against abandoned payloads. An
|
|
143
|
+
owner performing deferred work can renew the lease. If an expired payload is
|
|
144
|
+
required again, the caller must explicitly request reconstruction; lease
|
|
145
|
+
renewal and purging never fetch or reload source data. Dynamic or
|
|
146
|
+
non-reloadable resources must remain locked, retain the required payload, or
|
|
147
|
+
be able to recreate it.
|
|
148
|
+
|
|
149
|
+
Payload references are shared read-only by default. Preparing WebGL and
|
|
150
|
+
WebGPU adapters side by side should normally pass the same payload to both
|
|
151
|
+
consumers and retain it until both have finished. Copying is an explicit
|
|
152
|
+
consumer operation, justified when a consumer must mutate data, transfer and
|
|
153
|
+
detach an `ArrayBuffer`, or retain an independently writable snapshot. The
|
|
154
|
+
consumer should copy only the fields it requires; runtime-resource does not
|
|
155
|
+
automatically deep-clone payload or typed-array bundles.
|
|
156
|
+
|
|
157
|
+
## Read-cache provenance
|
|
158
|
+
|
|
159
|
+
Source and parsed-format caches use explicit provenance, separate from
|
|
160
|
+
path/output resource identity. `sourceRevision` is an opaque caller/source-
|
|
161
|
+
supplied string or finite number identifying source content for one source
|
|
162
|
+
object and normalized path. It scopes read caches only; it does not alter
|
|
163
|
+
MotherLode resource identity, and changing it does not replace a resident
|
|
164
|
+
payload without `reload: true`. Source and format records do not share across
|
|
165
|
+
revisions.
|
|
166
|
+
|
|
167
|
+
`cacheSource` and `cacheFormat` are tri-state per-call policies:
|
|
168
|
+
|
|
169
|
+
- omitted: share in-flight or explicitly retained work, then drop a newly
|
|
170
|
+
completed record;
|
|
171
|
+
- `true`: share and retain success; a joining caller upgrades the record;
|
|
172
|
+
- `false`: bypass sharing and retention.
|
|
173
|
+
|
|
174
|
+
Failures are never retained. Format records are additionally isolated by
|
|
175
|
+
selected source object, frozen registration descriptor, revision, and
|
|
176
|
+
effective format options, so another source or a re-registered default cannot
|
|
177
|
+
reuse a stale parse. Re-registering a format with new defaults therefore
|
|
178
|
+
cannot reuse an old descriptor's parse. Registered defaults are copied into
|
|
179
|
+
deeply frozen plain-object/array snapshots. Material format options that
|
|
180
|
+
cannot be represented safely (for example class instances with hidden mutable
|
|
181
|
+
state) bypass format-cache sharing instead of risking a false match;
|
|
182
|
+
functions and byte views use cache-local identity plus visible byte content
|
|
183
|
+
where applicable.
|
|
184
|
+
|
|
185
|
+
A resource loader retains the effective selected source and `sourceRevision`
|
|
186
|
+
for reconstruction, including the manager default selected at creation, but
|
|
187
|
+
not cache flags or one-shot reload.
|
|
188
|
+
|
|
189
|
+
## Explicit and automatic purging
|
|
190
|
+
|
|
191
|
+
`PurgeInactive(options)` performs an explicit deterministic sweep using
|
|
192
|
+
independent identity and payload frame/time limits. Locks skip both forms of
|
|
193
|
+
eviction. Identity expiry destroys adapter resources, releases the payload,
|
|
194
|
+
detaches lifecycle callbacks, marks compatible handles `PURGED`, and removes
|
|
195
|
+
the canonical key; payload expiry calls `ReleasePayload()` while retaining
|
|
196
|
+
identity and adapter allocations. Candidate failures are aggregated after the
|
|
197
|
+
sweep has continued over other entries. A sweep never fetches, prepares, or
|
|
198
|
+
reloads a resource.
|
|
199
|
+
|
|
200
|
+
Automatic scheduling is available only when a caller supplies
|
|
201
|
+
`autoPurgePolicy` to the constructor/`Register()` or calls
|
|
202
|
+
`SetAutoPurgePolicy()`. It is disabled by default and deliberately accepts
|
|
203
|
+
only millisecond limits: MotherLode activity frames count explicit
|
|
204
|
+
observations and are not renderer frames. A policy must set at least one of
|
|
205
|
+
`maxIdleMilliseconds` or `payloadMaxIdleMilliseconds`;
|
|
206
|
+
`intervalMilliseconds` defaults to 1000. The first
|
|
207
|
+
`PumpAutoPurge()`/`Update()` after configuration sweeps immediately, then the
|
|
208
|
+
interval sets the minimum cadence. `Update({ purge: false })` suppresses a
|
|
209
|
+
sweep for one update without changing cadence. A regressing clock rebases and
|
|
210
|
+
skips one pump; custom deterministic clocks should be shared with MotherLode.
|
|
211
|
+
Recorded-byte cache trimming is separate from this opt-in inactivity policy
|
|
212
|
+
and runs on ordinary updates unless `{ cache: false }` is supplied.
|
|
213
|
+
|
|
214
|
+
```js
|
|
215
|
+
const resMan = new CjsResMan({
|
|
216
|
+
source,
|
|
217
|
+
autoPurgePolicy: {
|
|
218
|
+
intervalMilliseconds: 1000,
|
|
219
|
+
maxIdleMilliseconds: 60_000,
|
|
220
|
+
payloadMaxIdleMilliseconds: 10_000
|
|
221
|
+
}
|
|
222
|
+
});
|
|
223
|
+
|
|
224
|
+
resMan.Update();
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Both queued `QueueResourceObject()` work and direct `LoadResourceObject()`
|
|
228
|
+
work hold one manager-owned lock from request/loading publication through
|
|
229
|
+
success or failure. The lock is balanced independently of caller locks, so
|
|
230
|
+
automatic or manual sweeps cannot detach a handle while its read/prepare
|
|
231
|
+
operation is still active. Lock release is conditional on the same captured
|
|
232
|
+
ownership generation, so stale work cannot decrement a newly rebound handle's
|
|
233
|
+
lock. Scheduling and active-work protection do not fetch or reload data.
|
|
234
|
+
|
|
235
|
+
Cache trimming and automatic inactivity sweeps retain the strict no-reload
|
|
236
|
+
rule. Application retention defaults, automatic resource/payload byte
|
|
237
|
+
estimation, separate CPU/adapter budgets, and purged-resource/device-loss
|
|
238
|
+
recovery policy are future work; see the [roadmap](../roadmap.md).
|
|
239
|
+
|
|
240
|
+
## Related documentation
|
|
241
|
+
|
|
242
|
+
- [Resource lifecycle concepts](../concepts/resource-lifecycle.md)
|
|
243
|
+
- [Candidate-first atomic reload](../reference/reload.md)
|
|
244
|
+
- [Queues and the Wait fence](../reference/queues.md)
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Queues, publication, and the Wait fence
|
|
2
|
+
|
|
3
|
+
Status: Evolving
|
|
4
|
+
Scope: `@carbonenginejs/runtime-resource`
|
|
5
|
+
Audience: Users and integrators
|
|
6
|
+
Summary: Defines the two manager-owned queues, publication budgeting, registration, and the `Wait()` fence semantics.
|
|
7
|
+
|
|
8
|
+
## Queued CPU load and publication
|
|
9
|
+
|
|
10
|
+
`GetObject()`, `LoadObject()`, and resource `Ready()` use two manager-owned
|
|
11
|
+
queues:
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
BACKGROUND: deduplicated source load, limited by maxConcurrentLoads
|
|
15
|
+
MAIN: reader/format conversion -> resource publication
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
The main reader/format operation and publication are separate queue items.
|
|
19
|
+
`maxPrepareTime` is a per-pump budget in seconds limiting synchronous
|
|
20
|
+
main-queue work started in one pump, and `maxPrepareItemsPerTick` can add an
|
|
21
|
+
item-count limit. Promise-returning format work remains in flight without
|
|
22
|
+
blocking the JavaScript event loop. The default scheduler keeps promise-based
|
|
23
|
+
calls working; a `CjsLibrary` or direct caller can provide its frame
|
|
24
|
+
scheduler:
|
|
25
|
+
|
|
26
|
+
```js
|
|
27
|
+
const resMan = new CjsResMan({
|
|
28
|
+
source,
|
|
29
|
+
maxConcurrentLoads: 8,
|
|
30
|
+
maxPrepareTime: 0.005,
|
|
31
|
+
queueScheduler: callback => requestAnimationFrame(callback)
|
|
32
|
+
});
|
|
33
|
+
|
|
34
|
+
await resMan.FetchResource("res:/model/ship.gr2", {
|
|
35
|
+
requirement: "geometry",
|
|
36
|
+
emit: "cmf"
|
|
37
|
+
});
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The selected format class owns conversion to the promised CPU output.
|
|
41
|
+
`CjsResMan` does not inspect WebGL, WebGPU, texture, geometry, or codec
|
|
42
|
+
support, and it does not run backend realization. An engine consumes the
|
|
43
|
+
published CPU resource afterward through its own explicit operation.
|
|
44
|
+
|
|
45
|
+
## Queue controls
|
|
46
|
+
|
|
47
|
+
Blue-compatible queue controls are exposed directly on `CjsResMan`:
|
|
48
|
+
`AddToQueue`, `CancelFromQueue`, `GetNextIdForQueue`, `PumpMainThreadQueue`,
|
|
49
|
+
`PauseQueue`, `ResumeQueue`, `GetPendingLoads`, and `GetPendingPrepares`.
|
|
50
|
+
`Update()`/`Tick()` pump work.
|
|
51
|
+
|
|
52
|
+
## The Wait fence
|
|
53
|
+
|
|
54
|
+
`Wait()` synchronously captures queued resource-operation roots and low-level
|
|
55
|
+
queue tasks that already exist when it is called. Captured resource roots
|
|
56
|
+
include publication work enqueued after an asynchronous read; unrelated later
|
|
57
|
+
roots/tasks do not postpone the fence. Failure and queued cancellation count
|
|
58
|
+
as settlement, remain observable through their original operation promises,
|
|
59
|
+
and cross the fence without making `Wait()` reject.
|
|
60
|
+
|
|
61
|
+
By default `Wait()` pumps the two queues directly within their ordinary
|
|
62
|
+
budgets and never runs automatic purge housekeeping. It preserves pause
|
|
63
|
+
state; `{ pump: false }` leaves all progress to an external driver. A
|
|
64
|
+
standalone canonical `PrepareResourceObjectQueued()` call is a queued root.
|
|
65
|
+
Direct `LoadResourceObject()`, direct `PrepareResourceObject()`, standalone
|
|
66
|
+
`ReadResource()`, and standalone `ReadFormatOnce()` calls bypass both queues
|
|
67
|
+
and are outside this fence unless they own a captured queue task, although
|
|
68
|
+
direct resource mutations are still tracked for safe MotherLode replacement.
|
|
69
|
+
`WaitUrgent()` remains deferred until the queue has real per-item priority
|
|
70
|
+
and urgent-membership semantics.
|
|
71
|
+
|
|
72
|
+
## Registration
|
|
73
|
+
|
|
74
|
+
Format classes own input extensions. Resource classes are registered by a
|
|
75
|
+
semantic requirement, never by file extension:
|
|
76
|
+
|
|
77
|
+
```js
|
|
78
|
+
const resMan = new CjsResMan().Register({
|
|
79
|
+
source,
|
|
80
|
+
formats: [ CjsDdsFormat, CjsPngFormat ],
|
|
81
|
+
resourceTypes: [ TriTextureRes, Tr2ImageRes ]
|
|
82
|
+
});
|
|
83
|
+
|
|
84
|
+
const texture = resMan.GetResource("res:/image/ship.png", {
|
|
85
|
+
requirement: "texture",
|
|
86
|
+
emit: "image"
|
|
87
|
+
});
|
|
88
|
+
const image = resMan.GetResource("res:/image/ship.png", {
|
|
89
|
+
requirement: "image",
|
|
90
|
+
emit: "image"
|
|
91
|
+
});
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Those are distinct resource identities but share the normalized source-byte
|
|
95
|
+
operation. The manager does not expose an extension-to-resource compatibility
|
|
96
|
+
registry.
|
|
97
|
+
|
|
98
|
+
## Related documentation
|
|
99
|
+
|
|
100
|
+
- [Resource lifecycle concepts](../concepts/resource-lifecycle.md)
|
|
101
|
+
- [MotherLode identity, cache, and retention](motherlode-cache.md)
|
|
102
|
+
- [Format subpaths](../formats/README.md)
|