@dcl/abgen-win32-x64 0.17.13 → 0.19.0

package/BUILD-INFO.txt

@@ -1,6 +1,6 @@
-ABGEN_VERSION=0.17.13
-ABGEN_RELEASE=v0.17.13
+ABGEN_VERSION=0.19.0
+ABGEN_RELEASE=v0.19.0
 ABGEN_TARGET=x86_64-pc-windows-gnu
-ABGEN_BUILD_ID=c7f413c2a0a5
-ABGEN_GIT_REV=2230291c5eedebff9b249d60bf70b857cb25568a
+ABGEN_BUILD_ID=96b6cff855db
+ABGEN_GIT_REV=7324c4aecc2998ee377d32b1e93b4008629dfae1
 SOURCE_DATE_EPOCH=315532800

package/README.md

@@ -13,7 +13,7 @@ resolves it for embedding tools - see [npm/abgen](npm/abgen/README.md)). From so
 ```bash
 git clone <this-repo> abgen && cd abgen
 cargo build --release            # no postgres, no openssl, no protobuf
-cargo build --release --examples # bundle dump tools (objdump/texdump/matdump/texcmp/texpng)
+cargo build --release --examples # bundle dump tools (objdump/texdump/matdump)
 scripts/bootstrap-runtime.sh     # integrity-check the vendored runtime data
 ```
 There is also a wasm build: `crate/abgen-wasm/` compiles the converter to wasm32 — see its README.
@@ -26,10 +26,10 @@ There is also a wasm build: `crate/abgen-wasm/` compiles the converter to wasm32
 | `abgen-build` | single-file local converter CLI (glb -> bundle, `--expect-hash` verify) |
 | `abgen-corpus` | batch corpus builder: manifest / `--entity-ids` (add `--fetch-missing` to pull content from a catalyst) / `--world <name>[,...]` (resolve + fetch + convert a world; upstream via `--worlds-url` or `ABGEN_WORLDS_URL`) / `--live-mode` / `--collection-urn` / `--from-reference`; full-corpus runbook in [docs/BATCH.md](docs/BATCH.md) |
 | `abgen-verify` | parity differ (ours vs reference bundles, ppm-bits, `--tolerant`) |
-| `abgen-lod` | LOD lane CLI: `bundle`, `compare`, `placements`, `assemble`, `atlas`, `simplify`, `generate` |
+| `abgen-lod` | LOD lane CLI: `bundle`, `placements`, `assemble`, `atlas`, `simplify`, `generate`, `compare` (structural diff of two LOD bundles: materials/textures/vertices/triangles plus a per-material property diff — shader, keywords, render queue, floats, clipping colours, texture bindings; local or CDN URL), `qualify-corpus` (Genesis City + Worlds corpus build; `--reference-cdn` records the same diff against production per scene) |
 | `abgen-lambda` | AWS Lambda handler (deployment event in, bundles + manifests to S3; from the `lambda/` workspace member) - see [lambda/README.md](lambda/README.md) |
 
-Plus bundle-inspection examples (`texdump`, `matdump`, `objdump`, `texcmp`, `texpng`) under
+Plus bundle-inspection examples (`texdump`, `matdump`, `objdump`) under
 `target/release/examples/`.
 ## Server
 ```bash
@@ -59,6 +59,9 @@ header semantics, and the DB-vs-proxy source selection: [docs/ROUTES.md](docs/RO
 ABGEN_ROOT="$PWD" cargo test --workspace --lib -- --test-threads=1
 ```
 `--test-threads=1` is required: the lib tests share process-wide `ABGEN_ROOT` state.
+LOD assembly frame conformance against the production `_1.glb` references: `cargo test --release
+--test lod_handedness` (offline fixtures; add `-- --ignored` to assemble from the catalyst).
+
 ## wasm
 `crate/abgen-wasm/` compiles the converter lib (default features off) to `wasm32-unknown-unknown` behind
 a hand-rolled C ABI; the JS runtime in `crate/abgen-wasm/js/` (worker pool + WebGPU bridge) is what a
@@ -71,20 +74,17 @@ template paths in `crate/src/builder/templates.rs` assume the source layout wher
 the crate — an in-repo wasm32 build needs those relative paths bumped one level (`../../template/` ->
 `../../../template/`). No workspace target compiles for wasm32, so the divergence never touches CI.
 ## Features
-One compile-time feature flag: `server` (on by default) gates the abcdn HTTP server + registry stack
-(axum/tokio/sqlx and the `dcl-contents` crate); the `abgen` bin has `required-features = ["server"]`.
-Library consumers - `lambda/`, `crate/abgen-native/`, `crate/abgen-wasm/` - build with
-`default-features = false` and get the converter without the server stack. Under the default
-features every capability below builds into every native binary; `wasm32` builds target-gate the
-server, content-DB, and GPU code off automatically. Each capability is activated at runtime, not at
-build time.
-
-| Capability | Built | Enable at runtime |
+The default `server` feature builds the complete service. Reusable consumers should select only the lanes they need; the smallest native converter is `cargo build --release --lib --no-default-features`.
+
+| Feature | Adds | Enabled by |
 |---|---|---|
-| tokio/axum HTTP JIT server (`abcdn` module + `abgen` bin) | always (native targets) | run the `abgen` bin |
-| catalyst content-DB index (sqlx/postgres, via the in-tree `dcl-contents` crate) for real timestamps + deployer on `/entities/*` and the unsigned registry routes (`/profiles*`, `/entities/status/{id}`, `/worlds/{name}/manifest`) | always | set a content-DB connection (`CONTENT_PG_CONNECTION_STRING` or `POSTGRES_CONTENT_*`); without one the built-in content-client fallback serves the index routes with `timestamp: 0`, empty deployer, and the registry routes proxy the upstream catalyst |
-| CUDA GPU BC7/BC5 encode path (`libcuda` dlopen'd at runtime, no toolkit needed to build; PTX kernels vendored in `crate/src/gpu/kernel.ptx`, regen from `crate/kernel-ptx/`) | always | opt in per run with `--gpu` / `ABGEN_GPU=1`, backend pick via `ABGEN_GPU_BACKEND=auto\|cuda\|wgpu\|off` |
-| portable `wgpu` compute backend (Vulkan/Metal/DX12), same dispatch + qualification gate | always | `ABGEN_GPU_BACKEND=wgpu` (or `auto`, which tries CUDA first) |
+| `server` (default) | HTTP/registry stack, `gpu`, `cli`, `web-pack`, and `scene-runtime` | `abgen` server |
+| `gpu` | native wgpu/CUDA dispatch | server, build/corpus/verify CLIs, Lambda |
+| `web-pack` | BV WebGPU pack emission plus DDS/glTF JSON dependencies | server, build/corpus CLIs |
+| `scene-runtime` | embedded QuickJS fallback for suspicious static LOD placements | server, LOD CLI, Lambda |
+| `cli` | tracing subscriber and CLI logging | server and LOD CLI |
+
+`crate/abgen-native`, `crate/abgen-node`, and converter WASM use no default features. The standalone `crate/wasm-gpu` package compiles the shared BC7/WebGPU source directly and does not depend on the full `abgen` graph; its worker ships only `gpu_init` and `gpu_encode`. Static CRDT placement interpretation remains available without `scene-runtime`; if that result is empty or suspicious, the minimal build fails explicitly instead of publishing an empty LOD.
 
 GPU backends self-qualify per device at enable time: the selected backend (`auto` tries CUDA, then
 wgpu) must reproduce the CPU BC7 encoder bit-for-bit on a probe matrix (sizes x srgb x perceptual x
@@ -131,6 +131,7 @@ restore it - it never fetches or re-downloads:
 | `ABGEN_JIT_FAIL_TTL_S` | `60` | failure negative-cache TTL for entity JIT builds; concurrent identical requests single-flight per `{entity}:{platform}` |
 | `ABGEN_JIT_CONTENT_DIGEST` | `0` | freshness for content servers whose declared hashes are not content-addressed (the `@dcl/sdk-commands` preview server keys them by file path). Defaults to ON when `ABGEN_CATALYST_URL` points at a loopback host (a dev preview server), OFF otherwise; explicit values always win. When on, manifest requests re-download the entity's convertible content, sha256 it, and prune stale conversions when bytes changed under an unchanged hash. Changed non-glb files (textures/`.bin`) also invalidate every glb bundle of the entity. Debounced per entity via `ABGEN_REVALIDATE_DEBOUNCE_S` (default `2`) |
 | `ABGEN_UPSTREAM_AB_CDN` | unset | read-through to a production ab-cdn (e.g. `https://ab-cdn.decentraland.org`): manifest/bundle/LOD/ISS requests that 404 through every local and JIT lane are streamed from upstream without persisting anything locally. Lets a client point its whole optimized-assets base URL here (wearables/emotes keep working) while only local scene entities are built locally. Paths containing `b64-` preview ids are never proxied |
+| `ABGEN_UPSTREAM_AB_REGISTRY` | see right | upstream registry URL: records for entities without local bundles are answered with its versions. Default: `asset-bundle-registry.<domain>` when the CDN is `ab-cdn.<domain>`. `off` disables |
 | `RUST_LOG` | `abgen=info,tower_http=info` | tracing filter |
 | `CONTENT_PG_CONNECTION_STRING` (or `POSTGRES_*` parts) | unset | content DB; URL form also mounts the signed registry routes |
 ### S3 space cache (read-through + write-back)
@@ -147,8 +148,7 @@ Uploads carry each key family's production writer's object metadata, derived fro
 (cdn-uploader's comma-joined spelling), scene sources (`.js`/`.json`/`.crdt`) the direct-upload
 spelling `public, max-age=31536000, immutable`, manifests (`manifest/…`) `application/json` +
 `private, max-age=0, no-cache`, ISS descriptors (`lods-unity/manifests/…`)
-`public, max-age=31536000` (lod-generator-unity's spelling), `.br` keys
-`public,no-transform,max-age=31536000,immutable` + `Content-Encoding: br` (as cdn-uploader sets).
+`public, max-age=31536000` (lod-generator-unity's spelling).
 
 ### Asset-reuse mode (upstream converter parity)
 ON by default, matching the ab-cdn deployment's asset-reuse naming from v49 onward: scene
@@ -168,6 +168,79 @@ non-digest names).
   `ABGEN_MAGENTA_MISSING` is on, in which case unresolvable deps are dropped from the digest and
   the build substitutes placeholder textures
 
+### Two version lanes: `AB_VERSION` and `WEARABLE_AB_VERSION`
+Scene bundles and wearable/emote bundles live under separate key prefixes. `AB_VERSION` is the scene
+lane; `WEARABLE_AB_VERSION` is wearables and emotes, and defaults to `AB_VERSION` when unset — so a
+deployment that does not set it behaves exactly as it did when the two shared one prefix.
+
+They are split because only a scene's name can carry an invalidation. Scene bundles are digest-named
+(`{hash}_{digest}_{platform}`), so a dependency change or a recipe bump moves them; wearable bundles
+are `{hash}_{platform}`, with nowhere to put one, which leaves the prefix as the only lever they
+have. Sharing one prefix therefore meant every scene-driven bump dragged the whole wearable corpus
+through a rebuild it had no use for, and every wearable-driven bump did the same to the world.
+
+- The conversion path resolves its lane from the entity type (`Proxy::version_for`); everything
+  that is not a scene rides the wearable lane.
+- The serving path has only a bundle name to go on, so it reads both lanes and then
+  `ABGEN_FALLBACK_VERSION` (`Proxy::read_versions`). Safe rather than merely convenient: an
+  entity's bundles exist under exactly one prefix and every name is content-addressed, so a hit is
+  the right bytes whichever lane answered. A miss costs one extra 404.
+- The lambda's already-converted gate accepts a manifest at *either* lane, because the entity type
+  is only known after the entity doc is fetched and the gate deliberately runs before that. Safe
+  while the two are distinct strings: a manifest carries whichever version wrote it, so a bump of
+  either stops matching.
+
+### Per-asset-type cache keys (recipes)
+`AB_VERSION` prefixes every space key, so bumping it orphans every bundle of every type at once —
+the right hammer for a change to the bundle container, far too big for the usual fix, which changes
+how one kind of asset is built and leaves the rest byte-identical. Baking skinned-renderer bounds
+over the animation clips (#119) rewrote nothing but animated rigs; an `AB_VERSION` bump behind it
+would have rebuilt every texture in the world.
+
+`crate/src/recipes.rs` holds a generation counter per build-affecting behaviour and folds the ones
+an asset uses into the digest that names its bundle. A bundle is rebuilt whole or not at all — a GLB
+bundle carries its meshes, skeleton, clips, materials *and* its resolved textures in one artifact —
+so there are only two base recipes, one per kind of bundle, plus two narrowing ones that earn their
+place by being rare:
+
+| Recipe | Applies to | Rebuilds on a bump |
+|---|---|---|
+| `glb` | every glTF | every glb bundle; no standalone image bundles |
+| `texture` | standalone images, and any glTF with `images` | every image bundle **and** every glb that embeds one |
+| `skin` | a glTF declaring `skins` | only rigs — a minority of glbs |
+| `animation` | a glTF carrying `animations` | only animated glbs — a minority |
+
+**To ship a fix without an `AB_VERSION` bump:** raise the matching counter by one, in the commit that
+changes the output, then re-enqueue as you would for a version bump. Bundles whose recipes moved get
+new names, miss the HEAD probe and rebuild; every other bundle keeps its name and is reused where it
+already sits. Never renumber a counter and never reuse a value — a generation that comes back around
+makes stale bundles look fresh.
+
+Default to `glb`. Reach for `skin` or `animation` only when the fix has a gate you can point at —
+#119 is the model, where a glTF with no clips serialized byte for byte as before. The asymmetry is
+the whole rule: too broad costs a rebuild you were going to pay anyway, too narrow ships stale
+bundles and says nothing. There is deliberately no counter per sub-asset (`mesh`, `material`) — the
+bundle is the unit, so those would rebuild the same set `glb` does — and none for normal maps, since
+"is this encoder fix normal-only?" is exactly the ambiguous question a recipe must not ask.
+
+Scope and cost:
+- Only the digest-named lane carries recipes. Wearables and emotes are `{hash}_{platform}`, with
+  nowhere to put a generation, so a fix to those still needs `AB_VERSION` — as does anything that
+  changes the bundle container, the manifest shape, or the client's side of the contract.
+- Generation `0` folds in as nothing *in a digest*, so adopting this cost no rebuild: every
+  bundle name is byte-for-byte what it was before. Manifests do gain a `recipes` key.
+- `AB_VERSION` stays total. It is the key prefix, never a digest input, and the conversion gate
+  gained a conjunct rather than a substitute, so a version bump still reconverts everything.
+- Each conversion records, in the per-platform manifest's `recipes` block, every recipe governing
+  its bundles and the generation each stood at — **baselines included** — and `platform_converted`
+  compares it, so the lambda's already-converted skip stays correct without a `force` flag. A scene
+  of static props records `{"glb":0,"texture":0}`: bumping `skin` leaves it current (that is the
+  saving), bumping `glb` does not. Recording only the *bumped* recipes would be blind to every
+  later `0 -> 1` bump, which is why the baselines are kept here and dropped in the digest.
+  A manifest predating recipes has no block at all and reconverts once.
+- The LOD lane solves the same problem with one counter of its own,
+  `crate/src/lodgen/pipeline.rs::LOD_GENERATION`, folded into the reuse state document.
+
 Display-only files — `autogenerated-thumbnail.png` anywhere in the content map, and whatever
 `metadata.display.navmapThumbnail` points at — are dropped from the conversion input, matching the
 prod converter's `-skippedHashes` flag: no bundle, no manifest entry, and no contribution to the
@@ -181,12 +254,17 @@ request waits up to the deadline, builds finish in the background. Knobs: `ABGEN
 `ABGEN_INDEX_BUILD_CONCURRENCY` (default: CPU count), `ABGEN_INDEX_BUILD_DEADLINE_MS` (default
 `20000`), `ABGEN_INDEX_BUILD_MAX_QUEUE` (default `0` = unlimited; when exceeded, new builds are skipped).
 ### LOD JIT lane
-`ABGEN_LOD_JIT=1` enables JIT LOD builds on `GET /LOD/{0|1}/...` misses; needs `gltfpack`
+`ABGEN_LOD_JIT=1` enables JIT LOD builds on `GET /LOD/1/...` misses (level 1 only, the production
+shape: level 0 is the client-side ISS assembly, so `LOD/0/...` answers `lod-level-unsupported` and
+`abgen-lod generate --level 0,1` is the offline opt-in); needs `gltfpack`
 (`ABGEN_GLTFPACK` or `$PATH`), fails closed without it. Knobs: `ABGEN_LOD_MANIFEST_BUILDER` (unset
 limits JIT to scenes with a published ISS descriptor), `ABGEN_LOD_CACHE_DIR` (default:
 `ABGEN_CACHE_DIR`), `ABGEN_LOD_JIT_TIMEOUT_S` / `ABGEN_LOD_JIT_FAIL_TTL_S` (defaults `600` / `3600`),
 `ABGEN_LOD_BUILD_CONCURRENCY` (default `1`). Builds stage in a per-build workdir; only gate-passed
-output is promoted into the serving root, so rejected bundles are never servable.
+output is promoted into the serving root, so rejected bundles are never servable. A passed build is
+written back to the space with the production key set: `LOD/1/{sid}_1_{platform}`,
+`lods-unity/manifests/{sid}_InitialSceneState.json` and the gltfpack-layout
+`lods-unity/lods/{sid}_1.glb`.
 ### Conversion / parity knobs
 - `ABGEN_SHADER_BUNDLE` - path to `scene_ignore_windows` (default `crate/shader/scene_ignore_windows`; siblings like `scene_ignore_mac` resolve from the same dir)
 - `ABGEN_CONTENT_ROOT` - local sharded content store root (default `./content`)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dcl/abgen-win32-x64",
-  "version": "0.17.13",
+  "version": "0.19.0",
   "description": "abgen prebuilt binary for win32 x64 (installed by @dcl/abgen)",
   "license": "AGPL-3.0-or-later",
   "repository": {