@displayxr/inline3d 1.6.1 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,104 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
5
5
  `./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
6
6
  which tier they touch, because that is what tells you whether an upgrade can move your pixels.
7
7
 
8
+ ## 1.7.0 — 2026-09-19
9
+
10
+ Touches the **preview tier** (`./splat`) only, and additively: `addSplat` with no new options
11
+ renders exactly as it did in 1.6.1 — every Spark default stays where Spark put it and the display
12
+ rig with its auto-frame is still what an asset without a `camera` block gets.
13
+
14
+ ### Added
15
+
16
+ - **`addSplat({ perf })` — cut a splat's overdraw** (preview tier). A splat scene's cost is the
17
+ per-fragment composite, and splat COUNT is the weakest axis on it: decimating the reference
18
+ 1.18M-gaussian capture to 25 % breaks it visibly while removing less cost than these settings,
19
+ which remove none of the picture. Two presets (`'balanced'`, `'aggressive'`) or an object of your
20
+ own over Spark's `minAlpha` / `maxStdDev` / `minPixelRadius` / `maxPixelRadius` / `falloff` and
21
+ its LOD budget, plus two of this SDK's own:
22
+ - **`alphaRadius`** shrinks each splat's quad to the radius where its own alpha reaches
23
+ `minAlpha`. Spark's fragment shader already discards everything past that radius, so this
24
+ removes work and not pixels — **bit-exact**, measured: 457 of 3,686,400 channel bytes differ
25
+ at 1280×720, every one of them by exactly 1. Spark 2.1.0 has no option for it (`maxStdDev` is
26
+ one global uniform), so the SDK patches Spark's splat vertex shader through its supported
27
+ `vertexShader` surface, rewriting Spark's OWN source off the live material rather than shipping
28
+ a copy — a Spark upgrade brings its shader fixes along, and if the lines stop matching the
29
+ patch declines with one warning and everything still renders.
30
+ - **`alphaFloor`** moves that cut up: each tail is dropped where IT reaches the floor rather than
31
+ where an 8-bit framebuffer stops representing it — the per-splat version of turning
32
+ `maxStdDev` down.
33
+
34
+ Two results from measuring it that are worth more than the options themselves, because both are
35
+ the opposite of the obvious move (M1 Pro, Chrome/ANGLE-Metal, GPU timer queries, configs
36
+ interleaved frame by frame):
37
+ - **Decimating the asset buys nothing.** 50 % and 25 % decimations measured within noise of the
38
+ full 1.18M-gaussian scene. Decimation drops the small gaussians and the few huge ones that
39
+ cover the frame survive it. A decimated `.sog` is a download win, not a render-cost win.
40
+ - **The bit-exact shrink buys little on a lifted photograph**, because 86 % of its gaussians are
41
+ near-opaque and an opaque splat's own 1/255 radius is already wider than the σ Spark draws it
42
+ at. It is exact and it stays — the scene it was built for is large low-alpha haze — but the
43
+ preset that pays on the web (`'balanced'`, −5…−20 %) tightens the quad extent instead.
44
+
45
+ `handle.perf` reports what was applied, and `applySplatPerf(spark, perf)` is exported for pages
46
+ that build their own `SparkRenderer` — the knobs are live, so a quality menu can call it at any
47
+ time. Measured numbers, and which knob is worth which pixels, are in
48
+ [docs/authoring-inline-3d.md](docs/authoring-inline-3d.md#gaussian-splats-performance-and-the-camera-block).
49
+
50
+ - **A `.sog`'s `camera` block now picks the view rig, and a WATERFALL fills in the rest**
51
+ (preview tier). A splat viewer needs BOTH rigs and the same call site loads both kinds of asset
52
+ — a product hero wants the display rig and its auto-frame, while a photograph lifted into 3D
53
+ wants the camera it was taken with. Nothing in the page can tell them apart; the file can.
54
+ `addSplat` reads the `camera` block out of the `.sog` (a PKZip — ~40 bytes of central directory,
55
+ never the webp planes, and only on the BYTES path) and resolves three questions from it:
56
+
57
+ | | 1st | 2nd | 3rd | last |
58
+ |---|---|---|---|---|
59
+ | **rig** | caller | the block's `rig` | a block at all ⇒ camera | display |
60
+ | **intrinsics** | the block | caller | **estimated from the cloud** | 28 mm-eq |
61
+ | **focus** | caller | the block's `focus.point` | **median disparity** | 2 m ahead |
62
+
63
+ Each resolved value carries the step that produced it (`handle.rig.focusSource`,
64
+ `intrinsicsSource`, `typeSource`), because a number from a lower step is not a wrong number, it
65
+ is a wrong SOURCE, and that is invisible in the picture.
66
+
67
+ The block is now a **v2 superset**: `rig`, `focus` (one point that is the orbit centre, the pivot
68
+ plane AND the convergence) and `dxr` (the camera rig's absolute scalars) join it, `intrinsics`
69
+ becomes optional, and a v1 block still reads. `rig: "display"` beside a `rest` is meaningful —
70
+ *a display rig, opened at this viewpoint*.
71
+
72
+ **Estimating the lens** works because a capture's gaussians only exist where its camera could see
73
+ them: P1/P99 of `x/z` and `y/z` about the rest camera ARE the frustum that made it, principal
74
+ point included. Measured against a capture whose true half-tangents are ±0.857 and ±0.482:
75
+ 0.8635 and 0.4827, +0.75 % and +0.12 %. The implied 35 mm-equivalent focal is gated to
76
+ [14, 85] mm, outside which the cloud is describing something that is not a camera. It matters
77
+ because a splat rendered through the wrong focal is drawn at the wrong SIZE and nothing else —
78
+ no artefact, just a picture that feels zoomed out.
79
+
80
+ **Estimating the focus** is the median of 1/z, inverted — not of z. On the reference capture that
81
+ is 2.17 m against the gallery's own 2.14 m; the centre of the measured bounds, which this
82
+ replaced, was 39.8 m, because an open scene's percentile bounds are 128 m wide.
83
+
84
+ On the camera path the subject is not reframed, the turntable is off, the mono camera is posed
85
+ and lensed as the capture, and the rig is **declared** with `cameraRigFromCamera` — the off-axis
86
+ projection stays in the runtime.
87
+
88
+ - **Pointing the window: double-click, Space and `handle.setFocus(point|null)`** (preview tier).
89
+ The focus is one point — the orbit centre, the pivot plane and the convergence — and it is now
90
+ something a viewer can move. Double-click focuses what was clicked (Spark's own
91
+ `SplatMesh.raycast`, ~57 ms over 1.18M gaussians, with a nearest-gaussian-to-the-ray fallback
92
+ documented as the approximation it is); Space returns to the resolved value; both ease at 0.18
93
+ per frame, and while the ease runs a camera rig re-declares its convergence every frame. What
94
+ moves depends on the rig and only on that: a camera rig moves the rotation centre and leaves the
95
+ capture where it was placed, a display rig brings the focused point to the middle of the tile.
96
+ `focusInput: false` turns the gestures off for a page that owns them itself, and
97
+ `handle.pick(x, y)` exposes the raycast.
98
+
99
+ New on the handle: **`handle.camera`** (the raw block), **`handle.rig`** (the resolved waterfall),
100
+ `handle.viewRig`, `handle.perf`, `handle.setFocus`, `handle.pick`. New exports from `./splat`:
101
+ `readSogCamera(bytes)`, `readSogMeta(bytes)`, `resolveRig`, `applySplatPerf`,
102
+ `SPLAT_PERF_PRESETS`. **`SceneViewer` gains `setFocus` / `getFocus` / `onFocusChange` / `onTick`**
103
+ (`./viewer`), and `fitTo` now goes through the focus, so a refit cannot leave the orbit turning
104
+ about somewhere the framing has moved away from.
105
+
8
106
  ## 1.6.1 — 2026-09-09
9
107
 
10
108
  Touches the **core tier** (`.`) with a behaviour fix only — no API changes — and the **preview tier**
package/README.md CHANGED
@@ -113,6 +113,7 @@ js/
113
113
  inline3d-viewer.js experimental: SceneViewer — framing, orbit, idle turntable, mono fallback,
114
114
  and the placement readback (getSubjectBounds / getPose / depthOffset)
115
115
  inline3d-splat.js experimental: addSplat() — a Gaussian splat window via Spark
116
+ (`perf` cuts overdraw; a `.sog`'s `camera` block picks the view rig)
116
117
  inline3d-model.js experimental: addModel() — a glTF/GLB window; wires Draco / meshopt / KTX2
117
118
  from what the asset declares (you serve the decoder files — see the guide)
118
119
  docs/
@@ -0,0 +1,295 @@
1
+ // inline3d-sog.js — read the `meta.json` out of a `.sog`, and the optional `camera` block in it.
2
+ //
3
+ // EXPERIMENTAL. Internal to `./splat`, which re-exports `readSogCamera`. Not covered by the SDK's
4
+ // 1.x semver promise — see docs/sdk-stability.md.
5
+ //
6
+ // WHY A ZIP READER IS IN THIS SDK AT ALL. A `.sog` from `splat-transform` is a PKZip of webp
7
+ // planes plus a `meta.json`, and Spark reads exactly the fields it needs to build splats out of
8
+ // it — it neither surfaces the rest of the file nor hands back the parsed metadata. But whether a
9
+ // splat is an OBJECT (a product hero, a scan, a turntable subject) or a PHOTOGRAPH LIFTED INTO 3D
10
+ // is not a rendering detail: it decides which VIEW RIG the window should be on, and getting that
11
+ // wrong is the difference between a picture you can lean into and an arbitrary cloud framed by a
12
+ // bounding box. The `camera` block records the recording camera so the viewer can conserve it.
13
+ //
14
+ // So the choice is between asking every page to parse its own assets and reading ~40 bytes of
15
+ // central directory here. The reader below does the second: it is deliberately the smallest thing
16
+ // that can find ONE named entry in a zip, and it never touches the webp planes (which are
17
+ // megabytes, and Spark's business).
18
+ //
19
+ // It is BYTES-ONLY on purpose. The gallery hands the SDK bytes rather than a URL (Spark infers a
20
+ // splat's format from the URL path, so an extension-less `blob:` URL fails inside a worker), and
21
+ // those same bytes are the ones this reads — one download, no second fetch, no range request, and
22
+ // no chance of reading metadata from a different build of the asset than the one on screen.
23
+
24
+ /** `PK\x05\x06` — end of central directory. */
25
+ const EOCD_SIG = 0x06054b50;
26
+ /** `PK\x01\x02` — a central-directory file header. */
27
+ const CEN_SIG = 0x02014b50;
28
+ /** `PK\x03\x04` — a local file header. */
29
+ const LOC_SIG = 0x04034b50;
30
+
31
+ /** EOCD is 22 bytes plus a comment of at most 64 KiB. */
32
+ const EOCD_MAX_BACK = 22 + 0xffff;
33
+
34
+ /**
35
+ * A `meta.json` this large is not a `meta.json`. The guard is against a malformed/hostile
36
+ * central directory, not against real assets: the largest one seen is ~4 KB.
37
+ */
38
+ const META_MAX_BYTES = 4 << 20;
39
+
40
+ /** Locate the end-of-central-directory record, scanning backwards. */
41
+ function findEocd(dv) {
42
+ const len = dv.byteLength;
43
+ if (len < 22) return -1;
44
+ const stop = Math.max(0, len - EOCD_MAX_BACK);
45
+ for (let i = len - 22; i >= stop; i--) {
46
+ if (dv.getUint32(i, true) === EOCD_SIG) return i;
47
+ }
48
+ return -1;
49
+ }
50
+
51
+ /** Inflate a raw deflate stream. Returns null where the platform has no DecompressionStream. */
52
+ async function inflateRaw(slice) {
53
+ if (typeof DecompressionStream !== 'function') return null;
54
+ const ds = new DecompressionStream('deflate-raw');
55
+ const stream = new Blob([slice]).stream().pipeThrough(ds);
56
+ return new Uint8Array(await new Response(stream).arrayBuffer());
57
+ }
58
+
59
+ /**
60
+ * Read one named entry out of a PKZip, as bytes.
61
+ *
62
+ * Handles the two compression methods a `.sog` actually uses — 0 (stored) and 8 (deflate) — and
63
+ * returns null for anything else rather than guessing. Zip64 is refused the same way: a `.sog`
64
+ * big enough to need it would have to be over 4 GB.
65
+ *
66
+ * @param {Uint8Array} bytes the whole archive.
67
+ * @param {string} name exact entry name, e.g. `meta.json`.
68
+ * @returns {Promise<Uint8Array|null>}
69
+ */
70
+ export async function readZipEntry(bytes, name) {
71
+ if (!bytes || bytes.byteLength < 22) return null;
72
+ const dv = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
73
+ const eocd = findEocd(dv);
74
+ if (eocd < 0) return null;
75
+
76
+ const count = dv.getUint16(eocd + 10, true);
77
+ let p = dv.getUint32(eocd + 16, true);
78
+ // 0xffffffff in either field is the zip64 escape; we do not follow it (see above).
79
+ if (p === 0xffffffff || p >= dv.byteLength) return null;
80
+
81
+ const dec = new TextDecoder();
82
+ for (let i = 0; i < count; i++) {
83
+ if (p + 46 > dv.byteLength || dv.getUint32(p, true) !== CEN_SIG) return null;
84
+ const method = dv.getUint16(p + 10, true);
85
+ const compSize = dv.getUint32(p + 20, true);
86
+ const rawSize = dv.getUint32(p + 24, true);
87
+ const nameLen = dv.getUint16(p + 28, true);
88
+ const extraLen = dv.getUint16(p + 30, true);
89
+ const commentLen = dv.getUint16(p + 32, true);
90
+ const localOff = dv.getUint32(p + 42, true);
91
+ const entry = dec.decode(bytes.subarray(p + 46, p + 46 + nameLen));
92
+ p += 46 + nameLen + extraLen + commentLen;
93
+ if (entry !== name) continue;
94
+
95
+ if (compSize === 0xffffffff || rawSize === 0xffffffff || localOff === 0xffffffff) return null;
96
+ if (rawSize > META_MAX_BYTES) return null;
97
+ // The central directory's sizes are authoritative; the LOCAL header's may be zeroed (a
98
+ // streaming writer defers them to a data descriptor). Only its two length fields are read.
99
+ if (localOff + 30 > dv.byteLength || dv.getUint32(localOff, true) !== LOC_SIG) return null;
100
+ const lNameLen = dv.getUint16(localOff + 26, true);
101
+ const lExtraLen = dv.getUint16(localOff + 28, true);
102
+ const start = localOff + 30 + lNameLen + lExtraLen;
103
+ if (start + compSize > dv.byteLength) return null;
104
+ const slice = bytes.subarray(start, start + compSize);
105
+ if (method === 0) return slice;
106
+ if (method === 8) return inflateRaw(slice);
107
+ return null;
108
+ }
109
+ return null;
110
+ }
111
+
112
+ /**
113
+ * Parse a `.sog`'s `meta.json`.
114
+ *
115
+ * @param {Uint8Array|ArrayBuffer} bytes
116
+ * @returns {Promise<object|null>} the parsed object, or null if this is not a `.sog`, has no
117
+ * `meta.json`, or the entry cannot be read on this platform.
118
+ */
119
+ export async function readSogMeta(bytes) {
120
+ const u8 = bytes instanceof Uint8Array ? bytes : bytes ? new Uint8Array(bytes) : null;
121
+ if (!u8 || u8.length < 4) return null;
122
+ // PK\x03\x04 — cheap reject before the backwards scan, so a `.ply`/`.spz`/`.splat` costs
123
+ // four byte comparisons.
124
+ if (!(u8[0] === 0x50 && u8[1] === 0x4b && u8[2] === 0x03 && u8[3] === 0x04)) return null;
125
+ let raw;
126
+ try {
127
+ raw = await readZipEntry(u8, 'meta.json');
128
+ } catch {
129
+ return null;
130
+ }
131
+ if (!raw) return null;
132
+ try {
133
+ return JSON.parse(new TextDecoder().decode(raw));
134
+ } catch {
135
+ return null;
136
+ }
137
+ }
138
+
139
+ const num = (v) => (typeof v === 'number' && Number.isFinite(v) ? v : null);
140
+
141
+ /**
142
+ * Validate the `camera` block of a `.sog` `meta.json`.
143
+ *
144
+ * THE BLOCK IS THE RIG SWITCH — present means "this splat was lifted from a photograph, and here
145
+ * is the camera that took it"; absent means "this is an object", which is the display rig and the
146
+ * behaviour every existing page already has. That is the whole contract, and it is why this is
147
+ * strict: a half-parsed block that silently keeps some defaults would put a photo scene on a
148
+ * plausible-looking rig that is not the capture, which is indistinguishable from a framing bug.
149
+ *
150
+ * Shape (`meta.json`, top level, right after `count`; `version` stays 2). v2 is a SUPERSET of
151
+ * v1 — every key below except `convention` is optional, and a v1 block still reads:
152
+ *
153
+ * "camera": {
154
+ * "convention": "opencv",
155
+ * "rig": "camera", // v2: which rig this asset wants
156
+ * "rest": { "position": [0,0,0], "rotation": [0,0,0,1] },
157
+ * "intrinsics": { "fx":…, "fy":…, "cx":…, "cy":…, "width":…, "height":… },
158
+ * "stereo": { "baseline_m": 0.063 },
159
+ * "focus": { "point": [0,0,1.68], "subject_m":…, "near_m":…, "far_m":…,
160
+ * "source": "convergence|manual|auto" }, // v2
161
+ * "dxr": { "ipd_factor": 1.0, "parallax_factor": 1.0 } // v2
162
+ * }
163
+ *
164
+ * `intrinsics` BECAME OPTIONAL IN v2, which is the change with teeth: a block can now say "this
165
+ * is a camera rig, open it at this viewpoint" without claiming a lens, and the consumer is
166
+ * expected to estimate one. So this returns a descriptor with null intrinsics rather than
167
+ * refusing the block — refusing it would silently demote a camera-rig asset to the display rig,
168
+ * which is the failure this whole mechanism exists to prevent.
169
+ *
170
+ * `focus.point` is THE point: the orbit centre, the pivot plane and the convergence distance,
171
+ * which are one thing and are stored once.
172
+ *
173
+ * `convention` is REQUIRED to be `opencv` (+x right, +y DOWN, +z forward, pixel (0,0) at the top
174
+ * left) rather than defaulted: it is the frame the intrinsics are expressed in, and a reader that
175
+ * assumed it would mis-sign the principal-point offset on any other convention — a wrong answer
176
+ * with no error, which is the one failure mode a metadata block must not have.
177
+ *
178
+ * @param {object|null} meta a parsed `meta.json`.
179
+ * @returns {object|null} a normalised camera descriptor, or null.
180
+ */
181
+ export function sogCameraFromMeta(meta) {
182
+ const c = meta && typeof meta === 'object' ? meta.camera : null;
183
+ if (!c || typeof c !== 'object') return null;
184
+ if (c.convention !== 'opencv') {
185
+ console.warn(
186
+ `[inline3d/splat] .sog camera block has convention "${c.convention}" — only "opencv" is ` +
187
+ 'understood, so the block is IGNORED and this asset stays on the display rig.',
188
+ );
189
+ return null;
190
+ }
191
+ const i = c.intrinsics;
192
+ let intrinsics = null;
193
+ if (i && typeof i === 'object') {
194
+ const fx = num(i.fx);
195
+ const fy = num(i.fy);
196
+ const cx = num(i.cx);
197
+ const cy = num(i.cy);
198
+ const width = num(i.width);
199
+ const height = num(i.height);
200
+ if (!(fx > 0) || !(fy > 0) || !(width > 0) || !(height > 0) || cx === null || cy === null) {
201
+ // Half-believing a lens is worse than having none: with intrinsics optional in v2 there is
202
+ // a well-defined thing to do instead, which is estimate one from the cloud.
203
+ console.warn(
204
+ '[inline3d/splat] .sog camera block has unusable intrinsics — they are DROPPED and the ' +
205
+ 'lens is estimated from the cloud instead; the rest of the block still applies.',
206
+ i,
207
+ );
208
+ } else {
209
+ intrinsics = { fx, fy, cx, cy, width, height };
210
+ }
211
+ }
212
+ const pos = Array.isArray(c.rest?.position) ? c.rest.position.map((v) => num(v) ?? 0) : [0, 0, 0];
213
+ const rot = Array.isArray(c.rest?.rotation) ? c.rest.rotation.map((v) => num(v) ?? 0) : [0, 0, 0, 1];
214
+ const baseline = num(c.stereo?.baseline_m);
215
+
216
+ // v2 `rig`. Anything unrecognised is dropped rather than guessed at — the waterfall's next
217
+ // step (a block means a camera) is a better answer than a typo taken literally.
218
+ let rig = null;
219
+ if (c.rig === 'camera' || c.rig === 'display') rig = c.rig;
220
+ else if (c.rig !== undefined) {
221
+ console.warn(`[inline3d/splat] .sog camera block has rig "${c.rig}" — ignored`, c.rig);
222
+ }
223
+
224
+ // v2 `focus`. The point is the only required part; the three distances are advisory and are
225
+ // carried through untouched for a host page that wants them (a depth budget, a HUD).
226
+ let focus = null;
227
+ const fp = c.focus?.point;
228
+ if (Array.isArray(fp) && fp.length >= 3 && fp.every((v) => num(v) !== null)) {
229
+ focus = {
230
+ point: [fp[0], fp[1], fp[2]],
231
+ subject_m: num(c.focus.subject_m),
232
+ near_m: num(c.focus.near_m),
233
+ far_m: num(c.focus.far_m),
234
+ source: typeof c.focus.source === 'string' ? c.focus.source : null,
235
+ };
236
+ } else if (c.focus !== undefined) {
237
+ console.warn('[inline3d/splat] .sog camera block has an unusable focus — ignored', c.focus);
238
+ }
239
+
240
+ // v2 `dxr`. These are the camera rig's ABSOLUTE scalars, and they stay absolute: normalising
241
+ // them against the convergence distance would make the scene's depth breathe every time the
242
+ // viewer re-focused.
243
+ const ipdFactor = num(c.dxr?.ipd_factor);
244
+ const parallaxFactor = num(c.dxr?.parallax_factor);
245
+
246
+ return {
247
+ convention: 'opencv',
248
+ rig,
249
+ focus,
250
+ dxr: {
251
+ ipdFactor: ipdFactor !== null && ipdFactor >= 0 ? ipdFactor : null,
252
+ parallaxFactor: parallaxFactor !== null && parallaxFactor >= 0 ? parallaxFactor : null,
253
+ },
254
+ rest: {
255
+ position: [pos[0] ?? 0, pos[1] ?? 0, pos[2] ?? 0],
256
+ rotation: [rot[0] ?? 0, rot[1] ?? 0, rot[2] ?? 0, rot[3] ?? 1],
257
+ },
258
+ intrinsics,
259
+ stereo: baseline > 0 ? { baseline_m: baseline } : null,
260
+ /**
261
+ * Derived, because every consumer needs them and each is one line to get subtly wrong.
262
+ *
263
+ * Null when the block carried no usable intrinsics (legal in v2) — the caller estimates a
264
+ * lens from the cloud instead.
265
+ *
266
+ * `verticalFov` is the FULL vertical angle the capture subtends, in RADIANS — the unit an
267
+ * XRViewRigInit wants (three's `camera.fov` is the same angle in degrees).
268
+ *
269
+ * `principalOffset` is the principal point's offset from the frame centre as a fraction of
270
+ * the frame, x rightwards and **y upwards** — i.e. already out of OpenCV's y-down frame and
271
+ * into the GL/three one, so a consumer never has to remember which way `cy` grows. A
272
+ * rectified stereo pair carries its deconvergence here: shifting the principal point is what
273
+ * "deconverging" DOES to a pair, so a non-zero x is the capture's zero-disparity plane
274
+ * expressed as a lens shift rather than as a distance.
275
+ */
276
+ verticalFov: intrinsics ? 2 * Math.atan(intrinsics.height / (2 * intrinsics.fy)) : null,
277
+ principalOffset: intrinsics
278
+ ? {
279
+ x: (intrinsics.cx - intrinsics.width / 2) / intrinsics.width,
280
+ y: -(intrinsics.cy - intrinsics.height / 2) / intrinsics.height,
281
+ }
282
+ : null,
283
+ };
284
+ }
285
+
286
+ /**
287
+ * Read the `camera` block straight out of `.sog` bytes. Convenience over
288
+ * {@link readSogMeta} + {@link sogCameraFromMeta}.
289
+ *
290
+ * @param {Uint8Array|ArrayBuffer} bytes
291
+ * @returns {Promise<object|null>}
292
+ */
293
+ export async function readSogCamera(bytes) {
294
+ return sogCameraFromMeta(await readSogMeta(bytes));
295
+ }
@@ -0,0 +1,255 @@
1
+ // inline3d-splat-perf.js — cut a Gaussian splat's OVERDRAW, for `./splat`.
2
+ //
3
+ // EXPERIMENTAL. Internal to `./splat`, which re-exports `applySplatPerf` and takes `perf` as an
4
+ // option. Not covered by the SDK's 1.x semver promise — see docs/sdk-stability.md.
5
+ //
6
+ // WHAT COSTS WHAT. On a 1.18M-gaussian photo-lifted scene the per-fragment composite is 75–85 %
7
+ // of the frame (measured natively on an M1 Pro, one eye at 1920×1080), and it is OVERDRAW, not
8
+ // resolution: a handful of enormous, nearly transparent sky splats cover the frame many times
9
+ // over. Splat COUNT is the axis everyone reaches for first and it is the wrong one — decimating
10
+ // this asset to 25 % breaks it visibly (bright stipple on near lit surfaces) while removing far
11
+ // less fill than the two changes below, which remove none of the picture at all.
12
+ //
13
+ // The two that matter, in the native renderer's words: **shrink each splat's quad to the radius
14
+ // where its alpha falls below 1/255**, instead of a fixed 3σ, and **cull gaussians whose peak
15
+ // opacity is already below 1/255**. This module is both of those on Spark.
16
+ //
17
+ // WHY THE FIRST ONE IS FREE. Spark draws every splat as a quad of `maxStdDev` σ (default √8 ≈
18
+ // 2.83σ) and its fragment shader then discards any fragment whose alpha has fallen under
19
+ // `minAlpha` — so for a splat of peak alpha `a` every fragment beyond
20
+ //
21
+ // r = sqrt(2 · ln(a / minAlpha))
22
+ //
23
+ // is ALREADY being discarded. It is rasterised, interpolated, shaded, and thrown away. Shrinking
24
+ // the quad to exactly that radius therefore removes fragments that contributed nothing: the
25
+ // output is bit-identical (see `alphaRadius` below for the two conditions), and the saving is
26
+ // biggest on exactly the splats that dominate the cost — a haze splat of a = 0.02 needs 1.81σ,
27
+ // not 2.83σ, which is 2.4× fewer fragments for the same pixels.
28
+ //
29
+ // Spark 2.1.0 has no option for that (`maxStdDev` is ONE global uniform, and nothing in the
30
+ // shader derives a radius from alpha), so this patches the vertex shader — through the supported
31
+ // `vertexShader` surface, and by REWRITING SPARK'S OWN SOURCE off the live material rather than
32
+ // shipping a copy of it, so a Spark upgrade brings its shader fixes with it instead of silently
33
+ // pinning this SDK to a fork of a 2.1.0 file. If the anchors ever stop matching, the patch
34
+ // declines with one warning and everything still renders.
35
+
36
+ /**
37
+ * Where a splat's alpha has to fall before its quad may be cut, as a fraction of full opacity.
38
+ *
39
+ * 1/255 is the native renderer's threshold and the point below which an 8-bit framebuffer cannot
40
+ * represent the contribution at all. Note Spark's own `minAlpha` default is HALF of this
41
+ * (0.5/255) — deliberately, so its discard is a shade conservative — which is why `minAlpha` is
42
+ * a knob here and not an assumption.
43
+ */
44
+ const ALPHA_FLOOR = 1 / 255;
45
+
46
+ /**
47
+ * Presets — chosen from measurements on this exact asset class, not from first principles.
48
+ *
49
+ * What the measurement said (1.18M-gaussian SHARP capture, M1 Pro, Chrome/ANGLE-Metal, GPU timer
50
+ * queries, configs interleaved PER FRAME so clock drift cannot bias one against another; full
51
+ * table in docs/authoring-inline-3d.md):
52
+ *
53
+ * - **Splat COUNT is not the cost.** 50 % and 25 % decimations of the same scene measured
54
+ * within noise of the full one (+5 %, +1 % at 1920×1080). Decimation drops the small
55
+ * gaussians; the few enormous ones that cover the frame survive it, and they are the bill.
56
+ * A decimated asset is a download and memory win, not a render-cost win.
57
+ * - **Quad extent is the cost.** `maxStdDev` √8→√6 is −5…−20 % and √8→√4 is −22 %.
58
+ * - **The bit-exact `alphaRadius` buys ~nothing HERE**, because it has nothing to shrink: 86 %
59
+ * of this asset's gaussians are near-opaque (mean peak alpha 0.86; Spark doubles the stored
60
+ * alpha on top), and an opaque splat's own 1/255 radius is 3.53σ, wider than the √8 ≈ 2.83σ
61
+ * it is already drawn at. It stays available and stays exact — a scene of large, low-alpha
62
+ * haze is exactly where it pays, and that is the scene the native renderer was tuned on.
63
+ *
64
+ * So the presets are honest about which axis works, and nothing is applied unless a caller asks:
65
+ * `addSplat` with no `perf` leaves every Spark default exactly where Spark put it.
66
+ */
67
+ export const SPLAT_PERF_PRESETS = {
68
+ /**
69
+ * The bit-exact one. No measurable win on a mostly-opaque capture; real on a scene whose cost is
70
+ * large low-alpha splats. Costs a little vertex ALU, so on a scene with nothing to shrink it can
71
+ * read as a wash or a shade slower.
72
+ */
73
+ exact: {
74
+ alphaRadius: true,
75
+ minAlpha: ALPHA_FLOOR,
76
+ },
77
+ /** −5…−20 % measured. Truncates every splat's tail at 2.45σ instead of 2.83σ. */
78
+ balanced: {
79
+ minAlpha: ALPHA_FLOOR,
80
+ maxStdDev: Math.sqrt(6),
81
+ },
82
+ /** −22 % measured. 2σ, plus the sub-pixel cull. For a phone, not for a hero. */
83
+ aggressive: {
84
+ minAlpha: ALPHA_FLOOR,
85
+ maxStdDev: 2,
86
+ // Both eigenaxes under a pixel. On the reference capture at 1280×720 and 1920×1080 this
87
+ // changed ZERO channel bytes — a quad that small usually covers no sample point at all — but
88
+ // it is framing-dependent by nature, so it is here and not in `balanced`.
89
+ minPixelRadius: 1,
90
+ },
91
+ };
92
+
93
+ /** Fields that are plain properties on SparkRenderer, copied into uniforms every frame. */
94
+ const SPARK_FIELDS = [
95
+ 'minAlpha',
96
+ 'maxStdDev',
97
+ 'minPixelRadius',
98
+ 'maxPixelRadius',
99
+ 'falloff',
100
+ // LOD budget knobs. Live, but inert unless the MESH was loaded with LOD data — see
101
+ // splatPerfMeshOptions().
102
+ 'lodSplatCount',
103
+ 'lodSplatScale',
104
+ 'lodRenderScale',
105
+ ];
106
+
107
+ // The two anchors the shader patch needs, quoted from Spark 2.1.0's `splatVertex.glsl`.
108
+ //
109
+ // `vRgba.a = rgba.a;` is the line AFTER the anti-aliasing blur has been folded into the alpha
110
+ // (`rgba.a *= blurAdjust`) — so at that point `rgba.a` is exactly the alpha the fragment shader
111
+ // will interpolate, which is what makes the derived radius exact rather than approximate. It is
112
+ // also after the eigen-decomposition's inputs are final and BEFORE `scale1`/`scale2` are taken
113
+ // from `adjustedStdDev`, which is the only window where writing it has any effect.
114
+ const ANCHOR_ALPHA = ' vRgba.a = rgba.a;\n';
115
+ const ANCHOR_UNIFORM = 'uniform float minAlpha;\n';
116
+
117
+ /**
118
+ * Rewrite Spark's splat vertex shader so each quad is only as big as its own alpha justifies.
119
+ *
120
+ * `falloff` is read (not assumed) because the whole argument rests on the fragment shader's
121
+ * `a·exp(−z²/2)` decay: at `falloff < 1` the alpha does NOT decay across the quad, nothing is
122
+ * being discarded, and cutting the quad would cut the picture. The uniform is shared between the
123
+ * two stages, so the vertex shader only has to declare it.
124
+ *
125
+ * @returns {boolean} whether the patch went in.
126
+ */
127
+ function patchAlphaRadius(material) {
128
+ const src = material?.vertexShader;
129
+ if (typeof src !== 'string') return false;
130
+ if (src.includes('dxrAlphaRadius')) return true; // idempotent — a shared material, or a re-apply
131
+ if (!src.includes(ANCHOR_ALPHA) || !src.includes(ANCHOR_UNIFORM)) {
132
+ console.warn(
133
+ '[inline3d/splat] perf.alphaRadius: this build of Spark does not have the shader lines ' +
134
+ 'this patch rewrites, so the quad-shrink is SKIPPED (everything else still applies, and ' +
135
+ 'the picture is unchanged). Report the Spark version — the anchors are versioned in ' +
136
+ 'js/inline3d-splat-perf.js.',
137
+ );
138
+ return false;
139
+ }
140
+ material.vertexShader = src
141
+ .replace(
142
+ ANCHOR_UNIFORM,
143
+ `${ANCHOR_UNIFORM}uniform float falloff;\nuniform bool dxrAlphaRadius;\nuniform float dxrAlphaFloor;\n`,
144
+ )
145
+ .replace(
146
+ ANCHOR_ALPHA,
147
+ `${ANCHOR_ALPHA}
148
+ // @displayxr/inline3d: shrink the quad to where this splat's own alpha reaches minAlpha.
149
+ // Every fragment outside that radius is discarded by the fragment shader anyway, so this
150
+ // removes work and not pixels. Guarded on falloff == 1, which is what makes that true.
151
+ if (dxrAlphaRadius && (falloff == 1.0) && (rgba.a <= 1.0)) {
152
+ float floorA = max(dxrAlphaFloor > 0.0 ? dxrAlphaFloor : minAlpha, 1e-6);
153
+ adjustedStdDev = min(adjustedStdDev, sqrt(max(0.0, 2.0 * log(rgba.a / floorA))));
154
+ vSplatUv = position.xy * adjustedStdDev;
155
+ }
156
+ `,
157
+ );
158
+ material.uniforms.dxrAlphaRadius = { value: true };
159
+ // 0 means "use minAlpha", which is the bit-exact cut. See `alphaFloor`.
160
+ material.uniforms.dxrAlphaFloor = { value: 0 };
161
+ material.needsUpdate = true;
162
+ return true;
163
+ }
164
+
165
+ /**
166
+ * Apply a perf profile to a live `SparkRenderer`.
167
+ *
168
+ * Exported for pages that build their own Spark renderer instead of going through `addSplat`
169
+ * — the knobs are all live, so this can be called at any time (a quality menu, a battery-saver
170
+ * toggle) and takes effect on the next frame.
171
+ *
172
+ * | option | default (Spark 2.1.0) | effect | safety |
173
+ * |---|---|---|---|
174
+ * | `alphaRadius` | — (no such thing) | quad shrunk to the splat's own `alphaFloor` radius | **bit-exact** at the default floor, see below |
175
+ * | `alphaFloor` | — (= `minAlpha`) | the alpha the tail may be cut at, PER SPLAT | lossy above `minAlpha`, and gently: it spends radius where the splat is opaque and takes it where it is not |
176
+ * | `minAlpha` | `0.5/255` | splats and fragments under this alpha are dropped | lossy under 1 LSB |
177
+ * | `maxStdDev` | `√8` | quad extent in σ, globally | lossy: truncates opaque tails |
178
+ * | `minPixelRadius` | `0` | drop splats under this size in px | lossy: drops fine grain |
179
+ * | `maxPixelRadius` | `512` | clamp on quad size in px | lossy, and it SQUASHES: the profile is compressed into the smaller quad, not clipped |
180
+ * | `falloff` | `1` | 1 = Gaussian, 0 = flat | NOT a perf knob — 0 makes the fragment discard stop firing, which costs MORE |
181
+ *
182
+ * BIT-EXACT, and the two conditions on that word. `alphaRadius` only removes fragments the
183
+ * fragment shader was already discarding, so the composited image does not change — provided
184
+ * `falloff` is 1 (the shader guards this itself) and `minPixelRadius` is 0. With a non-zero
185
+ * `minPixelRadius` the shrunken quad can fall under it and the splat is then dropped outright,
186
+ * which is a real (small) change; that is why `balanced` leaves it at 0. At the discard boundary
187
+ * itself the two sides can disagree by one float ULP, where the fragment's own contribution is
188
+ * below 1/255 by construction — invisible in 8 bits, but "bit-exact" is stated with that caveat
189
+ * rather than without it.
190
+ *
191
+ * @param {object} spark a SparkRenderer.
192
+ * @param {true|'balanced'|'aggressive'|object} perf
193
+ * @returns {object|null} the profile actually applied.
194
+ */
195
+ export function applySplatPerf(spark, perf) {
196
+ if (!spark || !perf) return null;
197
+ let profile;
198
+ if (perf === true) profile = SPLAT_PERF_PRESETS.balanced;
199
+ else if (typeof perf === 'string') profile = SPLAT_PERF_PRESETS[perf];
200
+ else if (typeof perf === 'object') profile = perf;
201
+ if (!profile) {
202
+ console.warn(
203
+ `[inline3d/splat] unknown perf preset "${perf}" — ignored. ` +
204
+ `Known: ${Object.keys(SPLAT_PERF_PRESETS).join(', ')}, or an options object.`,
205
+ );
206
+ return null;
207
+ }
208
+
209
+ const applied = {};
210
+ for (const key of SPARK_FIELDS) {
211
+ if (typeof profile[key] === 'number' && Number.isFinite(profile[key])) {
212
+ spark[key] = profile[key];
213
+ applied[key] = profile[key];
214
+ }
215
+ }
216
+ // `alphaFloor` is meaningless on its own — it is the floor the shrink cuts at — so asking for
217
+ // one asks for the shrink, unless the caller said otherwise in the same breath.
218
+ const wantRadius = profile.alphaRadius ?? (profile.alphaFloor !== undefined ? true : undefined);
219
+ if (wantRadius) patchAlphaRadius(spark.material);
220
+ const u = spark.material?.uniforms;
221
+ if (u?.dxrAlphaRadius && wantRadius !== undefined) {
222
+ // Set the value every time rather than relying on the patch: the patch is idempotent, so a
223
+ // second call asking to turn it back ON would otherwise return early and leave it off.
224
+ u.dxrAlphaRadius.value = !!wantRadius;
225
+ applied.alphaRadius = !!wantRadius;
226
+ }
227
+ if (u?.dxrAlphaFloor && profile.alphaFloor !== undefined) {
228
+ u.dxrAlphaFloor.value = Number.isFinite(profile.alphaFloor) ? profile.alphaFloor : 0;
229
+ applied.alphaFloor = u.dxrAlphaFloor.value;
230
+ }
231
+ return applied;
232
+ }
233
+
234
+ /**
235
+ * The half of a perf profile that has to go into the `SplatMesh` CONSTRUCTOR rather than onto the
236
+ * renderer — Spark builds level-of-detail data at load time or not at all.
237
+ *
238
+ * LOD is the splat-COUNT axis: Spark keeps a merged, decimated pyramid and picks a level against
239
+ * a budget (2.5M splats on desktop, 1M on Android) and a minimum on-screen splat size. It is
240
+ * genuinely lossy — it substitutes merged splats — and it is inert unless the mesh was loaded
241
+ * with it, which is why it cannot be switched on later from `applySplatPerf`.
242
+ *
243
+ * @param {object} perf
244
+ * @returns {object} extra SplatMesh options (empty when LOD was not asked for).
245
+ */
246
+ export function splatPerfMeshOptions(perf) {
247
+ const profile =
248
+ perf === true
249
+ ? SPLAT_PERF_PRESETS.balanced
250
+ : typeof perf === 'string'
251
+ ? SPLAT_PERF_PRESETS[perf]
252
+ : perf;
253
+ if (!profile || !profile.lod) return {};
254
+ return { lod: profile.lod === 'quality' ? 'quality' : true };
255
+ }