@displayxr/inline3d 1.6.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +136 -0
- package/README.md +1 -0
- package/index.d.ts +9 -0
- package/js/inline3d-model.js +3 -0
- package/js/inline3d-sog.js +295 -0
- package/js/inline3d-splat-perf.js +255 -0
- package/js/inline3d-splat-rig.js +330 -0
- package/js/inline3d-splat.js +484 -8
- package/js/inline3d-viewer.js +146 -4
- package/js/inline3d.js +138 -6
- package/package.json +4 -1
- package/splat.d.ts +164 -0
- package/viewer.d.ts +22 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,142 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
|
|
|
5
5
|
`./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
|
|
6
6
|
which tier they touch, because that is what tells you whether an upgrade can move your pixels.
|
|
7
7
|
|
|
8
|
+
## 1.7.0 — 2026-09-19
|
|
9
|
+
|
|
10
|
+
Touches the **preview tier** (`./splat`) only, and additively: `addSplat` with no new options
|
|
11
|
+
renders exactly as it did in 1.6.1 — every Spark default stays where Spark put it and the display
|
|
12
|
+
rig with its auto-frame is still what an asset without a `camera` block gets.
|
|
13
|
+
|
|
14
|
+
### Added
|
|
15
|
+
|
|
16
|
+
- **`addSplat({ perf })` — cut a splat's overdraw** (preview tier). A splat scene's cost is the
|
|
17
|
+
per-fragment composite, and splat COUNT is the weakest axis on it: decimating the reference
|
|
18
|
+
1.18M-gaussian capture to 25 % breaks it visibly while removing less cost than these settings,
|
|
19
|
+
which remove none of the picture. Two presets (`'balanced'`, `'aggressive'`) or an object of your
|
|
20
|
+
own over Spark's `minAlpha` / `maxStdDev` / `minPixelRadius` / `maxPixelRadius` / `falloff` and
|
|
21
|
+
its LOD budget, plus two of this SDK's own:
|
|
22
|
+
- **`alphaRadius`** shrinks each splat's quad to the radius where its own alpha reaches
|
|
23
|
+
`minAlpha`. Spark's fragment shader already discards everything past that radius, so this
|
|
24
|
+
removes work and not pixels — **bit-exact**, measured: 457 of 3,686,400 channel bytes differ
|
|
25
|
+
at 1280×720, every one of them by exactly 1. Spark 2.1.0 has no option for it (`maxStdDev` is
|
|
26
|
+
one global uniform), so the SDK patches Spark's splat vertex shader through its supported
|
|
27
|
+
`vertexShader` surface, rewriting Spark's OWN source off the live material rather than shipping
|
|
28
|
+
a copy — a Spark upgrade brings its shader fixes along, and if the lines stop matching the
|
|
29
|
+
patch declines with one warning and everything still renders.
|
|
30
|
+
- **`alphaFloor`** moves that cut up: each tail is dropped where IT reaches the floor rather than
|
|
31
|
+
where an 8-bit framebuffer stops representing it — the per-splat version of turning
|
|
32
|
+
`maxStdDev` down.
|
|
33
|
+
|
|
34
|
+
Two results from measuring it that are worth more than the options themselves, because both are
|
|
35
|
+
the opposite of the obvious move (M1 Pro, Chrome/ANGLE-Metal, GPU timer queries, configs
|
|
36
|
+
interleaved frame by frame):
|
|
37
|
+
- **Decimating the asset buys nothing.** 50 % and 25 % decimations measured within noise of the
|
|
38
|
+
full 1.18M-gaussian scene. Decimation drops the small gaussians and the few huge ones that
|
|
39
|
+
cover the frame survive it. A decimated `.sog` is a download win, not a render-cost win.
|
|
40
|
+
- **The bit-exact shrink buys little on a lifted photograph**, because 86 % of its gaussians are
|
|
41
|
+
near-opaque and an opaque splat's own 1/255 radius is already wider than the σ Spark draws it
|
|
42
|
+
at. It is exact and it stays — the scene it was built for is large low-alpha haze — but the
|
|
43
|
+
preset that pays on the web (`'balanced'`, −5…−20 %) tightens the quad extent instead.
|
|
44
|
+
|
|
45
|
+
`handle.perf` reports what was applied, and `applySplatPerf(spark, perf)` is exported for pages
|
|
46
|
+
that build their own `SparkRenderer` — the knobs are live, so a quality menu can call it at any
|
|
47
|
+
time. Measured numbers, and which knob is worth which pixels, are in
|
|
48
|
+
[docs/authoring-inline-3d.md](docs/authoring-inline-3d.md#gaussian-splats-performance-and-the-camera-block).
|
|
49
|
+
|
|
50
|
+
- **A `.sog`'s `camera` block now picks the view rig, and a WATERFALL fills in the rest**
|
|
51
|
+
(preview tier). A splat viewer needs BOTH rigs and the same call site loads both kinds of asset
|
|
52
|
+
— a product hero wants the display rig and its auto-frame, while a photograph lifted into 3D
|
|
53
|
+
wants the camera it was taken with. Nothing in the page can tell them apart; the file can.
|
|
54
|
+
`addSplat` reads the `camera` block out of the `.sog` (a PKZip — ~40 bytes of central directory,
|
|
55
|
+
never the webp planes, and only on the BYTES path) and resolves three questions from it:
|
|
56
|
+
|
|
57
|
+
| | 1st | 2nd | 3rd | last |
|
|
58
|
+
|---|---|---|---|---|
|
|
59
|
+
| **rig** | caller | the block's `rig` | a block at all ⇒ camera | display |
|
|
60
|
+
| **intrinsics** | the block | caller | **estimated from the cloud** | 28 mm-eq |
|
|
61
|
+
| **focus** | caller | the block's `focus.point` | **median disparity** | 2 m ahead |
|
|
62
|
+
|
|
63
|
+
Each resolved value carries the step that produced it (`handle.rig.focusSource`,
|
|
64
|
+
`intrinsicsSource`, `typeSource`), because a number from a lower step is not a wrong number, it
|
|
65
|
+
is a wrong SOURCE, and that is invisible in the picture.
|
|
66
|
+
|
|
67
|
+
The block is now a **v2 superset**: `rig`, `focus` (one point that is the orbit centre, the pivot
|
|
68
|
+
plane AND the convergence) and `dxr` (the camera rig's absolute scalars) join it, `intrinsics`
|
|
69
|
+
becomes optional, and a v1 block still reads. `rig: "display"` beside a `rest` is meaningful —
|
|
70
|
+
*a display rig, opened at this viewpoint*.
|
|
71
|
+
|
|
72
|
+
**Estimating the lens** works because a capture's gaussians only exist where its camera could see
|
|
73
|
+
them: P1/P99 of `x/z` and `y/z` about the rest camera ARE the frustum that made it, principal
|
|
74
|
+
point included. Measured against a capture whose true half-tangents are ±0.857 and ±0.482:
|
|
75
|
+
0.8635 and 0.4827, +0.75 % and +0.12 %. The implied 35 mm-equivalent focal is gated to
|
|
76
|
+
[14, 85] mm, outside which the cloud is describing something that is not a camera. It matters
|
|
77
|
+
because a splat rendered through the wrong focal is drawn at the wrong SIZE and nothing else —
|
|
78
|
+
no artefact, just a picture that feels zoomed out.
|
|
79
|
+
|
|
80
|
+
**Estimating the focus** is the median of 1/z, inverted — not of z. On the reference capture that
|
|
81
|
+
is 2.17 m against the gallery's own 2.14 m; the centre of the measured bounds, which this
|
|
82
|
+
replaced, was 39.8 m, because an open scene's percentile bounds are 128 m wide.
|
|
83
|
+
|
|
84
|
+
On the camera path the subject is not reframed, the turntable is off, the mono camera is posed
|
|
85
|
+
and lensed as the capture, and the rig is **declared** with `cameraRigFromCamera` — the off-axis
|
|
86
|
+
projection stays in the runtime.
|
|
87
|
+
|
|
88
|
+
- **Pointing the window: double-click, Space and `handle.setFocus(point|null)`** (preview tier).
|
|
89
|
+
The focus is one point — the orbit centre, the pivot plane and the convergence — and it is now
|
|
90
|
+
something a viewer can move. Double-click focuses what was clicked (Spark's own
|
|
91
|
+
`SplatMesh.raycast`, ~57 ms over 1.18M gaussians, with a nearest-gaussian-to-the-ray fallback
|
|
92
|
+
documented as the approximation it is); Space returns to the resolved value; both ease at 0.18
|
|
93
|
+
per frame, and while the ease runs a camera rig re-declares its convergence every frame. What
|
|
94
|
+
moves depends on the rig and only on that: a camera rig moves the rotation centre and leaves the
|
|
95
|
+
capture where it was placed, a display rig brings the focused point to the middle of the tile.
|
|
96
|
+
`focusInput: false` turns the gestures off for a page that owns them itself, and
|
|
97
|
+
`handle.pick(x, y)` exposes the raycast.
|
|
98
|
+
|
|
99
|
+
New on the handle: **`handle.camera`** (the raw block), **`handle.rig`** (the resolved waterfall),
|
|
100
|
+
`handle.viewRig`, `handle.perf`, `handle.setFocus`, `handle.pick`. New exports from `./splat`:
|
|
101
|
+
`readSogCamera(bytes)`, `readSogMeta(bytes)`, `resolveRig`, `applySplatPerf`,
|
|
102
|
+
`SPLAT_PERF_PRESETS`. **`SceneViewer` gains `setFocus` / `getFocus` / `onFocusChange` / `onTick`**
|
|
103
|
+
(`./viewer`), and `fitTo` now goes through the focus, so a refit cannot leave the orbit turning
|
|
104
|
+
about somewhere the framing has moved away from.
|
|
105
|
+
|
|
106
|
+
## 1.6.1 — 2026-09-09
|
|
107
|
+
|
|
108
|
+
Touches the **core tier** (`.`) with a behaviour fix only — no API changes — and the **preview tier**
|
|
109
|
+
(`./viewer`, `./splat`, `./model`) with one additive option. A page that ignores everything below
|
|
110
|
+
renders identically while it is weaving; the changes only decide what a canvas shows once nothing
|
|
111
|
+
weaves it any more.
|
|
112
|
+
|
|
113
|
+
### Fixed
|
|
114
|
+
|
|
115
|
+
- **A canvas nothing is weaving can no longer be left holding a raw side-by-side pair**
|
|
116
|
+
([#28](https://github.com/DisplayXR/displayxr-web/issues/28), field report
|
|
117
|
+
[browser-pvt#99](https://github.com/DisplayXR/displayxr-browser-pvt/issues/99)). Every fallback in
|
|
118
|
+
this SDK was decided once at boot, so four paths ended with a tile showing a flat squeezed
|
|
119
|
+
left|right pair, permanently — the symptom users report as "3D element shows SBS", usually on a
|
|
120
|
+
slow connection:
|
|
121
|
+
- session `end` tore down without the mono repaint `_deactivate` does — every image and video
|
|
122
|
+
canvas kept its last SBS frame; teardown, deactivate and a failed activate now share one
|
|
123
|
+
`_paintMono`;
|
|
124
|
+
- an image whose download landed **after** teardown painted a fresh SBS pair into a canvas with no
|
|
125
|
+
layer — `_paint` now forces the mono branch (and a 1:1 buffer) whenever the manager is stopped or
|
|
126
|
+
the window has no live layer, whatever `win.sbs` says;
|
|
127
|
+
- a throwing `new XRDisplayLayer()` was swallowed silently and left the canvas as it was — it now
|
|
128
|
+
warns once per window (with the error) and repaints mono; still no retry;
|
|
129
|
+
- a buffering video (`readyState < 2`) skipped its paint entirely, so its canvas layer went idle and
|
|
130
|
+
could drop out of the browser's aggregated frame — it now re-commits its last decoded frame with
|
|
131
|
+
an identity blit.
|
|
132
|
+
- **One throwing scene no longer stops the windows after it from repainting.** `onFrame` is contained
|
|
133
|
+
per window and warned about once; an un-redrawn canvas is exactly what the browser's weave join
|
|
134
|
+
loses.
|
|
135
|
+
|
|
136
|
+
### Added
|
|
137
|
+
|
|
138
|
+
- **`addScene({ onLayerLost })`** (preview tier): called once when a scene window's layer goes away
|
|
139
|
+
for good (session end, or the layer could not be created) — not when a lazy tile scrolls off. The
|
|
140
|
+
SDK does not own a scene canvas's pixels, so this is how it tells the owner to go flat.
|
|
141
|
+
`SceneViewer.onLayerLost` is the ready-made handler (`startMono()`), and `./splat` / `./model` wire
|
|
142
|
+
it for you.
|
|
143
|
+
|
|
8
144
|
## 1.6.0 — 2026-09-08
|
|
9
145
|
|
|
10
146
|
Touches the **preview tier** (`./viewer`) and fixes a **documentation error in the core tier**.
|
package/README.md
CHANGED
|
@@ -113,6 +113,7 @@ js/
|
|
|
113
113
|
inline3d-viewer.js experimental: SceneViewer — framing, orbit, idle turntable, mono fallback,
|
|
114
114
|
and the placement readback (getSubjectBounds / getPose / depthOffset)
|
|
115
115
|
inline3d-splat.js experimental: addSplat() — a Gaussian splat window via Spark
|
|
116
|
+
(`perf` cuts overdraw; a `.sog`'s `camera` block picks the view rig)
|
|
116
117
|
inline3d-model.js experimental: addModel() — a glTF/GLB window; wires Draco / meshopt / KTX2
|
|
117
118
|
from what the asset declares (you serve the decoder files — see the guide)
|
|
118
119
|
docs/
|
package/index.d.ts
CHANGED
|
@@ -179,6 +179,15 @@ export interface SceneOptions extends TileOptions {
|
|
|
179
179
|
viewRig?: XRViewRigInit;
|
|
180
180
|
/** Element whose visibility drives the lazy create/close lifecycle (defaults to the canvas). */
|
|
181
181
|
observe?: Element;
|
|
182
|
+
/**
|
|
183
|
+
* Called once when this window's weave layer goes away for good — the session ended, or the
|
|
184
|
+
* layer could not be created. You own a scene canvas's pixels, so this is the SDK's only way
|
|
185
|
+
* to tell you that the side-by-side pair in it is no longer being woven and is now just
|
|
186
|
+
* squeezed 2D on the page: take the canvas flat here (`SceneViewer.startMono`, or your own
|
|
187
|
+
* mono path). NOT called when a lazy tile merely scrolls off screen — that layer is coming
|
|
188
|
+
* back. A throw is caught and warned about.
|
|
189
|
+
*/
|
|
190
|
+
onLayerLost?: () => void;
|
|
182
191
|
}
|
|
183
192
|
|
|
184
193
|
/** The per-frame render callback passed to {@link Inline3D.addScene}. */
|
package/js/inline3d-model.js
CHANGED
|
@@ -393,6 +393,9 @@ export function addModel(wall, canvas, src, opts = {}) {
|
|
|
393
393
|
if (wall && wall.supported) {
|
|
394
394
|
handle = wall.addScene(canvas, viewer.onFrame, {
|
|
395
395
|
virtualDisplayHeight,
|
|
396
|
+
// The layer can go away for good (the session ends, the constructor refuses): take the
|
|
397
|
+
// canvas flat rather than leave its last side-by-side frame on the page (web#28).
|
|
398
|
+
onLayerLost: viewer.onLayerLost,
|
|
396
399
|
...(observe ? { observe } : {}),
|
|
397
400
|
});
|
|
398
401
|
} else {
|
|
@@ -0,0 +1,295 @@
|
|
|
1
|
+
// inline3d-sog.js — read the `meta.json` out of a `.sog`, and the optional `camera` block in it.
|
|
2
|
+
//
|
|
3
|
+
// EXPERIMENTAL. Internal to `./splat`, which re-exports `readSogCamera`. Not covered by the SDK's
|
|
4
|
+
// 1.x semver promise — see docs/sdk-stability.md.
|
|
5
|
+
//
|
|
6
|
+
// WHY A ZIP READER IS IN THIS SDK AT ALL. A `.sog` from `splat-transform` is a PKZip of webp
|
|
7
|
+
// planes plus a `meta.json`, and Spark reads exactly the fields it needs to build splats out of
|
|
8
|
+
// it — it neither surfaces the rest of the file nor hands back the parsed metadata. But whether a
|
|
9
|
+
// splat is an OBJECT (a product hero, a scan, a turntable subject) or a PHOTOGRAPH LIFTED INTO 3D
|
|
10
|
+
// is not a rendering detail: it decides which VIEW RIG the window should be on, and getting that
|
|
11
|
+
// wrong is the difference between a picture you can lean into and an arbitrary cloud framed by a
|
|
12
|
+
// bounding box. The `camera` block records the recording camera so the viewer can conserve it.
|
|
13
|
+
//
|
|
14
|
+
// So the choice is between asking every page to parse its own assets and reading ~40 bytes of
|
|
15
|
+
// central directory here. The reader below does the second: it is deliberately the smallest thing
|
|
16
|
+
// that can find ONE named entry in a zip, and it never touches the webp planes (which are
|
|
17
|
+
// megabytes, and Spark's business).
|
|
18
|
+
//
|
|
19
|
+
// It is BYTES-ONLY on purpose. The gallery hands the SDK bytes rather than a URL (Spark infers a
|
|
20
|
+
// splat's format from the URL path, so an extension-less `blob:` URL fails inside a worker), and
|
|
21
|
+
// those same bytes are the ones this reads — one download, no second fetch, no range request, and
|
|
22
|
+
// no chance of reading metadata from a different build of the asset than the one on screen.
|
|
23
|
+
|
|
24
|
+
/** `PK\x05\x06` — end of central directory. */
|
|
25
|
+
const EOCD_SIG = 0x06054b50;
|
|
26
|
+
/** `PK\x01\x02` — a central-directory file header. */
|
|
27
|
+
const CEN_SIG = 0x02014b50;
|
|
28
|
+
/** `PK\x03\x04` — a local file header. */
|
|
29
|
+
const LOC_SIG = 0x04034b50;
|
|
30
|
+
|
|
31
|
+
/** EOCD is 22 bytes plus a comment of at most 64 KiB. */
|
|
32
|
+
const EOCD_MAX_BACK = 22 + 0xffff;
|
|
33
|
+
|
|
34
|
+
/**
|
|
35
|
+
* A `meta.json` this large is not a `meta.json`. The guard is against a malformed/hostile
|
|
36
|
+
* central directory, not against real assets: the largest one seen is ~4 KB.
|
|
37
|
+
*/
|
|
38
|
+
const META_MAX_BYTES = 4 << 20;
|
|
39
|
+
|
|
40
|
+
/** Locate the end-of-central-directory record, scanning backwards. */
|
|
41
|
+
function findEocd(dv) {
|
|
42
|
+
const len = dv.byteLength;
|
|
43
|
+
if (len < 22) return -1;
|
|
44
|
+
const stop = Math.max(0, len - EOCD_MAX_BACK);
|
|
45
|
+
for (let i = len - 22; i >= stop; i--) {
|
|
46
|
+
if (dv.getUint32(i, true) === EOCD_SIG) return i;
|
|
47
|
+
}
|
|
48
|
+
return -1;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
/** Inflate a raw deflate stream. Returns null where the platform has no DecompressionStream. */
|
|
52
|
+
async function inflateRaw(slice) {
|
|
53
|
+
if (typeof DecompressionStream !== 'function') return null;
|
|
54
|
+
const ds = new DecompressionStream('deflate-raw');
|
|
55
|
+
const stream = new Blob([slice]).stream().pipeThrough(ds);
|
|
56
|
+
return new Uint8Array(await new Response(stream).arrayBuffer());
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
/**
|
|
60
|
+
* Read one named entry out of a PKZip, as bytes.
|
|
61
|
+
*
|
|
62
|
+
* Handles the two compression methods a `.sog` actually uses — 0 (stored) and 8 (deflate) — and
|
|
63
|
+
* returns null for anything else rather than guessing. Zip64 is refused the same way: a `.sog`
|
|
64
|
+
* big enough to need it would have to be over 4 GB.
|
|
65
|
+
*
|
|
66
|
+
* @param {Uint8Array} bytes the whole archive.
|
|
67
|
+
* @param {string} name exact entry name, e.g. `meta.json`.
|
|
68
|
+
* @returns {Promise<Uint8Array|null>}
|
|
69
|
+
*/
|
|
70
|
+
export async function readZipEntry(bytes, name) {
|
|
71
|
+
if (!bytes || bytes.byteLength < 22) return null;
|
|
72
|
+
const dv = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
|
73
|
+
const eocd = findEocd(dv);
|
|
74
|
+
if (eocd < 0) return null;
|
|
75
|
+
|
|
76
|
+
const count = dv.getUint16(eocd + 10, true);
|
|
77
|
+
let p = dv.getUint32(eocd + 16, true);
|
|
78
|
+
// 0xffffffff in either field is the zip64 escape; we do not follow it (see above).
|
|
79
|
+
if (p === 0xffffffff || p >= dv.byteLength) return null;
|
|
80
|
+
|
|
81
|
+
const dec = new TextDecoder();
|
|
82
|
+
for (let i = 0; i < count; i++) {
|
|
83
|
+
if (p + 46 > dv.byteLength || dv.getUint32(p, true) !== CEN_SIG) return null;
|
|
84
|
+
const method = dv.getUint16(p + 10, true);
|
|
85
|
+
const compSize = dv.getUint32(p + 20, true);
|
|
86
|
+
const rawSize = dv.getUint32(p + 24, true);
|
|
87
|
+
const nameLen = dv.getUint16(p + 28, true);
|
|
88
|
+
const extraLen = dv.getUint16(p + 30, true);
|
|
89
|
+
const commentLen = dv.getUint16(p + 32, true);
|
|
90
|
+
const localOff = dv.getUint32(p + 42, true);
|
|
91
|
+
const entry = dec.decode(bytes.subarray(p + 46, p + 46 + nameLen));
|
|
92
|
+
p += 46 + nameLen + extraLen + commentLen;
|
|
93
|
+
if (entry !== name) continue;
|
|
94
|
+
|
|
95
|
+
if (compSize === 0xffffffff || rawSize === 0xffffffff || localOff === 0xffffffff) return null;
|
|
96
|
+
if (rawSize > META_MAX_BYTES) return null;
|
|
97
|
+
// The central directory's sizes are authoritative; the LOCAL header's may be zeroed (a
|
|
98
|
+
// streaming writer defers them to a data descriptor). Only its two length fields are read.
|
|
99
|
+
if (localOff + 30 > dv.byteLength || dv.getUint32(localOff, true) !== LOC_SIG) return null;
|
|
100
|
+
const lNameLen = dv.getUint16(localOff + 26, true);
|
|
101
|
+
const lExtraLen = dv.getUint16(localOff + 28, true);
|
|
102
|
+
const start = localOff + 30 + lNameLen + lExtraLen;
|
|
103
|
+
if (start + compSize > dv.byteLength) return null;
|
|
104
|
+
const slice = bytes.subarray(start, start + compSize);
|
|
105
|
+
if (method === 0) return slice;
|
|
106
|
+
if (method === 8) return inflateRaw(slice);
|
|
107
|
+
return null;
|
|
108
|
+
}
|
|
109
|
+
return null;
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
/**
|
|
113
|
+
* Parse a `.sog`'s `meta.json`.
|
|
114
|
+
*
|
|
115
|
+
* @param {Uint8Array|ArrayBuffer} bytes
|
|
116
|
+
* @returns {Promise<object|null>} the parsed object, or null if this is not a `.sog`, has no
|
|
117
|
+
* `meta.json`, or the entry cannot be read on this platform.
|
|
118
|
+
*/
|
|
119
|
+
export async function readSogMeta(bytes) {
|
|
120
|
+
const u8 = bytes instanceof Uint8Array ? bytes : bytes ? new Uint8Array(bytes) : null;
|
|
121
|
+
if (!u8 || u8.length < 4) return null;
|
|
122
|
+
// PK\x03\x04 — cheap reject before the backwards scan, so a `.ply`/`.spz`/`.splat` costs
|
|
123
|
+
// four byte comparisons.
|
|
124
|
+
if (!(u8[0] === 0x50 && u8[1] === 0x4b && u8[2] === 0x03 && u8[3] === 0x04)) return null;
|
|
125
|
+
let raw;
|
|
126
|
+
try {
|
|
127
|
+
raw = await readZipEntry(u8, 'meta.json');
|
|
128
|
+
} catch {
|
|
129
|
+
return null;
|
|
130
|
+
}
|
|
131
|
+
if (!raw) return null;
|
|
132
|
+
try {
|
|
133
|
+
return JSON.parse(new TextDecoder().decode(raw));
|
|
134
|
+
} catch {
|
|
135
|
+
return null;
|
|
136
|
+
}
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
const num = (v) => (typeof v === 'number' && Number.isFinite(v) ? v : null);
|
|
140
|
+
|
|
141
|
+
/**
|
|
142
|
+
* Validate the `camera` block of a `.sog` `meta.json`.
|
|
143
|
+
*
|
|
144
|
+
* THE BLOCK IS THE RIG SWITCH — present means "this splat was lifted from a photograph, and here
|
|
145
|
+
* is the camera that took it"; absent means "this is an object", which is the display rig and the
|
|
146
|
+
* behaviour every existing page already has. That is the whole contract, and it is why this is
|
|
147
|
+
* strict: a half-parsed block that silently keeps some defaults would put a photo scene on a
|
|
148
|
+
* plausible-looking rig that is not the capture, which is indistinguishable from a framing bug.
|
|
149
|
+
*
|
|
150
|
+
* Shape (`meta.json`, top level, right after `count`; `version` stays 2). v2 is a SUPERSET of
|
|
151
|
+
* v1 — every key below except `convention` is optional, and a v1 block still reads:
|
|
152
|
+
*
|
|
153
|
+
* "camera": {
|
|
154
|
+
* "convention": "opencv",
|
|
155
|
+
* "rig": "camera", // v2: which rig this asset wants
|
|
156
|
+
* "rest": { "position": [0,0,0], "rotation": [0,0,0,1] },
|
|
157
|
+
* "intrinsics": { "fx":…, "fy":…, "cx":…, "cy":…, "width":…, "height":… },
|
|
158
|
+
* "stereo": { "baseline_m": 0.063 },
|
|
159
|
+
* "focus": { "point": [0,0,1.68], "subject_m":…, "near_m":…, "far_m":…,
|
|
160
|
+
* "source": "convergence|manual|auto" }, // v2
|
|
161
|
+
* "dxr": { "ipd_factor": 1.0, "parallax_factor": 1.0 } // v2
|
|
162
|
+
* }
|
|
163
|
+
*
|
|
164
|
+
* `intrinsics` BECAME OPTIONAL IN v2, which is the change with teeth: a block can now say "this
|
|
165
|
+
* is a camera rig, open it at this viewpoint" without claiming a lens, and the consumer is
|
|
166
|
+
* expected to estimate one. So this returns a descriptor with null intrinsics rather than
|
|
167
|
+
* refusing the block — refusing it would silently demote a camera-rig asset to the display rig,
|
|
168
|
+
* which is the failure this whole mechanism exists to prevent.
|
|
169
|
+
*
|
|
170
|
+
* `focus.point` is THE point: the orbit centre, the pivot plane and the convergence distance,
|
|
171
|
+
* which are one thing and are stored once.
|
|
172
|
+
*
|
|
173
|
+
* `convention` is REQUIRED to be `opencv` (+x right, +y DOWN, +z forward, pixel (0,0) at the top
|
|
174
|
+
* left) rather than defaulted: it is the frame the intrinsics are expressed in, and a reader that
|
|
175
|
+
* assumed it would mis-sign the principal-point offset on any other convention — a wrong answer
|
|
176
|
+
* with no error, which is the one failure mode a metadata block must not have.
|
|
177
|
+
*
|
|
178
|
+
* @param {object|null} meta a parsed `meta.json`.
|
|
179
|
+
* @returns {object|null} a normalised camera descriptor, or null.
|
|
180
|
+
*/
|
|
181
|
+
export function sogCameraFromMeta(meta) {
|
|
182
|
+
const c = meta && typeof meta === 'object' ? meta.camera : null;
|
|
183
|
+
if (!c || typeof c !== 'object') return null;
|
|
184
|
+
if (c.convention !== 'opencv') {
|
|
185
|
+
console.warn(
|
|
186
|
+
`[inline3d/splat] .sog camera block has convention "${c.convention}" — only "opencv" is ` +
|
|
187
|
+
'understood, so the block is IGNORED and this asset stays on the display rig.',
|
|
188
|
+
);
|
|
189
|
+
return null;
|
|
190
|
+
}
|
|
191
|
+
const i = c.intrinsics;
|
|
192
|
+
let intrinsics = null;
|
|
193
|
+
if (i && typeof i === 'object') {
|
|
194
|
+
const fx = num(i.fx);
|
|
195
|
+
const fy = num(i.fy);
|
|
196
|
+
const cx = num(i.cx);
|
|
197
|
+
const cy = num(i.cy);
|
|
198
|
+
const width = num(i.width);
|
|
199
|
+
const height = num(i.height);
|
|
200
|
+
if (!(fx > 0) || !(fy > 0) || !(width > 0) || !(height > 0) || cx === null || cy === null) {
|
|
201
|
+
// Half-believing a lens is worse than having none: with intrinsics optional in v2 there is
|
|
202
|
+
// a well-defined thing to do instead, which is estimate one from the cloud.
|
|
203
|
+
console.warn(
|
|
204
|
+
'[inline3d/splat] .sog camera block has unusable intrinsics — they are DROPPED and the ' +
|
|
205
|
+
'lens is estimated from the cloud instead; the rest of the block still applies.',
|
|
206
|
+
i,
|
|
207
|
+
);
|
|
208
|
+
} else {
|
|
209
|
+
intrinsics = { fx, fy, cx, cy, width, height };
|
|
210
|
+
}
|
|
211
|
+
}
|
|
212
|
+
const pos = Array.isArray(c.rest?.position) ? c.rest.position.map((v) => num(v) ?? 0) : [0, 0, 0];
|
|
213
|
+
const rot = Array.isArray(c.rest?.rotation) ? c.rest.rotation.map((v) => num(v) ?? 0) : [0, 0, 0, 1];
|
|
214
|
+
const baseline = num(c.stereo?.baseline_m);
|
|
215
|
+
|
|
216
|
+
// v2 `rig`. Anything unrecognised is dropped rather than guessed at — the waterfall's next
|
|
217
|
+
// step (a block means a camera) is a better answer than a typo taken literally.
|
|
218
|
+
let rig = null;
|
|
219
|
+
if (c.rig === 'camera' || c.rig === 'display') rig = c.rig;
|
|
220
|
+
else if (c.rig !== undefined) {
|
|
221
|
+
console.warn(`[inline3d/splat] .sog camera block has rig "${c.rig}" — ignored`, c.rig);
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
// v2 `focus`. The point is the only required part; the three distances are advisory and are
|
|
225
|
+
// carried through untouched for a host page that wants them (a depth budget, a HUD).
|
|
226
|
+
let focus = null;
|
|
227
|
+
const fp = c.focus?.point;
|
|
228
|
+
if (Array.isArray(fp) && fp.length >= 3 && fp.every((v) => num(v) !== null)) {
|
|
229
|
+
focus = {
|
|
230
|
+
point: [fp[0], fp[1], fp[2]],
|
|
231
|
+
subject_m: num(c.focus.subject_m),
|
|
232
|
+
near_m: num(c.focus.near_m),
|
|
233
|
+
far_m: num(c.focus.far_m),
|
|
234
|
+
source: typeof c.focus.source === 'string' ? c.focus.source : null,
|
|
235
|
+
};
|
|
236
|
+
} else if (c.focus !== undefined) {
|
|
237
|
+
console.warn('[inline3d/splat] .sog camera block has an unusable focus — ignored', c.focus);
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
// v2 `dxr`. These are the camera rig's ABSOLUTE scalars, and they stay absolute: normalising
|
|
241
|
+
// them against the convergence distance would make the scene's depth breathe every time the
|
|
242
|
+
// viewer re-focused.
|
|
243
|
+
const ipdFactor = num(c.dxr?.ipd_factor);
|
|
244
|
+
const parallaxFactor = num(c.dxr?.parallax_factor);
|
|
245
|
+
|
|
246
|
+
return {
|
|
247
|
+
convention: 'opencv',
|
|
248
|
+
rig,
|
|
249
|
+
focus,
|
|
250
|
+
dxr: {
|
|
251
|
+
ipdFactor: ipdFactor !== null && ipdFactor >= 0 ? ipdFactor : null,
|
|
252
|
+
parallaxFactor: parallaxFactor !== null && parallaxFactor >= 0 ? parallaxFactor : null,
|
|
253
|
+
},
|
|
254
|
+
rest: {
|
|
255
|
+
position: [pos[0] ?? 0, pos[1] ?? 0, pos[2] ?? 0],
|
|
256
|
+
rotation: [rot[0] ?? 0, rot[1] ?? 0, rot[2] ?? 0, rot[3] ?? 1],
|
|
257
|
+
},
|
|
258
|
+
intrinsics,
|
|
259
|
+
stereo: baseline > 0 ? { baseline_m: baseline } : null,
|
|
260
|
+
/**
|
|
261
|
+
* Derived, because every consumer needs them and each is one line to get subtly wrong.
|
|
262
|
+
*
|
|
263
|
+
* Null when the block carried no usable intrinsics (legal in v2) — the caller estimates a
|
|
264
|
+
* lens from the cloud instead.
|
|
265
|
+
*
|
|
266
|
+
* `verticalFov` is the FULL vertical angle the capture subtends, in RADIANS — the unit an
|
|
267
|
+
* XRViewRigInit wants (three's `camera.fov` is the same angle in degrees).
|
|
268
|
+
*
|
|
269
|
+
* `principalOffset` is the principal point's offset from the frame centre as a fraction of
|
|
270
|
+
* the frame, x rightwards and **y upwards** — i.e. already out of OpenCV's y-down frame and
|
|
271
|
+
* into the GL/three one, so a consumer never has to remember which way `cy` grows. A
|
|
272
|
+
* rectified stereo pair carries its deconvergence here: shifting the principal point is what
|
|
273
|
+
* "deconverging" DOES to a pair, so a non-zero x is the capture's zero-disparity plane
|
|
274
|
+
* expressed as a lens shift rather than as a distance.
|
|
275
|
+
*/
|
|
276
|
+
verticalFov: intrinsics ? 2 * Math.atan(intrinsics.height / (2 * intrinsics.fy)) : null,
|
|
277
|
+
principalOffset: intrinsics
|
|
278
|
+
? {
|
|
279
|
+
x: (intrinsics.cx - intrinsics.width / 2) / intrinsics.width,
|
|
280
|
+
y: -(intrinsics.cy - intrinsics.height / 2) / intrinsics.height,
|
|
281
|
+
}
|
|
282
|
+
: null,
|
|
283
|
+
};
|
|
284
|
+
}
|
|
285
|
+
|
|
286
|
+
/**
|
|
287
|
+
* Read the `camera` block straight out of `.sog` bytes. Convenience over
|
|
288
|
+
* {@link readSogMeta} + {@link sogCameraFromMeta}.
|
|
289
|
+
*
|
|
290
|
+
* @param {Uint8Array|ArrayBuffer} bytes
|
|
291
|
+
* @returns {Promise<object|null>}
|
|
292
|
+
*/
|
|
293
|
+
export async function readSogCamera(bytes) {
|
|
294
|
+
return sogCameraFromMeta(await readSogMeta(bytes));
|
|
295
|
+
}
|