@displayxr/inline3d 1.17.0 → 1.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,55 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
5
5
  `./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
6
6
  which tier they touch, because that is what tells you whether an upgrade can move your pixels.
7
7
 
8
+ ## 1.19.0 — 2026-09-24
9
+
10
+ Touches the **preview tier** (`./splat` and `./model`, `engine: 'playcanvas'` only). The defaults
11
+ keep the wheel's pixels. One default does change: a two-finger touch now pinch-zooms (within the
12
+ old 0.2–6 range, with no relax) where it used to act as an erratic one-finger drag.
13
+
14
+ ### Added
15
+
16
+ - **`zoom: { min, max, relax, ease }`** (#36), bounded zoom that relaxes back to rest.
17
+ - Wheel, pinch and `setPose` clamp to `[min, max]`. Defaults are 0.2 and 6, as before.
18
+ - `relax: true` eases back to rest (1×, or the last `setPose` zoom) once the wheel has been idle
19
+ 150 ms or the pinch ends. It uses the orbit's τ 0.6 s, with a landing floor so 2× is exactly
20
+ home in about 2.9 s, and it never fights a live gesture.
21
+ - The zoom scales about the focus, so the focus keeps its place and its disparity in both eyes.
22
+ - The gallery's Spatial View passes `{ min: 1, max: 2, relax: true }`: you can't zoom out past
23
+ the frame's edges, and a peek comes back on its own.
24
+ - **Two-finger pinch zoom** on the PlayCanvas viewer, with Pointer Events and the same bounds.
25
+ - Gates, on a real GPU (1280×720):
26
+ - Zooming out is refused: MAE 0 against the rest render.
27
+ - Wheel and pinch reach the 2× cap and return to **MAE 0** against rest 3 s after release.
28
+ - In fake stereo, the focus's disparity is unchanged at 1×, 1.5× and 2×.
29
+
30
+ ## 1.18.0 — 2026-09-24
31
+
32
+ Touches the **preview tier** (`./splat`, `engine: 'playcanvas'` only). Additive: a page that never
33
+ calls `setVideo` renders the same pixels.
34
+
35
+ ### Added
36
+
37
+ - **`handle.setVideo(src, options?)`** (#36): a stereo video on the persistent PlayCanvas splat
38
+ handle, with no second canvas, layer or session. This is for pages that keep one woven canvas for
39
+ the whole app, where a fresh `addVideo` canvas after a navigation is the side-by-side flash.
40
+ - `src` is a URL (an SDK-owned `<video>`, autoplaying) or your `HTMLVideoElement`. Options:
41
+ `format: 'sbs' | 'tb' | 'mono'`, `fit: 'contain' | 'cover'`, `rig: 'display'`,
42
+ `virtualDisplayHeight`, `loop`, `muted`, `autoplay`.
43
+ - It resolves on the video's first frame to `{ video, format, fit, remove(), stats() }`. The page
44
+ drives transport through `video`.
45
+ - The splat is hidden and a screen-locked plane on the display rig shows the video. Each eye
46
+ samples only its own half. Flat (mono), it shows the left half at full resolution.
47
+ - Upload is gated on `requestVideoFrameCallback`. The colour is `addVideo`'s own.
48
+ - `setVideo(null)` restores the splat, the pose, the lens and the declared view rig exactly.
49
+ - It throws during an in-flight `setSource`. While a video is on, `setSource` rejects and
50
+ `setRig` throws. `controls:'page'` throws.
51
+ - Gates, on a real GPU: each eye shows 0 px of the other half, exit renders **MAE 0.0000**
52
+ against the pre-video frame with the same declared rig, and colour matches `addVideo`'s paint
53
+ at MAE 0.003. At 3840×1080 the upload runs 29.9/s with `texImage2D` at p50 0.1 ms, and 0 video
54
+ frames drop on a 120 Hz panel.
55
+ - See [`docs/playcanvas-adapter.md` § setVideo](docs/playcanvas-adapter.md#setvideo--a-stereo-video-on-the-persistent-handle-36).
56
+
8
57
  ## 1.17.0 — 2026-09-24
9
58
 
10
59
  Touches the **preview tier** (`./model` and `./splat`, `engine: 'playcanvas'` only). The default look
package/README.md CHANGED
@@ -45,10 +45,13 @@ API is not yet covered by the semver promise below.
45
45
  renders the same splat window with the [PlayCanvas](https://playcanvas.com/) engine instead of
46
46
  Spark — same handle, same rig/focus/camera-block behaviour, same `perf` presets (mapped onto the
47
47
  engine's knobs) — and adds `setSource(src, { fadeMs })` (crossfading asset swaps), `feather`, a
48
- tilt-and-relax orbit, `handle.engine` (the engine objects, for advanced pages) and
48
+ tilt-and-relax orbit, two-finger pinch plus bounded zoom that relaxes back to rest
49
+ (`zoom: { min, max, relax }`, [§Zoom bounds and relax](docs/playcanvas-adapter.md#zoom-bounds-and-relax)), `handle.engine` (the engine objects, for advanced pages) and
49
50
  `handle.setRig('display' | 'camera' | 'auto')`: a live, reversible rig switch, so one persistent tile
50
51
  can show a glTF under `handle.engine.root` exactly as `addModel` would and then go back to the photo's
51
- capture rig ([§setRig](docs/playcanvas-adapter.md#setrig--switching-between-the-display-rig-and-the-camera-rig-36)). It needs the
52
+ capture rig ([§setRig](docs/playcanvas-adapter.md#setrig--switching-between-the-display-rig-and-the-camera-rig-36)), and
53
+ `handle.setVideo(src, { format: 'sbs' | 'tb' | 'mono' })`: a stereo video on that same tile, each eye
54
+ its own half, with no second canvas ([§setVideo](docs/playcanvas-adapter.md#setvideo--a-stereo-video-on-the-persistent-handle-36)). It needs the
52
55
  optional peer `playcanvas` (`>=2.22.3 <3`) and reads `.sog`, `.ply` and a Streamed-SOG
53
56
  `lod-meta.json`. **Spark stays the default**; a page that never passes `engine` never loads
54
57
  `playcanvas`. What differs, and why: [`docs/playcanvas-adapter.md`](docs/playcanvas-adapter.md).
@@ -54,7 +54,7 @@ export function resolveModelEngine(opts = {}) {
54
54
  }
55
55
 
56
56
  /** Options that exist only on the PlayCanvas backend: an error on an explicit engine:'three'. */
57
- const PLAYCANVAS_ONLY = ['playcanvas', 'environmentRotation', 'controls', 'comfortDepth', 'onBeforeFrame', 'antialias', 'preserveDrawingBuffer', 'nearClip', 'farClip', 'orbitMaxDeg', 'orbitEase'];
57
+ const PLAYCANVAS_ONLY = ['playcanvas', 'environmentRotation', 'controls', 'comfortDepth', 'onBeforeFrame', 'antialias', 'preserveDrawingBuffer', 'nearClip', 'farClip', 'orbitMaxDeg', 'orbitEase', 'zoom'];
58
58
 
59
59
  /**
60
60
  * Everything that can be checked before a module loads. Throws at CALL time — a page bug is true
@@ -903,6 +903,7 @@ export function attachPlayCanvasModel(out, wall, canvas, src, opts, pending = []
903
903
  flipY: false, // glTF is already Y-up
904
904
  orbitMaxDeg: opts.orbitMaxDeg,
905
905
  orbitEase: opts.orbitEase,
906
+ zoom: opts.zoom,
906
907
  feather,
907
908
  nearClip: opts.nearClip,
908
909
  farClip: opts.farClip,
@@ -63,6 +63,7 @@ import {
63
63
  particleSpan,
64
64
  } from './inline3d-splat-effects.js';
65
65
  import { LiveOutgoing, resolveOutgoingOption, defaultOutgoing, yieldIdle } from './inline3d-splat-live.js';
66
+ import { VideoPlane, validateSetVideo, PAGE_VIDEO_ERROR } from './inline3d-splat-video.js';
66
67
  import {
67
68
  clamp,
68
69
  finite,
@@ -83,8 +84,9 @@ import {
83
84
  WHEEL_PAGE_PX,
84
85
  WHEEL_MAX_PX,
85
86
  ZOOM_PER_PX,
86
- ZOOM_MIN,
87
- ZOOM_MAX,
87
+ ZOOM_WHEEL_IDLE_MS,
88
+ ZOOM_RELAX_MIN_RATE,
89
+ resolveZoomOption,
88
90
  MONO_FOV,
89
91
  MONO_NEAR,
90
92
  MONO_FAR,
@@ -627,6 +629,7 @@ export class PlayCanvasSplatViewer {
627
629
  pitchLimit = PITCH_LIMIT,
628
630
  orbitMaxDeg = ORBIT_MAX_DEG,
629
631
  orbitEase = {},
632
+ zoom,
630
633
  feather = 0,
631
634
  captureFit = 'height',
632
635
  nearClip,
@@ -664,6 +667,20 @@ export class PlayCanvasSplatViewer {
664
667
  this.orbitMaxDeg = orbitMaxDeg;
665
668
  this.orbitEase = { drag: orbitEase.drag ?? ORBIT_TAU_DRAG_S, rest: orbitEase.rest ?? ORBIT_TAU_REST_S };
666
669
  this._orbitMode = null;
670
+ // Zoom bounds + relax (./inline3d-splat-shared.js §ZOOM). Wheel and pinch clamp to
671
+ // [zoomOpts.min, zoomOpts.max]; with relax, the zoom eases back to `_restZoom` (1×, or the
672
+ // last setPose zoom) once the wheel is idle or the pinch ends. `_zoomMode` is 'rest' while
673
+ // relaxing, else null (the ordinary damping). `_pinching` holds the relax off.
674
+ this.zoomOpts = resolveZoomOption(zoom);
675
+ this._zoomMode = null;
676
+ this._restZoom = clamp(1, this.zoomOpts.min, this.zoomOpts.max);
677
+ this._lastWheel = 0;
678
+ this._pinching = false;
679
+ /** handle.setVideo: orbit/zoom input ignored while a video plane is on. */
680
+ this.inputLocked = false;
681
+ /** handle.setVideo's plane (./inline3d-splat-video.js) while a video is on, else null. */
682
+ this._videoPlane = null;
683
+ this.boxAspect = 1;
667
684
  this.featherPx = feather > 0 ? feather : 0;
668
685
  this.captureFit = captureFit;
669
686
  /** Called when the capture camera's vertical FOV changes (captureFit 'cover' on a resize). */
@@ -687,8 +704,8 @@ export class PlayCanvasSplatViewer {
687
704
  this.flipY = opts.flipY !== false;
688
705
 
689
706
  this._fitScale = 1;
690
- this._zoom = 1;
691
- this._targetZoom = 1;
707
+ this._zoom = this._restZoom;
708
+ this._targetZoom = this._restZoom;
692
709
  this._depthOffset = 0;
693
710
  this._subjectHalf = [0, 0, 0];
694
711
  this._yaw = 0;
@@ -809,7 +826,11 @@ export class PlayCanvasSplatViewer {
809
826
  if (pitch !== undefined) {
810
827
  this._targetPitch = this._pitch = clamp(pitch, this.pitchLimit[0], this.pitchLimit[1]);
811
828
  }
812
- if (zoom !== undefined) this._targetZoom = this._zoom = clamp(zoom, ZOOM_MIN, ZOOM_MAX);
829
+ if (zoom !== undefined) {
830
+ // A snap, and the new rest: a relax comes home HERE, not to 1×.
831
+ this._targetZoom = this._zoom = this._restZoom = clamp(zoom, this.zoomOpts.min, this.zoomOpts.max);
832
+ this._zoomMode = null;
833
+ }
813
834
  if (depthOffset !== undefined) this._depthOffset = finite(depthOffset, this._depthOffset);
814
835
  this._applyTransform();
815
836
  }
@@ -1024,6 +1045,8 @@ export class PlayCanvasSplatViewer {
1024
1045
  }
1025
1046
  this._live?.destroy();
1026
1047
  this._live = null;
1048
+ this._videoPlane?.destroy();
1049
+ this._videoPlane = null;
1027
1050
  try {
1028
1051
  this.app?.destroy();
1029
1052
  } catch (err) {
@@ -1544,6 +1567,7 @@ export class PlayCanvasSplatViewer {
1544
1567
  _updateMonoProjection() {
1545
1568
  const box = this.canvas.getBoundingClientRect();
1546
1569
  const aspect = box.height > 0 ? box.width / box.height : 1;
1570
+ this.boxAspect = aspect; // setVideo's plane reads it per frame (no layout read in the draw)
1547
1571
  if (this.mono.capture) {
1548
1572
  captureProjection(this.mono.capture, aspect, this.mono.near, this.mono.far, this.mono.proj, this.captureFit);
1549
1573
  if (this.captureFit !== 'height') {
@@ -1652,6 +1676,8 @@ export class PlayCanvasSplatViewer {
1652
1676
  }
1653
1677
  }
1654
1678
  this._updateFeather(entries[0].width * sx, entries[0].height * sy);
1679
+ // handle.setVideo's plane: size, eye split, and a new frame's upload (./inline3d-splat-video.js).
1680
+ this._videoPlane?.beforeDraw(entries, rect);
1655
1681
  app.tick(now());
1656
1682
  this._afterTick();
1657
1683
  return true;
@@ -1737,7 +1763,33 @@ export class PlayCanvasSplatViewer {
1737
1763
  ) {
1738
1764
  this._orbitMode = null; // at rest: the ordinary damping (and the idle turntable) take over
1739
1765
  }
1740
- if (Math.abs(this._targetZoom - this._zoom) > 1e-4) {
1766
+ // Zoom relax: the wheel has gone idle (or the pinch ended) away from rest — head home.
1767
+ if (
1768
+ this.zoomOpts.relax &&
1769
+ !this._zoomMode &&
1770
+ !this._pinching &&
1771
+ t - this._lastWheel > ZOOM_WHEEL_IDLE_MS &&
1772
+ Math.abs(this._targetZoom - this._restZoom) > 1e-6
1773
+ ) {
1774
+ this._targetZoom = this._restZoom;
1775
+ this._zoomMode = 'rest';
1776
+ }
1777
+ // Relaxing eases with τ = zoomOpts.ease (the orbit's 0.6 s by default); the wheel's own
1778
+ // damping otherwise. Both in log space, so 2× → 1× reads like 1× → ½×.
1779
+ if (this._zoomMode === 'rest') {
1780
+ // The exponential (τ = zoomOpts.ease) with a landing floor: it never moves slower than
1781
+ // ZOOM_RELAX_MIN_RATE (log-zoom per second), so it ARRIVES — 2× is home in ≈3 s — instead of
1782
+ // creeping in forever or popping at a snap. Above ~1 % from home the floor never binds.
1783
+ const gap = Math.log(this._targetZoom / this._zoom);
1784
+ const ko = dt > 0 ? 1 - Math.exp(-dt / this.zoomOpts.ease) : 0;
1785
+ const step = Math.min(Math.abs(gap), Math.max(Math.abs(gap) * ko, ZOOM_RELAX_MIN_RATE * dt));
1786
+ if (Math.abs(gap) - step < 1e-6) {
1787
+ this._zoom = this._targetZoom;
1788
+ this._zoomMode = null;
1789
+ } else {
1790
+ this._zoom *= Math.exp(Math.sign(gap) * step);
1791
+ }
1792
+ } else if (Math.abs(this._targetZoom - this._zoom) > 1e-4) {
1741
1793
  this._zoom *= Math.pow(this._targetZoom / this._zoom, k);
1742
1794
  } else {
1743
1795
  this._zoom = this._targetZoom;
@@ -1858,7 +1910,41 @@ export class PlayCanvasSplatViewer {
1858
1910
  let startY = 0;
1859
1911
  let restYaw = 0;
1860
1912
  let restPitch = 0;
1913
+ // Pinch: two pointers down zoom by the ratio of their spread, about the focus (the pivot's
1914
+ // scale is S(fit × zoom) about it), within zoomOpts' bounds. The drag ends when the second
1915
+ // finger lands (its tilt relaxes); the finger left after a pinch does not orbit.
1916
+ const pointers = new Map(); // pointerId → [clientX, clientY]
1917
+ let pinchD0 = 0;
1918
+ let pinchZ0 = 1;
1919
+ const spread = () => {
1920
+ const [a, b] = [...pointers.values()];
1921
+ return Math.hypot(a[0] - b[0], a[1] - b[1]);
1922
+ };
1923
+ const endDrag = () => {
1924
+ dragging = false;
1925
+ this._targetYaw = restYaw;
1926
+ this._targetPitch = clamp(restPitch, this.pitchLimit[0], this.pitchLimit[1]);
1927
+ this._orbitMode = 'rest';
1928
+ };
1861
1929
  this._onDown = (ev) => {
1930
+ if (this.inputLocked) return; // handle.setVideo: a screen-locked plane has nothing to orbit
1931
+ if (ev.pointerId !== undefined) pointers.set(ev.pointerId, [ev.clientX, ev.clientY]);
1932
+ try {
1933
+ el.setPointerCapture?.(ev.pointerId);
1934
+ } catch {
1935
+ // An inactive pointer id (a synthetic event) throws; the gesture works without capture.
1936
+ }
1937
+ if (pointers.size === 2) {
1938
+ if (dragging) endDrag();
1939
+ this._pinching = true;
1940
+ this._zoomMode = null; // a new gesture takes over from a relax
1941
+ pinchD0 = Math.max(spread(), 1);
1942
+ pinchZ0 = this._zoom;
1943
+ this._targetZoom = this._zoom;
1944
+ this._lastInput = now();
1945
+ return;
1946
+ }
1947
+ if (pointers.size > 2) return;
1862
1948
  dragging = true;
1863
1949
  startX = ev.clientX;
1864
1950
  startY = ev.clientY;
@@ -1867,9 +1953,16 @@ export class PlayCanvasSplatViewer {
1867
1953
  restPitch = this._orbitMode ? restPitch : this._targetPitch;
1868
1954
  this._orbitMode = 'drag';
1869
1955
  this._lastInput = now();
1870
- el.setPointerCapture?.(ev.pointerId);
1871
1956
  };
1872
1957
  this._onMove = (ev) => {
1958
+ if (pointers.has(ev.pointerId)) pointers.set(ev.pointerId, [ev.clientX, ev.clientY]);
1959
+ if (this._pinching) {
1960
+ if (pointers.size < 2) return;
1961
+ const { min, max } = this.zoomOpts;
1962
+ this._targetZoom = clamp(pinchZ0 * (spread() / pinchD0), min, max);
1963
+ this._lastInput = now();
1964
+ return;
1965
+ }
1873
1966
  if (!dragging) return;
1874
1967
  const box = el.getBoundingClientRect();
1875
1968
  const dx = (ev.clientX - startX) / Math.max(box.width, 1);
@@ -1884,21 +1977,34 @@ export class PlayCanvasSplatViewer {
1884
1977
  this._lastInput = now();
1885
1978
  };
1886
1979
  this._onUp = (ev) => {
1980
+ const had = pointers.delete(ev.pointerId);
1981
+ if (this._pinching && pointers.size < 2) {
1982
+ // The pinch ends when either finger lifts: the zoom relaxes (zoomOpts.relax) from here.
1983
+ this._pinching = false;
1984
+ this._lastWheel = now() - ZOOM_WHEEL_IDLE_MS - 1;
1985
+ this._lastInput = now();
1986
+ }
1987
+ try {
1988
+ if (had || ev.pointerId === undefined) el.releasePointerCapture?.(ev.pointerId);
1989
+ } catch {
1990
+ // see pointerdown
1991
+ }
1887
1992
  if (!dragging) return;
1888
- dragging = false;
1889
- this._targetYaw = restYaw;
1890
- this._targetPitch = clamp(restPitch, this.pitchLimit[0], this.pitchLimit[1]);
1891
- this._orbitMode = 'rest';
1993
+ endDrag();
1892
1994
  this._lastInput = now();
1893
- el.releasePointerCapture?.(ev.pointerId);
1894
1995
  };
1895
1996
  this._onWheel = (ev) => {
1997
+ if (this.inputLocked) return;
1896
1998
  ev.preventDefault();
1897
1999
  let px = ev.deltaY;
1898
2000
  if (ev.deltaMode === 1) px *= WHEEL_LINE_PX;
1899
2001
  else if (ev.deltaMode === 2) px *= WHEEL_PAGE_PX;
1900
2002
  px = clamp(px, -WHEEL_MAX_PX, WHEEL_MAX_PX);
1901
- this._targetZoom = clamp(this._targetZoom * Math.exp(-px * ZOOM_PER_PX), ZOOM_MIN, ZOOM_MAX);
2003
+ // A wheel tick during a relax restarts from where the zoom IS, not from the rest target.
2004
+ const from = this._zoomMode === 'rest' ? this._zoom : this._targetZoom;
2005
+ this._zoomMode = null;
2006
+ this._targetZoom = clamp(from * Math.exp(-px * ZOOM_PER_PX), this.zoomOpts.min, this.zoomOpts.max);
2007
+ this._lastWheel = now();
1902
2008
  this._lastInput = now();
1903
2009
  };
1904
2010
  if (el.style) el.style.touchAction = 'none';
@@ -1920,6 +2026,7 @@ export class PlayCanvasSplatViewer {
1920
2026
  el.removeEventListener('pointerleave', this._onUp);
1921
2027
  el.removeEventListener('wheel', this._onWheel);
1922
2028
  this._onDown = null;
2029
+ this._pinching = false;
1923
2030
  }
1924
2031
 
1925
2032
  /** The camera currently on screen: [proj, pose] of the first eye in 3D, the mono camera else. */
@@ -2210,6 +2317,10 @@ export const SET_RIG_TYPES = Object.freeze(['display', 'camera', 'auto']);
2210
2317
  const SET_RIG_FITS = ['contain', 'cover', 'height', 'none'];
2211
2318
  let warnedSetRigKeys = false;
2212
2319
 
2320
+ const VIDEO_BUSY = (what) =>
2321
+ `@displayxr/inline3d/splat: ${what}() while a video is on — call handle.setVideo(null) first ` +
2322
+ '(the video holds the display rig and hides the splat until it exits).';
2323
+
2213
2324
  const PAGE_SETRIG_ERROR =
2214
2325
  "@displayxr/inline3d/splat: setRig() is not available with controls:'page' — the page owns the " +
2215
2326
  'camera (its camera IS the rig). Drive it with handle.setCameraPose(matrixWorld, { verticalFovDeg, near, far }).';
@@ -2408,6 +2519,7 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
2408
2519
  flipY,
2409
2520
  orbitMaxDeg: opts.orbitMaxDeg,
2410
2521
  orbitEase: opts.orbitEase,
2522
+ zoom: opts.zoom,
2411
2523
  feather,
2412
2524
  captureFit,
2413
2525
  nearClip: opts.nearClip,
@@ -2423,6 +2535,12 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
2423
2535
  const livePrepared = new Set();
2424
2536
  /** The asset on screen: its resource, kind, the pick data it keeps. */
2425
2537
  let current = null; // { asset, entity, res, kind, streamedBounds, alpha8, pickCentres }
2538
+ // handle.setVideo (below). Declared up here: a remove() queued before load runs during attach.
2539
+ let vid = null;
2540
+ let videoSeq = 0;
2541
+ /** The pending setVideo's canceller (a newer call, setVideo(null) or remove() supersedes it). */
2542
+ let cancelPendingVideo = null;
2543
+ let warnedAutoplayMuted = false;
2426
2544
 
2427
2545
  Object.assign(out, {
2428
2546
  backend: 'playcanvas',
@@ -2491,6 +2609,7 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
2491
2609
  setSource,
2492
2610
  prepareSource,
2493
2611
  setRig,
2612
+ setVideo,
2494
2613
  /**
2495
2614
  * Play a transition effect (inflate, deflate, sweep, dissolve, fade, pulse, custom) — see
2496
2615
  * docs/splat-effects.md. Validated now; runs once the first asset is on screen. Resolves
@@ -2517,6 +2636,7 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
2517
2636
  effects: () => (fx ? fx.list() : []),
2518
2637
  remove() {
2519
2638
  removed = true;
2639
+ dropVideo(); // before the app goes: the plane's rVFC, and a <video> this handle made
2520
2640
  current = null;
2521
2641
  for (const p of [...livePrepared]) p.dispose();
2522
2642
  fx?.dispose();
@@ -2727,6 +2847,7 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
2727
2847
  let declaredDisplay = pageMode ? 'page' : 'shorthand';
2728
2848
  function pushViewRig(force) {
2729
2849
  if (pageMode) return; // pageTick declares the rig every frame
2850
+ if (vid?.on) return; // setVideo holds the display rig; exit re-declares what was there
2730
2851
  if (!out.rig || out.rig.type !== 'camera') return;
2731
2852
  const pose = viewer.mono.pose;
2732
2853
  const f = viewer.getFocus();
@@ -2841,6 +2962,7 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
2841
2962
  if (pageMode || focusInput === false || unbindFocusInput) return;
2842
2963
  unbindFocusInput = bindFocusGestures(canvas, {
2843
2964
  onDoubleClick: (e) => {
2965
+ if (vid?.on) return false; // a video plane: nothing to focus on
2844
2966
  const m = pickModel(e.clientX, e.clientY);
2845
2967
  if (!m) return false;
2846
2968
  out.rig.focus = m;
@@ -3239,6 +3361,7 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
3239
3361
  */
3240
3362
  function setRig(type, o = {}) {
3241
3363
  const req = validateSetRig(type, o, pageMode);
3364
+ if (vid) throw new Error(VIDEO_BUSY('setRig'));
3242
3365
  const seq = ++rigSeq;
3243
3366
  return first.then(async () => {
3244
3367
  if (removed || seq !== rigSeq || !current) return out;
@@ -3312,6 +3435,231 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
3312
3435
  return out;
3313
3436
  });
3314
3437
 
3438
+ // ── handle.setVideo: a stereo video on this handle (docs/playcanvas-adapter.md §setVideo) ──
3439
+ /**
3440
+ * The video state: null = no video. `on` once the plane is up (the splat hidden, the display rig
3441
+ * declared); `saved` is everything the entry changed, restored exactly by setVideo(null).
3442
+ */
3443
+ // (state: `vid`, `videoSeq`, `cancelPendingVideo` — declared with `current`, above: remove() reads them)
3444
+
3445
+ /** The <video> for a URL: made here, owned (and released) by this handle. */
3446
+ function makeVideoElement(url) {
3447
+ const v = document.createElement('video');
3448
+ v.crossOrigin = 'anonymous'; // WebGL refuses a tainted frame; same-origin is unaffected
3449
+ v.playsInline = true;
3450
+ v.preload = 'auto';
3451
+ v.src = url;
3452
+ return v;
3453
+ }
3454
+ /** Let go of a <video> this handle made: stop it and free its decoder. */
3455
+ function releaseOwned(v) {
3456
+ try {
3457
+ v.pause();
3458
+ v.removeAttribute('src');
3459
+ v.load();
3460
+ } catch {
3461
+ /* already gone */
3462
+ }
3463
+ }
3464
+ /** Resolves once `v` has a current frame (readyState ≥ 2); rejects on error or cancel. */
3465
+ function waitForFrame(v, seq) {
3466
+ return new Promise((resolve, reject) => {
3467
+ const off = () => {
3468
+ v.removeEventListener('loadeddata', ok);
3469
+ v.removeEventListener('error', bad);
3470
+ if (cancelPendingVideo === cancel) cancelPendingVideo = null;
3471
+ };
3472
+ const ok = () => (off(), resolve());
3473
+ const bad = () => {
3474
+ off();
3475
+ const e = v.error;
3476
+ reject(new Error(`@displayxr/inline3d/splat: setVideo — the video failed to load${e ? ` (${e.message || `code ${e.code}`})` : ''}.`));
3477
+ };
3478
+ const cancel = () => {
3479
+ off();
3480
+ const err = new Error('@displayxr/inline3d/splat: setVideo — superseded by a later setVideo / remove() before its first frame.');
3481
+ err.name = 'AbortError';
3482
+ reject(err);
3483
+ };
3484
+ cancelPendingVideo = cancel;
3485
+ if (seq !== videoSeq) return cancel();
3486
+ if ((v.readyState || 0) >= 2 && v.videoWidth > 0) return ok();
3487
+ v.addEventListener('loadeddata', ok);
3488
+ v.addEventListener('error', bad);
3489
+ });
3490
+ }
3491
+ function autoplay(v) {
3492
+ const p = v.play?.();
3493
+ if (!p || typeof p.catch !== 'function') return;
3494
+ p.catch((err) => {
3495
+ // No user gesture yet: the browser allows a MUTED play. The page unmutes on its next gesture.
3496
+ if (err?.name !== 'NotAllowedError' || v.muted) return;
3497
+ v.muted = true;
3498
+ if (!warnedAutoplayMuted) {
3499
+ warnedAutoplayMuted = true;
3500
+ console.info('[inline3d/splat] setVideo: autoplay with sound was refused (no user gesture yet) — playing MUTED; set video.muted = false on a gesture.');
3501
+ }
3502
+ v.play().catch(() => {});
3503
+ });
3504
+ }
3505
+
3506
+ /** Everything the entry changes, for an exact restore. */
3507
+ function snapshotForVideo() {
3508
+ const v = viewer;
3509
+ const entity = current?.entity || null;
3510
+ return {
3511
+ entity,
3512
+ splatEnabled: entity ? entity.enabled : null,
3513
+ vH: v.vH,
3514
+ idleSpin: v.idleSpin,
3515
+ mono: { fov: v.mono.fov, near: v.mono.near, far: v.mono.far, pose: Float64Array.from(v.mono.pose), capture: v.mono.capture },
3516
+ pose: {
3517
+ _yaw: v._yaw, _pitch: v._pitch, _targetYaw: v._targetYaw, _targetPitch: v._targetPitch,
3518
+ _zoom: v._zoom, _targetZoom: v._targetZoom, _depthOffset: v._depthOffset,
3519
+ _focusSettled: v._focusSettled, _focusRecentres: v._focusRecentres, _orbitMode: v._orbitMode, _lastInput: v._lastInput,
3520
+ },
3521
+ focus: { ...v._focus },
3522
+ targetFocus: { ...v._targetFocus },
3523
+ viewRig: out.viewRig,
3524
+ declaredDisplay,
3525
+ lastConvergence,
3526
+ };
3527
+ }
3528
+
3529
+ /** Enter: the splat hidden, the display camera + rig, the plane up. One task: one frame sees it all. */
3530
+ function enterVideo(state) {
3531
+ const v = viewer;
3532
+ state.saved = snapshotForVideo();
3533
+ if (state.saved.entity) state.saved.entity.enabled = false;
3534
+ v.inputLocked = true;
3535
+ v.idleSpin = 0;
3536
+ v.vH = state.vH;
3537
+ if (v.mono.capture) v.useDisplayCamera(); // camera rig → the display rig's mono camera
3538
+ else {
3539
+ v._placeMonoForFit();
3540
+ v._updateMonoProjection();
3541
+ }
3542
+ declareDisplay({ vH: state.vH });
3543
+ state.on = true;
3544
+ }
3545
+
3546
+ /** Exit: put back exactly what enterVideo changed, and the declaration that was there. */
3547
+ function exitVideo(state) {
3548
+ const v = viewer;
3549
+ const s = state.saved;
3550
+ v._videoPlane?.destroy();
3551
+ v._videoPlane = null;
3552
+ if (s.entity) s.entity.enabled = s.splatEnabled;
3553
+ v.vH = s.vH;
3554
+ v.idleSpin = s.idleSpin;
3555
+ Object.assign(v.mono, { fov: s.mono.fov, near: s.mono.near, far: s.mono.far, pose: s.mono.pose, capture: s.mono.capture });
3556
+ Object.assign(v, s.pose);
3557
+ v._focus = { ...s.focus };
3558
+ v._targetFocus = { ...s.targetFocus };
3559
+ v._updateMonoProjection(); // recomputed, not copied: the box may have been resized meanwhile
3560
+ v._applyTransform();
3561
+ v.inputLocked = false;
3562
+ lastConvergence = s.lastConvergence;
3563
+ if (declaredDisplay !== s.declaredDisplay) {
3564
+ if (s.declaredDisplay === 'shorthand') {
3565
+ // The boot shorthand cannot be restored: its explicit form (what it is shorthand for).
3566
+ declaredDisplay = null;
3567
+ declareDisplay({ vH: bootFraming.vH });
3568
+ } else {
3569
+ declaredDisplay = s.declaredDisplay;
3570
+ out.viewRig = s.viewRig;
3571
+ handle?.setViewRig?.(out.viewRig);
3572
+ }
3573
+ } else out.viewRig = s.viewRig;
3574
+ lastConvergence = s.lastConvergence;
3575
+ state.on = false;
3576
+ }
3577
+
3578
+ /** Tear the video down now (setVideo(null) / remove()). */
3579
+ function dropVideo() {
3580
+ videoSeq++;
3581
+ cancelPendingVideo?.();
3582
+ const state = vid;
3583
+ vid = null;
3584
+ if (!state) return;
3585
+ if (state.on) exitVideo(state);
3586
+ if (state.owned) releaseOwned(state.el);
3587
+ }
3588
+
3589
+ /**
3590
+ * handle.setVideo(src, options) — play a stereo video on THIS handle: no new canvas, layer or
3591
+ * session. The splat is hidden and a screen-locked plane on the display rig shows the video, each
3592
+ * eye its own half (sbs: left/right, tb: top/bottom); flat (mono), the left half at full
3593
+ * resolution. Applies on the first frame the video has. Resolves to `{ video, remove(), stats() }`;
3594
+ * the page drives transport through `video`. setVideo(null) exits and restores the splat, the rig
3595
+ * and its declaration exactly as they were. Throws during an in-flight setSource.
3596
+ */
3597
+ function setVideo(src, o = {}) {
3598
+ if (src === null || src === undefined) {
3599
+ if (pageMode) throw new Error(PAGE_VIDEO_ERROR);
3600
+ dropVideo();
3601
+ return Promise.resolve(null);
3602
+ }
3603
+ const r = validateSetVideo(src, o, pageMode);
3604
+ if (sourceInFlight > 0) {
3605
+ throw new Error(
3606
+ '@displayxr/inline3d/splat: setVideo() during an in-flight setSource() — await the swap first ' +
3607
+ '(a video and a transition cannot share the tile).',
3608
+ );
3609
+ }
3610
+ const seq = ++videoSeq;
3611
+ cancelPendingVideo?.();
3612
+ const prev = vid;
3613
+ const owned = typeof r.src === 'string';
3614
+ const el = owned ? makeVideoElement(r.src) : r.src;
3615
+ if (r.loop !== undefined) el.loop = r.loop;
3616
+ if (r.muted !== undefined) el.muted = r.muted;
3617
+ if (owned ? r.autoplay !== false : r.autoplay === true) autoplay(el);
3618
+ // The guard state is taken NOW (setSource / setRig refuse from this call on), the pixels change
3619
+ // on the first frame the video has.
3620
+ const state = { seq, el, owned, format: r.format, fit: r.fit, vH: r.vH ?? bootFraming.vH, on: false, saved: null, pending: true };
3621
+ if (!prev || !prev.on) vid = state; // a pending one it supersedes is cancelled above
3622
+ return first
3623
+ .catch(() => null)
3624
+ .then(() => waitForFrame(el, seq))
3625
+ .then(() => {
3626
+ if (removed || seq !== videoSeq || !viewer.app) throw Object.assign(new Error('@displayxr/inline3d/splat: setVideo — superseded.'), { name: 'AbortError' });
3627
+ const live = vid && vid.on ? vid : null;
3628
+ if (live) {
3629
+ // Another video on screen: keep ITS snapshot (the pre-video state), swap the source.
3630
+ state.saved = live.saved;
3631
+ state.on = true;
3632
+ if (live.owned && live.el !== el) releaseOwned(live.el);
3633
+ if (state.vH !== live.vH) {
3634
+ viewer.vH = state.vH;
3635
+ viewer._placeMonoForFit();
3636
+ viewer._updateMonoProjection();
3637
+ declareDisplay({ vH: state.vH });
3638
+ }
3639
+ } else enterVideo(state);
3640
+ vid = state;
3641
+ state.pending = false;
3642
+ viewer._videoPlane ||= new VideoPlane(viewer);
3643
+ viewer._videoPlane.setSource(el, { format: state.format, fit: state.fit, vH: state.vH });
3644
+ const plane = viewer._videoPlane;
3645
+ return Object.freeze({
3646
+ video: el,
3647
+ format: state.format,
3648
+ fit: state.fit,
3649
+ /** Exit (setVideo(null)) — a no-op once another setVideo replaced this one. */
3650
+ remove: () => (vid === state ? setVideo(null) : Promise.resolve(null)),
3651
+ /** Upload accounting: frames drawn with the plane up, and texture uploads (new frames). */
3652
+ stats: () => ({ frames: plane.frames, uploads: plane.uploads }),
3653
+ });
3654
+ })
3655
+ .catch((err) => {
3656
+ // Failed or superseded before it showed: nothing changed on screen; free what we made.
3657
+ if (vid === state) vid = null;
3658
+ if (owned && (!vid || vid.el !== el)) releaseOwned(el);
3659
+ throw err;
3660
+ });
3661
+ }
3662
+
3315
3663
  // ── prepareSource: load the next asset in the background, for a setSource with no load ──
3316
3664
  /**
3317
3665
  * Fetch, decode and upload `src` now, without rendering it: the engine's load (its own unpack
@@ -3453,6 +3801,8 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
3453
3801
 
3454
3802
  // ── setSource: swap the asset — a cut, a crossfade, or a transition ──
3455
3803
  let sourceGen = 0;
3804
+ /** setSource calls between their call and their settle (setVideo refuses to start inside one). */
3805
+ let sourceInFlight = 0;
3456
3806
  let pendingSwap = null;
3457
3807
  /** setSource's crossfade FALLBACK: the coverage remap on one entity (null clears it). */
3458
3808
  const setFade = (entity, k) => fx?.setInternal(entity, 'xfade', k === null ? null : { k });
@@ -3469,7 +3819,13 @@ export function attachPlayCanvasSplat(out, wall, canvas, src, opts, pending = []
3469
3819
  * `onFocusChange` fires); the pose is kept unless `resetPose: true`. A newer call supersedes an
3470
3820
  * older one still loading. Resolves to the handle once the swap has finished.
3471
3821
  */
3472
- async function setSource(next, o = {}) {
3822
+ function setSource(next, o = {}) {
3823
+ // A video owns the tile (display rig, splat hidden) until setVideo(null).
3824
+ if (vid) return Promise.reject(new Error(VIDEO_BUSY('setSource')));
3825
+ sourceInFlight++;
3826
+ return setSourceNow(next, o).finally(() => sourceInFlight--);
3827
+ }
3828
+ async function setSourceNow(next, o = {}) {
3473
3829
  const plan = resolveSwap(o);
3474
3830
  const { resetPose = false } = o;
3475
3831
  const gen = ++sourceGen;
@@ -238,6 +238,49 @@ export const ORBIT_TAU_DRAG_S = 0.2;
238
238
  /** Time constant of the relax back to rest after release, seconds. */
239
239
  export const ORBIT_TAU_REST_S = 0.6;
240
240
 
241
+ // ── ZOOM: bounds + relax (the PlayCanvas backend's `zoom` option) ───────────────────────────
242
+
243
+ /** Wheel "release": this long with no wheel event ends the gesture and starts the relax, ms. */
244
+ export const ZOOM_WHEEL_IDLE_MS = 150;
245
+ /**
246
+ * The zoom relax's landing floor, log-zoom per second. The relax is the orbit's exponential
247
+ * (τ = 0.6 s), except it never moves slower than this, so it lands in finite time: from 2× the
248
+ * exponential carries it to 1.5 % from home in ≈2.3 s and the floor finishes in ≈0.6 s more.
249
+ */
250
+ export const ZOOM_RELAX_MIN_RATE = 0.025;
251
+
252
+ /**
253
+ * Validate + default the `zoom` option: `{ min, max, relax, ease }`.
254
+ *
255
+ * Defaults are today's behaviour — ZOOM_MIN..ZOOM_MAX, no relax. `relax: true` eases the zoom
256
+ * back to its rest (1×, or the last setPose zoom) once the wheel has been idle
257
+ * ZOOM_WHEEL_IDLE_MS or a pinch ends, with τ = `ease` (default ORBIT_TAU_REST_S, the orbit's
258
+ * relax — the two gestures come home together).
259
+ *
260
+ * @returns {{min:number, max:number, relax:boolean, ease:number}}
261
+ */
262
+ export function resolveZoomOption(zoom) {
263
+ if (zoom === undefined || zoom === null) return { min: ZOOM_MIN, max: ZOOM_MAX, relax: false, ease: ORBIT_TAU_REST_S };
264
+ if (typeof zoom !== 'object') {
265
+ throw new Error(`@displayxr/inline3d/splat: zoom ${zoom} — expected { min, max, relax, ease }.`);
266
+ }
267
+ const num = (k, v, d) => {
268
+ if (v === undefined) return d;
269
+ if (typeof v !== 'number' || !Number.isFinite(v) || v <= 0) {
270
+ throw new Error(`@displayxr/inline3d/splat: zoom.${k} ${v} — expected a positive number.`);
271
+ }
272
+ return v;
273
+ };
274
+ const min = num('min', zoom.min, ZOOM_MIN);
275
+ const max = num('max', zoom.max, ZOOM_MAX);
276
+ if (max < min) throw new Error(`@displayxr/inline3d/splat: zoom.max ${max} is below zoom.min ${min}.`);
277
+ const ease = num('ease', zoom.ease, ORBIT_TAU_REST_S);
278
+ if (zoom.relax !== undefined && typeof zoom.relax !== 'boolean') {
279
+ throw new Error(`@displayxr/inline3d/splat: zoom.relax ${zoom.relax} — expected true or false.`);
280
+ }
281
+ return { min, max, relax: zoom.relax === true, ease };
282
+ }
283
+
241
284
  /**
242
285
  * The capture camera's off-axis WINDOW at the near plane — the one projection both backends'
243
286
  * camera rigs draw the mono (flat) view through. Principal point honoured, so a deconverged
@@ -346,6 +389,7 @@ export const PAGE_IGNORED_OPTIONS = Object.freeze([
346
389
  'depthLimit',
347
390
  'orbitMaxDeg',
348
391
  'orbitEase',
392
+ 'zoom',
349
393
  'captureFit',
350
394
  ]);
351
395
 
@@ -0,0 +1,305 @@
1
+ // inline3d-splat-video.js — handle.setVideo(): a stereo video ON the persistent PlayCanvas splat
2
+ // handle (docs/playcanvas-adapter.md §setVideo). Internal to ./inline3d-splat-playcanvas.js.
3
+ //
4
+ // WHY ON THE HANDLE. A page that renders everything through ONE persistent woven canvas (rule 3 of
5
+ // docs/woven-canvas-rules.md) cannot put a movie in a second canvas without re-creating the very
6
+ // window the persistent stage exists to remove (a fresh canvas is fresh to the compositor: the raw
7
+ // side-by-side flash). So the video is drawn by the tile's own engine, in the tile's own frame.
8
+ //
9
+ // HOW IT IS DRAWN. One quad, parented under the RIG NODE (the eye camera's parent), at display
10
+ // space z = 0: RenderView composes the rig node into every view, so a child of it is fixed relative
11
+ // to the eyes whatever the pose (orbit, zoom, focus) — the plane is screen-locked by construction,
12
+ // and exit restores the pose untouched. On the display rig the z = 0 plane spans the element
13
+ // (docs/authoring-inline-3d.md §display rig), so the quad is the element box contained / covered by
14
+ // the video's per-eye aspect. Zero disparity for the quad itself: the depth is the video's own.
15
+ //
16
+ // EACH EYE SEES ONLY ITS HALF. One camera draws every view (the RenderView path), so the shader
17
+ // cannot be told "this is the right eye" by a uniform per view. It does not need to be: the eye
18
+ // viewports sit side by side in the buffer, so gl_FragCoord.x against the first right-eye
19
+ // viewport's x says which eye this fragment belongs to. Mono (not woven): the split is +∞, every
20
+ // fragment samples the LEFT half, at the full buffer resolution. The sample is clamped half a texel
21
+ // inside its half, so linear filtering never bleeds the other eye across the seam.
22
+ //
23
+ // UPLOAD. Gated on requestVideoFrameCallback (a new frame was presented), with `seeked` and a
24
+ // currentTime change as fallbacks (no rVFC; a paused seek on a build that does not fire it). One
25
+ // texture, re-specified from the <video> by the engine (texImage2D; GPU-to-GPU in Chromium): no
26
+ // per-frame allocation on our side. RGBA8, sampled and written UNCHANGED — the same encoded sRGB
27
+ // values a 2D canvas drawImage() of the frame puts in the buffer (addVideo's paint), so the woven
28
+ // video and the flat one are the same pixels.
29
+
30
+ /** handle.setVideo's formats, fits and rigs. */
31
+ export const VIDEO_FORMATS = Object.freeze(['sbs', 'tb', 'mono']);
32
+ export const VIDEO_FITS = Object.freeze(['contain', 'cover']);
33
+ const VIDEO_OPTION_KEYS = new Set(['format', 'fit', 'rig', 'virtualDisplayHeight', 'loop', 'muted', 'autoplay']);
34
+ let warnedVideoKeys = false;
35
+
36
+ export const PAGE_VIDEO_ERROR =
37
+ "@displayxr/inline3d/splat: setVideo() is not available with controls:'page' — the page owns the " +
38
+ 'camera, and a video plane needs the display rig. Draw the video into your own scene instead.';
39
+
40
+ /**
41
+ * setVideo's arguments, validated and resolved (throws at the call, before anything runs).
42
+ * `src` null/undefined is the exit call and is not validated here.
43
+ * @returns {{ src: string|HTMLVideoElement, format: 'sbs'|'tb'|'mono', fit: 'contain'|'cover',
44
+ * vH: number|undefined, loop?: boolean, muted?: boolean, autoplay?: boolean }}
45
+ */
46
+ export function validateSetVideo(src, o = {}, pageMode = false) {
47
+ if (pageMode) throw new Error(PAGE_VIDEO_ERROR);
48
+ const isVideo = typeof HTMLVideoElement !== 'undefined' ? src instanceof HTMLVideoElement : !!src && typeof src === 'object' && 'videoWidth' in src;
49
+ if (!(typeof src === 'string' && src.length > 0) && !isVideo) {
50
+ throw new TypeError('@displayxr/inline3d/splat: setVideo(src) — expected a URL string, an HTMLVideoElement, or null to exit.');
51
+ }
52
+ if (o === null || typeof o !== 'object') throw new TypeError('@displayxr/inline3d/splat: setVideo options must be an object.');
53
+ const unknown = Object.keys(o).filter((k) => !VIDEO_OPTION_KEYS.has(k));
54
+ if (unknown.length && !warnedVideoKeys) {
55
+ warnedVideoKeys = true;
56
+ console.warn(`[inline3d/splat] setVideo ignores ${unknown.join(', ')}.`);
57
+ }
58
+ const format = o.format === undefined ? 'sbs' : o.format;
59
+ if (!VIDEO_FORMATS.includes(format)) {
60
+ throw new Error(`@displayxr/inline3d/splat: setVideo — format "${format}", expected ${VIDEO_FORMATS.join(' | ')}.`);
61
+ }
62
+ const fit = o.fit === undefined ? 'contain' : o.fit;
63
+ if (!VIDEO_FITS.includes(fit)) throw new Error(`@displayxr/inline3d/splat: setVideo — fit "${fit}", expected ${VIDEO_FITS.join(' | ')}.`);
64
+ if (o.rig !== undefined && o.rig !== 'display') {
65
+ throw new Error(`@displayxr/inline3d/splat: setVideo — rig "${o.rig}"; a video plays on the display rig only ('display').`);
66
+ }
67
+ let vH;
68
+ if (o.virtualDisplayHeight !== undefined) {
69
+ vH = o.virtualDisplayHeight;
70
+ if (!Number.isFinite(vH) || !(vH > 0)) throw new Error(`@displayxr/inline3d/splat: setVideo — bad virtualDisplayHeight: ${o.virtualDisplayHeight}.`);
71
+ }
72
+ const bool = (k) => (o[k] === undefined ? undefined : !!o[k]);
73
+ return { src, format, fit, vH, loop: bool('loop'), muted: bool('muted'), autoplay: bool('autoplay') };
74
+ }
75
+
76
+ /**
77
+ * The per-eye source regions in texture space (st: s → right, t → DOWN, i.e. image rows), as
78
+ * [s0, t0, ds, dt] for the left and the right eye.
79
+ */
80
+ export function eyeRegions(format) {
81
+ if (format === 'tb') return { L: [0, 0, 1, 0.5], R: [0, 0.5, 1, 0.5] };
82
+ if (format === 'mono') return { L: [0, 0, 1, 1], R: [0, 0, 1, 1] };
83
+ return { L: [0, 0, 0.5, 1], R: [0.5, 0, 0.5, 1] };
84
+ }
85
+
86
+ /** One eye's aspect (width / height) of a `videoWidth × videoHeight` frame in `format`. */
87
+ export function eyeAspect(format, w, h) {
88
+ if (!(w > 0) || !(h > 0)) return 0;
89
+ if (format === 'sbs') return w / 2 / h;
90
+ if (format === 'tb') return w / (h / 2);
91
+ return w / h;
92
+ }
93
+
94
+ /**
95
+ * The quad's size at display-space z = 0, where the window is `vH · boxAspect` wide and `vH` tall:
96
+ * the window contained ('contain': all of the frame, bars where the aspects differ) or covered
97
+ * ('cover': the window full, the overflow cut by the window's own frustum edges).
98
+ */
99
+ export function videoPlaneSize({ boxAspect, eyeAspect: a, vH, fit = 'contain' }) {
100
+ const W = vH * boxAspect;
101
+ const H = vH;
102
+ if (!(a > 0) || !(boxAspect > 0)) return { w: W, h: H };
103
+ const wider = a >= boxAspect;
104
+ if ((fit === 'contain') === wider) return { w: W, h: W / a };
105
+ return { w: H * a, h: H };
106
+ }
107
+
108
+ /**
109
+ * The first right-eye pixel column in the buffer for this frame's views (entries in BUFFER px,
110
+ * through `rect`), or +∞ for one view (mono: every fragment is the left eye). With N > 2 views the
111
+ * first half of them are left eyes.
112
+ */
113
+ export function eyeSplit(entries, rect) {
114
+ if (!entries || entries.length < 2) return 1e9;
115
+ return rect(entries[entries.length >> 1])[0];
116
+ }
117
+
118
+ const VERT = `
119
+ attribute vec3 vertex_position;
120
+ attribute vec2 vertex_texCoord0;
121
+ uniform mat4 matrix_model;
122
+ uniform mat4 matrix_viewProjection;
123
+ varying vec2 vUv;
124
+ void main() {
125
+ vUv = vertex_texCoord0;
126
+ gl_Position = matrix_viewProjection * matrix_model * vec4(vertex_position, 1.0);
127
+ }`;
128
+ const FRAG = `
129
+ varying vec2 vUv;
130
+ uniform sampler2D dxrVid;
131
+ uniform vec4 dxrVidL; // left eye's region: s0, t0, ds, dt (t = image rows, top = 0)
132
+ uniform vec4 dxrVidR; // right eye's
133
+ uniform vec2 dxrVidTexel; // half a texel, per axis
134
+ uniform float dxrVidSplit; // first right-eye pixel column in the buffer (1e9 = mono)
135
+ void main() {
136
+ vec4 r = gl_FragCoord.x >= dxrVidSplit ? dxrVidR : dxrVidL;
137
+ vec2 st = r.xy + vec2(vUv.x, 1.0 - vUv.y) * r.zw;
138
+ st = clamp(st, r.xy + dxrVidTexel, r.xy + r.zw - dxrVidTexel);
139
+ gl_FragColor = vec4(texture2D(dxrVid, st).rgb, 1.0);
140
+ }`;
141
+
142
+ /**
143
+ * The quad, its material and its texture, owned by the viewer (`viewer._videoPlane`) while a video
144
+ * is on. `beforeDraw` is called by the viewer's draw, once per engine tick, before the tick.
145
+ */
146
+ export class VideoPlane {
147
+ constructor(viewer) {
148
+ this.viewer = viewer;
149
+ const pc = (this.pc = viewer.pc);
150
+ const device = viewer.app.graphicsDevice;
151
+ const mesh = new pc.Mesh(device);
152
+ mesh.setPositions(new Float32Array([-0.5, -0.5, 0, 0.5, -0.5, 0, 0.5, 0.5, 0, -0.5, 0.5, 0]));
153
+ mesh.setUvs(0, new Float32Array([0, 0, 1, 0, 1, 1, 0, 1]));
154
+ mesh.setIndices([0, 1, 2, 0, 2, 3]);
155
+ mesh.update();
156
+ this.mesh = mesh;
157
+ const mat = new pc.ShaderMaterial({
158
+ uniqueName: 'inline3dVideoPlane',
159
+ attributes: { vertex_position: pc.SEMANTIC_POSITION, vertex_texCoord0: pc.SEMANTIC_TEXCOORD0 },
160
+ vertexGLSL: VERT,
161
+ fragmentGLSL: FRAG,
162
+ });
163
+ mat.cull = pc.CULLFACE_NONE;
164
+ mat.setParameter('dxrVidSplit', 1e9);
165
+ mat.update();
166
+ this.mat = mat;
167
+ // Under the rig node: fixed relative to the eyes (see the header).
168
+ this.node = new pc.GraphNode('inline3d-video');
169
+ viewer.rigNode.addChild(this.node);
170
+ const mi = new pc.MeshInstance(mesh, mat, this.node);
171
+ mi.cull = false;
172
+ mi.visible = false;
173
+ this.mi = mi;
174
+ this.layer = viewer.app.scene.layers.getLayerById(pc.LAYERID_WORLD ?? 0);
175
+ this.layer.addMeshInstances([mi]);
176
+
177
+ this.video = null;
178
+ this.tex = null;
179
+ this.format = 'sbs';
180
+ this.fit = 'contain';
181
+ this.vH = viewer.vH;
182
+ this._dirty = false;
183
+ this._lastT = -1;
184
+ this._rvfc = 0;
185
+ this._sizeKey = '';
186
+ this._split = NaN;
187
+ this._onSeeked = () => (this._dirty = true);
188
+ /** Upload accounting (handle.setVideo(...).stats()). */
189
+ this.uploads = 0;
190
+ this.frames = 0;
191
+ }
192
+
193
+ /** Play `video` on the quad (a new element, format or fit). The element must have a frame. */
194
+ setSource(video, { format, fit, vH }) {
195
+ this.format = format;
196
+ this.fit = fit;
197
+ this.vH = vH;
198
+ const r = eyeRegions(format);
199
+ this.mat.setParameter('dxrVidL', r.L);
200
+ this.mat.setParameter('dxrVidR', r.R);
201
+ if (video !== this.video) {
202
+ this._unwatch();
203
+ this.video = video;
204
+ this._watch();
205
+ }
206
+ this._ensureTexture();
207
+ this._sizeKey = '';
208
+ this._dirty = true;
209
+ this.mat.update();
210
+ this.mi.visible = true;
211
+ }
212
+
213
+ _watch() {
214
+ const v = this.video;
215
+ v.addEventListener?.('seeked', this._onSeeked);
216
+ if (typeof v.requestVideoFrameCallback === 'function') {
217
+ const cb = () => {
218
+ this._dirty = true;
219
+ if (this.video === v) this._rvfc = v.requestVideoFrameCallback(cb);
220
+ };
221
+ this._rvfc = v.requestVideoFrameCallback(cb);
222
+ }
223
+ }
224
+
225
+ _unwatch() {
226
+ const v = this.video;
227
+ if (!v) return;
228
+ v.removeEventListener?.('seeked', this._onSeeked);
229
+ if (this._rvfc && typeof v.cancelVideoFrameCallback === 'function') v.cancelVideoFrameCallback(this._rvfc);
230
+ this._rvfc = 0;
231
+ }
232
+
233
+ /** One texture per video size; re-made only when the frame size changes. */
234
+ _ensureTexture() {
235
+ const pc = this.pc;
236
+ const v = this.video;
237
+ const w = v.videoWidth || 0;
238
+ const h = v.videoHeight || 0;
239
+ if (this.tex && this.tex.width === w && this.tex.height === h) return;
240
+ this.tex?.destroy?.();
241
+ this.tex = new pc.Texture(this.viewer.app.graphicsDevice, {
242
+ name: 'inline3d-video',
243
+ width: Math.max(1, w),
244
+ height: Math.max(1, h),
245
+ format: pc.PIXELFORMAT_RGBA8,
246
+ mipmaps: false,
247
+ flipY: false,
248
+ minFilter: pc.FILTER_LINEAR,
249
+ magFilter: pc.FILTER_LINEAR,
250
+ addressU: pc.ADDRESS_CLAMP_TO_EDGE,
251
+ addressV: pc.ADDRESS_CLAMP_TO_EDGE,
252
+ });
253
+ this.tex.setSource?.(v);
254
+ this.mat.setParameter('dxrVid', this.tex);
255
+ this.mat.setParameter('dxrVidTexel', [0.5 / Math.max(1, w), 0.5 / Math.max(1, h)]);
256
+ this._lastT = v.currentTime;
257
+ this._dirty = false;
258
+ this.uploads++; // setSource uploads
259
+ }
260
+
261
+ /** Per engine tick, before it: size the quad, place the eye split, upload a new frame. */
262
+ beforeDraw(entries, rect) {
263
+ const v = this.video;
264
+ if (!v || !this.mi.visible) return;
265
+ this.frames++;
266
+ const split = eyeSplit(entries, rect);
267
+ if (split !== this._split) {
268
+ this._split = split;
269
+ this.mat.setParameter('dxrVidSplit', split);
270
+ }
271
+ const boxAspect = this.viewer.boxAspect;
272
+ const key = `${boxAspect}|${v.videoWidth}x${v.videoHeight}|${this.format}|${this.fit}|${this.vH}`;
273
+ if (key !== this._sizeKey) {
274
+ this._sizeKey = key;
275
+ this._ensureTexture();
276
+ const s = videoPlaneSize({ boxAspect, eyeAspect: eyeAspect(this.format, v.videoWidth, v.videoHeight), vH: this.vH, fit: this.fit });
277
+ this.node.setLocalScale(s.w, s.h, 1);
278
+ }
279
+ // A new frame: rVFC said so, a seek landed, or (no rVFC) the clock moved.
280
+ const t = v.currentTime;
281
+ if (this._dirty || (!this._rvfc && t !== this._lastT)) {
282
+ if ((v.readyState || 0) >= 2) {
283
+ this.tex.upload();
284
+ this.uploads++;
285
+ }
286
+ this._dirty = false;
287
+ this._lastT = t;
288
+ }
289
+ }
290
+
291
+ hide() {
292
+ this.mi.visible = false;
293
+ }
294
+
295
+ destroy() {
296
+ this._unwatch();
297
+ this.video = null;
298
+ this.layer?.removeMeshInstances?.([this.mi]);
299
+ this.node.parent?.removeChild?.(this.node);
300
+ this.tex?.destroy?.();
301
+ this.tex = null;
302
+ this.mat.destroy?.();
303
+ this.mesh.destroy?.();
304
+ }
305
+ }
@@ -373,6 +373,13 @@ export function addSplat(wall, canvas, src, opts = {}) {
373
373
  prepareSource() {
374
374
  throw new Error("@displayxr/inline3d/splat: prepareSource() is implemented on the PlayCanvas backend only.");
375
375
  },
376
+ /** Not on this backend: a video on the splat handle is a PlayCanvas-backend feature. */
377
+ setVideo() {
378
+ throw new Error(
379
+ "@displayxr/inline3d/splat: setVideo() is implemented on the PlayCanvas backend only " +
380
+ "(addSplat(…, { engine:'playcanvas' })); with Spark, use wall.addVideo on its own canvas.",
381
+ );
382
+ },
376
383
  /** Not on this backend: the live rig switch is a PlayCanvas-backend feature. */
377
384
  setRig() {
378
385
  throw new Error(
@@ -736,6 +743,9 @@ function addSplatDeferred(wall, canvas, src, opts) {
736
743
  validateSetRigArgs(type, o);
737
744
  return out.ready.then(() => out.setRig(type, o));
738
745
  },
746
+ // A video before the first asset has landed: controls:'page' throws at the call's own line,
747
+ // the rest is validated by the adapter's setVideo, which runs once the asset is on screen.
748
+ setVideo: page ? pageOnly('setVideo') : (src, o) => out.ready.then(() => out.setVideo(src, o)),
739
749
  // Effects before the adapter has loaded: validated NOW (a bad call throws at its own line),
740
750
  // then run once the first asset is on screen.
741
751
  playEffect: (name, o) => {
package/model.d.ts CHANGED
@@ -117,6 +117,8 @@ export interface ModelOptions {
117
117
  orbitMaxDeg?: number;
118
118
  /** PlayCanvas only: the orbit's easing time constants, seconds. */
119
119
  orbitEase?: { drag?: number; rest?: number };
120
+ /** PlayCanvas only: zoom bounds + relax (wheel, pinch, setPose) — see the splat option. */
121
+ zoom?: { min?: number; max?: number; relax?: boolean; ease?: number };
120
122
  /**
121
123
  * PlayCanvas only: who owns the camera — the SDK's viewer (default) or the page, through
122
124
  * {@link ModelHandle.setCameraPose} every frame (the attach rig; converges on the model's bounds
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@displayxr/inline3d",
3
- "version": "1.17.0",
3
+ "version": "1.19.0",
4
4
  "description": "Turn any HTML <canvas> into a glasses-free-3D window on a DisplayXR display, inside an ordinary web page. Dependency-free; progressive enhancement (falls back to plain 2D everywhere else).",
5
5
  "type": "module",
6
6
  "types": "./index.d.ts",
@@ -52,6 +52,7 @@
52
52
  "js/inline3d-splat-shared.js",
53
53
  "js/inline3d-splat-effects.js",
54
54
  "js/inline3d-splat-live.js",
55
+ "js/inline3d-splat-video.js",
55
56
  "js/inline3d-model.js",
56
57
  "js/inline3d-model-entry.js",
57
58
  "js/inline3d-model-playcanvas.js",
package/splat.d.ts CHANGED
@@ -178,6 +178,35 @@ export interface ResolvedRig {
178
178
  parallaxFactor: number;
179
179
  }
180
180
 
181
+ /** `handle.setVideo` options. */
182
+ export interface SplatVideoOptions {
183
+ /** Stereo layout of the frame: `'sbs'` (default, left eye = left half), `'tb'` (left = top), `'mono'` (both eyes the whole frame). */
184
+ format?: 'sbs' | 'tb' | 'mono';
185
+ /** `'contain'` (default): the whole eye image, bars where the aspects differ. `'cover'`: the window full, the overflow cropped. */
186
+ fit?: 'contain' | 'cover';
187
+ /** Only `'display'`. */
188
+ rig?: 'display';
189
+ /** The display rig's height while the video is on (default: the tile's own `virtualDisplayHeight`). No visible effect on a flat plane at the window. */
190
+ virtualDisplayHeight?: number;
191
+ /** Applied to the element when given. */
192
+ loop?: boolean;
193
+ muted?: boolean;
194
+ /** Default true for a URL (the SDK's own element), false for an element you pass. */
195
+ autoplay?: boolean;
196
+ }
197
+
198
+ /** What `handle.setVideo(src)` resolves to. */
199
+ export interface SplatVideo {
200
+ /** The element: play / pause / currentTime / duration / ended / events are the page's to drive. */
201
+ readonly video: HTMLVideoElement;
202
+ readonly format: 'sbs' | 'tb' | 'mono';
203
+ readonly fit: 'contain' | 'cover';
204
+ /** `handle.setVideo(null)`, or a no-op if another setVideo has replaced this one. */
205
+ remove(): Promise<null>;
206
+ /** Frames drawn with the plane up, and texture uploads (new video frames). */
207
+ stats(): { frames: number; uploads: number };
208
+ }
209
+
181
210
  export interface SplatOptions {
182
211
  /**
183
212
  * Which renderer. `'spark'` (the default) is three.js + Spark. `'playcanvas'` is the PlayCanvas
@@ -230,6 +259,15 @@ export interface SplatOptions {
230
259
  orbitMaxDeg?: number;
231
260
  /** PlayCanvas: easing time constants, seconds — `drag` while held (0.2), `rest` after release (0.6). */
232
261
  orbitEase?: { drag?: number; rest?: number };
262
+ /**
263
+ * PlayCanvas: zoom bounds + relax, for the wheel, the two-finger pinch and `setPose`. `min` /
264
+ * `max` clamp the zoom (defaults 0.2 / 6, the pre-1.19 range); `relax: true` eases it back to
265
+ * rest — 1×, or the last `setPose` zoom — once the wheel has been idle 150 ms or the pinch ends,
266
+ * with the orbit's time constant (`ease`, default 0.6 s) and a landing floor so it arrives (2×
267
+ * is home in ≈3 s). The zoom scales the subject about the focus, which keeps its screen position
268
+ * and its disparity. `{ min: 1, max: 2, relax: true }` is a peek that never shows the splat's edges.
269
+ */
270
+ zoom?: { min?: number; max?: number; relax?: boolean; ease?: number };
233
271
  fit?: 'contain' | 'height' | 'cover' | 'none';
234
272
  /** Fraction of the tile the subject may occupy (default 0.8) — width AND height. */
235
273
  margin?: number;
@@ -800,6 +838,25 @@ export interface SplatHandle {
800
838
  */
801
839
  setRig(type: 'display', opts?: SplatSetRigDisplayOptions): Promise<SplatHandle>;
802
840
  setRig(type: 'camera' | 'auto'): Promise<SplatHandle>;
841
+ /**
842
+ * PlayCanvas backend only — throws on Spark, and with `controls:'page'`. Play a stereo video ON
843
+ * this handle: no new canvas, layer or session (docs/playcanvas-adapter.md §setVideo). The splat
844
+ * is hidden and a screen-locked plane on the display rig shows the video, sized to its per-eye
845
+ * aspect by `fit`; each eye samples only its own half (`'sbs'`: left/right, `'tb'`: top/bottom).
846
+ * Not woven (the mono path): the LEFT half at the full buffer resolution. The texture is
847
+ * re-uploaded only when the video presents a new frame (requestVideoFrameCallback).
848
+ *
849
+ * Applies on the first frame the video has; resolves to a {@link SplatVideo}. The page drives
850
+ * transport through `v.video`. A URL makes an SDK-owned `<video>` (autoplays unless
851
+ * `autoplay: false`; muted automatically if the browser refuses sound without a gesture); an
852
+ * element you pass stays yours (the SDK never plays or pauses it unless `autoplay: true`).
853
+ * `setVideo(null)` exits and restores the splat's visibility, the pose, the lens and the declared
854
+ * view rig exactly as they were. A second setVideo replaces the video and keeps the pre-video
855
+ * state. Throws during an in-flight `setSource`; while a video is on, `setSource` rejects and
856
+ * `setRig` throws (`setVideo(null)` first). `prepareSource` stays available.
857
+ */
858
+ setVideo(src: string | HTMLVideoElement, opts?: SplatVideoOptions): Promise<SplatVideo>;
859
+ setVideo(src: null): Promise<null>;
803
860
  /**
804
861
  * `engine: 'playcanvas'` only (throws on Spark). Play a transition/pulse/custom effect;
805
862
  * validated at the call, run once the first asset is on screen. Resolves `{ finished }` —