@displayxr/inline3d 0.0.1 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +91 -8
- package/index.d.ts +164 -0
- package/js/inline3d-three.js +154 -0
- package/js/inline3d.js +612 -0
- package/package.json +59 -8
- package/three.d.ts +37 -0
package/README.md
CHANGED
|
@@ -1,16 +1,99 @@
|
|
|
1
|
-
#
|
|
1
|
+
# displayxr-web
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
Inline-3D **web samples** and a small JS SDK for the
|
|
4
|
+
[DisplayXR Browser](https://github.com/DisplayXR/displayxr-browser) — the DisplayXR analog of
|
|
5
|
+
[`immersive-web/webxr-samples`](https://github.com/immersive-web/webxr-samples). This is the canonical
|
|
6
|
+
repo web developers clone to build glasses-free 3D pages, and the site the browser navigates to for the
|
|
7
|
+
live demos.
|
|
4
8
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
+
**▶ See it live:** <https://displayxr.github.io/displayxr-web/> — open in the
|
|
10
|
+
[DisplayXR Browser](https://github.com/DisplayXR/displayxr-browser/releases) on DisplayXR hardware for
|
|
11
|
+
glasses-free 3D; in any other browser the pages render as a normal 2D fallback, so they're safe to view
|
|
12
|
+
anywhere.
|
|
9
13
|
|
|
10
|
-
Install
|
|
14
|
+
## Install
|
|
15
|
+
|
|
16
|
+
The SDK is published as **[`@displayxr/inline3d`](https://www.npmjs.com/package/@displayxr/inline3d)**
|
|
17
|
+
(dependency-free ESM, ships its own TypeScript types):
|
|
11
18
|
|
|
12
19
|
```sh
|
|
13
20
|
npm install @displayxr/inline3d
|
|
14
21
|
```
|
|
15
22
|
|
|
16
|
-
|
|
23
|
+
```js
|
|
24
|
+
import { createInline3D } from '@displayxr/inline3d';
|
|
25
|
+
import { EyeCamera, EdgeFeather } from '@displayxr/inline3d/three'; // optional three.js glue
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
No build step or bundler required — it's plain ES modules. You can also import a pinned version by
|
|
29
|
+
URL from a CDN (jsDelivr / unpkg) without npm. The samples in this repo import the SDK by relative
|
|
30
|
+
path (`./js/inline3d.js`) so they run straight off GitHub Pages; in your own app prefer the package.
|
|
31
|
+
|
|
32
|
+
`three` is an **optional peer dependency** — only the `@displayxr/inline3d/three` helpers need it.
|
|
33
|
+
|
|
34
|
+
Stability & what's covered by semver (and the deferred N-view / web-components / CSS-native roadmap
|
|
35
|
+
that is intentionally **not** in 1.0): [`docs/sdk-stability.md`](docs/sdk-stability.md).
|
|
36
|
+
|
|
37
|
+
## Quick start
|
|
38
|
+
|
|
39
|
+
One SDK call turns a `<canvas>` into a glasses-free-3D window. Everything degrades to plain 2D on a
|
|
40
|
+
non-DisplayXR browser, so a page is safe to ship anywhere.
|
|
41
|
+
|
|
42
|
+
```js
|
|
43
|
+
import { createInline3D } from '@displayxr/inline3d';
|
|
44
|
+
|
|
45
|
+
const wall = await createInline3D(); // opens an inline-3d session (detects support)
|
|
46
|
+
if (!wall.supported) {
|
|
47
|
+
// Not the DisplayXR Browser (or no 3D display) — your page's normal 2D content shows. Done.
|
|
48
|
+
} else {
|
|
49
|
+
// Woven, glasses-free 3D. Add content — one call per element:
|
|
50
|
+
wall.addImage(canvas, 'photo-sbs.png'); // a still side-by-side 3D photo
|
|
51
|
+
wall.addVideo(canvas, videoEl); // an SBS 3D video
|
|
52
|
+
wall.addScene(canvas, (views, layer) => { /* render */ });// a live three.js / WebGL stereo scene
|
|
53
|
+
}
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The browser weaves each element's stereo pair at its on-screen rect; the surrounding DOM stays flat.
|
|
57
|
+
The runtime batches every visible window into one weave per frame, so it scales to a wall of elements.
|
|
58
|
+
|
|
59
|
+
> **Detection:** call `createInline3D()` and check `wall.supported` — do **not** gate on
|
|
60
|
+
> `navigator.xr.isSessionSupported('inline-3d')`. That async probe resolves `false` if it runs before the
|
|
61
|
+
> OS weave service has bound (typically at page load), a false-negative that silently drops you to 2D.
|
|
62
|
+
> `createInline3D()` detects by actually acquiring a session, which is authoritative.
|
|
63
|
+
|
|
64
|
+
Full API + authoring guidance: [`docs/authoring-inline-3d.md`](docs/authoring-inline-3d.md).
|
|
65
|
+
Three.js glue (an off-axis `EyeCamera`) in [`js/inline3d-three.js`](js/inline3d-three.js).
|
|
66
|
+
|
|
67
|
+
## What's here
|
|
68
|
+
|
|
69
|
+
```
|
|
70
|
+
index.html landing (Pages entry point)
|
|
71
|
+
samples/
|
|
72
|
+
windows/ mixed 3D windows — still photos + a live video + a real-time three.js scene,
|
|
73
|
+
each woven with one SDK call, all on one session
|
|
74
|
+
js/
|
|
75
|
+
inline3d.js the SDK: createInline3D() → { addImage, addVideo, addScene }, feature-detect,
|
|
76
|
+
SBS buffer management, and a lazy create/close lifecycle for many windows
|
|
77
|
+
inline3d-three.js optional three.js helper (EyeCamera: off-axis projection from the session's eyes)
|
|
78
|
+
docs/
|
|
79
|
+
authoring-inline-3d.md the authoring guide
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## The inline-3D model (under the SDK)
|
|
83
|
+
|
|
84
|
+
If you want the raw WebXR surface the SDK wraps, an inline-3d element:
|
|
85
|
+
|
|
86
|
+
1. `const session = await navigator.xr.requestSession('inline-3d')` — a sensorless inline session
|
|
87
|
+
(feature-detect by whether this resolves; falls back to plain 2D).
|
|
88
|
+
2. `const layer = new XRDisplayLayer(session, canvas)` — binds the weave to that element.
|
|
89
|
+
3. Each XR frame: render the scene as a **side-by-side stereo pair** into the canvas, re-projected
|
|
90
|
+
**off-axis** (asymmetric-frustum / Kooima) from the eye positions the session reports that frame —
|
|
91
|
+
so moving your head looks *around* the 3D content.
|
|
92
|
+
|
|
93
|
+
See the [WebXR inline-3D explainer](https://github.com/DisplayXR/displayxr-runtime/blob/main/docs/roadmap/webxr-displayxr-explainer.md).
|
|
94
|
+
|
|
95
|
+
## Local preview
|
|
96
|
+
|
|
97
|
+
Any static server, e.g. `python -m http.server 8080`, then open `http://localhost:8080/`.
|
|
98
|
+
(Loading over `file://` is fine for pure-2D, but WebXR requires a **secure context** — use
|
|
99
|
+
`http://localhost` or `https://`.)
|
package/index.d.ts
ADDED
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
// Type definitions for @displayxr/inline3d — the DisplayXR inline-3D SDK.
|
|
2
|
+
// Public 1.0 surface. See docs/sdk-stability.md for the semver contract.
|
|
3
|
+
|
|
4
|
+
/** Options shared by every add*() call. */
|
|
5
|
+
export interface TileOptions {
|
|
6
|
+
/** Per-eye buffer resolution in px (defaults to the CSS box × devicePixelRatio, dpr capped at 2). */
|
|
7
|
+
width?: number;
|
|
8
|
+
/** Per-eye buffer height in px (see `width`). */
|
|
9
|
+
height?: number;
|
|
10
|
+
/** Round each eye's corners, in BUFFER px (CSS radii can't cross the packed side-by-side pair). */
|
|
11
|
+
cornerRadius?: number;
|
|
12
|
+
/** Fade each eye's outer edges to transparent over this many buffer px. */
|
|
13
|
+
feather?: number;
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
/** Extra options for {@link Inline3D.addScene}. */
|
|
17
|
+
export interface SceneOptions extends TileOptions {
|
|
18
|
+
/**
|
|
19
|
+
* Metre height of the virtual display this scene is authored for (default 0.24). The runtime
|
|
20
|
+
* scales the eye poses it reports so the z=0 plane spans a display this tall — author in metres
|
|
21
|
+
* and render the reported views as-is. Halving it doubles how much of the window an object fills.
|
|
22
|
+
*/
|
|
23
|
+
virtualDisplayHeight?: number;
|
|
24
|
+
/** Element whose visibility drives the lazy create/close lifecycle (defaults to the canvas). */
|
|
25
|
+
observe?: Element;
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
/** The per-frame render callback passed to {@link Inline3D.addScene}. */
|
|
29
|
+
export type SceneFrameCallback = (
|
|
30
|
+
views: readonly XRView[],
|
|
31
|
+
layer: XRDisplayLayer,
|
|
32
|
+
frame: XRFrame,
|
|
33
|
+
) => void;
|
|
34
|
+
|
|
35
|
+
/** The handle returned by every add*() call. */
|
|
36
|
+
export interface TileHandle {
|
|
37
|
+
/** Remove this window: close its weave layer and stop driving it. */
|
|
38
|
+
remove(): void;
|
|
39
|
+
/**
|
|
40
|
+
* Mark a 2D element painted OVER this window so the weave leaves it crisp 2D instead of
|
|
41
|
+
* garbling it (browser#18). No-op on browsers without overlay exclusion.
|
|
42
|
+
*/
|
|
43
|
+
exclude(el: Element): void;
|
|
44
|
+
/** Stop excluding `el` from this window's weave. */
|
|
45
|
+
unexclude(el: Element): void;
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/** An open inline-3D session you add weaved windows to. Returned by {@link createInline3D}. */
|
|
49
|
+
export interface Inline3D {
|
|
50
|
+
readonly supported: true;
|
|
51
|
+
/** The underlying WebXR session. */
|
|
52
|
+
readonly session: XRSession;
|
|
53
|
+
/** The reference space the eye poses are reported in (may be null if none could be acquired). */
|
|
54
|
+
readonly refSpace: XRReferenceSpace | null;
|
|
55
|
+
/** Number of currently-active (weaving) windows. */
|
|
56
|
+
readonly liveCount: number;
|
|
57
|
+
|
|
58
|
+
/** Weave a still side-by-side 3D photo from a URL or decoded image source. */
|
|
59
|
+
addImage(
|
|
60
|
+
canvas: HTMLCanvasElement,
|
|
61
|
+
source: string | HTMLImageElement | ImageBitmap | HTMLCanvasElement,
|
|
62
|
+
opts?: TileOptions,
|
|
63
|
+
): TileHandle;
|
|
64
|
+
|
|
65
|
+
/** Weave a side-by-side 3D video element (re-drawn each decoded frame). */
|
|
66
|
+
addVideo(
|
|
67
|
+
canvas: HTMLCanvasElement,
|
|
68
|
+
video: HTMLVideoElement,
|
|
69
|
+
opts?: TileOptions,
|
|
70
|
+
): TileHandle;
|
|
71
|
+
|
|
72
|
+
/**
|
|
73
|
+
* Weave a live-rendered stereo scene. Your callback receives the two eye views + the layer;
|
|
74
|
+
* render each `layer.getViewport(view)` into the canvas's SBS backing (three.js: see the
|
|
75
|
+
* `@displayxr/inline3d/three` helpers).
|
|
76
|
+
*/
|
|
77
|
+
addScene(
|
|
78
|
+
canvas: HTMLCanvasElement,
|
|
79
|
+
onFrame: SceneFrameCallback,
|
|
80
|
+
opts?: SceneOptions,
|
|
81
|
+
): TileHandle;
|
|
82
|
+
|
|
83
|
+
/**
|
|
84
|
+
* Register a PAGE-GLOBAL 2D overlay (a fixed/sticky header, a floating toolbar) excluded from
|
|
85
|
+
* EVERY window's weave and re-applied when a window lazily re-activates. Register once instead
|
|
86
|
+
* of calling {@link TileHandle.exclude} per tile. No-op without overlay exclusion (browser#18).
|
|
87
|
+
*/
|
|
88
|
+
addGlobalOverlay(el: Element): void;
|
|
89
|
+
/** Stop treating `el` as a page-global overlay and drop it from every live window. */
|
|
90
|
+
removeGlobalOverlay(el: Element): void;
|
|
91
|
+
|
|
92
|
+
/** Close the session and remove every window. */
|
|
93
|
+
close(): void;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
/** The shape {@link createInline3D} resolves to when inline-3D is unavailable. */
|
|
97
|
+
export interface Inline3DUnsupported {
|
|
98
|
+
supported: false;
|
|
99
|
+
error?: unknown;
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
/** Options for {@link createInline3D}. */
|
|
103
|
+
export interface CreateInline3DOptions {
|
|
104
|
+
/** WebXR reference space for the eye poses (default `"viewer"`). */
|
|
105
|
+
referenceSpace?: string;
|
|
106
|
+
/**
|
|
107
|
+
* Create each window's weave layer only while it is (near-)visible and close it when it scrolls
|
|
108
|
+
* away, so a long wall only pays for what's on screen (default `true`). Set `false` for a single
|
|
109
|
+
* always-on element.
|
|
110
|
+
*/
|
|
111
|
+
lazy?: boolean;
|
|
112
|
+
/** IntersectionObserver margin for lazy mode (default `"50% 0px"`). */
|
|
113
|
+
rootMargin?: string;
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
/** The return of {@link startInline3D}. */
|
|
117
|
+
export interface StartInline3DResult {
|
|
118
|
+
supported: boolean;
|
|
119
|
+
/** The manager (present when supported). */
|
|
120
|
+
wall?: Inline3D;
|
|
121
|
+
/** The underlying WebXR session (present when supported). */
|
|
122
|
+
session?: XRSession;
|
|
123
|
+
/** Close the session (present when supported). */
|
|
124
|
+
close?: () => void;
|
|
125
|
+
error?: unknown;
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
/**
|
|
129
|
+
* Cheap, synchronous "can this browser even attempt inline-3D?" gate — true only in the DisplayXR
|
|
130
|
+
* Browser with the feature enabled. Use it to decide page UI up front.
|
|
131
|
+
*/
|
|
132
|
+
export function inline3DAvailable(): boolean;
|
|
133
|
+
|
|
134
|
+
/**
|
|
135
|
+
* True when this browser supports 2D-overlay exclusion (browser#18) — putting a 2D element ON a
|
|
136
|
+
* woven tile so it composites as crisp 2D over the woven 3D. Implies {@link inline3DAvailable}.
|
|
137
|
+
* Sync + cheap.
|
|
138
|
+
*/
|
|
139
|
+
export function inline3dOverlaySupported(): boolean;
|
|
140
|
+
|
|
141
|
+
/** Open the page's inline-3D session and return a manager you add windows to. */
|
|
142
|
+
export function createInline3D(
|
|
143
|
+
opts?: CreateInline3DOptions,
|
|
144
|
+
): Promise<Inline3D | Inline3DUnsupported>;
|
|
145
|
+
|
|
146
|
+
/**
|
|
147
|
+
* Back-compatible single-scene helper: open a session, weave one canvas, drive a render callback
|
|
148
|
+
* each frame. Equivalent to `createInline3D({lazy:false})` then `addScene(canvas, onFrame)`.
|
|
149
|
+
*/
|
|
150
|
+
export function startInline3D(
|
|
151
|
+
canvas: HTMLCanvasElement,
|
|
152
|
+
opts?: {
|
|
153
|
+
onFrame?: SceneFrameCallback;
|
|
154
|
+
referenceSpace?: string;
|
|
155
|
+
virtualDisplayHeight?: number;
|
|
156
|
+
},
|
|
157
|
+
): Promise<StartInline3DResult>;
|
|
158
|
+
|
|
159
|
+
// XRDisplayLayer is a DisplayXR-Browser extension to WebXR; declare the minimum the SDK exposes.
|
|
160
|
+
export interface XRDisplayLayer {
|
|
161
|
+
getViewport(view: XRView): { x: number; y: number; width: number; height: number } | null;
|
|
162
|
+
excludeElement?(el: Element): void;
|
|
163
|
+
close(): void;
|
|
164
|
+
}
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
// inline3d-three.js — optional three.js glue for the inline-3D SDK.
|
|
2
|
+
//
|
|
3
|
+
// The core inline3d.js is dependency-free and hands a scene window the two eye XRViews each
|
|
4
|
+
// frame. This module removes the three.js-specific boilerplate: driving a camera from an
|
|
5
|
+
// XRView, and the one non-obvious bit — SCALING the scene to the canvas element's physical
|
|
6
|
+
// size.
|
|
7
|
+
//
|
|
8
|
+
// import * as THREE from 'three';
|
|
9
|
+
// import { createInline3D } from '../js/inline3d.js';
|
|
10
|
+
// import { EyeCamera } from '../js/inline3d-three.js';
|
|
11
|
+
//
|
|
12
|
+
// // TWO bits of renderer setup are load-bearing (see "VIEWPORTS" below):
|
|
13
|
+
// renderer.setPixelRatio(1); // getViewport() is already in device px
|
|
14
|
+
// const dpr = window.devicePixelRatio || 1; // SBS store: DOUBLE-WIDTH, device-res
|
|
15
|
+
// renderer.setSize(canvas.clientWidth * dpr * 2, canvas.clientHeight * dpr, false);
|
|
16
|
+
//
|
|
17
|
+
// const eye = new EyeCamera(THREE); // one reusable off-axis camera
|
|
18
|
+
// wall.addScene(canvas, (views, layer) => { // addScene sets virtualDisplayHeight = 0.24 m
|
|
19
|
+
// renderer.clear();
|
|
20
|
+
// renderer.setScissorTest(true);
|
|
21
|
+
// for (const view of views) {
|
|
22
|
+
// const vp = layer.getViewport(view);
|
|
23
|
+
// renderer.setViewport(vp.x, vp.y, vp.width, vp.height);
|
|
24
|
+
// renderer.setScissor(vp.x, vp.y, vp.width, vp.height);
|
|
25
|
+
// eye.setFromView(view); // projection + pose straight from the view
|
|
26
|
+
// renderer.render(scene, eye.camera); // author at metre scale; NO scaling here
|
|
27
|
+
// }
|
|
28
|
+
// renderer.setScissorTest(false);
|
|
29
|
+
// });
|
|
30
|
+
//
|
|
31
|
+
// VIEWPORTS — the one trap. layer.getViewport() returns BACKING-STORE pixels, but three.js's
|
|
32
|
+
// setViewport()/setScissor() multiply what you pass them by the renderer's pixelRatio. So
|
|
33
|
+
// setPixelRatio(anything but 1) silently scales every eye viewport: at dpr 2 the left eye
|
|
34
|
+
// covers the WHOLE canvas and overflows vertically, and the weave then shows you a stretched
|
|
35
|
+
// slice of it. The tell is nasty — the scene still head-tracks perfectly (the pose and the
|
|
36
|
+
// off-axis projection are untouched), it is just zoomed and off-centre — so it looks like a
|
|
37
|
+
// projection/rig bug when it is purely a viewport one. Keep pixelRatio at 1 and size the
|
|
38
|
+
// backing store in device pixels yourself.
|
|
39
|
+
//
|
|
40
|
+
// SCENE SCALE IS THE RUNTIME'S JOB (display-rig m2v). The inline-3D views the session reports
|
|
41
|
+
// are already scaled to your scene by the layer's `virtualDisplayHeight` (see addScene) — the
|
|
42
|
+
// runtime places each eye at eye_physical × (virtualDisplayHeight / element_physical_height),
|
|
43
|
+
// so the z=0 plane spans that virtual display. Author your scene in metres for a display that
|
|
44
|
+
// tall (0.24 m by default), put focused content at z=0 (positive z behind the glass, negative
|
|
45
|
+
// in front), and render `eye.camera` directly. No per-frame world scaling — that is the whole
|
|
46
|
+
// point of using the rig instead of re-deriving it in the app, and it mirrors the native
|
|
47
|
+
// reference apps (cube_handle), which supply one scale number and consume render-ready views.
|
|
48
|
+
|
|
49
|
+
/**
|
|
50
|
+
* A reusable three.js camera driven directly by an XRView's matrices. Construct once with
|
|
51
|
+
* your THREE namespace and reuse across frames/windows.
|
|
52
|
+
*/
|
|
53
|
+
export class EyeCamera {
|
|
54
|
+
/** @param {object} THREE your imported three.js module namespace. */
|
|
55
|
+
constructor(THREE) {
|
|
56
|
+
this._THREE = THREE;
|
|
57
|
+
this.camera = new THREE.PerspectiveCamera();
|
|
58
|
+
this.camera.matrixAutoUpdate = false; // matrices come straight from the XRView
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
/** Set the camera's projection + world pose from an XRView (call once per eye per frame). */
|
|
62
|
+
setFromView(view) {
|
|
63
|
+
const cam = this.camera;
|
|
64
|
+
cam.projectionMatrix.fromArray(view.projectionMatrix);
|
|
65
|
+
cam.projectionMatrixInverse.copy(cam.projectionMatrix).invert();
|
|
66
|
+
cam.matrix.fromArray(view.transform.matrix);
|
|
67
|
+
cam.matrixWorld.copy(cam.matrix);
|
|
68
|
+
cam.matrixWorldInverse.copy(cam.matrixWorld).invert();
|
|
69
|
+
return cam;
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
/**
|
|
74
|
+
* Fade a rendered eye's edges to transparent, so a 3D window dissolves into the page instead of
|
|
75
|
+
* ending at a hard rectangle. The WebGL counterpart of the SDK's `feather` option for
|
|
76
|
+
* image/video windows (which the SDK bakes itself, since it owns those 2D buffers — for a scene,
|
|
77
|
+
* YOU own the canvas, so the pass has to run here).
|
|
78
|
+
*
|
|
79
|
+
* PER EYE, and that is not a detail: each eye's image spans the WHOLE window, so each needs a
|
|
80
|
+
* fade on all four of ITS OWN edges. A CSS mask/filter on the canvas fades only the element
|
|
81
|
+
* box's outer edges — the left eye would get a fade on its left and none on its right, and the
|
|
82
|
+
* split line would fade when it must not. Same reason cornerRadius is per-eye.
|
|
83
|
+
*
|
|
84
|
+
* Call once per eye, straight after renderer.render(scene, eye.camera), with the SAME viewport
|
|
85
|
+
* still set. Multiplies the framebuffer by an edge ramp (dst *= ramp) via ZeroFactor/SrcAlpha
|
|
86
|
+
* blending, so it works on whatever you drew without knowing anything about it.
|
|
87
|
+
*
|
|
88
|
+
* Requires a transparent canvas to fade INTO: WebGLRenderer({ alpha: true }),
|
|
89
|
+
* renderer.setClearColor(0x000000, 0), and no opaque scene.background.
|
|
90
|
+
*
|
|
91
|
+
* const feather = new EdgeFeather(THREE, { px: 28 });
|
|
92
|
+
* ...
|
|
93
|
+
* renderer.render(scene, eye.camera);
|
|
94
|
+
* feather.render(renderer, vp); // vp = layer.getViewport(view)
|
|
95
|
+
*/
|
|
96
|
+
export class EdgeFeather {
|
|
97
|
+
/**
|
|
98
|
+
* @param {object} THREE your imported three.js module namespace.
|
|
99
|
+
* @param {object} [opts]
|
|
100
|
+
* @param {number} [opts.px=24] fade width in BUFFER px (the same units getViewport reports).
|
|
101
|
+
*/
|
|
102
|
+
constructor(THREE, { px = 24 } = {}) {
|
|
103
|
+
this._THREE = THREE;
|
|
104
|
+
this.px = px;
|
|
105
|
+
this._cam = new THREE.OrthographicCamera(-1, 1, 1, -1, 0, 1);
|
|
106
|
+
this._mat = new THREE.ShaderMaterial({
|
|
107
|
+
uniforms: { fx: { value: 0.1 }, fy: { value: 0.1 } },
|
|
108
|
+
vertexShader: `
|
|
109
|
+
varying vec2 vUv;
|
|
110
|
+
void main() { vUv = uv; gl_Position = vec4(position.xy, 0.0, 1.0); }
|
|
111
|
+
`,
|
|
112
|
+
fragmentShader: `
|
|
113
|
+
varying vec2 vUv;
|
|
114
|
+
uniform float fx;
|
|
115
|
+
uniform float fy;
|
|
116
|
+
void main() {
|
|
117
|
+
// 1 inside, ramping to 0 at each edge. smoothstep gives a soft, banding-free falloff.
|
|
118
|
+
float ax = smoothstep(0.0, fx, vUv.x) * smoothstep(0.0, fx, 1.0 - vUv.x);
|
|
119
|
+
float ay = smoothstep(0.0, fy, vUv.y) * smoothstep(0.0, fy, 1.0 - vUv.y);
|
|
120
|
+
gl_FragColor = vec4(1.0, 1.0, 1.0, ax * ay);
|
|
121
|
+
}
|
|
122
|
+
`,
|
|
123
|
+
// dst_new = src*0 + dst*src.a => multiply the framebuffer (colour AND alpha) by the ramp.
|
|
124
|
+
transparent: true,
|
|
125
|
+
depthTest: false,
|
|
126
|
+
depthWrite: false,
|
|
127
|
+
blending: THREE.CustomBlending,
|
|
128
|
+
blendSrc: THREE.ZeroFactor,
|
|
129
|
+
blendDst: THREE.SrcAlphaFactor,
|
|
130
|
+
blendSrcAlpha: THREE.ZeroFactor,
|
|
131
|
+
blendDstAlpha: THREE.SrcAlphaFactor,
|
|
132
|
+
});
|
|
133
|
+
this._quad = new THREE.Mesh(new THREE.PlaneGeometry(2, 2), this._mat);
|
|
134
|
+
this._quad.frustumCulled = false;
|
|
135
|
+
this._scene = new THREE.Scene();
|
|
136
|
+
this._scene.add(this._quad);
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
/**
|
|
140
|
+
* @param {THREE.WebGLRenderer} renderer
|
|
141
|
+
* @param {{x:number,y:number,width:number,height:number}} vp this eye's viewport.
|
|
142
|
+
*/
|
|
143
|
+
render(renderer, vp) {
|
|
144
|
+
if (!vp || this.px <= 0) return;
|
|
145
|
+
// Ramp width as a fraction of THIS eye's viewport, so the fade is px-uniform on screen even
|
|
146
|
+
// though the eye is horizontally squeezed (a half-width viewport stretched 2x by the weave).
|
|
147
|
+
this._mat.uniforms.fx.value = Math.min(0.5, this.px / Math.max(1, vp.width));
|
|
148
|
+
this._mat.uniforms.fy.value = Math.min(0.5, this.px / Math.max(1, vp.height));
|
|
149
|
+
const prevAutoClear = renderer.autoClear;
|
|
150
|
+
renderer.autoClear = false;
|
|
151
|
+
renderer.render(this._scene, this._cam);
|
|
152
|
+
renderer.autoClear = prevAutoClear;
|
|
153
|
+
}
|
|
154
|
+
}
|
package/js/inline3d.js
ADDED
|
@@ -0,0 +1,612 @@
|
|
|
1
|
+
// inline3d.js — the DisplayXR inline-3D SDK. Dependency-free.
|
|
2
|
+
//
|
|
3
|
+
// Turn any HTML <canvas> into a glasses-free-3D "window" on a DisplayXR display, inside an
|
|
4
|
+
// otherwise ordinary web page. One page, one WebXR session, MANY weaved windows — and any
|
|
5
|
+
// content:
|
|
6
|
+
// • a still side-by-side (SBS) 3D photo → wall.addImage(canvas, url)
|
|
7
|
+
// • an SBS 3D video / movie → wall.addVideo(canvas, videoEl)
|
|
8
|
+
// • a live-rendered stereo scene (three.js, WebGL) → wall.addScene(canvas, onFrame)
|
|
9
|
+
//
|
|
10
|
+
// The DisplayXR runtime batches every visible window into ONE weave call per frame, so a
|
|
11
|
+
// scrolling wall of many 3D windows stays cheap. This SDK keeps that easy: it owns the
|
|
12
|
+
// fiddly parts (the SBS buffer contract, correct feature-detection, the compositor-layer
|
|
13
|
+
// hint, and — for many windows — a lazy create/close lifecycle) so your page code is short.
|
|
14
|
+
//
|
|
15
|
+
// ── THE ONE CONTRACT ────────────────────────────────────────────────────────────────────
|
|
16
|
+
// A weaved window is a <canvas> whose BACKING BUFFER holds side-by-side stereo — the left
|
|
17
|
+
// eye in the left half, the right eye in the right half — while its on-screen CSS box is
|
|
18
|
+
// whatever shape you want the viewer to see. The weave un-squishes the two halves back onto
|
|
19
|
+
// the box. So a square 3D photo is a 2:1 buffer in a square box; a 16:9 3D movie is a 32:9
|
|
20
|
+
// buffer in a 16:9 box. addImage/addVideo maintain this for you; addScene hands you the two
|
|
21
|
+
// eye viewports and you render into them.
|
|
22
|
+
//
|
|
23
|
+
// On any non-DisplayXR browser (or a 2D monitor) createInline3D() resolves to
|
|
24
|
+
// { supported:false } and your page shows its normal 2D content — inline-3D is progressive
|
|
25
|
+
// enhancement, never a hard dependency.
|
|
26
|
+
|
|
27
|
+
const hasWebXR = () => typeof navigator !== 'undefined' && !!navigator.xr;
|
|
28
|
+
const hasLayer = () =>
|
|
29
|
+
typeof window !== 'undefined' && typeof window.XRDisplayLayer === 'function';
|
|
30
|
+
// Overlay exclusion (browser#18): 2D DOM painted OVER a weaved window (hover plates,
|
|
31
|
+
// badges) would otherwise be woven along with the content and come out garbled. Browsers
|
|
32
|
+
// with XRDisplayLayer.excludeElement punch a per-pixel 2D hole in the weave there
|
|
33
|
+
// (final = M·weave + (1−M)·2D, M=0 inside the overlay rect). Older browsers: silent
|
|
34
|
+
// no-op — the page still works, the overlay just weaves like before.
|
|
35
|
+
const hasExclusion = () =>
|
|
36
|
+
hasLayer() && 'excludeElement' in window.XRDisplayLayer.prototype;
|
|
37
|
+
|
|
38
|
+
/**
|
|
39
|
+
* Cheap, synchronous "can this browser even attempt inline-3D?" gate — true only in the
|
|
40
|
+
* DisplayXR Browser with the feature enabled. Use it to decide page UI up front.
|
|
41
|
+
*
|
|
42
|
+
* It deliberately does NOT call navigator.xr.isSessionSupported('inline-3d'): that is an
|
|
43
|
+
* async round-trip to the OS weave service which resolves FALSE if it runs before the
|
|
44
|
+
* service has bound (typically at page load), a false-negative that silently drops you to
|
|
45
|
+
* 2D. The authoritative signal is whether createInline3D() actually acquires a session.
|
|
46
|
+
*/
|
|
47
|
+
export function inline3DAvailable() {
|
|
48
|
+
return hasWebXR() && hasLayer();
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
/**
|
|
52
|
+
* True when this browser supports 2D-overlay exclusion (browser#18) — putting a
|
|
53
|
+
* 2D element ON a woven tile (hover plate, badge) so it composites as crisp 2D
|
|
54
|
+
* over the woven 3D instead of being woven. Use it to choose the on-image
|
|
55
|
+
* overlay path when available and a weave-safe fallback (e.g. a caption band
|
|
56
|
+
* below the tile) otherwise. Implies inline3DAvailable(). Sync + cheap.
|
|
57
|
+
*/
|
|
58
|
+
export function inline3dOverlaySupported() {
|
|
59
|
+
return hasExclusion();
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
/**
|
|
63
|
+
* Open the page's inline-3D session and return a manager you add windows to.
|
|
64
|
+
*
|
|
65
|
+
* @param {object} [opts]
|
|
66
|
+
* @param {string} [opts.referenceSpace='viewer'] WebXR reference space for the eye poses.
|
|
67
|
+
* @param {boolean} [opts.lazy=true] Create each window's weave layer only while it is
|
|
68
|
+
* (near-)visible and close it when it scrolls away — so a long wall only pays for
|
|
69
|
+
* what's on screen. Set false for a single always-on element.
|
|
70
|
+
* @param {string} [opts.rootMargin='50% 0px'] IntersectionObserver margin for lazy mode;
|
|
71
|
+
* the default pre-arms a window half a viewport early so a fast scroll never shows a
|
|
72
|
+
* raw (un-woven) frame.
|
|
73
|
+
* @returns {Promise<Inline3D | {supported:false, error?:Error}>}
|
|
74
|
+
*/
|
|
75
|
+
export async function createInline3D(opts = {}) {
|
|
76
|
+
const { referenceSpace = 'viewer', lazy = true, rootMargin = '50% 0px' } = opts;
|
|
77
|
+
if (!inline3DAvailable()) return { supported: false };
|
|
78
|
+
let session;
|
|
79
|
+
try {
|
|
80
|
+
// requestSession is Blink-local and resolves immediately when the feature is present —
|
|
81
|
+
// the correct detection path (see inline3DAvailable's note on isSessionSupported).
|
|
82
|
+
session = await navigator.xr.requestSession('inline-3d');
|
|
83
|
+
} catch (e) {
|
|
84
|
+
return { supported: false, error: e };
|
|
85
|
+
}
|
|
86
|
+
let refSpace = null;
|
|
87
|
+
try {
|
|
88
|
+
refSpace = await session.requestReferenceSpace(referenceSpace);
|
|
89
|
+
} catch {
|
|
90
|
+
/* rAF still fires without a ref space; views are just null (fine for image/video). */
|
|
91
|
+
}
|
|
92
|
+
return new Inline3D(session, refSpace, { lazy, rootMargin });
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
/**
|
|
96
|
+
* Back-compatible single-scene helper: open a session, weave one canvas, drive a render
|
|
97
|
+
* callback with the two eye views each frame. Equivalent to
|
|
98
|
+
* createInline3D({lazy:false}) → addScene(canvas, onFrame).
|
|
99
|
+
* Returns { supported, close() } (plus the manager as .wall) or { supported:false }.
|
|
100
|
+
*/
|
|
101
|
+
export async function startInline3D(
|
|
102
|
+
canvas,
|
|
103
|
+
{ onFrame, referenceSpace = 'viewer', virtualDisplayHeight = 0.24 } = {}
|
|
104
|
+
) {
|
|
105
|
+
const wall = await createInline3D({ referenceSpace, lazy: false });
|
|
106
|
+
if (!wall.supported) return wall;
|
|
107
|
+
// Forward the scene-scale knob: without it addScene's default applies, and a caller who
|
|
108
|
+
// authored for a different virtual display size has no way to say so.
|
|
109
|
+
wall.addScene(canvas, onFrame, { virtualDisplayHeight });
|
|
110
|
+
return { supported: true, wall, session: wall.session, close: () => wall.close() };
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
class Inline3D {
|
|
114
|
+
constructor(session, refSpace, { lazy, rootMargin }) {
|
|
115
|
+
this.supported = true;
|
|
116
|
+
this.session = session;
|
|
117
|
+
this.refSpace = refSpace;
|
|
118
|
+
this._windows = new Map(); // canvas -> window record
|
|
119
|
+
this._globalOverlays = new Set(); // page-global overlays excluded from EVERY window
|
|
120
|
+
// el -> Set(window) currently excluding it. Isolation (will-change) is a GLOBAL
|
|
121
|
+
// property of the element while exclusion is PER-WINDOW, so the promotion has to
|
|
122
|
+
// be reference-counted: without this the first window to drop an element
|
|
123
|
+
// un-promotes it while other windows still need it isolated, and the element
|
|
124
|
+
// silently falls back into the canvas layer (→ it lands in that tile's SBS weave
|
|
125
|
+
// input and gets woven). Page-global overlays span many windows, so they are
|
|
126
|
+
// exactly the case that breaks.
|
|
127
|
+
this._isolatedBy = new WeakMap();
|
|
128
|
+
this._running = true;
|
|
129
|
+
this._lazy = lazy;
|
|
130
|
+
this._observer =
|
|
131
|
+
lazy && typeof IntersectionObserver === 'function'
|
|
132
|
+
? new IntersectionObserver((entries) => this._onIntersect(entries), { rootMargin })
|
|
133
|
+
: null;
|
|
134
|
+
session.addEventListener('end', () => this._teardown());
|
|
135
|
+
session.requestAnimationFrame((t, f) => this._frame(t, f));
|
|
136
|
+
}
|
|
137
|
+
|
|
138
|
+
/** Number of windows whose weave layer is currently live (on-screen in lazy mode). */
|
|
139
|
+
get liveCount() {
|
|
140
|
+
let n = 0;
|
|
141
|
+
for (const w of this._windows.values()) if (w.layer) n++;
|
|
142
|
+
return n;
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
/**
|
|
146
|
+
* Register a PAGE-GLOBAL 2D overlay (a fixed/sticky header, a floating toolbar) —
|
|
147
|
+
* an element that lives OUTSIDE any tile's container and can overlap MANY tiles as
|
|
148
|
+
* they scroll under it. It's excluded from every window's weave (current and future),
|
|
149
|
+
* re-applied automatically whenever a lazy window re-activates, so you register it ONCE
|
|
150
|
+
* instead of calling handle.exclude(el) per tile (which races window lifecycles).
|
|
151
|
+
*
|
|
152
|
+
* Note (browser#18, pre-#22): this keeps the element out of each tile's SBS weave input,
|
|
153
|
+
* but the per-tile present can still seam page-global chrome that spans tile gaps during
|
|
154
|
+
* scroll — the systematic fix is the DP-composited whole-window present (browser#22).
|
|
155
|
+
* No-op on browsers without excludeElement (progressive enhancement).
|
|
156
|
+
*/
|
|
157
|
+
addGlobalOverlay(el) {
|
|
158
|
+
if (!el || this._globalOverlays.has(el)) return;
|
|
159
|
+
this._globalOverlays.add(el);
|
|
160
|
+
for (const win of this._windows.values()) if (win.layer) this._applyExclusion(win, el);
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
/** Stop treating `el` as a page-global overlay and drop it from every live window. */
|
|
164
|
+
removeGlobalOverlay(el) {
|
|
165
|
+
if (!el || !this._globalOverlays.delete(el)) return;
|
|
166
|
+
for (const win of this._windows.values()) if (win.layer) this._dropExclusion(win, el);
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
/**
|
|
170
|
+
* Weave a still side-by-side 3D image into `canvas`.
|
|
171
|
+
* @param {HTMLCanvasElement} canvas a 2D canvas; the SDK owns its backing buffer.
|
|
172
|
+
* @param {string|HTMLImageElement|ImageBitmap|HTMLCanvasElement} source full SBS content
|
|
173
|
+
* (left eye = left half). A URL string is loaded for you.
|
|
174
|
+
* @param {object} [opts]
|
|
175
|
+
* @param {number} [opts.width] [opts.height] per-eye buffer resolution in px; defaults to
|
|
176
|
+
* the canvas's CSS box size × devicePixelRatio (so the box shape sets the aspect).
|
|
177
|
+
* @param {number} [opts.cornerRadius=0] round each eye's corners in buffer px (CSS
|
|
178
|
+
* border-radius can't: it would round the packed SBS square's outer corners and
|
|
179
|
+
* come out lopsided after the eye-split).
|
|
180
|
+
* @param {number} [opts.feather=0] fade each eye's outer edges to transparent over this
|
|
181
|
+
* many buffer px, so the 3D window dissolves into the page instead of ending at a
|
|
182
|
+
* hard rectangle. Same reason CSS can't do it: a mask/filter on the canvas applies
|
|
183
|
+
* across the packed SBS pair, so each eye would get an inner fade along the split
|
|
184
|
+
* line and only half its outer edge.
|
|
185
|
+
* @returns {{remove():void}}
|
|
186
|
+
*/
|
|
187
|
+
addImage(canvas, source, opts = {}) {
|
|
188
|
+
const win = this._register(canvas, 'image', opts);
|
|
189
|
+
win.ready = loadImage(source).then((img) => {
|
|
190
|
+
win.img = img;
|
|
191
|
+
win.repaint();
|
|
192
|
+
});
|
|
193
|
+
return this._handle(canvas, win);
|
|
194
|
+
}
|
|
195
|
+
|
|
196
|
+
/**
|
|
197
|
+
* Weave a playing SBS 3D video into `canvas` (redrawn every frame while visible).
|
|
198
|
+
* @param {HTMLCanvasElement} canvas a 2D canvas; the SDK owns its backing buffer.
|
|
199
|
+
* @param {HTMLVideoElement} video a full-SBS 3D video, already play()-ing (left = left).
|
|
200
|
+
* @param {object} [opts] same width/height/cornerRadius as addImage.
|
|
201
|
+
* @returns {{remove():void}}
|
|
202
|
+
*/
|
|
203
|
+
addVideo(canvas, video, opts = {}) {
|
|
204
|
+
const win = this._register(canvas, 'video', opts);
|
|
205
|
+
win.video = video;
|
|
206
|
+
return this._handle(canvas, win);
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
/**
|
|
210
|
+
* Weave a live-rendered stereo scene into `canvas`. YOU own the canvas (its size, its
|
|
211
|
+
* WebGL/2D context); the SDK only creates the weave layer and calls you each frame with
|
|
212
|
+
* the two eye views. Render each view into `layer.getViewport(view)` (an {x,y,width,
|
|
213
|
+
* height} into the canvas) using `view.projectionMatrix` + `view.transform.matrix`.
|
|
214
|
+
* See inline3d-three.js for three.js glue (camera + element-scale helpers).
|
|
215
|
+
* The session reports per-eye off-axis (Kooima) views already scaled to your scene by
|
|
216
|
+
* `virtualDisplayHeight` (the display-rig m2v knob): author your scene in metres for a
|
|
217
|
+
* display that tall, put focused content at z=0, and render the views DIRECTLY — the
|
|
218
|
+
* runtime owns the projection AND the scale, so there is no per-frame world scaling in
|
|
219
|
+
* your app.
|
|
220
|
+
* @param {HTMLCanvasElement} canvas
|
|
221
|
+
* @param {(views:XRView[], layer:XRDisplayLayer, frame:XRFrame)=>void} onFrame
|
|
222
|
+
* @param {object} [opts]
|
|
223
|
+
* @param {number} [opts.virtualDisplayHeight=0.24] metres of virtual display the scene is
|
|
224
|
+
* composed for. Larger = the element shows a bigger slice of the world.
|
|
225
|
+
* @param {Element} [opts.observe=canvas] element whose visibility gates lazy create/close.
|
|
226
|
+
* @returns {{remove():void}}
|
|
227
|
+
*/
|
|
228
|
+
addScene(canvas, onFrame, opts = {}) {
|
|
229
|
+
const win = this._register(canvas, 'scene', { virtualDisplayHeight: 0.24, ...opts });
|
|
230
|
+
win.onFrame = onFrame;
|
|
231
|
+
win.ownsBuffer = false; // the app sizes a scene canvas; we never touch canvas.width/height
|
|
232
|
+
return this._handle(canvas, win);
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
/**
|
|
236
|
+
* The handle every add*() returns. `exclude(el)` marks 2D DOM painted over this window
|
|
237
|
+
* (a hover plate, a play badge) so the weave leaves it crisp 2D instead of garbling it
|
|
238
|
+
* (browser#18). Queued if the layer isn't live yet (lazy mode) and re-applied on every
|
|
239
|
+
* re-activate; a browser without excludeElement silently ignores it (the overlay weaves
|
|
240
|
+
* like before — progressive enhancement, like the rest of this SDK). Prefer the
|
|
241
|
+
* declarative `data-inline3d-overlay` attribute (see _startOverlayScan) unless you need
|
|
242
|
+
* to exclude an element outside the window's container.
|
|
243
|
+
*/
|
|
244
|
+
_handle(canvas, win) {
|
|
245
|
+
return {
|
|
246
|
+
remove: () => this._remove(canvas),
|
|
247
|
+
exclude: (el) => {
|
|
248
|
+
if (!el) return;
|
|
249
|
+
win.excluded.add(el);
|
|
250
|
+
this._applyExclusion(win, el);
|
|
251
|
+
},
|
|
252
|
+
unexclude: (el) => {
|
|
253
|
+
if (!el || !win.excluded.delete(el)) return;
|
|
254
|
+
this._dropExclusion(win, el);
|
|
255
|
+
},
|
|
256
|
+
};
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
close() {
|
|
260
|
+
try {
|
|
261
|
+
this.session.end();
|
|
262
|
+
} catch {
|
|
263
|
+
/* end() also fires our 'end' handler → _teardown */
|
|
264
|
+
}
|
|
265
|
+
this._teardown();
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
// ── internals ───────────────────────────────────────────────────────────────────────
|
|
269
|
+
|
|
270
|
+
_register(canvas, kind, opts) {
|
|
271
|
+
if (this._windows.has(canvas)) this._remove(canvas);
|
|
272
|
+
// Own compositing layer: makes the canvas a distinct quad the weave can track. Harmless
|
|
273
|
+
// when the compositor would have promoted it anyway.
|
|
274
|
+
canvas.style.willChange = 'transform';
|
|
275
|
+
canvas.style.transform = 'translateZ(0)';
|
|
276
|
+
const win = {
|
|
277
|
+
canvas,
|
|
278
|
+
kind,
|
|
279
|
+
layer: null,
|
|
280
|
+
img: null,
|
|
281
|
+
video: null,
|
|
282
|
+
onFrame: null,
|
|
283
|
+
ready: null,
|
|
284
|
+
ownsBuffer: kind !== 'scene',
|
|
285
|
+
cornerRadius: opts.cornerRadius || 0,
|
|
286
|
+
feather: opts.feather || 0,
|
|
287
|
+
reqW: opts.width || 0,
|
|
288
|
+
reqH: opts.height || 0,
|
|
289
|
+
virtualDisplayHeight: opts.virtualDisplayHeight || 0,
|
|
290
|
+
observeEl: opts.observe || canvas,
|
|
291
|
+
ctx: kind === 'scene' ? null : canvas.getContext('2d'),
|
|
292
|
+
repaint: () => this._paint(win, null),
|
|
293
|
+
// Overlay exclusion (browser#18): explicit handle.exclude() elements and
|
|
294
|
+
// [data-inline3d-overlay] descendants found by the auto-scan. Applied to the
|
|
295
|
+
// layer on every (re-)activate; the browser clears its own set on layer close.
|
|
296
|
+
excluded: new Set(),
|
|
297
|
+
autoExcluded: new Set(),
|
|
298
|
+
overlayObserver: null,
|
|
299
|
+
};
|
|
300
|
+
this._windows.set(canvas, win);
|
|
301
|
+
if (this._lazy && this._observer) {
|
|
302
|
+
this._observer.observe(win.observeEl);
|
|
303
|
+
} else {
|
|
304
|
+
this._activate(win);
|
|
305
|
+
}
|
|
306
|
+
return win;
|
|
307
|
+
}
|
|
308
|
+
|
|
309
|
+
_remove(canvas) {
|
|
310
|
+
const win = this._windows.get(canvas);
|
|
311
|
+
if (!win) return;
|
|
312
|
+
if (this._observer) this._observer.unobserve(win.observeEl);
|
|
313
|
+
this._deactivate(win);
|
|
314
|
+
this._windows.delete(canvas);
|
|
315
|
+
}
|
|
316
|
+
|
|
317
|
+
_onIntersect(entries) {
|
|
318
|
+
for (const e of entries) {
|
|
319
|
+
// The observed element may be a wrapper; find the window it belongs to.
|
|
320
|
+
let win = null;
|
|
321
|
+
for (const w of this._windows.values()) {
|
|
322
|
+
if (w.observeEl === e.target) {
|
|
323
|
+
win = w;
|
|
324
|
+
break;
|
|
325
|
+
}
|
|
326
|
+
}
|
|
327
|
+
if (!win) continue;
|
|
328
|
+
if (e.isIntersecting) this._activate(win);
|
|
329
|
+
else this._deactivate(win);
|
|
330
|
+
}
|
|
331
|
+
}
|
|
332
|
+
|
|
333
|
+
_activate(win) {
|
|
334
|
+
if (win.layer) return;
|
|
335
|
+
try {
|
|
336
|
+
// virtualDisplayHeight (display-rig m2v) tells the runtime what scale this
|
|
337
|
+
// window's scene is authored at, so it returns render-ready scaled views.
|
|
338
|
+
const init =
|
|
339
|
+
win.virtualDisplayHeight > 0 ? { virtualDisplayHeight: win.virtualDisplayHeight } : {};
|
|
340
|
+
win.layer = new XRDisplayLayer(this.session, win.canvas, init);
|
|
341
|
+
} catch {
|
|
342
|
+
win.layer = null;
|
|
343
|
+
return;
|
|
344
|
+
}
|
|
345
|
+
// Re-apply overlay exclusions (browser#18): the browser's layer-side set died with
|
|
346
|
+
// the previous layer (lazy close), so a re-activated window must re-declare its own
|
|
347
|
+
// explicit exclusions, the page-global overlays, and the attribute-scanned overlays,
|
|
348
|
+
// then resume watching for changes.
|
|
349
|
+
for (const el of win.excluded) this._applyExclusion(win, el);
|
|
350
|
+
for (const el of this._globalOverlays) this._applyExclusion(win, el);
|
|
351
|
+
this._startOverlayScan(win);
|
|
352
|
+
if (win.ownsBuffer) {
|
|
353
|
+
this._sizeBuffer(win, /*sbs*/ true);
|
|
354
|
+
this._paint(win, null); // first SBS paint (video will refresh each frame)
|
|
355
|
+
}
|
|
356
|
+
}
|
|
357
|
+
|
|
358
|
+
_deactivate(win) {
|
|
359
|
+
this._stopOverlayScan(win);
|
|
360
|
+
if (win.layer) {
|
|
361
|
+
try {
|
|
362
|
+
win.layer.close();
|
|
363
|
+
} catch {
|
|
364
|
+
/* already closed */
|
|
365
|
+
}
|
|
366
|
+
win.layer = null;
|
|
367
|
+
}
|
|
368
|
+
// Leave a flat (left-eye-only) frame so an off-screen image/video still shows 2D.
|
|
369
|
+
if (win.ownsBuffer && win.kind !== 'scene') {
|
|
370
|
+
this._sizeBuffer(win, /*sbs*/ false);
|
|
371
|
+
this._paint(win, null);
|
|
372
|
+
}
|
|
373
|
+
}
|
|
374
|
+
|
|
375
|
+
// ── overlay exclusion (browser#18) ─────────────────────────────────────────────────
|
|
376
|
+
|
|
377
|
+
_applyExclusion(win, el) {
|
|
378
|
+
if (!win.layer || !hasExclusion()) return;
|
|
379
|
+
// Force the overlay onto its OWN composited layer so the browser can grab it
|
|
380
|
+
// as an isolated resource (the element rastered on transparency) and
|
|
381
|
+
// composite it OVER the woven 3D — final = plate + (1−plate.a)·woven, true
|
|
382
|
+
// 2D-over-3D. `will-change: transform` reliably promotes to a compositing
|
|
383
|
+
// layer even in the single-render-pass weave config (a CSS filter does NOT —
|
|
384
|
+
// its render surface is flattened away there). Remember we set it so
|
|
385
|
+
// unexclude can restore.
|
|
386
|
+
let refs = this._isolatedBy.get(el);
|
|
387
|
+
if (!refs) {
|
|
388
|
+
refs = new Set();
|
|
389
|
+
this._isolatedBy.set(el, refs);
|
|
390
|
+
}
|
|
391
|
+
refs.add(win);
|
|
392
|
+
if (!el.dataset.inline3dIsolated) {
|
|
393
|
+
el.dataset.inline3dPriorWillChange = el.style.willChange || '';
|
|
394
|
+
const wc = el.style.willChange && el.style.willChange !== 'auto'
|
|
395
|
+
? el.style.willChange + ', transform'
|
|
396
|
+
: 'transform';
|
|
397
|
+
el.style.willChange = wc;
|
|
398
|
+
el.dataset.inline3dIsolated = '1';
|
|
399
|
+
}
|
|
400
|
+
try {
|
|
401
|
+
win.layer.excludeElement(el);
|
|
402
|
+
} catch {
|
|
403
|
+
/* closed layer / detached element — the per-frame report drops empties anyway */
|
|
404
|
+
}
|
|
405
|
+
}
|
|
406
|
+
|
|
407
|
+
_dropExclusion(win, el) {
|
|
408
|
+
const refs = this._isolatedBy.get(el);
|
|
409
|
+
if (refs) refs.delete(win);
|
|
410
|
+
// Only un-promote once NO window needs this element isolated any more.
|
|
411
|
+
if ((!refs || refs.size === 0) && el.dataset.inline3dIsolated) {
|
|
412
|
+
el.style.willChange = el.dataset.inline3dPriorWillChange || '';
|
|
413
|
+
delete el.dataset.inline3dPriorWillChange;
|
|
414
|
+
delete el.dataset.inline3dIsolated;
|
|
415
|
+
}
|
|
416
|
+
if (!win.layer || !hasExclusion()) return;
|
|
417
|
+
try {
|
|
418
|
+
win.layer.unexcludeElement(el);
|
|
419
|
+
} catch {
|
|
420
|
+
/* ignore */
|
|
421
|
+
}
|
|
422
|
+
}
|
|
423
|
+
|
|
424
|
+
// Declarative overlays: any element marked `data-inline3d-overlay` inside the window's
|
|
425
|
+
// container (the canvas's parent — where an over-the-window plate must live to be
|
|
426
|
+
// positioned over it) is auto-excluded while the window is live, and tracked through
|
|
427
|
+
// add/remove/toggle by one MutationObserver per active window. Hidden overlays cost
|
|
428
|
+
// nothing: a display:none element reports an empty rect browser-side, so show/hide of a
|
|
429
|
+
// hover plate needs no attribute churn — mark it once, toggle `display` freely. (Hide
|
|
430
|
+
// with display, not opacity/visibility: those still report a full rect, so the weave
|
|
431
|
+
// hole would stay punched under an invisible plate.)
|
|
432
|
+
_startOverlayScan(win) {
|
|
433
|
+
if (!hasExclusion() || typeof MutationObserver !== 'function') return;
|
|
434
|
+
const container = win.canvas.parentElement;
|
|
435
|
+
if (!container) return;
|
|
436
|
+
const sync = () => {
|
|
437
|
+
const marked = new Set(container.querySelectorAll('[data-inline3d-overlay]'));
|
|
438
|
+
for (const el of win.autoExcluded) {
|
|
439
|
+
if (!marked.has(el)) {
|
|
440
|
+
win.autoExcluded.delete(el);
|
|
441
|
+
this._dropExclusion(win, el);
|
|
442
|
+
}
|
|
443
|
+
}
|
|
444
|
+
for (const el of marked) {
|
|
445
|
+
if (!win.autoExcluded.has(el)) {
|
|
446
|
+
win.autoExcluded.add(el);
|
|
447
|
+
this._applyExclusion(win, el);
|
|
448
|
+
}
|
|
449
|
+
}
|
|
450
|
+
};
|
|
451
|
+
sync();
|
|
452
|
+
win.overlayObserver = new MutationObserver(sync);
|
|
453
|
+
win.overlayObserver.observe(container, {
|
|
454
|
+
childList: true,
|
|
455
|
+
subtree: true,
|
|
456
|
+
attributes: true,
|
|
457
|
+
attributeFilter: ['data-inline3d-overlay'],
|
|
458
|
+
});
|
|
459
|
+
}
|
|
460
|
+
|
|
461
|
+
_stopOverlayScan(win) {
|
|
462
|
+
if (win.overlayObserver) {
|
|
463
|
+
win.overlayObserver.disconnect();
|
|
464
|
+
win.overlayObserver = null;
|
|
465
|
+
}
|
|
466
|
+
// The browser clears the layer-side set on close; mirror that so a re-activate
|
|
467
|
+
// re-scans from scratch (the container's overlays may have changed while dark).
|
|
468
|
+
win.autoExcluded.clear();
|
|
469
|
+
}
|
|
470
|
+
|
|
471
|
+
_sizeBuffer(win, sbs) {
|
|
472
|
+
const dpr = Math.min(window.devicePixelRatio || 1, 2);
|
|
473
|
+
const boxW = win.reqW || Math.round((win.canvas.clientWidth || 256) * dpr);
|
|
474
|
+
const boxH = win.reqH || Math.round((win.canvas.clientHeight || 256) * dpr);
|
|
475
|
+
win.eyeW = boxW;
|
|
476
|
+
win.eyeH = boxH;
|
|
477
|
+
win.canvas.width = sbs ? boxW * 2 : boxW; // SBS = two eye tiles wide
|
|
478
|
+
win.canvas.height = boxH;
|
|
479
|
+
win.sbs = sbs;
|
|
480
|
+
}
|
|
481
|
+
|
|
482
|
+
_paint(win, _views) {
|
|
483
|
+
if (win.kind === 'scene' || !win.ctx) return;
|
|
484
|
+
const src = win.kind === 'video' ? win.video : win.img;
|
|
485
|
+
if (!src) return;
|
|
486
|
+
if (win.kind === 'video' && (src.readyState || 0) < 2) return; // no frame yet
|
|
487
|
+
const c = win.canvas;
|
|
488
|
+
const ctx = win.ctx;
|
|
489
|
+
const srcW = src.videoWidth || src.naturalWidth || src.width;
|
|
490
|
+
const srcH = src.videoHeight || src.naturalHeight || src.height;
|
|
491
|
+
if (!srcW || !srcH) return;
|
|
492
|
+
ctx.clearRect(0, 0, c.width, c.height);
|
|
493
|
+
if (!win.sbs) {
|
|
494
|
+
// Flat fallback: left eye only, stretched to the square buffer.
|
|
495
|
+
drawEye(ctx, src, 0, 0, srcW / 2, srcH, 0, 0, c.width, c.height, win.cornerRadius, win.feather);
|
|
496
|
+
return;
|
|
497
|
+
}
|
|
498
|
+
const halfDst = c.width / 2;
|
|
499
|
+
// A single stretched draw maps SBS source → SBS buffer (left→left, right→right); the
|
|
500
|
+
// per-eye path is only needed to bake decoration (rounded corners / edge feather), which
|
|
501
|
+
// MUST be applied to each eye separately — see drawEye/featherEye.
|
|
502
|
+
if (win.cornerRadius > 0 || win.feather > 0) {
|
|
503
|
+
drawEye(ctx, src, 0, 0, srcW / 2, srcH, 0, 0, halfDst, c.height, win.cornerRadius, win.feather); // L
|
|
504
|
+
drawEye(ctx, src, srcW / 2, 0, srcW / 2, srcH, halfDst, 0, halfDst, c.height, win.cornerRadius, win.feather); // R
|
|
505
|
+
} else {
|
|
506
|
+
ctx.drawImage(src, 0, 0, srcW, srcH, 0, 0, c.width, c.height);
|
|
507
|
+
}
|
|
508
|
+
}
|
|
509
|
+
|
|
510
|
+
_frame(t, f) {
|
|
511
|
+
if (!this._running) return;
|
|
512
|
+
this.session.requestAnimationFrame((t2, f2) => this._frame(t2, f2));
|
|
513
|
+
const pose = this.refSpace ? f.getViewerPose(this.refSpace) : null;
|
|
514
|
+
const views = pose ? pose.views : null;
|
|
515
|
+
for (const win of this._windows.values()) {
|
|
516
|
+
if (!win.layer) continue;
|
|
517
|
+
if (win.kind === 'scene') {
|
|
518
|
+
if (views && win.onFrame) win.onFrame(views, win.layer, f);
|
|
519
|
+
} else {
|
|
520
|
+
// Repaint image AND video every frame. The weave reads each window's
|
|
521
|
+
// composited canvas quad per frame; a canvas that isn't redrawn can have
|
|
522
|
+
// its layer dropped from the aggregated frame, so the weave reads a stale
|
|
523
|
+
// sub-rect and the window flickers to a horizontal smear. A still image's
|
|
524
|
+
// redraw is one cheap GPU drawImage — keep it live.
|
|
525
|
+
this._paint(win, views);
|
|
526
|
+
}
|
|
527
|
+
}
|
|
528
|
+
}
|
|
529
|
+
|
|
530
|
+
_teardown() {
|
|
531
|
+
if (!this._running) return;
|
|
532
|
+
this._running = false;
|
|
533
|
+
if (this._observer) this._observer.disconnect();
|
|
534
|
+
for (const win of this._windows.values()) {
|
|
535
|
+
this._stopOverlayScan(win);
|
|
536
|
+
if (win.layer) {
|
|
537
|
+
try {
|
|
538
|
+
win.layer.close();
|
|
539
|
+
} catch {
|
|
540
|
+
/* ignore */
|
|
541
|
+
}
|
|
542
|
+
win.layer = null;
|
|
543
|
+
}
|
|
544
|
+
}
|
|
545
|
+
this._windows.clear();
|
|
546
|
+
}
|
|
547
|
+
}
|
|
548
|
+
|
|
549
|
+
// ── small helpers ─────────────────────────────────────────────────────────────────────
|
|
550
|
+
|
|
551
|
+
function loadImage(source) {
|
|
552
|
+
if (typeof source !== 'string') return Promise.resolve(source); // element/bitmap/canvas
|
|
553
|
+
return new Promise((resolve, reject) => {
|
|
554
|
+
const img = new Image();
|
|
555
|
+
img.decoding = 'async';
|
|
556
|
+
img.onload = () => resolve(img);
|
|
557
|
+
img.onerror = reject;
|
|
558
|
+
img.src = source;
|
|
559
|
+
});
|
|
560
|
+
}
|
|
561
|
+
|
|
562
|
+
// Draw one eye region with optional baked rounded corners. Corners are left transparent so
|
|
563
|
+
// the canvas's page background shows through (as a CSS radius would have).
|
|
564
|
+
function drawEye(ctx, src, sx, sy, sw, sh, dx, dy, dw, dh, radius, feather) {
|
|
565
|
+
if (radius > 0 && ctx.roundRect) {
|
|
566
|
+
ctx.save();
|
|
567
|
+
ctx.beginPath();
|
|
568
|
+
ctx.roundRect(dx, dy, dw, dh, radius);
|
|
569
|
+
ctx.clip();
|
|
570
|
+
ctx.drawImage(src, sx, sy, sw, sh, dx, dy, dw, dh);
|
|
571
|
+
ctx.restore();
|
|
572
|
+
} else {
|
|
573
|
+
ctx.drawImage(src, sx, sy, sw, sh, dx, dy, dw, dh);
|
|
574
|
+
}
|
|
575
|
+
if (feather > 0) {
|
|
576
|
+
featherEye(ctx, dx, dy, dw, dh, feather);
|
|
577
|
+
}
|
|
578
|
+
}
|
|
579
|
+
|
|
580
|
+
// Fade this EYE's outer edges to transparent, so the 3D window dissolves into the page
|
|
581
|
+
// instead of ending at a hard rectangle. Same spirit as the runtime feathering a 3D zone's
|
|
582
|
+
// edge — but note that is the hardware WISH MASK (lens control, never content); this is the
|
|
583
|
+
// content-side equivalent, and the two are independent.
|
|
584
|
+
//
|
|
585
|
+
// Per-eye, like cornerRadius, and for the same reason: the weave splits the element's rect
|
|
586
|
+
// down the middle, so anything applied across the whole (side-by-side) buffer gets halved —
|
|
587
|
+
// each eye would get an inner fade along the split line that must not exist, and only half
|
|
588
|
+
// its outer edge. A CSS mask/filter on the canvas has exactly that bug.
|
|
589
|
+
//
|
|
590
|
+
// destination-out with an alpha ramp erases toward transparent, so it works on top of
|
|
591
|
+
// whatever was just drawn (image, video frame) without knowing the content.
|
|
592
|
+
function featherEye(ctx, x, y, w, h, px) {
|
|
593
|
+
const f = Math.min(px, Math.floor(Math.min(w, h) / 2));
|
|
594
|
+
if (f <= 0) return;
|
|
595
|
+
ctx.save();
|
|
596
|
+
ctx.globalCompositeOperation = 'destination-out';
|
|
597
|
+
const edges = [
|
|
598
|
+
// [x, y, w, h, gradient-from, gradient-to]
|
|
599
|
+
[x, y, w, f, [x, y], [x, y + f]], // top
|
|
600
|
+
[x, y + h - f, w, f, [x, y + h], [x, y + h - f]], // bottom
|
|
601
|
+
[x, y, f, h, [x, y], [x + f, y]], // left
|
|
602
|
+
[x + w - f, y, f, h, [x + w, y], [x + w - f, y]], // right
|
|
603
|
+
];
|
|
604
|
+
for (const [ex, ey, ew, eh, from, to] of edges) {
|
|
605
|
+
const g = ctx.createLinearGradient(from[0], from[1], to[0], to[1]);
|
|
606
|
+
g.addColorStop(0, 'rgba(0,0,0,1)'); // fully erased at the outer edge
|
|
607
|
+
g.addColorStop(1, 'rgba(0,0,0,0)'); // untouched inside
|
|
608
|
+
ctx.fillStyle = g;
|
|
609
|
+
ctx.fillRect(ex, ey, ew, eh);
|
|
610
|
+
}
|
|
611
|
+
ctx.restore();
|
|
612
|
+
}
|
package/package.json
CHANGED
|
@@ -1,18 +1,69 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@displayxr/inline3d",
|
|
3
|
-
"version": "0.0
|
|
4
|
-
"description": "
|
|
5
|
-
"
|
|
3
|
+
"version": "1.0.0",
|
|
4
|
+
"description": "Turn any HTML <canvas> into a glasses-free-3D window on a DisplayXR display, inside an ordinary web page. Dependency-free; progressive enhancement (falls back to plain 2D everywhere else).",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"types": "./index.d.ts",
|
|
7
|
+
"main": "./js/inline3d.js",
|
|
8
|
+
"module": "./js/inline3d.js",
|
|
9
|
+
"exports": {
|
|
10
|
+
".": {
|
|
11
|
+
"types": "./index.d.ts",
|
|
12
|
+
"import": "./js/inline3d.js"
|
|
13
|
+
},
|
|
14
|
+
"./three": {
|
|
15
|
+
"types": "./three.d.ts",
|
|
16
|
+
"import": "./js/inline3d-three.js"
|
|
17
|
+
}
|
|
18
|
+
},
|
|
19
|
+
"sideEffects": false,
|
|
20
|
+
"files": [
|
|
21
|
+
"js/inline3d.js",
|
|
22
|
+
"js/inline3d-three.js",
|
|
23
|
+
"index.d.ts",
|
|
24
|
+
"three.d.ts",
|
|
25
|
+
"README.md",
|
|
26
|
+
"LICENSE"
|
|
27
|
+
],
|
|
28
|
+
"keywords": [
|
|
29
|
+
"displayxr",
|
|
30
|
+
"webxr",
|
|
31
|
+
"inline-3d",
|
|
32
|
+
"glasses-free-3d",
|
|
33
|
+
"stereo",
|
|
34
|
+
"3d-display",
|
|
35
|
+
"lightfield",
|
|
36
|
+
"three"
|
|
37
|
+
],
|
|
38
|
+
"homepage": "https://displayxr.github.io/displayxr-web/",
|
|
6
39
|
"repository": {
|
|
7
40
|
"type": "git",
|
|
8
41
|
"url": "git+https://github.com/DisplayXR/displayxr-web.git"
|
|
9
42
|
},
|
|
10
|
-
"
|
|
43
|
+
"bugs": {
|
|
44
|
+
"url": "https://github.com/DisplayXR/displayxr-browser/issues"
|
|
45
|
+
},
|
|
46
|
+
"license": "Apache-2.0",
|
|
47
|
+
"peerDependencies": {
|
|
48
|
+
"three": ">=0.150.0"
|
|
49
|
+
},
|
|
50
|
+
"peerDependenciesMeta": {
|
|
51
|
+
"three": {
|
|
52
|
+
"optional": true
|
|
53
|
+
}
|
|
54
|
+
},
|
|
11
55
|
"publishConfig": {
|
|
12
56
|
"access": "public"
|
|
13
57
|
},
|
|
14
|
-
"
|
|
15
|
-
"
|
|
16
|
-
|
|
17
|
-
|
|
58
|
+
"engines": {
|
|
59
|
+
"node": ">=18"
|
|
60
|
+
},
|
|
61
|
+
"scripts": {
|
|
62
|
+
"typecheck": "tsc --noEmit -p tsconfig.json",
|
|
63
|
+
"test": "npm run typecheck"
|
|
64
|
+
},
|
|
65
|
+
"devDependencies": {
|
|
66
|
+
"typescript": "^5.4.0",
|
|
67
|
+
"@types/webxr": "^0.5.0"
|
|
68
|
+
}
|
|
18
69
|
}
|
package/three.d.ts
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
// Type definitions for @displayxr/inline3d/three — optional three.js helpers.
|
|
2
|
+
// These take your imported THREE namespace as a constructor arg, so the SDK never
|
|
3
|
+
// bundles three.js (it is an optional peer dependency).
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* A reusable three.js camera driven directly by an XRView's matrices. Construct once with your
|
|
7
|
+
* THREE namespace and reuse across frames/windows. Read `.camera` and render it as-is — author
|
|
8
|
+
* your scene at metre scale, do no per-frame world scaling (the runtime's rig already scaled the
|
|
9
|
+
* reported views).
|
|
10
|
+
*/
|
|
11
|
+
export class EyeCamera {
|
|
12
|
+
/** @param THREE your imported three.js module namespace. */
|
|
13
|
+
constructor(THREE: unknown);
|
|
14
|
+
/** The three.js PerspectiveCamera to render (`renderer.render(scene, eye.camera)`). */
|
|
15
|
+
readonly camera: unknown;
|
|
16
|
+
/** Set the camera's projection + world pose straight from an XRView (call once per eye). */
|
|
17
|
+
setFromView(view: XRView): void;
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
/**
|
|
21
|
+
* Fade a rendered eye's edges to transparent, so a 3D window dissolves into the page instead of
|
|
22
|
+
* ending at a hard rectangle. Call once per eye, straight after `renderer.render(scene, eye.camera)`,
|
|
23
|
+
* with the SAME viewport `layer.getViewport(view)`. Requires an alpha canvas
|
|
24
|
+
* (`{ alpha: true }` + `setClearColor(0x000000, 0)` + no opaque `scene.background`).
|
|
25
|
+
*/
|
|
26
|
+
export class EdgeFeather {
|
|
27
|
+
/**
|
|
28
|
+
* @param THREE your imported three.js module namespace.
|
|
29
|
+
* @param opts.px fade width in BUFFER px (the same units getViewport reports; default 24).
|
|
30
|
+
*/
|
|
31
|
+
constructor(THREE: unknown, opts?: { px?: number });
|
|
32
|
+
/** Fade this eye's edges. `vp` is the viewport rect from `layer.getViewport(view)`. */
|
|
33
|
+
render(
|
|
34
|
+
renderer: unknown,
|
|
35
|
+
vp: { x: number; y: number; width: number; height: number },
|
|
36
|
+
): void;
|
|
37
|
+
}
|