@displayxr/inline3d 1.46.0 → 1.47.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +75 -0
- package/README.md +3 -0
- package/THIRD_PARTY_NOTICES.md +1 -1
- package/depth.d.ts +184 -0
- package/dist/call.js +6 -6
- package/docs/making-3d-look-right.md +49 -0
- package/docs/woven-canvas-rules.md +378 -0
- package/index.d.ts +22 -3
- package/js/inline3d-depth-budget.js +342 -0
- package/js/inline3d-model-entry.js +15 -1
- package/js/inline3d-model-playcanvas.js +27 -2
- package/js/inline3d-model.js +22 -2
- package/js/inline3d-splat-deferred.js +2 -0
- package/js/inline3d-splat-pc-entry.js +2 -1
- package/js/inline3d-splat-playcanvas.js +42 -42
- package/js/inline3d-splat-shared.js +311 -0
- package/js/inline3d-splat.js +31 -3
- package/js/inline3d-three.js +30 -10
- package/js/inline3d-viewer.js +13 -0
- package/js/inline3d.js +35 -0
- package/llms.txt +21 -0
- package/model.d.ts +53 -0
- package/package.json +10 -1
- package/splat.d.ts +33 -0
- package/three.d.ts +7 -2
- package/viewer.d.ts +11 -0
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Depth and comfort on a 3D display
|
|
2
|
+
|
|
3
|
+
The SDK's defaults already give comfortable, real-looking 3D. These rules cover the mistakes seen
|
|
4
|
+
on real panels; everything else about how your scene looks is your call.
|
|
5
|
+
|
|
6
|
+
1. **Objects: keep the viewer's placement, and frame the first view.** `addModel`, `addSplat` and
|
|
7
|
+
`SceneViewer` scale the subject to its real proportions and centre it on the glass. That gives
|
|
8
|
+
it the smallest spread of disparity it can have. Don't push it back, shrink it or flatten it to
|
|
9
|
+
make it "more comfortable". The first view shows the whole subject, centred: a globe or a tank
|
|
10
|
+
that starts low and cut off by the frame reads as broken. If the user can zoom, cap zoom-in
|
|
11
|
+
before the subject comes uncomfortably far out, and leave room to zoom out.
|
|
12
|
+
2. **Lay flat arrangements back into depth.** Orbits, a map, a board game: tilt the plane they lie
|
|
13
|
+
in back from the glass so it recedes, rather than looking straight down on it, where it reads
|
|
14
|
+
as flat circles. A chart, a figure or a building starts upright.
|
|
15
|
+
3. **Moving things: keep the forward swing small.** Objects that orbit, fly or swing toward the
|
|
16
|
+
viewer are where pop-out becomes uncomfortable. Keep their paths mostly at or behind the glass,
|
|
17
|
+
and fade them out as they come close to the viewer. A follow camera keeps its subject in frame
|
|
18
|
+
on fast moves.
|
|
19
|
+
4. **Worlds: keep convergence fixed, and keep near things away.** On a camera rig, set
|
|
20
|
+
`convergence` once for the scene, or change it slowly when the scene changes. Never set it from
|
|
21
|
+
where the user is looking: whatever they look at lands on the glass and goes flat. Never set it
|
|
22
|
+
to 0. Near geometry is where a world goes wrong: a window frame, a wall or a crossbar close to
|
|
23
|
+
the camera comes far out of the glass and ghosts. Keep the visitor's camera well clear of
|
|
24
|
+
walls and furniture, or lower `ipdFactor` while it is close.
|
|
25
|
+
5. **Low contrast between the subject and what is behind it.** The display's crosstalk correction
|
|
26
|
+
works only within the range of the image, so a bright edge on black ghosts first. A light or
|
|
27
|
+
mid-tone, low-contrast background works best: a real texture or a soft gradient, not generated
|
|
28
|
+
noise. A dark grey is the minimum, and pure black is the worst case; keep the subject's darkest
|
|
29
|
+
faces above the background. Soften high-contrast detail far from the glass in the colours and
|
|
30
|
+
materials, not with fog, since camera-distance fog follows the viewer's head.
|
|
31
|
+
6. **Text: flat HTML near the glass, at a readable size.** Text drawn in the 3D scene, especially
|
|
32
|
+
white on dark, ghosts. Put titles, labels and values in HTML, or keep in-scene labels close to
|
|
33
|
+
the glass, facing the viewer, and show them on hover. Never shrink text to hide ghosting.
|
|
34
|
+
7. **No coplanar surfaces.** Z-fighting (two faces at the same depth flickering through each
|
|
35
|
+
other) is the most common defect seen on panels, and in 3D it reads as shimmer at one depth.
|
|
36
|
+
Offset decals, shadows, shelf faces and picture frames from the surface they sit on. Keep
|
|
37
|
+
geometry solid: no paper-thin or floating floors, no holes in rocks, nothing clipping through
|
|
38
|
+
glass.
|
|
39
|
+
8. **No detail finer than a few pixels.** Thin lines, single-pixel stars and fine repeating
|
|
40
|
+
patterns alias and shimmer as the head moves. Draw them thicker, softer or as sprites, and size
|
|
41
|
+
textures to their surfaces instead of stretching them.
|
|
42
|
+
9. **A cover over the canvas must always come off.** If you hide the canvas until `firstWoven`,
|
|
43
|
+
also remove the cover after a few seconds and if your setup throws. Otherwise a missed signal
|
|
44
|
+
leaves the page with its UI and no scene.
|
|
45
|
+
|
|
46
|
+
The mechanics (one view per eye, CSS that breaks the weave, the 2D/3D switch) are in
|
|
47
|
+
[woven-canvas-rules.md](woven-canvas-rules.md) and [authoring-inline-3d.md](authoring-inline-3d.md),
|
|
48
|
+
which also gives the comfort zone in diopters and how the same content changes from a large
|
|
49
|
+
monitor to a phone. The samples show how the API works, not how a page should look.
|
|
@@ -0,0 +1,378 @@
|
|
|
1
|
+
# Woven canvas rules: never show a raw side-by-side frame
|
|
2
|
+
|
|
3
|
+
Companion to [`authoring-inline-3d.md`](authoring-inline-3d.md). That page explains what a woven
|
|
4
|
+
window is. This one covers the failure that shows up on the first real deployment: for a moment,
|
|
5
|
+
sometimes a full second, a 3D tile shows **two squeezed halves side by side** instead of 3D. It
|
|
6
|
+
happens most often right after a navigation.
|
|
7
|
+
|
|
8
|
+
The rules below come from a production kiosk that chased this symptom for a week. They were then
|
|
9
|
+
checked against the DisplayXR Browser's compositor source. Every rule gives the reason for it
|
|
10
|
+
and the SDK call that satisfies it.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 1. What the page is actually seeing
|
|
15
|
+
|
|
16
|
+
A woven window's backing buffer holds side-by-side (SBS) stereo ([the one
|
|
17
|
+
contract](authoring-inline-3d.md#the-one-contract-you-must-understand)). The browser weaves that
|
|
18
|
+
buffer onto the display **only once its compositor has *joined* the canvas**: it has matched the
|
|
19
|
+
rect your layer reports to the quad the canvas actually drew in this frame. Until the match
|
|
20
|
+
lands, the browser **withholds** the rect. The weave does not touch it, and what reaches the
|
|
21
|
+
screen is the page's own raster of the canvas, which is the raw SBS pair squeezed into the box.
|
|
22
|
+
That is the flash.
|
|
23
|
+
|
|
24
|
+
Four facts about the join matter to a page:
|
|
25
|
+
|
|
26
|
+
- **Constructing the layer is not the join.** `new XRDisplayLayer()` (what `addImage` /
|
|
27
|
+
`addVideo` / `addScene` do for you) registers the canvas. The browser's compositor joins it
|
|
28
|
+
later, on its own schedule.
|
|
29
|
+
- **A canvas that is fresh to the compositor takes longest.** The browser measured the window at
|
|
30
|
+
**0.4–1.2 s** on a same-document navigation, where the outgoing screen's canvas dies in the same
|
|
31
|
+
frame the incoming one is born. In the frames captured during that window, the incoming
|
|
32
|
+
canvas had **not yet committed a real frame**: its backing store was still the default
|
|
33
|
+
`300×150`. The compositor's view of the page was also still half the previous screen, so
|
|
34
|
+
nothing in the frame *was* the new tile. No browser-side fallback can draw a frame it does not
|
|
35
|
+
have. The page owns this case.
|
|
36
|
+
- **Nothing tells the page when the join lands.** Session frames, views and layer methods all
|
|
37
|
+
look the same before and after the join. Views come from the runtime's eye locate, which knows
|
|
38
|
+
nothing about the compositor. See [the browser ask](proposals/layer-joined-signal.md).
|
|
39
|
+
- **Your content arriving fast makes it worse.** The flash can only happen when real SBS pixels
|
|
40
|
+
are already in the canvas while the join is still missing. A cached asset lands in tens of
|
|
41
|
+
milliseconds, well inside the window. A slow asset usually lands after it. This is why the
|
|
42
|
+
flash tends to affect one screen and not another screen built the same way.
|
|
43
|
+
|
|
44
|
+
So the page cannot make the join land sooner. What it controls is **whether stereo pixels are
|
|
45
|
+
on screen while the join is missing**.
|
|
46
|
+
|
|
47
|
+
## 2. The rules
|
|
48
|
+
|
|
49
|
+
### 1. One inline-3D session per document
|
|
50
|
+
|
|
51
|
+
The browser's element-rect channel is one whole-widget list. Two live managers overwrite each
|
|
52
|
+
other's list every frame, and a tile that is missing from the list is never joined. Call
|
|
53
|
+
`createInline3D()` **once** and add every window to it. The SDK warns if a second one goes live.
|
|
54
|
+
Between routes, `close()` the old manager before you open the next one, or better, keep one
|
|
55
|
+
manager for the life of the document.
|
|
56
|
+
|
|
57
|
+
### 2. Never remount a woven canvas inside a screen
|
|
58
|
+
|
|
59
|
+
A new `<canvas>` element brings a new WebGL context, and a new subject for the compositor to join.
|
|
60
|
+
Remounting on a state change (a React key change, or a Strict Mode double mount) re-creates the
|
|
61
|
+
window from §1 on purpose, every time. It can also hand the second renderer a context the first
|
|
62
|
+
one already disposed of, which freezes silently. Hold the canvas in a ref your framework never
|
|
63
|
+
re-renders over, and change what is **in** it:
|
|
64
|
+
|
|
65
|
+
- **Images and video:** register once and redraw the source. `addImage(canvas, sourceCanvas)`
|
|
66
|
+
repaints from a canvas you own every frame
|
|
67
|
+
([motion, §1](authoring-motion-and-effects.md#what-to-do-instead)).
|
|
68
|
+
- **Scenes:** swap the content inside your `onFrame`, never the canvas.
|
|
69
|
+
- **Splats on the PlayCanvas backend:** `handle.setSource(src, { fadeMs, resetPose })`.
|
|
70
|
+
|
|
71
|
+
Calling `add*()` again on the same canvas is not a content swap. It closes the window's layer and
|
|
72
|
+
builds a new one (a fresh identity gap), and on `./splat` / `./model` it puts a second renderer on
|
|
73
|
+
the context. Since 1.38 the SDK warns about it, once per canvas, while the first registration is
|
|
74
|
+
live (not `remove()`d), then proceeds as before. To re-register on purpose, `remove()` first.
|
|
75
|
+
|
|
76
|
+
### 3. Prefer one persistent canvas for the whole app
|
|
77
|
+
|
|
78
|
+
A canvas that survives every route change is fresh to the compositor exactly once, at boot. The
|
|
79
|
+
window from §1 then exists once instead of on every navigation. This is the only rule that
|
|
80
|
+
*removes* the race instead of shortening it. The cost is architectural: a shared "stage" in the
|
|
81
|
+
root layout that each screen attaches content to, instead of each screen owning a canvas. If you
|
|
82
|
+
can pay it, rules 4 and 5 become boot-time concerns only.
|
|
83
|
+
|
|
84
|
+
### 4. If a canvas must be created after a navigation, commit it before you register it
|
|
85
|
+
|
|
86
|
+
The measured flash frames had the new canvas at its default size, with no real frame committed.
|
|
87
|
+
Give the compositor a settled canvas before you ask it to join one:
|
|
88
|
+
|
|
89
|
+
1. Put the canvas in the DOM with its final CSS box and **`will-change: transform` from its first
|
|
90
|
+
paint**, in your stylesheet. The SDK sets `will-change: transform; transform: translateZ(0)`
|
|
91
|
+
at registration, which is one frame too late for a canvas's first appearance.
|
|
92
|
+
2. For `addScene`, **size the backing store yourself first**: 2× the box's aspect ratio, as the
|
|
93
|
+
buffer will be while woven. The SDK never sizes a scene canvas. `addImage` / `addVideo` size
|
|
94
|
+
theirs on activation.
|
|
95
|
+
3. Wait **at least two animation frames**, then call `add*()`.
|
|
96
|
+
|
|
97
|
+
This **shortens** the window. It does not close it: a page that deferred registration by four
|
|
98
|
+
frames still produced withheld frames on the browser's test rig. Rule 5 is the one that
|
|
99
|
+
guarantees the result.
|
|
100
|
+
|
|
101
|
+
### 5. Keep the canvas covered until `handle.firstWoven` resolves
|
|
102
|
+
|
|
103
|
+
Never let stereo pixels reach the screen before the first woven frame. Stack a cover over the
|
|
104
|
+
canvas: the poster, or an opaque block of the page background.
|
|
105
|
+
|
|
106
|
+
- **On top of the canvas, never behind it.** A canvas with opaque pixels hides whatever sits
|
|
107
|
+
underneath, whatever that layer's own opacity.
|
|
108
|
+
- **A hard cut, never a fade.** A congruent sibling fading over the tile is its own version of
|
|
109
|
+
the flash: a partial-opacity plate the size of the tile is exactly what rule 7 forbids.
|
|
110
|
+
- **Released on `handle.firstWoven`, not on first paint, and not on your own timer.**
|
|
111
|
+
|
|
112
|
+
```js
|
|
113
|
+
const handle = wall.addScene(canvas, onFrame); // or addImage / addVideo / addSplat / addModel
|
|
114
|
+
await Promise.all([contentReady, handle.firstWoven]); // your load + the SDK's woven signal
|
|
115
|
+
poster.remove(); // cut, never fade
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
`handle.firstWoven` resolves **once and never rejects** with `{ woven, confirmed, reason, ms }`:
|
|
119
|
+
|
|
120
|
+
| result | meaning | what to do |
|
|
121
|
+
|---|---|---|
|
|
122
|
+
| `woven: true`, `reason: 'woven'`, `confirmed: true` | the browser reported the layer woven (`XRDisplayLayer.wovenState`), after a real stereo frame | drop the cover |
|
|
123
|
+
| `woven: true`, `reason: 'hold-elapsed'` | a browser without that report: the window has drawn a real stereo frame on a layer that has existed for `firstWovenHoldMs` (default **1200**, the browser's measured worst case) | drop the cover |
|
|
124
|
+
| `woven: false`, `confirmed: true`, `reason` a withheld token (or `'pending'`) | the browser kept reporting the layer withheld for the whole safety cap (four holds, at least 4.8 s). The canvas is not taken flat | drop the cover, then read the token in §3: it names the page fix |
|
|
125
|
+
| `woven: false`, `reason: 'layer-failed'` / `'session-ended'` / `'removed'` | this window will not weave. An image or video canvas has already been painted flat, and a scene's `onLayerLost` has already run | drop the cover onto the 2D fallback |
|
|
126
|
+
| `woven: false`, `reason: 'unsupported'` (`addSplat` / `addModel` only) | no inline-3D session: the viewer is on its mono path | same |
|
|
127
|
+
|
|
128
|
+
**`confirmed` says which of the two you got.** A DisplayXR Browser build with the join report
|
|
129
|
+
(`XRDisplayLayer.wovenState`, Windows, behind a flag; detected with `'wovenState' in
|
|
130
|
+
XRDisplayLayer.prototype`) settles `firstWoven` on the report: `confirmed: true`, no hold. On
|
|
131
|
+
every other browser `firstWoven` is the documented worst case, measured by the SDK so that each
|
|
132
|
+
page does not measure it separately (`confirmed: false`). **The page code is the same on both**,
|
|
133
|
+
which is the reason to code against it rather than keep a local `setTimeout`. To see the report
|
|
134
|
+
yourself, read `handle.wovenState` (`'pending'`, `'woven'`, `'withheld'`, or `null` where absent)
|
|
135
|
+
and `handle.withheldReason`. Background: [`proposals/layer-joined-signal.md`](proposals/layer-joined-signal.md).
|
|
136
|
+
|
|
137
|
+
What counts as a "real stereo frame": for a scene, an `onFrame` that received two or more views
|
|
138
|
+
and did not throw. A frame with a short view list is the load fallback, and a frame that threw
|
|
139
|
+
drew nothing. For an image, an SBS paint of a loaded image. For a video, an SBS paint of a
|
|
140
|
+
`<video>` that has a current frame. A lazy tile that scrolls away before settling starts the
|
|
141
|
+
hold again on its next layer. `onFirstWoven(cb)` is the callback form.
|
|
142
|
+
|
|
143
|
+
Lower `firstWovenHoldMs` only for a canvas you know is not fresh to the compositor, for example
|
|
144
|
+
the persistent stage from rule 3 after its first ever reveal. `0` means "the first stereo frame".
|
|
145
|
+
|
|
146
|
+
**On a browser without draw-order occlusion** ([`inline3dOcclusionByDrawOrder()`](authoring-inline-3d.md#asking-which-mechanism-you-are-on)
|
|
147
|
+
is `false`), a cover over a *joined* tile is woven along with it. There, the cover looks
|
|
148
|
+
lens-scrambled from the moment the join lands until it is released. Do not mark a full-tile cover
|
|
149
|
+
`data-inline3d-overlay`: the SDK refuses a full-tile overlay with a warning on those browsers
|
|
150
|
+
(rule 8), and ignores it on draw-order browsers.
|
|
151
|
+
|
|
152
|
+
### 6. Hard-cut between screens: never have two woven canvases crossfading
|
|
153
|
+
|
|
154
|
+
A transition that blends the outgoing screen's canvas into the incoming one, by animating
|
|
155
|
+
`opacity` on either canvas or on their wrappers, breaks rule 7 on **both** canvases for the whole
|
|
156
|
+
fade. It also creates a case the compositor refuses outright. A canvas inside a render surface
|
|
157
|
+
can only be matched by shape, as the one 2:1 resource covering its rect. Two such candidates in
|
|
158
|
+
one region are **refused as ambiguous**, and neither is drawn from. Cut between screens. To blend
|
|
159
|
+
two pictures, blend them **inside one canvas**
|
|
160
|
+
([motion, §3 Crossfade](authoring-motion-and-effects.md#crossfade)).
|
|
161
|
+
|
|
162
|
+
Many woven canvases in one frame (a gallery wall) are fine. Each one joins by its own identity.
|
|
163
|
+
|
|
164
|
+
### 7. No CSS effects on a woven canvas or on any of its ancestors
|
|
165
|
+
|
|
166
|
+
`filter`, `opacity < 1`, `mask`, `clip-path`, `mix-blend-mode`, `backdrop-filter`: any property
|
|
167
|
+
that makes the browser give an element **its own render surface**. When the canvas or one of its
|
|
168
|
+
ancestors has one, the canvas is drawn into that surface instead of the page's root pass. Its
|
|
169
|
+
identity does not survive that, so the direct join misses. The best the browser can then do is
|
|
170
|
+
draw the tile **flat** (one eye, no 3D) in place, and sometimes it cannot find the tile at all.
|
|
171
|
+
The same goes for `border-radius`, `box-shadow` and `border` on the canvas: they are woven with
|
|
172
|
+
the packed pair and come out lopsided after the eye split. Use the SDK's options instead:
|
|
173
|
+
|
|
174
|
+
- rounded corners: `cornerRadius` on `addImage` / `addVideo`, or clip each viewport in a scene
|
|
175
|
+
([rounded corners](authoring-inline-3d.md#rounded-corners));
|
|
176
|
+
- a soft edge: `feather`;
|
|
177
|
+
- anything else: draw it into the canvas, per eye.
|
|
178
|
+
|
|
179
|
+
Scrolling and a *static* ancestor translate are fine. Animating the tile's geometry is a separate
|
|
180
|
+
rule ([motion, §1](authoring-motion-and-effects.md#1-why-a-woven-tile-must-not-move)).
|
|
181
|
+
|
|
182
|
+
### 8. Chrome over a woven canvas is a partial region
|
|
183
|
+
|
|
184
|
+
A caption band, a badge, a menu: on a browser with draw-order occlusion these need nothing
|
|
185
|
+
([2D over 3D](authoring-inline-3d.md#2d-over-3d--draw-order-occlusion)). On an older browser, mark
|
|
186
|
+
them `data-inline3d-overlay` (or call `handle.exclude(el)`), and keep them **smaller than the
|
|
187
|
+
tile**. An overlay congruent with the tile matches the canvas's own quad, so the SDK refuses it
|
|
188
|
+
with a warning rather than destroy the tile. Hide an overlay with `display: none`, never
|
|
189
|
+
`opacity: 0`. The declaration costs nothing on a draw-order browser, so one page works on both
|
|
190
|
+
generations.
|
|
191
|
+
|
|
192
|
+
### 9. Nothing with a background sits above the woven canvas; black goes under it
|
|
193
|
+
|
|
194
|
+
A section, wrapper or later sibling with an opaque background, painted over the canvas, hides the
|
|
195
|
+
tile while its content keeps running (for a video: sound, no picture). To put black around a
|
|
196
|
+
tile, give it to an element **behind** the canvas. A cover you mean to hide the tile with (rule 5,
|
|
197
|
+
rule 11) is the one deliberate exception, and it comes off on a signal.
|
|
198
|
+
|
|
199
|
+
### 10. No `backdrop-filter` on anything drawn over the weave
|
|
200
|
+
|
|
201
|
+
Rule 7 covers the canvas and its ancestors. This one covers the **siblings on top of it**: a
|
|
202
|
+
control bar, a caption, a badge, a menu. `backdrop-filter` is defined as a function of what is
|
|
203
|
+
behind the element, and behind it is the interlaced weave. A blur reads the interlaced pixels
|
|
204
|
+
and smears them, and the 3D image is disturbed across the element's **whole box**, not only
|
|
205
|
+
where the element is opaque (seen on the panel, 24 September). Use a plain translucent tint
|
|
206
|
+
(`background: rgba(0, 0, 0, 0.55)`) instead. The SDK's own player chrome follows this rule.
|
|
207
|
+
|
|
208
|
+
### 11. Treat a resize as a fresh-canvas moment
|
|
209
|
+
|
|
210
|
+
Resizing a woven canvas's rect re-registers it with the compositor, and the new rect can go
|
|
211
|
+
through the same **0.4–1.2 s** identity gap as a new canvas (§1): until the join lands, the
|
|
212
|
+
screen may show the page's own raster, which is the squeezed side-by-side pair (the panel A/B
|
|
213
|
+
below did not see one, so treat the cover as insurance, not a certainty). Entering or leaving
|
|
214
|
+
fullscreen, a responsive reflow that changes the tile's size, and a layout that swaps a small
|
|
215
|
+
tile for a large one all do this. Cover the canvas across the change exactly as in rule 5 (on
|
|
216
|
+
top, hard cut), and release on **`handle.rewoven()`**:
|
|
217
|
+
|
|
218
|
+
```js
|
|
219
|
+
cover.hidden = false; // before the rect changes
|
|
220
|
+
await el.requestFullscreen();
|
|
221
|
+
await handle.rewoven(); // a stereo frame on the moved rect, plus the hold
|
|
222
|
+
cover.hidden = true; // cut, never fade
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
`rewoven()` is `firstWoven` measured from the call, with the same result shape. A change of the
|
|
226
|
+
canvas's CSS size, devicePixelRatio or page position while it is pending restarts the hold
|
|
227
|
+
(checked every frame, for every window kind), so a resize or a move that settles over several
|
|
228
|
+
frames is covered until the last one. Position is measured in page coordinates, so scrolling the
|
|
229
|
+
document is not a move (since 1.38; before, a move without a resize was not detected at all). So
|
|
230
|
+
that a size that never stops animating cannot hold the cover up for good, it settles anyway
|
|
231
|
+
(`reason: 'hold-capped'`) four holds after the call. Before the first join it is `firstWoven`
|
|
232
|
+
itself. The SDK's player (`addPlayer`) does this for its own fullscreen button with
|
|
233
|
+
`fullscreenCover: true`.
|
|
234
|
+
|
|
235
|
+
**The SDK covers rect changes for you on `./splat` and `./model`** (1.38, `rectCover: 'auto'` by
|
|
236
|
+
default there). After the first join, a move or resize of the canvas raises a cover over it on
|
|
237
|
+
the frame the change is seen: a solid color with the last frame's left eye, cover-fit. The cover
|
|
238
|
+
comes down when `rewoven()` settles. Pass `rectCover: 'off'` if the page runs its own cover. On
|
|
239
|
+
core windows (`addScene` / `addImage` / `addVideo`) the same primitive is opt-in: pass
|
|
240
|
+
`rectCover: 'auto'` (or `{ color, snapshot }`), or keep your own cover and release it on
|
|
241
|
+
`handle.rewoven()`. A move is seen at the next session frame, so a move your page makes after that
|
|
242
|
+
frame's callback can show for one frame. Move the canvas from an event handler or before the frame,
|
|
243
|
+
not from a later animation callback.
|
|
244
|
+
|
|
245
|
+
**Measured on the Leia panel (26 September, browser test build with patch 0195, blind A/B, one
|
|
246
|
+
observer):** entering and leaving the player's fullscreen showed **no** raw side-by-side pair,
|
|
247
|
+
with or without the cover. Without it, leaving fullscreen showed the picture at once and then a
|
|
248
|
+
flash of the **whole browser viewport** about a second later. With it, the picture was hidden for
|
|
249
|
+
~1.2 s and the flash was still there. The flash is browser-side and wider than any tile, so no
|
|
250
|
+
page cover can hide it. That is why the player's cover is opt-in. Keep the rule for a page whose
|
|
251
|
+
resize does show the pair.
|
|
252
|
+
|
|
253
|
+
A scroll is not a resize. The canvas keeps its size and its identity, and nothing needs covering.
|
|
254
|
+
|
|
255
|
+
### Two rules that keep a joined canvas joined
|
|
256
|
+
|
|
257
|
+
- **Redraw every frame.** A canvas that is not redrawn can drop out of the frame the compositor
|
|
258
|
+
aggregates. The weave then reads a stale sub-rect and the tile smears. The SDK repaints
|
|
259
|
+
`addImage` / `addVideo` windows every frame, including a buffering video (it re-commits the
|
|
260
|
+
last frame). A scene must draw in every `onFrame`, and on a frame it cannot draw it must
|
|
261
|
+
**replay the last good one** rather than skip ([validate before you
|
|
262
|
+
clear](authoring-inline-3d.md#3-live-scene-threejs--webgl--addscenecanvas-onframe-opts)).
|
|
263
|
+
`./viewer`, `./splat` and `./model` already do this, and since 1.38 they also draw on the frames
|
|
264
|
+
before the first good one: the mono camera into both halves, the subject if it is loaded, or a
|
|
265
|
+
cleared frame if it is not. The only undrawn frames left are the ones before the engine has
|
|
266
|
+
created its context. The SDK cannot enforce this for an `addScene` window your page draws: draw
|
|
267
|
+
something on every `onFrame`, even with no content yet. When you keep the last good frame,
|
|
268
|
+
copy the viewport's and view's fields **by name**: `XRViewport` and `XRView` fields are
|
|
269
|
+
prototype getters, so `Object.assign` and spread copy nothing and the replay draws into a 0×0
|
|
270
|
+
viewport, which weaves black ([details](authoring-inline-3d.md#3-live-scene-threejs--webgl--addscenecanvas-onframe-opts)).
|
|
271
|
+
- **Known limitation: a transparent canvas weaves black.** A cleared (fully transparent) woven
|
|
272
|
+
frame comes out black on the panel, not see-through: a browser bug (browser-pvt #255), not
|
|
273
|
+
the SDK's. Until it is fixed, a frame with nothing to show is drawn but dark. If the page
|
|
274
|
+
behind the tile is not black, cover the tile until its content is in (rule 5).
|
|
275
|
+
- **Large scene canvases: consider `preserveDrawingBuffer: true`.** On older DisplayXR Browser
|
|
276
|
+
builds, the weave's zero-copy read of a full-window WebGL canvas can race the page's next
|
|
277
|
+
write, and the frame is dropped. Keep the SBS buffer no wider than the panel, and try
|
|
278
|
+
`preserveDrawingBuffer` on the renderer (`addSplat(…, { engine: 'playcanvas',
|
|
279
|
+
preserveDrawingBuffer: true })` on the PlayCanvas backend). See
|
|
280
|
+
[porting pitfall 26](porting-three-js-apps.md#9-pitfalls-register).
|
|
281
|
+
|
|
282
|
+
## 3. Verifying a new screen on hardware
|
|
283
|
+
|
|
284
|
+
A 2D browser can check your call order (the tests in this repo drive a stubbed `XRDisplayLayer`
|
|
285
|
+
for exactly that reason). It cannot check the join. On a DisplayXR display:
|
|
286
|
+
|
|
287
|
+
1. **Stereo is arriving.** `handle.stats()` reports `{ frames, monoFrames }` per scene window. A
|
|
288
|
+
high `monoFrames` share means the session is falling back under load.
|
|
289
|
+
2. **When the page thinks it is safe.** Log `(await handle.firstWoven).ms` and the time you
|
|
290
|
+
registered. Log per-frame canvas size for the first two seconds as well, to see when the
|
|
291
|
+
backing store actually became 2:1.
|
|
292
|
+
3. **What the browser actually did.** Launch the DisplayXR Browser with `--enable-logging` (add
|
|
293
|
+
`--v=1` for the render-surface lines below), reproduce, and read `chrome_debug.log` in the
|
|
294
|
+
browser's user-data directory:
|
|
295
|
+
|
|
296
|
+
```
|
|
297
|
+
[DisplayXR] inline-3D withheld N of M weave rects: … ids=[<token>=<why>@<rect> …]
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
The line is **throttled**: the first three occurrences, then one in every 300. A missing line
|
|
301
|
+
on a given frame proves nothing. Visible tiles are named, up to 16, and off-viewport tiles are
|
|
302
|
+
reduced to `+N offscreen`. What `<why>` means:
|
|
303
|
+
|
|
304
|
+
| `why` | what it means | whose fix |
|
|
305
|
+
|---|---|---|
|
|
306
|
+
| `no-identity` | the canvas's rect reached the browser without the identity the join needs. This is the raw-pair flash from §1: the page's own pixels are on screen | page: rules 3–5 |
|
|
307
|
+
| `no-quad` | the canvas drew nothing the compositor can see: it is not drawing, or it is occluded, or it is inside a render surface the browser could not trace | page: rule 7, or check that the canvas is drawing |
|
|
308
|
+
| `cross-pass:mono`, `cross-pass:mono(cover,2:1)` | the canvas is inside a render surface (rule 7). The browser found it and draws it **flat in place**: no raw pair, but no 3D either | page: remove the ancestor effect |
|
|
309
|
+
| `no-join` | nothing joined this rect. Seen at `@0,0 0x0` for lazy tiles pre-armed off screen, where it is harmless | page, if the rect is on screen |
|
|
310
|
+
| `resolve-dropped` | the canvas joined, but its GPU resource could not be resolved this frame | browser |
|
|
311
|
+
|
|
312
|
+
With `--v=1`, a separate `inline-3D cross-pass COVER … REFUSED ambiguous` line means rule 6:
|
|
313
|
+
two 2:1 candidates in one render-surface region. Remove one of them.
|
|
314
|
+
4. **Which browser build has which fix: read the device, not a version string.** A version
|
|
315
|
+
number does not tell you which compositor changes a build carries. The `why` token on the
|
|
316
|
+
unit tells you what that build did. Browser release notes that list such changes are tracked
|
|
317
|
+
separately.
|
|
318
|
+
|
|
319
|
+
## 4. What stays browser-owned
|
|
320
|
+
|
|
321
|
+
- **The join window itself.** Its 0.4–1.2 s length on a fresh canvas is compositor state that no
|
|
322
|
+
page-side sequencing controls. Rules 3–5 exist *because* of it. A browser-side mono fallback
|
|
323
|
+
for this exact case was built and measured, and it did not ship: in the measured frames nothing
|
|
324
|
+
in the compositor's frame was the incoming canvas, so there was nothing to draw flat from.
|
|
325
|
+
- **Reporting the join.** Only the browser knows when a canvas has been joined. Until it reports
|
|
326
|
+
that, `handle.firstWoven` is a timer with a stable contract. The request, and the shape it
|
|
327
|
+
asks for, is [`proposals/layer-joined-signal.md`](proposals/layer-joined-signal.md).
|
|
328
|
+
|
|
329
|
+
## 5. Pitfalls from production (September 2026)
|
|
330
|
+
|
|
331
|
+
Found on hardware in a production photo app after the rules above were written. Each one looked
|
|
332
|
+
like a runtime or browser bug on the panel and turned out to be the page.
|
|
333
|
+
|
|
334
|
+
- **No `backdrop-filter` on chrome that sits over the weave.** Rule 7 covers effects on the canvas
|
|
335
|
+
and its ancestors. A frosted bar or button placed *over* a woven tile is a different element, but
|
|
336
|
+
on a lenticular laptop panel it visibly disturbs the 3D image across that element's whole rect.
|
|
337
|
+
Use a plain translucent tint instead (a solid `rgba()` background). Keep an A/B switch for the
|
|
338
|
+
blurred variant if the design wants it back, and judge it on the panel, not on a 2D monitor.
|
|
339
|
+
- **Nothing opaque above the woven canvas.** Stacking order still applies: an element with an
|
|
340
|
+
opaque background painted *above* the canvas hides it, and on a draw-order browser that is
|
|
341
|
+
exactly what you get. The classic case is a full-bleed video plane under a page `<main>` with
|
|
342
|
+
`background: #000`: the sound plays and the picture is black. Dark or blurred backdrops go
|
|
343
|
+
*under* the canvas, as their own element.
|
|
344
|
+
- **Don't mount the woven canvas on 2D-only routes.** A route with no 3D content (an upload page, a
|
|
345
|
+
settings screen) should not host the stage at all. Where the woven canvas has to stay mounted
|
|
346
|
+
(one session, never remount: rules 1–3), request the 1-view mode for the flat screen instead:
|
|
347
|
+
`wall.setStereoEnabled(false)`, then `true` again before the next stereo content appears. The
|
|
348
|
+
1-view mode is real full-resolution 2D, not a flattened weave: the panel's 3D optics switch off,
|
|
349
|
+
nothing is woven, and DOM text is native-resolution. Confirm with
|
|
350
|
+
`wall.hardwareDisplayState === '2d'`. The SDK's eased mode switch ramps disparity out before the
|
|
351
|
+
panel flips and in only after it reports 3D. The mode is display-wide: every woven tile goes flat
|
|
352
|
+
while it is in force.
|
|
353
|
+
- **Keep the frames after `setSource` resolves free of per-splat CPU work.** A slideshow ran 105
|
|
354
|
+
`pick()` calls (24 to plan an animation path, 81 for a diagnostic depth grid) right after every
|
|
355
|
+
swap. On the Windows panel that was a 1.3 s main-thread stall at the end of every transition,
|
|
356
|
+
which reads on a tracked panel as "head tracking stops": no frames, so the image cannot follow
|
|
357
|
+
the head. SDK 1.22.0 made pick bursts cheap (a pick index), but the rule stands: defer or spread
|
|
358
|
+
work after a swap, and gate diagnostics behind a flag. `?dxrdiag=1` names what is on the main
|
|
359
|
+
thread per transition phase (`picks`, `gl`, `afterSettleTaskMs`, `longFrames`), so check it
|
|
360
|
+
before blaming the SDK or the browser.
|
|
361
|
+
- **Don't hold or blend rigs yourself around a transition.** Since SDK 1.24.0 each photo in a
|
|
362
|
+
two-splat transition is drawn through its own camera rig for the whole transition, including the
|
|
363
|
+
frame or two before the runtime delivers views for a newly declared rig. Declare each photo's rig
|
|
364
|
+
(or let the SOG camera block do it) and let `setSource` handle the swap: holding the old rig or
|
|
365
|
+
easing between the two moves one of the photos.
|
|
366
|
+
- **Don't assume `canvas.width` is the buffer.** A WebGL canvas larger than the device's limit is
|
|
367
|
+
clamped by the browser with no error: `canvas.width` keeps the value you set while
|
|
368
|
+
`gl.drawingBufferWidth` is smaller. An Android 3D tablet (Adreno 740) reports `MAX_TEXTURE_SIZE`
|
|
369
|
+
4096 where desktops report 16384 (the Chromium driver workaround tracked in browser-pvt#187), so a
|
|
370
|
+
full-screen SBS canvas at 5120 wide came out 4096 wide. `XRDisplayLayer.getViewport()` splits
|
|
371
|
+
`canvas.width`, so the eye boundary landed at 62.5% of the buffer while the weave split it at 50%:
|
|
372
|
+
a large double image. `./viewer`, `./splat`, `./model` and `addImage` / `addVideo` clamp the store
|
|
373
|
+
before sizing it (both axes by one factor) and warn once; `handle.renderScale` then reports the
|
|
374
|
+
scale in force. An `addScene` page sizes its own canvas: keep 2 × eye width and the height within
|
|
375
|
+
min(`MAX_TEXTURE_SIZE`, `MAX_RENDERBUFFER_SIZE`, `MAX_VIEWPORT_DIMS`), and draw into
|
|
376
|
+
`gl.drawingBufferWidth/Height` rather than `canvas.width/2`. The SDK warns once if it sees the
|
|
377
|
+
browser clamp a scene canvas. Test a large tile on Chrome with
|
|
378
|
+
`--webgl_or_caps_max_texture_size_limit_4096` to reproduce the tablet's limit on a desktop.
|
package/index.d.ts
CHANGED
|
@@ -125,8 +125,13 @@ export interface FirstWovenResult {
|
|
|
125
125
|
* `verticalFov`, offsets the eyes, and skews each frustum so `convergenceDiopters` lands on
|
|
126
126
|
* the zero-disparity plane.
|
|
127
127
|
*
|
|
128
|
-
* Every field is optional; the runtime **clamps**
|
|
129
|
-
*
|
|
128
|
+
* Every field is optional; the runtime **clamps** a value outside a field's sanity range (once,
|
|
129
|
+
* with a warning — e.g. `convergenceDiopters` to `[0, 20]`, `verticalFov` to `[0.01, 3.13]` rad,
|
|
130
|
+
* `perspectiveFactor` to `[0.1, 10]`, ipd/parallax to `[0, 10000]`) rather than rejecting the rig.
|
|
131
|
+
* It does **not** enforce comfort: a camera rig's gain and a display rig's factors above 1 are
|
|
132
|
+
* applied as given. `handle.setViewRig` logs a one-time console warning for the usual mistakes
|
|
133
|
+
* (a camera rig converged at infinity, a `verticalFov` that looks like degrees, display-rig
|
|
134
|
+
* factors above 1) and still sends the rig unchanged. A descriptor applies per-locate, so animating one means sending
|
|
130
135
|
* new values each frame — see {@link TileHandle.setViewRig} for the one-frame latency that
|
|
131
136
|
* implies. Build one with `cameraRigFromCamera` / `displayRig` from
|
|
132
137
|
* `@displayxr/inline3d/three`, or by hand.
|
|
@@ -140,7 +145,7 @@ export interface XRViewRigInit {
|
|
|
140
145
|
orientation?: DOMPointInit;
|
|
141
146
|
/** **Display rig.** Metres of virtual display; `m2v = this / the element's physical height`. */
|
|
142
147
|
virtualDisplayHeight?: number;
|
|
143
|
-
/** Eye separation — display rig: relative `[0,1]
|
|
148
|
+
/** Eye separation — display rig: relative, 1 = correct, `[0,1]` a comfort dial (above 1 exaggerates); camera rig: absolute `>= 0`. */
|
|
144
149
|
ipdFactor?: number;
|
|
145
150
|
/** Head-tracking response — display rig: `[0,1]`; camera rig: absolute `>= 0`. */
|
|
146
151
|
parallaxFactor?: number;
|
|
@@ -204,6 +209,20 @@ export interface DisplayMetrics {
|
|
|
204
209
|
}
|
|
205
210
|
|
|
206
211
|
/** What {@link TileHandle.displayMetrics} fills in when the display cannot say. */
|
|
212
|
+
// The depth budget (also `@displayxr/inline3d/depth`): pure measurements, see depth.d.ts.
|
|
213
|
+
export {
|
|
214
|
+
DEPTH_EYE_SEPARATION_M,
|
|
215
|
+
DEPTH_DEFAULT_VIEWER_M,
|
|
216
|
+
disparityPct,
|
|
217
|
+
cameraRigDisparityPct,
|
|
218
|
+
depthBudgetFromBounds,
|
|
219
|
+
comfortZone,
|
|
220
|
+
comfortDepthFactor,
|
|
221
|
+
cameraRigComfortFactor,
|
|
222
|
+
withheldReasonHint,
|
|
223
|
+
} from './depth.js';
|
|
224
|
+
export type { DepthReport, DepthReportBasis, ComfortZoneName, ComfortZoneReport } from './depth.js';
|
|
225
|
+
|
|
207
226
|
export const DISPLAY_METRICS_DEFAULTS: Readonly<{
|
|
208
227
|
displaySizeM: readonly [number, number];
|
|
209
228
|
displayPixels: readonly [number, number];
|