@displayxr/inline3d 1.27.0 → 1.29.0

package/CHANGELOG.md

@@ -5,6 +5,230 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
 `./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
 which tier they touch, because that is what tells you whether an upgrade can move your pixels.
 
+## 1.29.0 — 2026-09-30
+
+Touches the **core** (additive: `sharedInline3D()`, `handle.rewoven()`) and the **preview tier**
+(`./call` — RFC 0003 C1, the one-line path; `./player`, `./splat` `engine: 'playcanvas'`). One
+**removal** in the preview tier: `peerjsCloud`. Otherwise additive; no change for a page that does
+not call the new API.
+
+### `./call` — the one-line path (RFC 0003 §1, phase C1)
+
+- **`mountCall(el, opts?)`** — `addCall` with every argument optional: the wall is the document's
+  shared manager (`sharedInline3D()`, below), signalling defaults to the hosted server, the camera
+  to `'auto'`, the SDK chrome to on. `addCall(wall, container, opts)` stays as the explicit form.
+  A call's session recovery now re-opens through the shared manager too, so it joins a wall the
+  page re-created rather than opening a second session.
+- **`<dxr-call>`** — the same thing as markup, registered by importing `./call` (or `./call/full`,
+  or the CDN bundle): mounts on connect, `leave()`s on disconnect (a DOM move IS a teardown —
+  woven-canvas rule 2), re-dispatches every call event as a bubbling, composed `CustomEvent`
+  named `dxr-call:<event>` with the payload in `detail`, and exposes the handle as `el.call`
+  (plus a `ready` promise). Attributes are the string-typed options: `room`, `signaling` (a
+  URL), `key`, `camera`, `layout`, `accent`, `max-peers`, `no-ui`, `auto-join`, `mono3d="off"`,
+  `no-audio`, `no-self-view`, `no-auto-converge`, `tile-aspect`, `invite-base`, `browser-url`,
+  `debug`; anything else is `el.options = { … }`, set before connect (options win). Light DOM, no
+  shadow root, **no globals** (RFC 0003 Decision 11). `auto-join` joins during the mount, so the
+  element replays `joined` and one `peer` per participant already in the room once it can
+  subscribe. Exported: `DxrCallElement`, `defineCallElement`, `attrsToOpts`, `CALL_EVENT_PREFIX`.
+- **`@displayxr/inline3d/call/full`** — `./call` with `mono3D` defaulted to a **statically
+  imported** `lift`, the one shape a bundler can follow; importing it also points `<dxr-call>` at
+  that `mountCall`. `liftBundled` says whether this copy of the SDK carries the module. **Until
+  `./lift` lands on `main`, `js/lift/index.js` is a placeholder** (`lift === null`): the entry
+  builds and behaves like `./call`, and `liftBundled` is `false`.
+- **CDN bundle `dist/call.js`** — one pre-bundled ESM file (core + call + element, ~124 KB min,
+  no three/PlayCanvas), built by `npm run build:dist` (esbuild) and by `prepack` into every
+  published tarball, so jsDelivr serves it as `https://cdn.jsdelivr.net/npm/@displayxr/inline3d@1/dist/call.js`.
+  `<script type="module">` only; lift is a lazy sibling chunk (`dist/lift.js`) once the module
+  is present. `dist/` is not committed.
+- **`warning` event, code `'lift-not-bundled'`** — a 2D participant is on a 3D display, `mono3D`
+  is `'auto'`, and the lift module could not be imported from this build (a bundler that could
+  not follow the computed import, or a copy without `./lift`). Emitted once per call with one
+  `console.warn` naming the fix (`mono3D: lift` or `./call/full`); the lobby reads "2D→3D
+  unavailable in this build". Previously a silent flat tile and an info line. A display that
+  honestly cannot lift is unchanged (`handle.mono3D.reason`).
+- **`key`** — `dxrSignaling(url, { key })` / `CallOptions.key` / `<dxr-call key>`: a publishable
+  key for the hosted service (RFC 0003 §5a), sent as a query parameter on connect. Accepted and
+  forwarded now; the server ignores it until C3.
+- **Removed: `peerjsCloud`** (RFC 0003 Decision 12 — no known external user, so no warning
+  release). The demo adapter over the public PeerJS broker lives on as
+  `samples/call/peerjs-cloud.js`; the sample's `?signal=peerjs` still works. `dxrSignaling()`
+  (the hosted server) has been the default since 1.28.
+- **Types:** `mountCall`, `CallOptions.wall` / `.key`, `CallEvents.warning`, `DxrCallElement`,
+  `attrsToOpts`, `defineCallElement`, `DxrCallEventMap` (+ `HTMLElementTagNameMap` /
+  `HTMLElementEventMap` augmentations), `call-full.d.ts`.
+- **Tests:** element unit tests (attribute→option table, events, disconnect = leave); a
+  headless Chrome ↔ Chrome end-to-end run of `<dxr-call>` (source and CDN bundle) over the
+  hosted server (`npm run test:e2e`); Vite/webpack/esbuild smoke builds of `./call/full` and
+  `./call` that also run the bundles (`npm run test:bundlers`). Neither is part of `npm test`.
+
+### Core — additive
+
+- **`sharedInline3D(opts?)`** — the document's shared manager: the live `createInline3D()`
+  result if one exists, else a new one (concurrent callers share one creation; an unsupported
+  browser is never cached). Woven-canvas rule 1 as an API: a module that needs a wall it did not
+  create (`mountCall`, `<dxr-call>`) uses this and never opens a second session next to the
+  page's own; a page that made its wall first gets that same wall back here. Semver-covered.
+
+- **`./splat` `setVideo` crossfade, band and on-screen rect** (#105, RFC 0001 A5.1-A5.3,
+  `engine: 'playcanvas'`). `transition: 'crossfade'` (`durationMs` 600, `easing`
+  'easeInOutSine', the player's defaults) dissolves on the GPU from the video already on the plane:
+  its quad is frozen and drawn blended over the incoming one, the buffer opaque throughout; from the
+  splat and on `setVideo(null)` it stays a cut. `band: 2.39 | '2.39:1'` fits the picture into a
+  centred letterbox slot (`cover` fills the band and crops each eye about its centre). The result
+  gains `transition`, `band`, `rect` (the picture in CSS px, canvas-relative, clipped) and
+  `onRectChange(cb)`. `attachPlayer` now crossfades and takes `band` (controls in the bars) through
+  these instead of cutting / ignoring them. **Behaviour change for `attachPlayer(…, { skin: 'call' })`:**
+  that skin defaults `band` to 2.39, which surface mode used to ignore; the video now plays in a
+  2.39 letterbox. Pass `band: null` to keep the full tile.
+- **`handle.rewoven()`** — `firstWoven`, measured from the call, for a canvas that is already woven
+  but whose rect is about to change (fullscreen, a layout resize). Same `{ woven, confirmed,
+  reason, ms }` shape, never rejects; a change of the canvas's CSS size or devicePixelRatio while
+  pending restarts the hold (checked every frame, every window kind), and it settles anyway with
+  the new reason `'hold-capped'` four holds after the call if the size never stops changing;
+  before the first join it is `firstWoven`. Woven-canvas rule 11.
+- **Player: `fullscreenCover`** (opt-in, default false). Entering or leaving fullscreen puts an
+  opaque cover over the canvas (on top, hard cut, below the controls) until `rewoven()` settles.
+  Off by default because a blind A/B on the Leia panel showed no raw side-by-side pair without it
+  and ~1.2 s of no picture with it; the flash that did show is browser-wide (rule 11).
+- **Docs:** woven-canvas rules 9–11 (black goes under the canvas — previously only in the
+  authoring guide; no `backdrop-filter` over the weave; a resize is a fresh-canvas moment). The
+  authoring guide's short list now matches, 1–11.
+- **`attachPlayer(splat, src, opts)`** — surface mode (RFC 0001 Addendum A): the player's
+  transport, playlist and events on an existing PlayCanvas splat handle's `setVideo`, with no
+  canvas of its own. `detach()` gives the slot back (`setVideo(null)`: scene, pose, lens and rig
+  as they were); `'detached'` `{ reason: 'superseded' | 'released' }` when something else takes or
+  empties the slot. Cut transitions only for now; `fullscreen` defaults off. Panel-checked on one
+  persistent canvas: attach, next/back, detach, scene state identical after.
+- **`addPlayer` refactor:** one player core on a surface adapter. No behaviour change.
+- **`./splat`:** read-only `handle.videoElement` (the `<video>` the plane shows, or null) and
+  `handle.canvas`.
+- **Types:** `attachPlayer`, `AttachPlayerOptions`, `AttachedPlayerHandle`, `PlayerDetachedEvent`,
+  `'detached'` in `PlayerEvent`; `SplatHandle.videoElement` / `.canvas`.
+
+## 1.28.0 — 2026-09-27
+
+Touches the **preview tier** (a new subpath, `./call`; the SBS store clamp below) and two **core**
+fixes: image/video backing buffers are capped at the source's resolution, and every SBS store is
+clamped to the device's GL limits.
+
+### Fixed — core + preview: SBS backing stores clamped to the device's GL limits
+
+- A woven canvas wider than the device's WebGL limit was **silently** clamped by the browser, and
+  the SDK still laid the eyes out from `canvas.width / 2`. On an Android 3D tablet (Adreno 740,
+  DisplayXR Browser 154) `MAX_TEXTURE_SIZE` is 4096 (Chromium's
+  `webgl_or_caps_max_texture_size_limit_4096` workaround, browser-pvt#187) where Windows and Linux
+  report 16384. A full-screen SBS canvas at `renderScale` 1 asked for 5120×1348, got a 4096-wide
+  drawing buffer, and put the eye boundary at 62.5% of it while the weave split it at 50%: a large
+  double image on the panel, nothing logged.
+- Every path that sizes an SBS store now clamps the request **before** setting `canvas.width/height`,
+  so 2 × eye width and the height stay within min(`MAX_TEXTURE_SIZE`, `MAX_RENDERBUFFER_SIZE`,
+  `MAX_VIEWPORT_DIMS`). Both axes shrink by one factor, so the eye keeps its aspect and the store
+  stays 2:1 of the per-eye size. Mono (the 2D tier) is clamped with one eye across. Limits are
+  queried once per GL context and cached. One shared helper (`js/inline3d-buffer-limit.js`) serves
+  the core (`addImage` / `addVideo`, explicit `{width, height}` included), `./viewer`, `./splat` on
+  both engines and `./model` on both engines.
+- Belt and braces: if the drawing buffer still differs from `canvas.width/height` after a resize (a
+  browser clamp the query did not predict), the viewers map the layer's viewports onto the real
+  drawing buffer instead of `canvas.width / 2`, for live frames, replays and mono. An `addScene`
+  canvas is the page's to size; the core warns once if the browser clamped it.
+- One warning per handle when the clamp bites, naming the limit and the numbers, e.g.
+  `[inline3d/splat] SBS buffer 5120×1348 exceeds MAX_TEXTURE_SIZE 4096 on this device; rendering at
+  4096×1078 (renderScale 1 → 0.8)`.
+- `handle.renderScale` (`./splat`, both engines; `./model`, both engines) now reports the scale **in
+  force** (0.8 in the case above); the new `handle.renderScaleRequested` is the request as given.
+  `setRenderScale` still accepts (0, 4]. `SceneViewer` gains `effectiveRenderScale`. Nothing changes
+  on a device whose limit holds the store.
+
+### Added — call: auto-convergence on the point between the remote person's eyes
+
+- SBS tiles now converge by MEASUREMENT: the receiver finds the disparity of the point between the
+  remote person's eyes in the decoded frames and shifts each eye by half of it, so that person sits at
+  the display plane. No calibration, works for any SBS sender. Tracks a small template after the first
+  lock (~1 ms per measurement at 5 Hz). `autoConverge: false` restores the pair as sent; the depth
+  slider stays an offset on top; lifted (mono) tiles are untouched. `measureFocusDisparity` is
+  exported for pages that want to reuse it.
+
+### Fixed — core: image/video buffers capped at the source's per-eye resolution
+
+- An image or video window's backing buffer was sized box x devicePixelRatio per eye regardless of
+  the source, so a large tile over a small clip re-drew many times the clip's pixels every frame.
+  Measured on an NP02J tablet (DisplayXR Browser 1.0.6), the player sample's 90vw tile got a
+  4608x1296 buffer for a 640x360-per-eye clip: 25 fps, 47 of 97 video frames dropped. The buffer
+  is now capped at the source's per-eye size (box aspect kept, only ever shrinks), which on the same
+  page gives 1280x360, 58 fps, 0 dropped. The compositor scales the layer to the box either way, so
+  the picture is the same. An explicit `{width, height}` is never capped; `cornerRadius` / `feather`
+  keep their on-screen size; the buffer re-derives when a new title has a different size.
+
+### Docs — MP4 (H.264/AAC) plays in DisplayXR Browser 1.0.6+
+
+- The codec guidance no longer says the DisplayXR Browser cannot play H.264/AAC. From **1.0.6** it
+  plays MP4 through the operating system's decoders (Windows GPU + Media Foundation, Android
+  MediaCodec). VP9/AV1 + Opus in WebM stays the recommended first source: it also plays in 1.0.5 and
+  earlier, stock Chromium, and on machines with no hardware H.264 decode (where `canPlayType` still
+  answers "probably" for H.264). Updated in the authoring guide (media traps, hosting video), the
+  PlayCanvas adapter's `setVideo`, and `pickSource`'s doc comment and fallback log text (text only,
+  no behaviour change). WebRTC calls are unaffected: the browser still ships no H.264 for WebRTC,
+  so `./call` keeps preferring VP9.
+
+### Added — call: mono→3D through `lift()` (P2a of RFC 0002)
+
+- **Mono participants are lifted to 3D** on a 3D display: a mono peer on a woven wall is handed to
+  `lift()` (`@displayxr/inline3d/lift`) with `{ mode: 'live', wall }` — one lifted tile per peer,
+  in its own slot, lifted at once and re-created once when the weave session goes live (#172). `mono3D: 'auto'` (default) loads
+  the lift module lazily and falls back to flat, with one log line, when it is missing or there is
+  neither a native 2D→3D provider nor WebGPU; `'off'` keeps mono peers flat; a function injects your
+  own `lift`. `setMono3D(on)` switches at runtime; `liftOptions` passes `models` / `ort` /
+  `quality` through; `maxLifted` (default 4) caps concurrent lifts.
+- **One depth control**: the depth slider is lift's convergence on lifted tiles (0 = `'auto'`) and
+  the crop shift on stereo tiles.
+- **Priority**: the active speaker's lifted stream is `high`, the others `normal`, offscreen or
+  camera-off tiles `paused`, through `setLiftPriority()` — the hook for the native provider's
+  per-stream priority (a no-op on the web path).
+- **Badges** `3D` / `2D→3D` / `2D`; the lobby says what a mono participant will look like
+  (`liftCapabilities()` when available); `handle.mono3D` reports the state. On the web provider a
+  sustained frame-rate drop emits a `quality` event with `lift: { degraded, frameMs, tiles }`.
+- **Changed:** `badgeFor('lifted')` is now `'2D→3D'` (was `'3D'`); `routeFor()` takes
+  `overBudget` / `failed` and reports `mono3d: 'budget' | 'failed'`.
+
+### Added — call (`@displayxr/inline3d/call`, P1 of RFC 0002)
+
+- **`addCall(wall, container, opts)`** — a 3D video call in any page. Full-mesh WebRTC, up to 4
+  participants (`maxPeers`, enforced on the client and the server), behind an internal `Transport`
+  seam so an SFU can replace the mesh later. VP9 > VP8 > AV1 (never H.264, which the DisplayXR
+  Browser does not have); senders hold resolution (`contentHint: 'detail'`,
+  `degradationPreference: 'maintain-resolution'`) with a `maxBitrate` that falls as the mesh grows.
+  A dropped pair walks ICE restart → rebuild with backoff; a peer that leaves shows "Left the call"
+  instead of a black tile.
+- **Stereo from any side-by-side camera**: `camera: 'auto'` picks a device delivering > 2.5:1
+  frames (a raw 1280x480 stereo camera works — nothing is upscaled) and skips a camera held by an
+  eye tracker instead of failing. The pair is sent raw with `rectified: false`; `opts.rectify` is
+  the seam for a calibrated step (P2).
+- **The receiver decides** (RFC §3): an SBS peer is woven with a convergence shift
+  (`f·B/(2Z)` from `hello` + `hint`, low-passed, clamped, plus the depth slider) on the DisplayXR
+  Browser and shown as its left eye elsewhere; a mono peer is flat (the `mono3D` hook is where
+  `lift()` lands in P2). Plain Chrome/Safari/Firefox participants are first-class. Built on
+  `wall.addImage` over a module-owned canvas, so the core still owns every woven buffer; a failed
+  layer is retried and an ended session re-opened without a reload.
+- **Signalling is pluggable** (`SignalingAdapter`): `dxrSignaling(url)` speaks the new
+  `dxr-signal/1` protocol, with a reference Cloudflare Worker + Durable Object (template, not
+  deployed; mints short-lived TURN credentials server-side) and a zero-dependency Node dev server
+  in `signaling/`. `peerjsCloud()` is a demo-only zero-setup path. Rooms are 128-bit random ids
+  carried in the invite link's `#room=` fragment, which never reaches a server.
+- **SDK chrome** (`ui: true`; `ui: false` for headless): lobby with a 3D self-preview, invite link
+  + QR code (a small built-in encoder, no dependency), bottom bar (mute, camera, depth, invite,
+  leave), per-tile `3D`/`2D` badge, a "View in 3D with DisplayXR Browser" banner on other browsers,
+  `grid` or `speaker` layout (active speaker from inbound audio levels). The mirrored stereo self
+  view mirrors each half AND swaps them; the sent stream is never mirrored.
+- **Hardened after the first panel run**: woven tiles (self view included) register only once the
+  session delivers stereo frames, working around a browser bug where a tile registered before the
+  weave session is live is fed the whole side-by-side frame per eye (displayxr-browser-pvt#172). A
+  participant with no connection after ~10 s shows "Can't reach this participant — the network
+  needs a relay (TURN)" and emits `error` `unreachable` while retries continue. A camera held by
+  another app (e.g. eye tracking) gives `error` `camera-busy` and an audio-only join with
+  `retryCamera()`, and no 0x0 video track is ever sent. `peerjsCloud()` no longer counts stale
+  slots toward a full room and frees its slot on `pagehide`.
+- Sample: [`samples/call/`](samples/call/). Types: `call.d.ts`.
+
 ## 1.27.0 — 2026-09-26
 
 Touches the **preview tier** (`./splat`, `engine: 'playcanvas'` only) and adds one optional field

package/README.md

@@ -199,7 +199,8 @@ js/
   inline3d-splat-live.js        setSource's live outgoing photo: its own camera, layer and target (PlayCanvas)
                       from what the asset declares (you serve the decoder files — see the guide)
   inline3d-player.js  preview: addPlayer() — a media player window built on addVideo (stereo
-                      paint, display modes and the fallback are inherited) + an SDK transport bar
+                      paint, display modes and the fallback are inherited) + an SDK transport bar;
+                      attachPlayer() — the same player on a PlayCanvas splat handle's setVideo
 docs/
   authoring-inline-3d.md   the authoring guide
   authoring-motion-and-effects.md
@@ -207,8 +208,10 @@ docs/
                            that is not the API
   porting-three-js-apps.md porting an existing three.js app (WebXR or plain) to inline 3D —
                            the WebXR→inline-3d mapping table and the whole render loop
-  woven-canvas-rules.md    never show a raw side-by-side frame: the join, the eight rules,
-                           handle.firstWoven, reading the browser's `withheld` log line
+  woven-canvas-rules.md    never show a raw side-by-side frame: the join, the rules,
+                           handle.firstWoven / rewoven(), reading the browser's `withheld` log line
+  call.md                  3D video calls: the <dxr-call> / mountCall() one-line embed, options,
+                           events, headless, theming, React, self-hosting, troubleshooting
   proposals/               browser-side asks the SDK is waiting on
 ```
 

package/call-full.d.ts

@@ -0,0 +1,21 @@
+// Type definitions for @displayxr/inline3d/call/full — the call entry with 2D→3D bundled.
+// PREVIEW tier — not covered by the 1.x semver promise. See docs/sdk-stability.md.
+//
+// Same surface as `./call` (re-exported), except `mountCall` / `addCall` default `mono3D` to a
+// STATICALLY imported `lift` (`./lift`), which is what lets a bundler ship it. Use this entry in a
+// bundled app that wants 2D participants lifted to 3D; use `./call` when you do not want the
+// depth-model plumbing (it then warns `lift-not-bundled` the first time a 2D peer would have been
+// lifted on a 3D display). Importing this entry also points `<dxr-call>` at this `mountCall`.
+
+import type { CallHandle, CallOptions } from './call.js';
+
+export * from './call.js';
+
+/** True when this copy of the SDK carries the lift module; false on a build where `./lift` has not landed yet. */
+export const liftBundled: boolean;
+
+/** `mountCall` of `./call` with `mono3D` defaulted to the bundled `lift` (unless you pass your own). */
+export function mountCall(el: HTMLElement, opts?: CallOptions): Promise<CallHandle>;
+
+/** `addCall` of `./call` with `mono3D` defaulted to the bundled `lift` (unless you pass your own). */
+export function addCall(wall: unknown, container: HTMLElement, opts?: CallOptions): Promise<CallHandle>;