@displayxr/inline3d 1.28.0 → 1.29.0

package/CHANGELOG.md

@@ -5,6 +5,106 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
 `./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
 which tier they touch, because that is what tells you whether an upgrade can move your pixels.
 
+## 1.29.0 — 2026-09-30
+
+Touches the **core** (additive: `sharedInline3D()`, `handle.rewoven()`) and the **preview tier**
+(`./call` — RFC 0003 C1, the one-line path; `./player`, `./splat` `engine: 'playcanvas'`). One
+**removal** in the preview tier: `peerjsCloud`. Otherwise additive; no change for a page that does
+not call the new API.
+
+### `./call` — the one-line path (RFC 0003 §1, phase C1)
+
+- **`mountCall(el, opts?)`** — `addCall` with every argument optional: the wall is the document's
+  shared manager (`sharedInline3D()`, below), signalling defaults to the hosted server, the camera
+  to `'auto'`, the SDK chrome to on. `addCall(wall, container, opts)` stays as the explicit form.
+  A call's session recovery now re-opens through the shared manager too, so it joins a wall the
+  page re-created rather than opening a second session.
+- **`<dxr-call>`** — the same thing as markup, registered by importing `./call` (or `./call/full`,
+  or the CDN bundle): mounts on connect, `leave()`s on disconnect (a DOM move IS a teardown —
+  woven-canvas rule 2), re-dispatches every call event as a bubbling, composed `CustomEvent`
+  named `dxr-call:<event>` with the payload in `detail`, and exposes the handle as `el.call`
+  (plus a `ready` promise). Attributes are the string-typed options: `room`, `signaling` (a
+  URL), `key`, `camera`, `layout`, `accent`, `max-peers`, `no-ui`, `auto-join`, `mono3d="off"`,
+  `no-audio`, `no-self-view`, `no-auto-converge`, `tile-aspect`, `invite-base`, `browser-url`,
+  `debug`; anything else is `el.options = { … }`, set before connect (options win). Light DOM, no
+  shadow root, **no globals** (RFC 0003 Decision 11). `auto-join` joins during the mount, so the
+  element replays `joined` and one `peer` per participant already in the room once it can
+  subscribe. Exported: `DxrCallElement`, `defineCallElement`, `attrsToOpts`, `CALL_EVENT_PREFIX`.
+- **`@displayxr/inline3d/call/full`** — `./call` with `mono3D` defaulted to a **statically
+  imported** `lift`, the one shape a bundler can follow; importing it also points `<dxr-call>` at
+  that `mountCall`. `liftBundled` says whether this copy of the SDK carries the module. **Until
+  `./lift` lands on `main`, `js/lift/index.js` is a placeholder** (`lift === null`): the entry
+  builds and behaves like `./call`, and `liftBundled` is `false`.
+- **CDN bundle `dist/call.js`** — one pre-bundled ESM file (core + call + element, ~124 KB min,
+  no three/PlayCanvas), built by `npm run build:dist` (esbuild) and by `prepack` into every
+  published tarball, so jsDelivr serves it as `https://cdn.jsdelivr.net/npm/@displayxr/inline3d@1/dist/call.js`.
+  `<script type="module">` only; lift is a lazy sibling chunk (`dist/lift.js`) once the module
+  is present. `dist/` is not committed.
+- **`warning` event, code `'lift-not-bundled'`** — a 2D participant is on a 3D display, `mono3D`
+  is `'auto'`, and the lift module could not be imported from this build (a bundler that could
+  not follow the computed import, or a copy without `./lift`). Emitted once per call with one
+  `console.warn` naming the fix (`mono3D: lift` or `./call/full`); the lobby reads "2D→3D
+  unavailable in this build". Previously a silent flat tile and an info line. A display that
+  honestly cannot lift is unchanged (`handle.mono3D.reason`).
+- **`key`** — `dxrSignaling(url, { key })` / `CallOptions.key` / `<dxr-call key>`: a publishable
+  key for the hosted service (RFC 0003 §5a), sent as a query parameter on connect. Accepted and
+  forwarded now; the server ignores it until C3.
+- **Removed: `peerjsCloud`** (RFC 0003 Decision 12 — no known external user, so no warning
+  release). The demo adapter over the public PeerJS broker lives on as
+  `samples/call/peerjs-cloud.js`; the sample's `?signal=peerjs` still works. `dxrSignaling()`
+  (the hosted server) has been the default since 1.28.
+- **Types:** `mountCall`, `CallOptions.wall` / `.key`, `CallEvents.warning`, `DxrCallElement`,
+  `attrsToOpts`, `defineCallElement`, `DxrCallEventMap` (+ `HTMLElementTagNameMap` /
+  `HTMLElementEventMap` augmentations), `call-full.d.ts`.
+- **Tests:** element unit tests (attribute→option table, events, disconnect = leave); a
+  headless Chrome ↔ Chrome end-to-end run of `<dxr-call>` (source and CDN bundle) over the
+  hosted server (`npm run test:e2e`); Vite/webpack/esbuild smoke builds of `./call/full` and
+  `./call` that also run the bundles (`npm run test:bundlers`). Neither is part of `npm test`.
+
+### Core — additive
+
+- **`sharedInline3D(opts?)`** — the document's shared manager: the live `createInline3D()`
+  result if one exists, else a new one (concurrent callers share one creation; an unsupported
+  browser is never cached). Woven-canvas rule 1 as an API: a module that needs a wall it did not
+  create (`mountCall`, `<dxr-call>`) uses this and never opens a second session next to the
+  page's own; a page that made its wall first gets that same wall back here. Semver-covered.
+
+- **`./splat` `setVideo` crossfade, band and on-screen rect** (#105, RFC 0001 A5.1-A5.3,
+  `engine: 'playcanvas'`). `transition: 'crossfade'` (`durationMs` 600, `easing`
+  'easeInOutSine', the player's defaults) dissolves on the GPU from the video already on the plane:
+  its quad is frozen and drawn blended over the incoming one, the buffer opaque throughout; from the
+  splat and on `setVideo(null)` it stays a cut. `band: 2.39 | '2.39:1'` fits the picture into a
+  centred letterbox slot (`cover` fills the band and crops each eye about its centre). The result
+  gains `transition`, `band`, `rect` (the picture in CSS px, canvas-relative, clipped) and
+  `onRectChange(cb)`. `attachPlayer` now crossfades and takes `band` (controls in the bars) through
+  these instead of cutting / ignoring them. **Behaviour change for `attachPlayer(…, { skin: 'call' })`:**
+  that skin defaults `band` to 2.39, which surface mode used to ignore; the video now plays in a
+  2.39 letterbox. Pass `band: null` to keep the full tile.
+- **`handle.rewoven()`** — `firstWoven`, measured from the call, for a canvas that is already woven
+  but whose rect is about to change (fullscreen, a layout resize). Same `{ woven, confirmed,
+  reason, ms }` shape, never rejects; a change of the canvas's CSS size or devicePixelRatio while
+  pending restarts the hold (checked every frame, every window kind), and it settles anyway with
+  the new reason `'hold-capped'` four holds after the call if the size never stops changing;
+  before the first join it is `firstWoven`. Woven-canvas rule 11.
+- **Player: `fullscreenCover`** (opt-in, default false). Entering or leaving fullscreen puts an
+  opaque cover over the canvas (on top, hard cut, below the controls) until `rewoven()` settles.
+  Off by default because a blind A/B on the Leia panel showed no raw side-by-side pair without it
+  and ~1.2 s of no picture with it; the flash that did show is browser-wide (rule 11).
+- **Docs:** woven-canvas rules 9–11 (black goes under the canvas — previously only in the
+  authoring guide; no `backdrop-filter` over the weave; a resize is a fresh-canvas moment). The
+  authoring guide's short list now matches, 1–11.
+- **`attachPlayer(splat, src, opts)`** — surface mode (RFC 0001 Addendum A): the player's
+  transport, playlist and events on an existing PlayCanvas splat handle's `setVideo`, with no
+  canvas of its own. `detach()` gives the slot back (`setVideo(null)`: scene, pose, lens and rig
+  as they were); `'detached'` `{ reason: 'superseded' | 'released' }` when something else takes or
+  empties the slot. Cut transitions only for now; `fullscreen` defaults off. Panel-checked on one
+  persistent canvas: attach, next/back, detach, scene state identical after.
+- **`addPlayer` refactor:** one player core on a surface adapter. No behaviour change.
+- **`./splat`:** read-only `handle.videoElement` (the `<video>` the plane shows, or null) and
+  `handle.canvas`.
+- **Types:** `attachPlayer`, `AttachPlayerOptions`, `AttachedPlayerHandle`, `PlayerDetachedEvent`,
+  `'detached'` in `PlayerEvent`; `SplatHandle.videoElement` / `.canvas`.
+
 ## 1.28.0 — 2026-09-27
 
 Touches the **preview tier** (a new subpath, `./call`; the SBS store clamp below) and two **core**
@@ -39,6 +139,15 @@ clamped to the device's GL limits.
   `setRenderScale` still accepts (0, 4]. `SceneViewer` gains `effectiveRenderScale`. Nothing changes
   on a device whose limit holds the store.
 
+### Added — call: auto-convergence on the point between the remote person's eyes
+
+- SBS tiles now converge by MEASUREMENT: the receiver finds the disparity of the point between the
+  remote person's eyes in the decoded frames and shifts each eye by half of it, so that person sits at
+  the display plane. No calibration, works for any SBS sender. Tracks a small template after the first
+  lock (~1 ms per measurement at 5 Hz). `autoConverge: false` restores the pair as sent; the depth
+  slider stays an offset on top; lifted (mono) tiles are untouched. `measureFocusDisparity` is
+  exported for pages that want to reuse it.
+
 ### Fixed — core: image/video buffers capped at the source's per-eye resolution
 
 - An image or video window's backing buffer was sized box x devicePixelRatio per eye regardless of

package/README.md

@@ -199,7 +199,8 @@ js/
   inline3d-splat-live.js        setSource's live outgoing photo: its own camera, layer and target (PlayCanvas)
                       from what the asset declares (you serve the decoder files — see the guide)
   inline3d-player.js  preview: addPlayer() — a media player window built on addVideo (stereo
-                      paint, display modes and the fallback are inherited) + an SDK transport bar
+                      paint, display modes and the fallback are inherited) + an SDK transport bar;
+                      attachPlayer() — the same player on a PlayCanvas splat handle's setVideo
 docs/
   authoring-inline-3d.md   the authoring guide
   authoring-motion-and-effects.md
@@ -207,8 +208,10 @@ docs/
                            that is not the API
   porting-three-js-apps.md porting an existing three.js app (WebXR or plain) to inline 3D —
                            the WebXR→inline-3d mapping table and the whole render loop
-  woven-canvas-rules.md    never show a raw side-by-side frame: the join, the eight rules,
-                           handle.firstWoven, reading the browser's `withheld` log line
+  woven-canvas-rules.md    never show a raw side-by-side frame: the join, the rules,
+                           handle.firstWoven / rewoven(), reading the browser's `withheld` log line
+  call.md                  3D video calls: the <dxr-call> / mountCall() one-line embed, options,
+                           events, headless, theming, React, self-hosting, troubleshooting
   proposals/               browser-side asks the SDK is waiting on
 ```
 

package/call-full.d.ts

@@ -0,0 +1,21 @@
+// Type definitions for @displayxr/inline3d/call/full — the call entry with 2D→3D bundled.
+// PREVIEW tier — not covered by the 1.x semver promise. See docs/sdk-stability.md.
+//
+// Same surface as `./call` (re-exported), except `mountCall` / `addCall` default `mono3D` to a
+// STATICALLY imported `lift` (`./lift`), which is what lets a bundler ship it. Use this entry in a
+// bundled app that wants 2D participants lifted to 3D; use `./call` when you do not want the
+// depth-model plumbing (it then warns `lift-not-bundled` the first time a 2D peer would have been
+// lifted on a 3D display). Importing this entry also points `<dxr-call>` at this `mountCall`.
+
+import type { CallHandle, CallOptions } from './call.js';
+
+export * from './call.js';
+
+/** True when this copy of the SDK carries the lift module; false on a build where `./lift` has not landed yet. */
+export const liftBundled: boolean;
+
+/** `mountCall` of `./call` with `mono3D` defaulted to the bundled `lift` (unless you pass your own). */
+export function mountCall(el: HTMLElement, opts?: CallOptions): Promise<CallHandle>;
+
+/** `addCall` of `./call` with `mono3D` defaulted to the bundled `lift` (unless you pass your own). */
+export function addCall(wall: unknown, container: HTMLElement, opts?: CallOptions): Promise<CallHandle>;

package/call.d.ts

@@ -4,6 +4,11 @@
 // P1 of docs/rfcs/0002-video-call.md: full-mesh WebRTC (<= 4 participants), pluggable signalling,
 // side-by-side stereo from a stereo camera, convergence from `hello` + `hint`, SDK chrome or
 // headless. P2a: mono peers lifted to 3D through `@displayxr/inline3d/lift` (`mono3D`).
+// RFC 0003 C1: `mountCall(el, opts?)` (the one-line path, the wall from `sharedInline3D()`), the
+// `warning` event (`lift-not-bundled`), `key`; `<dxr-call>` is `./call/element`, and
+// `./call/full` is this entry with lift statically imported.
+
+import type { Inline3D, Inline3DUnsupported, CreateInline3DOptions } from './index.js';
 
 /**
  * The `lift()` of `@displayxr/inline3d/lift`, as the call uses it: `lift(video, { mode: 'live',
@@ -110,32 +115,24 @@ export interface SignalingSession {
  */
 /** The hosted DisplayXR signalling server (`dxrSignaling()` with no URL). */
 export const DXR_SIGNAL_DEFAULT: string;
-export function dxrSignaling(url?: string, opts?: { WebSocket?: any; pingMs?: number }): SignalingAdapter;
-
-/**
- * DEMO ONLY — the free public PeerJS broker. No uptime guarantee and not operated by DisplayXR.
- * `Peer` defaults to `globalThis.Peer`, else the ESM build is imported from jsDelivr.
- */
-export function peerjsCloud(opts?: {
-  Peer?: any;
-  peerOptions?: object;
-  url?: string;
-  /** Upper bound on the join-time wait for slots to say hello (ms, default 5000). */
-  helloMs?: number;
-  /** A taken slot silent this long is reported unreachable (ms, default 10000). */
-  unreachableMs?: number;
-  /** Background re-dial interval for silent slots (ms, default 10000). */
-  retryMs?: number;
-  heartbeatMs?: number;
-  settleMs?: number;
-}): SignalingAdapter;
+export function dxrSignaling(url?: string, opts?: { WebSocket?: any; pingMs?: number; /** A publishable key for the hosted service (public by design; sent on connect). */ key?: string }): SignalingAdapter;
 
 export type CallAccent = 'azure' | 'violet' | 'magenta' | 'sunset' | 'amber' | 'lime' | 'mint' | 'ice';
 
 export interface CallOptions {
+  /**
+   * The inline-3D manager to put the tiles on. `mountCall` defaults it to the document's shared
+   * one (`sharedInline3D()`); `addCall` takes it as its first argument instead.
+   */
+  wall?: Inline3D | Inline3DUnsupported | null;
   /** Default `dxrSignaling()` (the hosted DisplayXR server). Or `dxrSignaling(url)` to self-host,
-   *  `peerjsCloud()` (demo only), or your own adapter. */
+   *  or your own adapter. */
   signaling?: SignalingAdapter;
+  /**
+   * A publishable key for the hosted signalling service (`pk_…`, RFC 0003 §5a). Public by design
+   * — it sits in page source — and only used when `signaling` is left at the default.
+   */
+  key?: string;
   /**
    * `'auto'` (default): the room in this page's `#room=` fragment, else a new one on join. Or a
    * room id / invite link. Room ids are >= 96 random bits (generated ones: 128), base64url.
@@ -163,6 +160,13 @@ export interface CallOptions {
   rectify?: (stream: MediaStream, info: { width: number; height: number; deviceId: string | null; label: string }) => MediaStream | Promise<MediaStream>;
   /** Microphone (echo cancellation + noise suppression). Default true. */
   audio?: boolean;
+  /**
+   * Auto-convergence (default true): measure the disparity of the point between each SBS peer's
+   * eyes on the received frames and shift the two eyes by half of it each, so the remote person sits
+   * at the display plane. No calibration needed. The depth slider stays an offset on top; a peer's
+   * `hint` is used only while no measurement has succeeded. Lifted (mono) tiles are unaffected.
+   */
+  autoConverge?: boolean;
   /**
    * Lift mono peers to 3D on a 3D display (RFC §5). `'auto'` (default): import
    * `@displayxr/inline3d/lift` lazily from this copy of the SDK, and use it when it reports a
@@ -205,8 +209,8 @@ export interface CallOptions {
   /** Scroll the call block to the top of the viewport once, on join, so every tile (and the self
    *  view) is on screen and woven. Default true (only with `ui`). */
   scrollIntoView?: boolean;
-  /** `createInline3D()` options used for that re-open. */
-  wallOptions?: object;
+  /** `createInline3D()` options for the wall `mountCall` creates, and for a session re-open. */
+  wallOptions?: CreateInline3DOptions;
   debug?: boolean;
   log?: (tag: string, detail: object) => void;
 }
@@ -229,6 +233,8 @@ export interface CallPeer {
   readonly quality: CallQuality | null;
   /** The per-eye convergence shift currently painted, source px. */
   readonly convergencePx: number;
+  /** Auto-convergence: measured disparity (source px; null until a lock), mean ms per measurement, last method. */
+  readonly autoConverge: { disparityPx: number | null; ms: number | null; readMs: number | null; method: 'mode' | 'focus' | 'track' | null; via: 'videoframe' | 'canvas' | null };
   /** A lifted tile: whether lift is showing yet, native or web, and its current priority. */
   readonly lift: { readonly live: boolean; readonly native: boolean; readonly priority: LiftStreamPriority | null; readonly state: string | null } | null;
 }
@@ -259,6 +265,16 @@ export interface CallEvents {
   speaker: { id: string | null };
   /** Codes include `'camera-busy'`, `'no-camera'`, `'unreachable'` (error.peer = the tile id), `'room-full'`, `'session-ended'`. */
   error: { code: string; message: string; error: Error | null };
+  /**
+   * Degraded, not broken. `'lift-not-bundled'`: a 2D participant is on a 3D display, `mono3D`
+   * is `'auto'`, and the lift module could not be imported from this build (a bundler that could
+   * not follow the computed import, or a copy of the SDK without `./lift`) — they stay flat. Fix:
+   * `import { lift } from '@displayxr/inline3d/lift'` and pass `mono3D: lift`, or import
+   * `@displayxr/inline3d/call/full`. Emitted once per call, the first time such a peer routes,
+   * with one `console.warn`. A display that honestly cannot lift is NOT this: that is
+   * `handle.mono3D.reason` (`'no-provider'` / `'no-webgpu'`) and an info line.
+   */
+  warning: { code: 'lift-not-bundled' | string; message: string };
   state: { id: string; state: PeerState };
   joined: { room: string; id: string };
   left: { room: string };
@@ -303,7 +319,64 @@ export interface CallHandle {
   off<K extends keyof CallEvents>(type: K, cb: (e: CallEvents[K]) => void): void;
 }
 
-export function addCall(wall: unknown, container: HTMLElement, opts: CallOptions): Promise<CallHandle>;
+export function addCall(wall: unknown, container: HTMLElement, opts?: CallOptions): Promise<CallHandle>;
+
+/**
+ * The one-line path (RFC 0003 §1): `addCall` with every argument optional. The wall is
+ * `opts.wall` if given, else the document's shared manager (`sharedInline3D(opts.wallOptions)` —
+ * the page's existing session, never a second one). Hosted signalling, `camera: 'auto'` and the
+ * SDK chrome by default; resolves once the lobby shows (or, with `autoJoin` / `ui: false`, once
+ * joined). `<dxr-call>` (`@displayxr/inline3d/call/element`) is this as markup.
+ */
+export function mountCall(el: HTMLElement, opts?: CallOptions): Promise<CallHandle>;
+
+// ── <dxr-call> — the one-line path as markup ───────────────────────────────────────────────
+//
+// Importing this entry (or `./call/full`, or the CDN bundle `dist/call.js`) registers the element.
+// It mounts on connect (`mountCall(this, { ...attrsToOpts(this), ...this.options })`), leaves on
+// disconnect (a DOM move IS a teardown — woven-canvas rule 2), re-dispatches every call event as
+// a bubbling, composed `CustomEvent` named `dxr-call:<event>` with the payload in `detail`, and
+// exposes the handle as `el.call`. Attributes (read once, at connect): `room`, `signaling` (a
+// URL), `key`, `camera`, `layout`, `accent`, `max-peers`, `no-ui`, `auto-join`, `mono3d="off"`,
+// `no-audio`, `no-self-view`, `no-auto-converge`, `tile-aspect`, `invite-base`, `browser-url`,
+// `debug`. Everything else is `el.options`, set before connect; options win over attributes.
+
+export class DxrCallElement extends HTMLElement {
+  /** The mount seam: null = `mountCall` of `./call`; `./call/full` sets its own. */
+  static mount: ((el: HTMLElement, opts?: CallOptions) => Promise<CallHandle>) | null;
+  /** Non-string options (a `MediaStream`, an adapter, `lift`, a `wall`) — set BEFORE connecting. */
+  options: CallOptions | null;
+  /** The handle once mounted; null before, and again after disconnect. */
+  readonly call: CallHandle | null;
+  /** Resolves with the handle when the mount lands (null if the element left the DOM first); rejects if it failed. */
+  readonly ready: Promise<CallHandle | null> | null;
+  connectedCallback(): void;
+  disconnectedCallback(): void;
+}
+
+/** The attribute → option mapping the element applies at connect. Pure; unset attributes contribute nothing. */
+export function attrsToOpts(source: { getAttribute(name: string): string | null } | ((name: string) => string | null)): Partial<CallOptions>;
+
+/**
+ * Register the element (default name `dxr-call`) once. True when this call registered it; false
+ * when the name was taken or there is no `customElements` registry. The entries call it for you.
+ */
+export function defineCallElement(name?: string, registry?: CustomElementRegistry): boolean;
+
+/** `'dxr-call:'` — every DOM event the element dispatches starts with it. */
+export const CALL_EVENT_PREFIX: 'dxr-call:';
+
+/** The DOM events: one per {@link CallEvents} key, plus `dxr-call:ready` once the handle exists. */
+export type DxrCallEventMap = { [K in keyof CallEvents as `dxr-call:${K}`]: CustomEvent<CallEvents[K]> } & {
+  'dxr-call:ready': CustomEvent<{ call: CallHandle }>;
+};
+
+declare global {
+  interface HTMLElementTagNameMap {
+    'dxr-call': DxrCallElement;
+  }
+  interface HTMLElementEventMap extends DxrCallEventMap {}
+}
 
 // ── pure helpers (exported for tests and advanced pages) ──────────────────────────────────
 export function normalizeCallOptions(opts?: Partial<CallOptions>): Record<string, unknown>;
@@ -351,6 +424,13 @@ export interface CallLiftPool {
 export function createLiftPool(o: { lift: CallLiftFunction; max?: number; log?: (tag: string, o: object) => void; options?: Record<string, unknown> }): CallLiftPool;
 export function convergenceShiftPx(p: { eyeWidthPx: number; hfovDeg?: number | null; baselineMm?: number | null; subjectZmm?: number | null }): number;
 export function lowPass(prev: number, target: number, alpha?: number): number;
+/** The disparity (left x − right x, input px) of the point between the eyes in a grayscale SBS frame. */
+export function measureFocusDisparity(
+  img: ArrayLike<number>,
+  W: number,
+  H: number,
+  o?: { focus?: { x: number; y: number }; block?: number; dMin?: number; dMax?: number; dyMax?: number; minNcc?: number },
+): { d: number; x: number; y: number; c: number; method: 'focus' | 'mode'; blocks: number } | null;
 export function clampShift(px: number, eyeWidthPx: number, maxFraction?: number): number;
 export function eyeCropRect(eyeW: number, eyeH: number, aspect: number, shift: number, eye: 0 | 1): { sx: number; sy: number; sw: number; sh: number };
 export function mirrorSwapOps(W: number, H: number): Array<{ src: 'L' | 'R'; sx: number; sw: number; sy: number; sh: number; dx: number; dw: number; mirror: true }>;