@displayxr/inline3d 1.47.6 → 1.48.0

package/CHANGELOG.md

@@ -5,7 +5,87 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
 `./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
 which tier they touch, because that is what tells you whether an upgrade can move your pixels.
 
-## Unreleased
+## 1.48.0 — 2026-11-03
+
+### Changed — auto-convergence is measured on the sender's raw frames (`./call`)
+
+Preview tier. A stereo participant was converged by each RECEIVER, which measured the disparity of
+the point between the person's eyes on the frame it decoded: a compressed copy, often downscaled
+by WebRTC, measured again by every participant. The sender now measures it on its own raw camera
+frames, before encoding, and sends the result on the data channel (`conv { d, c }`, `d` a
+fraction of the per-eye width; at most 5 a second while the person moves, one every 400 ms while
+they hold still). Each receiver still applies it as a per-eye crop, so the stream keeps the full
+picture and every receiver keeps its own depth slider and display. The sender measures whatever
+its self view does, including `selfView: false`, and sends nothing while its camera is off or from
+a mono camera.
+
+- New option **`sendConvergence`** (default `true`; `<dxr-call no-send-convergence>`). `false`
+  sends nothing and leaves the measuring to the receivers.
+- Additive on the wire, so the wire version stays 1. A sender advertises it with `hello.conv:
+  true`. A sender without it (1.47 and earlier), or one silent for 1.2 s, is measured by the
+  receiver exactly as before, and an older receiver ignores `conv`. The depth cursor still copies
+  the received frames while it hovers a tile.
+- `diagnostics().peers[i].autoConverge` gains `source` (`'sender'` / `'local'` / `null`),
+  `senderFrac` and `samples`. A failed local measurement no longer drops a value held from the
+  sender.
+- `docs/call.md` § *Auto-convergence is measured by the sender*.
+
+### Added — headless-mode hooks for an app that owns its layout (`./call`)
+
+Preview tier. Everything a chat app that lays out the call itself (`ui: false`, `layout: 'none'`)
+needed and had to work around; the tiles stay module-owned. Docs: `docs/call.md` § *Hooks for an
+app that owns its layout*.
+
+- **Names.** Option `name` (≤ 64 characters, control and bidi characters stripped), sent in
+  `hello`; `CallPeer.name`; `handle.setName()` re-sends it; event `name { id, name }`. Additive on
+  the wire (wire version stays 1): an older peer ignores it.
+- **App messages.** `handle.send(peerId, data)` / `handle.broadcast(data)` on the call's data
+  channel (`{ type: 'app', d }`): JSON, ≤ 16 KiB serialised, ≤ 20 a second per participant (over a
+  cap: `false`, nothing sent); event `message { from, data }`. Only app messages reach the page.
+  An older peer drops them silently.
+- **Events.** `signal { state, code? }` (+ `handle.signal`): connected / reconnecting / closed.
+  `weave { id, state, reason?, ms }`: a tile's first join and its re-join after a resize, per
+  participant and `'self'`. `speaking { id, on }`, for participants and `'self'`.
+- **Layout.** `tileCover` (default `'auto'` with `layout: 'none'`): the core's `rectCover` on every
+  woven tile and a stereo self view. `theme.tileAspect: 'fill'`: no aspect imposed, each tile's
+  crop follows its own box (the numeric default is unchanged). `handle.tile('self')`
+  (`data-dxr-peer="self"`); under `layout: 'none'` the footer and self slot are
+  `display: contents` so the page places the self tile. `handle.setSelfView(on)` removes / restores
+  its layer. A hidden (0×0) tile is no longer painted or measured.
+- **Devices.** `handle.setMic(deviceId)`, `handle.setAudioOutput(sinkId)` (tiles that join later
+  too), `handle.localStream`.
+- **Quality.** The `quality` event gains `path` (`'host'` / `'srflx'` / `'prflx'` / `'relay'`, from
+  the selected ICE candidate pair) and `rttMs`.
+- `<dxr-call>`: `name` attribute, `tile-aspect="fill"`, and the five new events as `dxr-call:*`.
+
+### Fixed — the camera probe opened a hand-tracking camera (`./camera`, `./call`)
+
+Preview tier. Found on hardware (Windows, DisplayXR Browser 1.9.1): with no stereo pair open,
+`camera: 'auto'`'s last-resort "try every device" loop opened a hand-tracking camera — an ordinary
+video input, wide enough to pass for a side-by-side pair — and the call sent its picture as 3D.
+Every probe path (the named-pair fast path, the default camera, the last resort) now skips devices
+whose label marks a sensor (hand tracking, eye tracking, a "tracking camera", a depth camera), plus
+the new `openCamera({ exclude })` / call `cameraExclude` list; `false` turns the built-in list off.
+Left-out devices appear in `skipped` with `excluded: true`; an explicit `deviceId` is opened
+regardless. Also: the browser's "3D Camera (DisplayXR)" is listed LAST on Windows, so the probe no
+longer assumes it is the default — a default that is a pair only by its shape no longer wins over a
+camera that names itself a pair, and with an excluded default the first plain camera is the
+fallback. `addCameraView` gains `aspect: 'fill'` and `rectCover`.
+
+### Added — the dev signalling server ships in the package
+
+`signaling/dev-server.mjs` and the five modules it imports are in the published files (not an
+`exports` entry): run it by path, or import `startDevServer` from the file. Copy it unbundled
+(`signaling/README.md`).
+
+### Docs — frosted chrome over a tile works on DisplayXR Browser 0.1.16+
+
+`docs/woven-canvas-rules.md` rule 10 and §5, and `docs/call.md`: since DisplayXR Browser 0.1.16 the
+compositor draws a tile mono inside each `backdrop-filter` rect (displayxr-browser#120;
+`samples/composition` is the validated case), so partial frosted chrome over a tile is fine. Never
+frost a whole visible tile; effects on a tile or its ancestors (rule 7) are unchanged. The SDK's
+own chrome keeps its near-solid tint. `docs/call.md` also notes that changing only the `#room=`
+fragment of a page with a live call does not re-join (reload, or navigate with a changed query).
 
 ## 1.47.6 — 2026-11-03
 

package/call.d.ts

@@ -154,8 +154,12 @@ export interface CallTheme {
   radius?: number | string;
   /** Chrome font family → `--dxr-font`. */
   font?: string;
-  /** Tile aspect (w/h) = the woven buffer's per-eye aspect. Default 16/9. */
-  tileAspect?: number;
+  /**
+   * Tile aspect (w/h) = the woven buffer's per-eye aspect, the same for every tile. Default 16/9.
+   * `'fill'`: no aspect is imposed — the page sizes each tile's box (layout: 'none') and each
+   * tile's crop follows its own stage box (clamped to 0.3–4), the self view's too.
+   */
+  tileAspect?: number | 'fill';
   /** Overrides for the chrome's text (also localisation). */
   strings?: CallStrings;
 }
@@ -193,6 +197,24 @@ export interface CallOptions {
   format?: CallFormat;
   /** Microphone (echo cancellation + noise suppression). Default true. */
   audio?: boolean;
+  /**
+   * Your display name, sent to every participant in `hello` (their `CallPeer.name`). At most 64
+   * characters; control characters are stripped. Change it with `setName()`. Default none.
+   */
+  name?: string;
+  /**
+   * Devices `camera: 'auto' | 'stereo' | 'mono'` never opens — `openCamera`'s `exclude`: strings
+   * match a label (case-insensitive substring) or a deviceId, RegExps the label. Labels of
+   * sensors (hand tracking, eye tracking, a "tracking camera", a depth camera) are always left out
+   * too; `false` turns that built-in list off. An explicit deviceId is opened regardless.
+   */
+  cameraExclude?: ReadonlyArray<string | RegExp> | false;
+  /**
+   * Cover each woven tile (and a stereo self view) across a resize or move until it is woven
+   * again — the core's `rectCover: 'auto'` on its registration. Default `'auto'` with
+   * `layout: 'none'` (your layout moves the tiles), `'off'` otherwise.
+   */
+  tileCover?: 'auto' | 'off';
   /**
    * Lift mono peers to 3D on a 3D display (RFC §5). `'auto'` (default): import
    * `@displayxr/inline3d/lift` lazily from this copy of the SDK, and use it when it reports a
@@ -210,11 +232,20 @@ export interface CallOptions {
    */
   liftOptions?: Record<string, unknown> & { max?: number };
   /**
-   * Auto-convergence (default true): measure the disparity of the point between each SBS peer's
-   * eyes on the received frames and shift the two eyes by half of it each, so the remote person sits
-   * at the display plane. No calibration needed. The depth slider stays an offset on top.
+   * Auto-convergence (default true): shift each SBS peer's two eyes by half the disparity of the
+   * point between that person's eyes, so the remote person sits at the display plane. No
+   * calibration needed. The depth slider stays an offset on top. The disparity comes from the
+   * sender when it measures it (`sendConvergence`, 1.48+); from an older sender, this side measures
+   * the received frames itself.
    */
   autoConverge?: boolean;
+  /**
+   * With a stereo camera (default true): measure the disparity of the point between your eyes on
+   * your own raw camera frames, before compression, and send it to every peer a few times a second
+   * (a few bytes on the data channel). Their `autoConverge` applies it, so nobody measures the
+   * decoded frames. `false`: send nothing; receivers measure, as before 1.48.
+   */
+  sendConvergence?: boolean;
   /**
    * `'depth'`: over a 3D (stereo) tile — a peer's or your self view — the cursor is never
    * overlapped by what comes out of the screen: where the person under the pointer pops out, the
@@ -277,6 +308,14 @@ export type LiftFunction = ((element: HTMLVideoElement, opts: Record<string, unk
 export interface CallQuality {
   in: { width: number; height: number; fps: number; codec: string | null; kbps: number | null; dropped: number; decoder: string | null } | null;
   out: { width: number; height: number; fps: number; codec: string | null; kbps: number | null; limitation: string | null; encoder: string | null } | null;
+  /**
+   * The network path of the selected ICE candidate pair: `'relay'` (through TURN, either end),
+   * else the local candidate's type — `'host'` (direct, same network), `'srflx'` / `'prflx'`
+   * (direct through NAT). Null before a pair is selected.
+   */
+  path?: 'host' | 'srflx' | 'prflx' | 'relay' | null;
+  /** The selected pair's current round-trip time, ms; null when unknown. */
+  rttMs?: number | null;
 }
 
 /** A remote participant, as a page needs it. Debug data (hello, route, convergence, lift) is in `diagnostics()`. */
@@ -288,6 +327,8 @@ export interface CallPeer {
   readonly muted: boolean;
   readonly cameraOff: boolean;
   readonly speaking: boolean;
+  /** Their display name (`name` option / `setName()`), sanitised; null when they sent none (or an older SDK). */
+  readonly name: string | null;
 }
 
 /**
@@ -354,6 +395,26 @@ export interface CallEvents {
    * address this hour); the call runs STUN-only. A publishable key keeps TURN.
    */
   warning: { code: CallWarningCode; message: string };
+  /** A participant's display name arrived or changed (`name: null` = they cleared it). */
+  name: { id: string; name: string | null };
+  /**
+   * An app message from a participant (`send` / `broadcast` on their side): `data` as they sent it
+   * (JSON). Only app messages surface here — the call's own wire messages never do.
+   */
+  message: { from: string; data: unknown };
+  /**
+   * The signalling connection: `'connected'` on join and after a reconnect, `'reconnecting'` while
+   * it retries (media keeps flowing), `'closed'` when it gave up — `code` says why (e.g. `'expired'`).
+   */
+  signal: { state: 'connected' | 'reconnecting' | 'closed'; code?: string };
+  /**
+   * A tile's weave state settled: its first join (`firstWoven`), or its re-join after a resize
+   * (`rewoven()`). `id` is a participant's, or `'self'` for your self view. `'flat'`: the wall
+   * says the tile will not weave (`reason`, e.g. `'layer-failed'`) — it is shown flat.
+   */
+  weave: { id: string; state: 'woven' | 'flat'; reason?: string; ms: number };
+  /** A participant (or `'self'`) started / stopped speaking. */
+  speaking: { id: string; on: boolean };
 }
 
 export interface CallHandle {
@@ -391,8 +452,37 @@ export interface CallHandle {
   setDepth(v: number | null): number;
   /** Turn mono→3D on/off at runtime (no argument toggles). Off releases every lift; tiles go flat. */
   setMono3D(on?: boolean): boolean;
-  /** A participant's tile element (for `layout: 'none'`), or null. Style it in place; never move it. */
+  /**
+   * A participant's tile element (for `layout: 'none'`), or null; `'self'` is your self view
+   * (`data-dxr-peer="self"`). Style it in place; never move it.
+   */
   tile(peerId: string): HTMLElement | null;
+  /** Your display name, or null. */
+  readonly name: string | null;
+  /** Change your display name (re-sent to everyone in `hello`). Returns it sanitised (null = none). */
+  setName(name: string | null): string | null;
+  /**
+   * An app message to one participant over the call's data channel (arrives as their `message`
+   * event). `data` must be JSON-serialisable and at most 16 KiB serialised; at most 20 messages a
+   * second per participant. Returns false (nothing sent) when over a cap, not serialisable, or the
+   * participant is not connected. A participant on an SDK before 1.48 drops it silently.
+   */
+  send(peerId: string, data: unknown): boolean;
+  /** {@link CallHandle.send} to everyone. True only if every connected participant got it. */
+  broadcast(data: unknown): boolean;
+  /** The signalling connection — see the `signal` event. Null before join and after leave. */
+  readonly signal: 'connected' | 'reconnecting' | 'closed' | null;
+  /**
+   * Show (true) or hide (false) your self view; no argument toggles. Hiding removes its layer
+   * from the wall (never a hidden live registration). Returns the new state.
+   */
+  setSelfView(on?: boolean): boolean;
+  /** Switch the microphone (a `deviceId`; null = the default). Resolves false, keeping the old one, if it cannot open. */
+  setMic(deviceId: string | null): Promise<boolean>;
+  /** Play every participant's audio (also later ones) to `sinkId` (`HTMLMediaElement.setSinkId`). Resolves false if any tile refused. */
+  setAudioOutput(sinkId: string): Promise<boolean>;
+  /** What you send (camera + microphone). Read-only — use `setCamera` / `setMic` to change it. */
+  readonly localStream: MediaStream | null;
   on<K extends keyof CallEvents>(type: K, cb: (e: CallEvents[K]) => void): () => void;
   off<K extends keyof CallEvents>(type: K, cb: (e: CallEvents[K]) => void): void;
   /** Explicitly UNSTABLE: the wall, the transport, per-peer hello / route / convergence / lift / quality, the lift pool. For devtools. */
@@ -421,8 +511,8 @@ export function parseInviteLink(link: string | { hash?: string; href?: string }
 // a bubbling, composed `CustomEvent` named `dxr-call:<event>` with the payload in `detail`, and
 // exposes the handle as `el.call`. Attributes (read once, at connect): `room`, `signaling` (a
 // URL), `key`, `camera`, `layout`, `accent`, `max-peers`, `no-ui`, `ui="tiles"`, `auto-join`,
-// `mono3d="off"`, `no-audio`, `no-self-view`, `no-auto-converge`, `tile-aspect`, `invite-base`,
-// `browser-url`, `debug`. Everything else is `el.options`, set before connect; options win.
+// `mono3d="off"`, `no-audio`, `no-self-view`, `no-auto-converge`, `no-send-convergence`,
+// `tile-aspect`, `invite-base`, `browser-url`, `debug`. Everything else is `el.options`, set before connect; options win.
 
 export class DxrCallElement extends HTMLElement {
   /** The mount seam: null = `mountCall` of `./call`; `./call/full` sets its own. */

package/camera.d.ts

@@ -11,7 +11,7 @@
 // The public surface of this file is pinned by test/api-snapshot: a change here fails `npm test`
 // until the snapshot is updated in the same change (`node tools/api-surface.mjs --update`).
 
-import type { Inline3D, Inline3DUnsupported, TileHandle } from './index.js';
+import type { Inline3D, Inline3DUnsupported, TileHandle, RectCoverOption } from './index.js';
 import type { CursorShape, CursorShapeHit, CursorShapeOption, CursorSoftenOption } from './cursor-depth.js';
 
 /** What the camera delivers: a side-by-side pair (left eye left) or one flat picture. */
@@ -56,6 +56,17 @@ export interface CameraOptions {
    * is then true. A throwing hook keeps the raw pair (`cam.rectifyError`).
    */
   rectify?: (stream: MediaStream, info: { width: number; height: number; deviceId: string | null; label: string }) => MediaStream | Promise<MediaStream>;
+  /**
+   * Devices no probe may open. Built in, always (unless `false`): video inputs whose label marks a
+   * sensor rather than a camera — hand tracking, an eye tracker's or a "tracking camera"'s stream,
+   * a depth camera (`/\b(hand[- ]?track|tracking camera|eye[- ]?track|depth camera)/i`). Add your
+   * own: a string matches a label (case-insensitive substring) or a deviceId exactly; a RegExp
+   * tests the label. Applied on every probe path, the last-resort "try every device" one
+   * included; a default camera that turns out to be excluded is closed again. Left-out devices
+   * are listed in `skipped` with `excluded: true`. `false` turns the built-in list off. An explicit
+   * `deviceId` (`prefer`) is your choice and is opened regardless.
+   */
+  exclude?: ReadonlyArray<string | RegExp> | false;
   /** Verbose console logging. */
   debug?: boolean;
 }
@@ -64,7 +75,16 @@ export interface CameraOptions {
 export type CameraErrorCode = 'camera-busy' | 'permission-denied' | 'no-camera';
 export interface CameraError extends Error {
   code: CameraErrorCode;
-  skipped: ReadonlyArray<{ label: string; error: string; busy?: boolean; denied?: boolean }>;
+  skipped: ReadonlyArray<CameraSkip>;
+}
+
+/** A device {@link openCamera} tried and did not use: busy, refused, or left out (`excluded`, `error: 'excluded'`). */
+export interface CameraSkip {
+  label: string;
+  error: string;
+  busy?: boolean;
+  denied?: boolean;
+  excluded?: boolean;
 }
 
 export interface CameraEvents {
@@ -162,7 +182,7 @@ export interface StereoCamera {
   /** True when this module opened the device (and `close()` stops it); false for a page-supplied stream. */
   readonly owned: boolean;
   /** Devices tried and not used (busy, denied, not a pair) — the clue a `'camera-busy'` report carries. */
-  readonly skipped: ReadonlyArray<{ label: string; error: string; busy?: boolean; denied?: boolean }>;
+  readonly skipped: ReadonlyArray<CameraSkip>;
   readonly state: 'live' | 'ended' | 'closed';
   /** A detached, playing `<video>` on the stream (never in the DOM); null without a document. */
   readonly video: HTMLVideoElement | null;
@@ -214,8 +234,17 @@ export interface CameraViewOptions {
   autoConverge?: boolean;
   /** [-1, 1], + = push back: an offset on top of the convergence. Default 0. */
   depth?: number;
-  /** The tile's aspect (w/h) = the woven buffer's per-eye aspect. Default 16/9. */
-  aspect?: number;
+  /**
+   * The tile's aspect (w/h) = the woven buffer's per-eye aspect. Default 16/9. `'fill'`: the
+   * canvas's own CSS box, read every frame (clamped to 0.3–4), for a page that sizes the tile.
+   */
+  aspect?: number | 'fill';
+  /**
+   * The core's rect cover (`addImage`'s `rectCover`) on the woven registration: `'auto'`
+   * covers the tile across a later resize or move until it is woven again. Default: the core's
+   * (`'off'`).
+   */
+  rectCover?: RectCoverOption;
   /**
    * `'depth'`: while the pointer is over the woven view and what is under it comes out of the
    * screen, the normal arrow is drawn just in front of it (runtime ADR-046, measured off the pair's