@displayxr/inline3d 1.48.0 → 1.49.0

package/CHANGELOG.md

@@ -5,6 +5,61 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
 `./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
 which tier they touch, because that is what tells you whether an upgrade can move your pixels.
 
+## 1.49.0 — 2026-11-03
+
+### Added — the content tile: share a running 3D app into a call (`./call`, `./camera`)
+
+Preview tier. In the DisplayXR Browser a participant can share a running DisplayXR app, and every
+other participant sees it in 3D on their own display (runtime ADR-061, "3D screen share"). What is
+sent is the app's two views before weaving, side by side, on a **second video track** next to the
+camera; each receiver weaves it for its own panel. See [`docs/call.md` § Sharing a 3D app](docs/call.md#sharing-a-3d-app).
+
+- **Discovery** (`./call`): `listShareableApps()`, `watchShareableApps(cb)`, `isAppShareDevice()`,
+  `openAppShare(deviceId)`. The browser lists each app as a `videoinput` labelled
+  `DisplayXR: <app> (3D)` (id `dxr-app:…`); detection takes the label, the id prefix or the track's
+  `displayxrStereo.source === 'app'`, so browsers that do not expose the hint fields yet work, and a
+  missing `zeroDisparityShiftFrac` reads as 0. `openAppShare` is when the runtime asks the user on
+  the sender's machine; a Deny rejects with `code: 'share-denied'`.
+- **Sharing**: `handle.shareContent(stream, descriptor)`, `handle.stopContent({ stopTrack })`,
+  `handle.content`; the `content` event (`{ id | 'self', on, appName }`); `CallPeer.content`;
+  `tile(id, 'content')` (`data-dxr-kind="content"`); `selfContentView` (`<dxr-call self-content-view>`)
+  for a woven preview of your own share.
+- **Display-referred convergence**: the content message carries the disparity of the sender's
+  display plane (`convergence: { known: true, frac }`, 0 for every app today); a receiver applies
+  it directly, never measures the frames, and its depth slider still adds.
+- **Wire**: `hello.content: true` (capability) and a `content` message, both additive (wire version
+  stays 1). The content transceiver is added only when both hellos say `content`, by the pair's
+  offerer with a same-generation re-offer, so an older participant never gets a second video
+  m-line (nor a second camera sender) — they keep seeing your camera, and you get
+  `warning { code: 'content-unsupported' }`. While content is on, the camera sends at 60% of its
+  bitrate; the content track gets the side-by-side budget and holds its resolution.
+- `weave`, `speaking` and `display` events now carry `kind: 'camera' | 'content'`.
+- **`./camera`: a running app is never a camera.** Its label says "3D", so `stereoLabelHint` would
+  have preferred it: no probe path of `openCamera` (and so `camera: 'auto'`) opens an app-share
+  device, `exclude: false` included, and `label:` searches skip them. They are in `skipped` with
+  `reason: 'app-share'` (`CameraSkip.reason`, new).
+
+### Fixed — the Ultraleap hand tracker is excluded by its bare brand name; the default camera is not opened blind (`./camera`, `./call`)
+
+1.48.0's built-in exclusion list is generic words (hand tracking, tracking camera, eye tracking,
+depth camera). On Windows an Ultraleap (Leap 2 / Leap Motion Controller) enumerates as a UVC device
+labelled just **"Ultraleap"**, with modes of 2:1 or squarer, so neither the list nor the pair shape
+caught it; with its tracking service stopped it was free and, as the browser's *default* camera on
+that box, became the call's self view (win box, 2026-11-04). The list now also names `ultraleap`,
+`leap motion`, `infrared` and `ir camera`.
+
+While an excluded device is listed and labels are visible, the default camera is no longer opened
+blind and closed again: the first camera that is not excluded is opened by id, so a sensor is never
+started (an eye tracker's camera must not be). With labels hidden the prime stays blind, as before.
+
+### Added — `prefer: 'label:<text>'` opens the first camera whose label contains the text (`./camera`, `./call`)
+
+For a URL or a config, where a per-origin `deviceId` is useless: `openCamera({ prefer: 'label:hd webcam' })`,
+`addCall(wall, el, { camera: 'label:brio' })`. Case-insensitive; with no permission yet the default
+camera is primed once and let go so the labels become visible; no match rejects with `no-camera`.
+Explicit like a `deviceId`, so it opens an excluded device too. The call sample passes
+`?camera=label:…` (and a bare `?camera=<deviceId>`) straight through.
+
 ## 1.48.0 — 2026-11-03
 
 ### Changed — auto-convergence is measured on the sender's raw frames (`./call`)

package/call.d.ts

@@ -273,6 +273,11 @@ export interface CallOptions {
   autoJoin?: boolean;
   /** The small self view (mirrored; a stereo self view is mirrored AND eye-swapped — an `addCameraView`). Default true. */
   selfView?: boolean;
+  /**
+   * While you share content (`shareContent`), also show it to yourself: a woven preview tile,
+   * `tile('self', 'content')`. Default false (you have the app itself on your display).
+   */
+  selfContentView?: boolean;
   /** Colours, radius, font, tile aspect and strings — see {@link CallTheme}. */
   theme?: CallTheme;
   /** Invite links: `base` (default this page's URL, query kept, fragment replaced); `updateUrl` writes `#room=` into the page URL on join (default: `ui === true`). */
@@ -329,8 +334,85 @@ export interface CallPeer {
   readonly speaking: boolean;
   /** Their display name (`name` option / `setName()`), sanitised; null when they sent none (or an older SDK). */
   readonly name: string | null;
+  /** What they share (3D app share): `on`, its format and the app's name (null while `on` is false). */
+  readonly content: { readonly on: boolean; readonly format: CallFormat | null; readonly appName: string | null };
+}
+
+/** Which of a participant's tiles: their camera (the person) or the content they share. */
+export type CallTileKind = 'camera' | 'content';
+
+/**
+ * The convergence of shared content. `known: true`: the frames are DISPLAY-REFERRED — `frac` is the
+ * disparity of the sender's display plane as a fraction of the per-eye width (`zeroDisparityShiftFrac`
+ * of a DisplayXR app, 0 in v1). Receivers apply it directly and never measure the frames; their
+ * depth control adds to it. `known: false`: shown as sent.
+ */
+export interface ContentConvergence {
+  known: boolean;
+  frac: number | null;
+}
+
+/** What `shareContent` announces (also `openAppShare()`'s `descriptor`, which fits it as is). */
+export interface ContentDescriptor {
+  format?: CallFormat;
+  width?: number;
+  height?: number;
+  convergence?: ContentConvergence;
+  appName?: string | null;
+}
+
+/** `handle.content`: what you share right now. */
+export interface CallContent {
+  readonly on: true;
+  readonly format: CallFormat;
+  readonly width: number;
+  readonly height: number;
+  readonly appName: string | null;
+  readonly convergence: Readonly<ContentConvergence>;
+}
+
+/** A running DisplayXR app the browser offers to share (`listShareableApps`). */
+export interface ShareableApp {
+  readonly deviceId: string;
+  /** `DisplayXR: <app> (3D)`. */
+  readonly label: string;
+  readonly appName: string | null;
+}
+
+/** An opened app share (`openAppShare`): pass `stream` (+ `descriptor`) to `shareContent`. The track is yours to stop. */
+export interface AppShare {
+  stream: MediaStream;
+  track: MediaStreamTrack;
+  descriptor: { kind: 'content'; format: CallFormat; width: number; height: number; convergence: { known: true; frac: number }; appName: string | null };
 }
 
+/** Why `openAppShare` rejected (`error.code`): the user said no at the runtime's prompt, it failed, or the device is not an app. */
+export type AppShareErrorCode = 'share-denied' | 'share-failed' | 'not-app-share';
+
+/**
+ * Is this a running DisplayXR app offered for 3D screen share (runtime ADR-061) rather than a
+ * camera? A `MediaDeviceInfo`, a label, a track, or its settings: true on the label
+ * `DisplayXR: <app> (3D)`, a `dxr-app:` device id, or `displayxrStereo.source === 'app'`. No camera
+ * probe (`camera: 'auto'`, `openCamera`) ever opens one.
+ */
+export function isAppShareDevice(device: MediaDeviceInfo | MediaStreamTrack | MediaTrackSettings | { label?: string; deviceId?: string } | string | null | undefined): boolean;
+
+/**
+ * The running apps this browser offers to share. Empty before the page holds camera permission
+ * (the browser lists none until then), on any other browser, or when none is shareable.
+ */
+export function listShareableApps(opts?: { mediaDevices?: MediaDevices }): Promise<ShareableApp[]>;
+
+/** `cb(apps)` now and whenever the list changes (`devicechange`). Returns the unsubscribe. */
+export function watchShareableApps(cb: (apps: ShareableApp[]) => void, opts?: { mediaDevices?: MediaDevices }): () => void;
+
+/**
+ * Open a shareable app as a side-by-side track (default 2560×720 at 30 fps). The runtime asks the
+ * user on THIS machine now; Deny rejects with `code: 'share-denied'`. The track ends when the app
+ * exits. Stop it yourself when you stop sharing.
+ */
+export function openAppShare(deviceId: string, opts?: { width?: number; height?: number; frameRate?: number; mediaDevices?: MediaDevices }): Promise<AppShare>;
+
 /**
  * `error` codes — fatal for a feature, never for the widget. `'camera-busy'` (held by another
  * app: audio-only until `retryCamera()`), `'no-camera'`, `'permission-denied'`, `'unreachable'`
@@ -367,7 +449,7 @@ export type CallErrorCode =
   | (string & {});
 
 /** `warning` codes — degraded, not broken. */
-export type CallWarningCode = 'lift-not-bundled' | 'turn-shed' | (string & {});
+export type CallWarningCode = 'lift-not-bundled' | 'turn-shed' | 'content-unsupported' | (string & {});
 
 export interface CallEvents {
   joined: { room: string; id: string };
@@ -376,8 +458,8 @@ export interface CallEvents {
   peer: { id: string };
   peerleft: { id: string; reason: string };
   state: { id: string; state: PeerState };
-  /** How this side shows a participant changed (a hello arrived, lift went live, the wall came back). */
-  display: { id: string; display: CallDisplay };
+  /** How this side shows a participant changed (a hello arrived, lift went live, the wall came back). `kind`: which of their tiles. */
+  display: { id: string; display: CallDisplay; kind: CallTileKind };
   speaker: { id: string | null };
   /**
    * Per peer (`id`) every 2 s; or, with `id: null` and `lift`, when lifted tiles on the web
@@ -393,6 +475,8 @@ export interface CallEvents {
    * `'turn-shed'`: only THIS session got no TURN from the hosted service (an anonymous session
    * while the monthly relay budget is nearly spent, or too many credential requests from this
    * address this hour); the call runs STUN-only. A publishable key keeps TURN.
+   * `'content-unsupported'`: you share content and a participant's SDK predates it (no `content` in
+   * their hello) — they keep seeing your camera only. Once per participant per share.
    */
   warning: { code: CallWarningCode; message: string };
   /** A participant's display name arrived or changed (`name: null` = they cleared it). */
@@ -412,9 +496,14 @@ export interface CallEvents {
    * (`rewoven()`). `id` is a participant's, or `'self'` for your self view. `'flat'`: the wall
    * says the tile will not weave (`reason`, e.g. `'layer-failed'`) — it is shown flat.
    */
-  weave: { id: string; state: 'woven' | 'flat'; reason?: string; ms: number };
+  weave: { id: string; state: 'woven' | 'flat'; reason?: string; ms: number; kind: CallTileKind };
   /** A participant (or `'self'`) started / stopped speaking. */
-  speaking: { id: string; on: boolean };
+  speaking: { id: string; on: boolean; kind: CallTileKind };
+  /**
+   * A participant (or `'self'`) started or stopped sharing content (3D app share). `reason` on your
+   * own stop: `'stopped'` (`stopContent()`), `'ended'` (the track ended — the app exited), `'left'`.
+   */
+  content: { id: string; on: boolean; appName: string | null; reason?: 'stopped' | 'ended' | 'left' };
 }
 
 export interface CallHandle {
@@ -456,7 +545,18 @@ export interface CallHandle {
    * A participant's tile element (for `layout: 'none'`), or null; `'self'` is your self view
    * (`data-dxr-peer="self"`). Style it in place; never move it.
    */
-  tile(peerId: string): HTMLElement | null;
+  tile(peerId: string, kind?: CallTileKind): HTMLElement | null;
+  /**
+   * Share content on a second video track (3D app share): a running DisplayXR app's views from
+   * `openAppShare()` (pass its `descriptor` too), or any side-by-side track. Every participant on a
+   * DisplayXR SDK with content support gets a content tile; older ones keep seeing your camera only
+   * (`warning` `'content-unsupported'`). Replaces a previous share. Resolves with what is announced.
+   */
+  shareContent(src: MediaStream | MediaStreamTrack, opts?: ContentDescriptor): Promise<CallContent>;
+  /** Stop sharing. `stopTrack: true` also stops the track (ends the runtime's share). Resolves false when nothing was shared. */
+  stopContent(opts?: { stopTrack?: boolean }): Promise<boolean>;
+  /** What you share right now, or null. */
+  readonly content: CallContent | null;
   /** Your display name, or null. */
   readonly name: string | null;
   /** Change your display name (re-sent to everyone in `hello`). Returns it sanitised (null = none). */
@@ -511,7 +611,7 @@ export function parseInviteLink(link: string | { hash?: string; href?: string }
 // a bubbling, composed `CustomEvent` named `dxr-call:<event>` with the payload in `detail`, and
 // exposes the handle as `el.call`. Attributes (read once, at connect): `room`, `signaling` (a
 // URL), `key`, `camera`, `layout`, `accent`, `max-peers`, `no-ui`, `ui="tiles"`, `auto-join`,
-// `mono3d="off"`, `no-audio`, `no-self-view`, `no-auto-converge`, `no-send-convergence`,
+// `mono3d="off"`, `no-audio`, `no-self-view`, `self-content-view`, `no-auto-converge`, `no-send-convergence`,
 // `tile-aspect`, `invite-base`, `browser-url`, `debug`. Everything else is `el.options`, set before connect; options win.
 
 export class DxrCallElement extends HTMLElement {

package/camera.d.ts

@@ -21,8 +21,11 @@ export type CameraFormat = 'sbs' | 'mono';
  * Which camera to open. `'auto'` (default): a stereo device when one is present — the DisplayXR
  * Browser's "3D Camera" (recognised by its `displayxrStereo` track settings, never by its label),
  * or a device delivering wider than 2.5:1 frames — else the default webcam. `'stereo'` prefers
- * the pair and falls back to mono; `'mono'` never probes; a string is a `deviceId`; a
- * `MediaStream` is used as given, with {@link CameraOptions.format} declaring what it is.
+ * the pair and falls back to mono; `'mono'` never probes; a string is a `deviceId`, or
+ * `'label:<text>'` for the first camera whose label contains the text (case-insensitive; for a URL
+ * or a config, where a per-origin deviceId is useless); a `MediaStream` is used as given, with
+ * {@link CameraOptions.format} declaring what it is. A `deviceId` or `label:` is explicit and
+ * opens an {@link CameraOptions.exclude excluded} device too.
  */
 export type CameraPreference = 'auto' | 'stereo' | 'mono' | (string & {}) | MediaStream;
 
@@ -59,12 +62,15 @@ export interface CameraOptions {
   /**
    * Devices no probe may open. Built in, always (unless `false`): video inputs whose label marks a
    * sensor rather than a camera — hand tracking, an eye tracker's or a "tracking camera"'s stream,
-   * a depth camera (`/\b(hand[- ]?track|tracking camera|eye[- ]?track|depth camera)/i`). Add your
-   * own: a string matches a label (case-insensitive substring) or a deviceId exactly; a RegExp
-   * tests the label. Applied on every probe path, the last-resort "try every device" one
-   * included; a default camera that turns out to be excluded is closed again. Left-out devices
-   * are listed in `skipped` with `excluded: true`. `false` turns the built-in list off. An explicit
-   * `deviceId` (`prefer`) is your choice and is opened regardless.
+   * a depth camera, an infrared / IR camera, and the Ultraleap / Leap Motion hand tracker, which
+   * enumerates under its bare brand name
+   * (`/\b(hand[- ]?track|tracking camera|eye[- ]?track|depth camera|ultraleap|leap motion|infrared|ir camera)/i`).
+   * Add your own: a string matches a label (case-insensitive substring) or a deviceId exactly; a
+   * RegExp tests the label. Applied on every probe path, the last-resort "try every device" one
+   * included; while an excluded device is listed the default camera is not opened blind (the first
+   * other camera is opened by id), and a default that turns out to be excluded is closed again.
+   * Left-out devices are listed in `skipped` with `excluded: true`. `false` turns the built-in
+   * list off. An explicit `deviceId` or `label:` (`prefer`) is your choice and is opened regardless.
    */
   exclude?: ReadonlyArray<string | RegExp> | false;
   /** Verbose console logging. */
@@ -85,6 +91,8 @@ export interface CameraSkip {
   busy?: boolean;
   denied?: boolean;
   excluded?: boolean;
+  /** Why it was left out: `'excluded'` (a sensor, or your `exclude`), or `'app-share'` (a running DisplayXR app offered for 3D screen share — never a camera). */
+  reason?: 'excluded' | 'app-share';
 }
 
 export interface CameraEvents {