@displayxr/inline3d 1.47.6 → 1.49.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +136 -1
- package/call.d.ts +202 -12
- package/camera.d.ts +44 -7
- package/dist/call.js +11 -6
- package/docs/woven-canvas-rules.md +25 -13
- package/js/call/element.js +9 -3
- package/js/call/options.js +22 -3
- package/js/call/share.js +147 -0
- package/js/call/transport.js +160 -9
- package/js/call/ui.js +11 -1
- package/js/call/wire.js +227 -6
- package/js/camera/app-share.js +54 -0
- package/js/camera/capture.js +196 -32
- package/js/camera/conv-meter.js +152 -0
- package/js/inline3d-call.js +737 -91
- package/js/inline3d-camera.js +34 -7
- package/js/inline3d-player.js +8 -4
- package/package.json +8 -1
- package/signaling/README.md +289 -0
- package/signaling/dev-server.mjs +320 -0
- package/signaling/gate.mjs +112 -0
- package/signaling/keys.mjs +123 -0
- package/signaling/limits.mjs +221 -0
- package/signaling/room.mjs +322 -0
- package/signaling/turn-budget.mjs +254 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,7 +5,142 @@ entry points (`.`, `./three`) are frozen for 1.x, while the **scene subpaths** (
|
|
|
5
5
|
`./splat`, `./model`) are a preview tier whose options may change in any release. Entries below say
|
|
6
6
|
which tier they touch, because that is what tells you whether an upgrade can move your pixels.
|
|
7
7
|
|
|
8
|
-
##
|
|
8
|
+
## 1.49.0 — 2026-11-03
|
|
9
|
+
|
|
10
|
+
### Added — the content tile: share a running 3D app into a call (`./call`, `./camera`)
|
|
11
|
+
|
|
12
|
+
Preview tier. In the DisplayXR Browser a participant can share a running DisplayXR app, and every
|
|
13
|
+
other participant sees it in 3D on their own display (runtime ADR-061, "3D screen share"). What is
|
|
14
|
+
sent is the app's two views before weaving, side by side, on a **second video track** next to the
|
|
15
|
+
camera; each receiver weaves it for its own panel. See [`docs/call.md` § Sharing a 3D app](docs/call.md#sharing-a-3d-app).
|
|
16
|
+
|
|
17
|
+
- **Discovery** (`./call`): `listShareableApps()`, `watchShareableApps(cb)`, `isAppShareDevice()`,
|
|
18
|
+
`openAppShare(deviceId)`. The browser lists each app as a `videoinput` labelled
|
|
19
|
+
`DisplayXR: <app> (3D)` (id `dxr-app:…`); detection takes the label, the id prefix or the track's
|
|
20
|
+
`displayxrStereo.source === 'app'`, so browsers that do not expose the hint fields yet work, and a
|
|
21
|
+
missing `zeroDisparityShiftFrac` reads as 0. `openAppShare` is when the runtime asks the user on
|
|
22
|
+
the sender's machine; a Deny rejects with `code: 'share-denied'`.
|
|
23
|
+
- **Sharing**: `handle.shareContent(stream, descriptor)`, `handle.stopContent({ stopTrack })`,
|
|
24
|
+
`handle.content`; the `content` event (`{ id | 'self', on, appName }`); `CallPeer.content`;
|
|
25
|
+
`tile(id, 'content')` (`data-dxr-kind="content"`); `selfContentView` (`<dxr-call self-content-view>`)
|
|
26
|
+
for a woven preview of your own share.
|
|
27
|
+
- **Display-referred convergence**: the content message carries the disparity of the sender's
|
|
28
|
+
display plane (`convergence: { known: true, frac }`, 0 for every app today); a receiver applies
|
|
29
|
+
it directly, never measures the frames, and its depth slider still adds.
|
|
30
|
+
- **Wire**: `hello.content: true` (capability) and a `content` message, both additive (wire version
|
|
31
|
+
stays 1). The content transceiver is added only when both hellos say `content`, by the pair's
|
|
32
|
+
offerer with a same-generation re-offer, so an older participant never gets a second video
|
|
33
|
+
m-line (nor a second camera sender) — they keep seeing your camera, and you get
|
|
34
|
+
`warning { code: 'content-unsupported' }`. While content is on, the camera sends at 60% of its
|
|
35
|
+
bitrate; the content track gets the side-by-side budget and holds its resolution.
|
|
36
|
+
- `weave`, `speaking` and `display` events now carry `kind: 'camera' | 'content'`.
|
|
37
|
+
- **`./camera`: a running app is never a camera.** Its label says "3D", so `stereoLabelHint` would
|
|
38
|
+
have preferred it: no probe path of `openCamera` (and so `camera: 'auto'`) opens an app-share
|
|
39
|
+
device, `exclude: false` included, and `label:` searches skip them. They are in `skipped` with
|
|
40
|
+
`reason: 'app-share'` (`CameraSkip.reason`, new).
|
|
41
|
+
|
|
42
|
+
### Fixed — the Ultraleap hand tracker is excluded by its bare brand name; the default camera is not opened blind (`./camera`, `./call`)
|
|
43
|
+
|
|
44
|
+
1.48.0's built-in exclusion list is generic words (hand tracking, tracking camera, eye tracking,
|
|
45
|
+
depth camera). On Windows an Ultraleap (Leap 2 / Leap Motion Controller) enumerates as a UVC device
|
|
46
|
+
labelled just **"Ultraleap"**, with modes of 2:1 or squarer, so neither the list nor the pair shape
|
|
47
|
+
caught it; with its tracking service stopped it was free and, as the browser's *default* camera on
|
|
48
|
+
that box, became the call's self view (win box, 2026-11-04). The list now also names `ultraleap`,
|
|
49
|
+
`leap motion`, `infrared` and `ir camera`.
|
|
50
|
+
|
|
51
|
+
While an excluded device is listed and labels are visible, the default camera is no longer opened
|
|
52
|
+
blind and closed again: the first camera that is not excluded is opened by id, so a sensor is never
|
|
53
|
+
started (an eye tracker's camera must not be). With labels hidden the prime stays blind, as before.
|
|
54
|
+
|
|
55
|
+
### Added — `prefer: 'label:<text>'` opens the first camera whose label contains the text (`./camera`, `./call`)
|
|
56
|
+
|
|
57
|
+
For a URL or a config, where a per-origin `deviceId` is useless: `openCamera({ prefer: 'label:hd webcam' })`,
|
|
58
|
+
`addCall(wall, el, { camera: 'label:brio' })`. Case-insensitive; with no permission yet the default
|
|
59
|
+
camera is primed once and let go so the labels become visible; no match rejects with `no-camera`.
|
|
60
|
+
Explicit like a `deviceId`, so it opens an excluded device too. The call sample passes
|
|
61
|
+
`?camera=label:…` (and a bare `?camera=<deviceId>`) straight through.
|
|
62
|
+
|
|
63
|
+
## 1.48.0 — 2026-11-03
|
|
64
|
+
|
|
65
|
+
### Changed — auto-convergence is measured on the sender's raw frames (`./call`)
|
|
66
|
+
|
|
67
|
+
Preview tier. A stereo participant was converged by each RECEIVER, which measured the disparity of
|
|
68
|
+
the point between the person's eyes on the frame it decoded: a compressed copy, often downscaled
|
|
69
|
+
by WebRTC, measured again by every participant. The sender now measures it on its own raw camera
|
|
70
|
+
frames, before encoding, and sends the result on the data channel (`conv { d, c }`, `d` a
|
|
71
|
+
fraction of the per-eye width; at most 5 a second while the person moves, one every 400 ms while
|
|
72
|
+
they hold still). Each receiver still applies it as a per-eye crop, so the stream keeps the full
|
|
73
|
+
picture and every receiver keeps its own depth slider and display. The sender measures whatever
|
|
74
|
+
its self view does, including `selfView: false`, and sends nothing while its camera is off or from
|
|
75
|
+
a mono camera.
|
|
76
|
+
|
|
77
|
+
- New option **`sendConvergence`** (default `true`; `<dxr-call no-send-convergence>`). `false`
|
|
78
|
+
sends nothing and leaves the measuring to the receivers.
|
|
79
|
+
- Additive on the wire, so the wire version stays 1. A sender advertises it with `hello.conv:
|
|
80
|
+
true`. A sender without it (1.47 and earlier), or one silent for 1.2 s, is measured by the
|
|
81
|
+
receiver exactly as before, and an older receiver ignores `conv`. The depth cursor still copies
|
|
82
|
+
the received frames while it hovers a tile.
|
|
83
|
+
- `diagnostics().peers[i].autoConverge` gains `source` (`'sender'` / `'local'` / `null`),
|
|
84
|
+
`senderFrac` and `samples`. A failed local measurement no longer drops a value held from the
|
|
85
|
+
sender.
|
|
86
|
+
- `docs/call.md` § *Auto-convergence is measured by the sender*.
|
|
87
|
+
|
|
88
|
+
### Added — headless-mode hooks for an app that owns its layout (`./call`)
|
|
89
|
+
|
|
90
|
+
Preview tier. Everything a chat app that lays out the call itself (`ui: false`, `layout: 'none'`)
|
|
91
|
+
needed and had to work around; the tiles stay module-owned. Docs: `docs/call.md` § *Hooks for an
|
|
92
|
+
app that owns its layout*.
|
|
93
|
+
|
|
94
|
+
- **Names.** Option `name` (≤ 64 characters, control and bidi characters stripped), sent in
|
|
95
|
+
`hello`; `CallPeer.name`; `handle.setName()` re-sends it; event `name { id, name }`. Additive on
|
|
96
|
+
the wire (wire version stays 1): an older peer ignores it.
|
|
97
|
+
- **App messages.** `handle.send(peerId, data)` / `handle.broadcast(data)` on the call's data
|
|
98
|
+
channel (`{ type: 'app', d }`): JSON, ≤ 16 KiB serialised, ≤ 20 a second per participant (over a
|
|
99
|
+
cap: `false`, nothing sent); event `message { from, data }`. Only app messages reach the page.
|
|
100
|
+
An older peer drops them silently.
|
|
101
|
+
- **Events.** `signal { state, code? }` (+ `handle.signal`): connected / reconnecting / closed.
|
|
102
|
+
`weave { id, state, reason?, ms }`: a tile's first join and its re-join after a resize, per
|
|
103
|
+
participant and `'self'`. `speaking { id, on }`, for participants and `'self'`.
|
|
104
|
+
- **Layout.** `tileCover` (default `'auto'` with `layout: 'none'`): the core's `rectCover` on every
|
|
105
|
+
woven tile and a stereo self view. `theme.tileAspect: 'fill'`: no aspect imposed, each tile's
|
|
106
|
+
crop follows its own box (the numeric default is unchanged). `handle.tile('self')`
|
|
107
|
+
(`data-dxr-peer="self"`); under `layout: 'none'` the footer and self slot are
|
|
108
|
+
`display: contents` so the page places the self tile. `handle.setSelfView(on)` removes / restores
|
|
109
|
+
its layer. A hidden (0×0) tile is no longer painted or measured.
|
|
110
|
+
- **Devices.** `handle.setMic(deviceId)`, `handle.setAudioOutput(sinkId)` (tiles that join later
|
|
111
|
+
too), `handle.localStream`.
|
|
112
|
+
- **Quality.** The `quality` event gains `path` (`'host'` / `'srflx'` / `'prflx'` / `'relay'`, from
|
|
113
|
+
the selected ICE candidate pair) and `rttMs`.
|
|
114
|
+
- `<dxr-call>`: `name` attribute, `tile-aspect="fill"`, and the five new events as `dxr-call:*`.
|
|
115
|
+
|
|
116
|
+
### Fixed — the camera probe opened a hand-tracking camera (`./camera`, `./call`)
|
|
117
|
+
|
|
118
|
+
Preview tier. Found on hardware (Windows, DisplayXR Browser 1.9.1): with no stereo pair open,
|
|
119
|
+
`camera: 'auto'`'s last-resort "try every device" loop opened a hand-tracking camera — an ordinary
|
|
120
|
+
video input, wide enough to pass for a side-by-side pair — and the call sent its picture as 3D.
|
|
121
|
+
Every probe path (the named-pair fast path, the default camera, the last resort) now skips devices
|
|
122
|
+
whose label marks a sensor (hand tracking, eye tracking, a "tracking camera", a depth camera), plus
|
|
123
|
+
the new `openCamera({ exclude })` / call `cameraExclude` list; `false` turns the built-in list off.
|
|
124
|
+
Left-out devices appear in `skipped` with `excluded: true`; an explicit `deviceId` is opened
|
|
125
|
+
regardless. Also: the browser's "3D Camera (DisplayXR)" is listed LAST on Windows, so the probe no
|
|
126
|
+
longer assumes it is the default — a default that is a pair only by its shape no longer wins over a
|
|
127
|
+
camera that names itself a pair, and with an excluded default the first plain camera is the
|
|
128
|
+
fallback. `addCameraView` gains `aspect: 'fill'` and `rectCover`.
|
|
129
|
+
|
|
130
|
+
### Added — the dev signalling server ships in the package
|
|
131
|
+
|
|
132
|
+
`signaling/dev-server.mjs` and the five modules it imports are in the published files (not an
|
|
133
|
+
`exports` entry): run it by path, or import `startDevServer` from the file. Copy it unbundled
|
|
134
|
+
(`signaling/README.md`).
|
|
135
|
+
|
|
136
|
+
### Docs — frosted chrome over a tile works on DisplayXR Browser 0.1.16+
|
|
137
|
+
|
|
138
|
+
`docs/woven-canvas-rules.md` rule 10 and §5, and `docs/call.md`: since DisplayXR Browser 0.1.16 the
|
|
139
|
+
compositor draws a tile mono inside each `backdrop-filter` rect (displayxr-browser#120;
|
|
140
|
+
`samples/composition` is the validated case), so partial frosted chrome over a tile is fine. Never
|
|
141
|
+
frost a whole visible tile; effects on a tile or its ancestors (rule 7) are unchanged. The SDK's
|
|
142
|
+
own chrome keeps its near-solid tint. `docs/call.md` also notes that changing only the `#room=`
|
|
143
|
+
fragment of a page with a live call does not re-join (reload, or navigate with a changed query).
|
|
9
144
|
|
|
10
145
|
## 1.47.6 — 2026-11-03
|
|
11
146
|
|
package/call.d.ts
CHANGED
|
@@ -154,8 +154,12 @@ export interface CallTheme {
|
|
|
154
154
|
radius?: number | string;
|
|
155
155
|
/** Chrome font family → `--dxr-font`. */
|
|
156
156
|
font?: string;
|
|
157
|
-
/**
|
|
158
|
-
|
|
157
|
+
/**
|
|
158
|
+
* Tile aspect (w/h) = the woven buffer's per-eye aspect, the same for every tile. Default 16/9.
|
|
159
|
+
* `'fill'`: no aspect is imposed — the page sizes each tile's box (layout: 'none') and each
|
|
160
|
+
* tile's crop follows its own stage box (clamped to 0.3–4), the self view's too.
|
|
161
|
+
*/
|
|
162
|
+
tileAspect?: number | 'fill';
|
|
159
163
|
/** Overrides for the chrome's text (also localisation). */
|
|
160
164
|
strings?: CallStrings;
|
|
161
165
|
}
|
|
@@ -193,6 +197,24 @@ export interface CallOptions {
|
|
|
193
197
|
format?: CallFormat;
|
|
194
198
|
/** Microphone (echo cancellation + noise suppression). Default true. */
|
|
195
199
|
audio?: boolean;
|
|
200
|
+
/**
|
|
201
|
+
* Your display name, sent to every participant in `hello` (their `CallPeer.name`). At most 64
|
|
202
|
+
* characters; control characters are stripped. Change it with `setName()`. Default none.
|
|
203
|
+
*/
|
|
204
|
+
name?: string;
|
|
205
|
+
/**
|
|
206
|
+
* Devices `camera: 'auto' | 'stereo' | 'mono'` never opens — `openCamera`'s `exclude`: strings
|
|
207
|
+
* match a label (case-insensitive substring) or a deviceId, RegExps the label. Labels of
|
|
208
|
+
* sensors (hand tracking, eye tracking, a "tracking camera", a depth camera) are always left out
|
|
209
|
+
* too; `false` turns that built-in list off. An explicit deviceId is opened regardless.
|
|
210
|
+
*/
|
|
211
|
+
cameraExclude?: ReadonlyArray<string | RegExp> | false;
|
|
212
|
+
/**
|
|
213
|
+
* Cover each woven tile (and a stereo self view) across a resize or move until it is woven
|
|
214
|
+
* again — the core's `rectCover: 'auto'` on its registration. Default `'auto'` with
|
|
215
|
+
* `layout: 'none'` (your layout moves the tiles), `'off'` otherwise.
|
|
216
|
+
*/
|
|
217
|
+
tileCover?: 'auto' | 'off';
|
|
196
218
|
/**
|
|
197
219
|
* Lift mono peers to 3D on a 3D display (RFC §5). `'auto'` (default): import
|
|
198
220
|
* `@displayxr/inline3d/lift` lazily from this copy of the SDK, and use it when it reports a
|
|
@@ -210,11 +232,20 @@ export interface CallOptions {
|
|
|
210
232
|
*/
|
|
211
233
|
liftOptions?: Record<string, unknown> & { max?: number };
|
|
212
234
|
/**
|
|
213
|
-
* Auto-convergence (default true):
|
|
214
|
-
*
|
|
215
|
-
*
|
|
235
|
+
* Auto-convergence (default true): shift each SBS peer's two eyes by half the disparity of the
|
|
236
|
+
* point between that person's eyes, so the remote person sits at the display plane. No
|
|
237
|
+
* calibration needed. The depth slider stays an offset on top. The disparity comes from the
|
|
238
|
+
* sender when it measures it (`sendConvergence`, 1.48+); from an older sender, this side measures
|
|
239
|
+
* the received frames itself.
|
|
216
240
|
*/
|
|
217
241
|
autoConverge?: boolean;
|
|
242
|
+
/**
|
|
243
|
+
* With a stereo camera (default true): measure the disparity of the point between your eyes on
|
|
244
|
+
* your own raw camera frames, before compression, and send it to every peer a few times a second
|
|
245
|
+
* (a few bytes on the data channel). Their `autoConverge` applies it, so nobody measures the
|
|
246
|
+
* decoded frames. `false`: send nothing; receivers measure, as before 1.48.
|
|
247
|
+
*/
|
|
248
|
+
sendConvergence?: boolean;
|
|
218
249
|
/**
|
|
219
250
|
* `'depth'`: over a 3D (stereo) tile — a peer's or your self view — the cursor is never
|
|
220
251
|
* overlapped by what comes out of the screen: where the person under the pointer pops out, the
|
|
@@ -242,6 +273,11 @@ export interface CallOptions {
|
|
|
242
273
|
autoJoin?: boolean;
|
|
243
274
|
/** The small self view (mirrored; a stereo self view is mirrored AND eye-swapped — an `addCameraView`). Default true. */
|
|
244
275
|
selfView?: boolean;
|
|
276
|
+
/**
|
|
277
|
+
* While you share content (`shareContent`), also show it to yourself: a woven preview tile,
|
|
278
|
+
* `tile('self', 'content')`. Default false (you have the app itself on your display).
|
|
279
|
+
*/
|
|
280
|
+
selfContentView?: boolean;
|
|
245
281
|
/** Colours, radius, font, tile aspect and strings — see {@link CallTheme}. */
|
|
246
282
|
theme?: CallTheme;
|
|
247
283
|
/** Invite links: `base` (default this page's URL, query kept, fragment replaced); `updateUrl` writes `#room=` into the page URL on join (default: `ui === true`). */
|
|
@@ -277,6 +313,14 @@ export type LiftFunction = ((element: HTMLVideoElement, opts: Record<string, unk
|
|
|
277
313
|
export interface CallQuality {
|
|
278
314
|
in: { width: number; height: number; fps: number; codec: string | null; kbps: number | null; dropped: number; decoder: string | null } | null;
|
|
279
315
|
out: { width: number; height: number; fps: number; codec: string | null; kbps: number | null; limitation: string | null; encoder: string | null } | null;
|
|
316
|
+
/**
|
|
317
|
+
* The network path of the selected ICE candidate pair: `'relay'` (through TURN, either end),
|
|
318
|
+
* else the local candidate's type — `'host'` (direct, same network), `'srflx'` / `'prflx'`
|
|
319
|
+
* (direct through NAT). Null before a pair is selected.
|
|
320
|
+
*/
|
|
321
|
+
path?: 'host' | 'srflx' | 'prflx' | 'relay' | null;
|
|
322
|
+
/** The selected pair's current round-trip time, ms; null when unknown. */
|
|
323
|
+
rttMs?: number | null;
|
|
280
324
|
}
|
|
281
325
|
|
|
282
326
|
/** A remote participant, as a page needs it. Debug data (hello, route, convergence, lift) is in `diagnostics()`. */
|
|
@@ -288,8 +332,87 @@ export interface CallPeer {
|
|
|
288
332
|
readonly muted: boolean;
|
|
289
333
|
readonly cameraOff: boolean;
|
|
290
334
|
readonly speaking: boolean;
|
|
335
|
+
/** Their display name (`name` option / `setName()`), sanitised; null when they sent none (or an older SDK). */
|
|
336
|
+
readonly name: string | null;
|
|
337
|
+
/** What they share (3D app share): `on`, its format and the app's name (null while `on` is false). */
|
|
338
|
+
readonly content: { readonly on: boolean; readonly format: CallFormat | null; readonly appName: string | null };
|
|
339
|
+
}
|
|
340
|
+
|
|
341
|
+
/** Which of a participant's tiles: their camera (the person) or the content they share. */
|
|
342
|
+
export type CallTileKind = 'camera' | 'content';
|
|
343
|
+
|
|
344
|
+
/**
|
|
345
|
+
* The convergence of shared content. `known: true`: the frames are DISPLAY-REFERRED — `frac` is the
|
|
346
|
+
* disparity of the sender's display plane as a fraction of the per-eye width (`zeroDisparityShiftFrac`
|
|
347
|
+
* of a DisplayXR app, 0 in v1). Receivers apply it directly and never measure the frames; their
|
|
348
|
+
* depth control adds to it. `known: false`: shown as sent.
|
|
349
|
+
*/
|
|
350
|
+
export interface ContentConvergence {
|
|
351
|
+
known: boolean;
|
|
352
|
+
frac: number | null;
|
|
353
|
+
}
|
|
354
|
+
|
|
355
|
+
/** What `shareContent` announces (also `openAppShare()`'s `descriptor`, which fits it as is). */
|
|
356
|
+
export interface ContentDescriptor {
|
|
357
|
+
format?: CallFormat;
|
|
358
|
+
width?: number;
|
|
359
|
+
height?: number;
|
|
360
|
+
convergence?: ContentConvergence;
|
|
361
|
+
appName?: string | null;
|
|
362
|
+
}
|
|
363
|
+
|
|
364
|
+
/** `handle.content`: what you share right now. */
|
|
365
|
+
export interface CallContent {
|
|
366
|
+
readonly on: true;
|
|
367
|
+
readonly format: CallFormat;
|
|
368
|
+
readonly width: number;
|
|
369
|
+
readonly height: number;
|
|
370
|
+
readonly appName: string | null;
|
|
371
|
+
readonly convergence: Readonly<ContentConvergence>;
|
|
372
|
+
}
|
|
373
|
+
|
|
374
|
+
/** A running DisplayXR app the browser offers to share (`listShareableApps`). */
|
|
375
|
+
export interface ShareableApp {
|
|
376
|
+
readonly deviceId: string;
|
|
377
|
+
/** `DisplayXR: <app> (3D)`. */
|
|
378
|
+
readonly label: string;
|
|
379
|
+
readonly appName: string | null;
|
|
380
|
+
}
|
|
381
|
+
|
|
382
|
+
/** An opened app share (`openAppShare`): pass `stream` (+ `descriptor`) to `shareContent`. The track is yours to stop. */
|
|
383
|
+
export interface AppShare {
|
|
384
|
+
stream: MediaStream;
|
|
385
|
+
track: MediaStreamTrack;
|
|
386
|
+
descriptor: { kind: 'content'; format: CallFormat; width: number; height: number; convergence: { known: true; frac: number }; appName: string | null };
|
|
291
387
|
}
|
|
292
388
|
|
|
389
|
+
/** Why `openAppShare` rejected (`error.code`): the user said no at the runtime's prompt, it failed, or the device is not an app. */
|
|
390
|
+
export type AppShareErrorCode = 'share-denied' | 'share-failed' | 'not-app-share';
|
|
391
|
+
|
|
392
|
+
/**
|
|
393
|
+
* Is this a running DisplayXR app offered for 3D screen share (runtime ADR-061) rather than a
|
|
394
|
+
* camera? A `MediaDeviceInfo`, a label, a track, or its settings: true on the label
|
|
395
|
+
* `DisplayXR: <app> (3D)`, a `dxr-app:` device id, or `displayxrStereo.source === 'app'`. No camera
|
|
396
|
+
* probe (`camera: 'auto'`, `openCamera`) ever opens one.
|
|
397
|
+
*/
|
|
398
|
+
export function isAppShareDevice(device: MediaDeviceInfo | MediaStreamTrack | MediaTrackSettings | { label?: string; deviceId?: string } | string | null | undefined): boolean;
|
|
399
|
+
|
|
400
|
+
/**
|
|
401
|
+
* The running apps this browser offers to share. Empty before the page holds camera permission
|
|
402
|
+
* (the browser lists none until then), on any other browser, or when none is shareable.
|
|
403
|
+
*/
|
|
404
|
+
export function listShareableApps(opts?: { mediaDevices?: MediaDevices }): Promise<ShareableApp[]>;
|
|
405
|
+
|
|
406
|
+
/** `cb(apps)` now and whenever the list changes (`devicechange`). Returns the unsubscribe. */
|
|
407
|
+
export function watchShareableApps(cb: (apps: ShareableApp[]) => void, opts?: { mediaDevices?: MediaDevices }): () => void;
|
|
408
|
+
|
|
409
|
+
/**
|
|
410
|
+
* Open a shareable app as a side-by-side track (default 2560×720 at 30 fps). The runtime asks the
|
|
411
|
+
* user on THIS machine now; Deny rejects with `code: 'share-denied'`. The track ends when the app
|
|
412
|
+
* exits. Stop it yourself when you stop sharing.
|
|
413
|
+
*/
|
|
414
|
+
export function openAppShare(deviceId: string, opts?: { width?: number; height?: number; frameRate?: number; mediaDevices?: MediaDevices }): Promise<AppShare>;
|
|
415
|
+
|
|
293
416
|
/**
|
|
294
417
|
* `error` codes — fatal for a feature, never for the widget. `'camera-busy'` (held by another
|
|
295
418
|
* app: audio-only until `retryCamera()`), `'no-camera'`, `'permission-denied'`, `'unreachable'`
|
|
@@ -326,7 +449,7 @@ export type CallErrorCode =
|
|
|
326
449
|
| (string & {});
|
|
327
450
|
|
|
328
451
|
/** `warning` codes — degraded, not broken. */
|
|
329
|
-
export type CallWarningCode = 'lift-not-bundled' | 'turn-shed' | (string & {});
|
|
452
|
+
export type CallWarningCode = 'lift-not-bundled' | 'turn-shed' | 'content-unsupported' | (string & {});
|
|
330
453
|
|
|
331
454
|
export interface CallEvents {
|
|
332
455
|
joined: { room: string; id: string };
|
|
@@ -335,8 +458,8 @@ export interface CallEvents {
|
|
|
335
458
|
peer: { id: string };
|
|
336
459
|
peerleft: { id: string; reason: string };
|
|
337
460
|
state: { id: string; state: PeerState };
|
|
338
|
-
/** How this side shows a participant changed (a hello arrived, lift went live, the wall came back). */
|
|
339
|
-
display: { id: string; display: CallDisplay };
|
|
461
|
+
/** How this side shows a participant changed (a hello arrived, lift went live, the wall came back). `kind`: which of their tiles. */
|
|
462
|
+
display: { id: string; display: CallDisplay; kind: CallTileKind };
|
|
340
463
|
speaker: { id: string | null };
|
|
341
464
|
/**
|
|
342
465
|
* Per peer (`id`) every 2 s; or, with `id: null` and `lift`, when lifted tiles on the web
|
|
@@ -352,8 +475,35 @@ export interface CallEvents {
|
|
|
352
475
|
* `'turn-shed'`: only THIS session got no TURN from the hosted service (an anonymous session
|
|
353
476
|
* while the monthly relay budget is nearly spent, or too many credential requests from this
|
|
354
477
|
* address this hour); the call runs STUN-only. A publishable key keeps TURN.
|
|
478
|
+
* `'content-unsupported'`: you share content and a participant's SDK predates it (no `content` in
|
|
479
|
+
* their hello) — they keep seeing your camera only. Once per participant per share.
|
|
355
480
|
*/
|
|
356
481
|
warning: { code: CallWarningCode; message: string };
|
|
482
|
+
/** A participant's display name arrived or changed (`name: null` = they cleared it). */
|
|
483
|
+
name: { id: string; name: string | null };
|
|
484
|
+
/**
|
|
485
|
+
* An app message from a participant (`send` / `broadcast` on their side): `data` as they sent it
|
|
486
|
+
* (JSON). Only app messages surface here — the call's own wire messages never do.
|
|
487
|
+
*/
|
|
488
|
+
message: { from: string; data: unknown };
|
|
489
|
+
/**
|
|
490
|
+
* The signalling connection: `'connected'` on join and after a reconnect, `'reconnecting'` while
|
|
491
|
+
* it retries (media keeps flowing), `'closed'` when it gave up — `code` says why (e.g. `'expired'`).
|
|
492
|
+
*/
|
|
493
|
+
signal: { state: 'connected' | 'reconnecting' | 'closed'; code?: string };
|
|
494
|
+
/**
|
|
495
|
+
* A tile's weave state settled: its first join (`firstWoven`), or its re-join after a resize
|
|
496
|
+
* (`rewoven()`). `id` is a participant's, or `'self'` for your self view. `'flat'`: the wall
|
|
497
|
+
* says the tile will not weave (`reason`, e.g. `'layer-failed'`) — it is shown flat.
|
|
498
|
+
*/
|
|
499
|
+
weave: { id: string; state: 'woven' | 'flat'; reason?: string; ms: number; kind: CallTileKind };
|
|
500
|
+
/** A participant (or `'self'`) started / stopped speaking. */
|
|
501
|
+
speaking: { id: string; on: boolean; kind: CallTileKind };
|
|
502
|
+
/**
|
|
503
|
+
* A participant (or `'self'`) started or stopped sharing content (3D app share). `reason` on your
|
|
504
|
+
* own stop: `'stopped'` (`stopContent()`), `'ended'` (the track ended — the app exited), `'left'`.
|
|
505
|
+
*/
|
|
506
|
+
content: { id: string; on: boolean; appName: string | null; reason?: 'stopped' | 'ended' | 'left' };
|
|
357
507
|
}
|
|
358
508
|
|
|
359
509
|
export interface CallHandle {
|
|
@@ -391,8 +541,48 @@ export interface CallHandle {
|
|
|
391
541
|
setDepth(v: number | null): number;
|
|
392
542
|
/** Turn mono→3D on/off at runtime (no argument toggles). Off releases every lift; tiles go flat. */
|
|
393
543
|
setMono3D(on?: boolean): boolean;
|
|
394
|
-
/**
|
|
395
|
-
|
|
544
|
+
/**
|
|
545
|
+
* A participant's tile element (for `layout: 'none'`), or null; `'self'` is your self view
|
|
546
|
+
* (`data-dxr-peer="self"`). Style it in place; never move it.
|
|
547
|
+
*/
|
|
548
|
+
tile(peerId: string, kind?: CallTileKind): HTMLElement | null;
|
|
549
|
+
/**
|
|
550
|
+
* Share content on a second video track (3D app share): a running DisplayXR app's views from
|
|
551
|
+
* `openAppShare()` (pass its `descriptor` too), or any side-by-side track. Every participant on a
|
|
552
|
+
* DisplayXR SDK with content support gets a content tile; older ones keep seeing your camera only
|
|
553
|
+
* (`warning` `'content-unsupported'`). Replaces a previous share. Resolves with what is announced.
|
|
554
|
+
*/
|
|
555
|
+
shareContent(src: MediaStream | MediaStreamTrack, opts?: ContentDescriptor): Promise<CallContent>;
|
|
556
|
+
/** Stop sharing. `stopTrack: true` also stops the track (ends the runtime's share). Resolves false when nothing was shared. */
|
|
557
|
+
stopContent(opts?: { stopTrack?: boolean }): Promise<boolean>;
|
|
558
|
+
/** What you share right now, or null. */
|
|
559
|
+
readonly content: CallContent | null;
|
|
560
|
+
/** Your display name, or null. */
|
|
561
|
+
readonly name: string | null;
|
|
562
|
+
/** Change your display name (re-sent to everyone in `hello`). Returns it sanitised (null = none). */
|
|
563
|
+
setName(name: string | null): string | null;
|
|
564
|
+
/**
|
|
565
|
+
* An app message to one participant over the call's data channel (arrives as their `message`
|
|
566
|
+
* event). `data` must be JSON-serialisable and at most 16 KiB serialised; at most 20 messages a
|
|
567
|
+
* second per participant. Returns false (nothing sent) when over a cap, not serialisable, or the
|
|
568
|
+
* participant is not connected. A participant on an SDK before 1.48 drops it silently.
|
|
569
|
+
*/
|
|
570
|
+
send(peerId: string, data: unknown): boolean;
|
|
571
|
+
/** {@link CallHandle.send} to everyone. True only if every connected participant got it. */
|
|
572
|
+
broadcast(data: unknown): boolean;
|
|
573
|
+
/** The signalling connection — see the `signal` event. Null before join and after leave. */
|
|
574
|
+
readonly signal: 'connected' | 'reconnecting' | 'closed' | null;
|
|
575
|
+
/**
|
|
576
|
+
* Show (true) or hide (false) your self view; no argument toggles. Hiding removes its layer
|
|
577
|
+
* from the wall (never a hidden live registration). Returns the new state.
|
|
578
|
+
*/
|
|
579
|
+
setSelfView(on?: boolean): boolean;
|
|
580
|
+
/** Switch the microphone (a `deviceId`; null = the default). Resolves false, keeping the old one, if it cannot open. */
|
|
581
|
+
setMic(deviceId: string | null): Promise<boolean>;
|
|
582
|
+
/** Play every participant's audio (also later ones) to `sinkId` (`HTMLMediaElement.setSinkId`). Resolves false if any tile refused. */
|
|
583
|
+
setAudioOutput(sinkId: string): Promise<boolean>;
|
|
584
|
+
/** What you send (camera + microphone). Read-only — use `setCamera` / `setMic` to change it. */
|
|
585
|
+
readonly localStream: MediaStream | null;
|
|
396
586
|
on<K extends keyof CallEvents>(type: K, cb: (e: CallEvents[K]) => void): () => void;
|
|
397
587
|
off<K extends keyof CallEvents>(type: K, cb: (e: CallEvents[K]) => void): void;
|
|
398
588
|
/** Explicitly UNSTABLE: the wall, the transport, per-peer hello / route / convergence / lift / quality, the lift pool. For devtools. */
|
|
@@ -421,8 +611,8 @@ export function parseInviteLink(link: string | { hash?: string; href?: string }
|
|
|
421
611
|
// a bubbling, composed `CustomEvent` named `dxr-call:<event>` with the payload in `detail`, and
|
|
422
612
|
// exposes the handle as `el.call`. Attributes (read once, at connect): `room`, `signaling` (a
|
|
423
613
|
// URL), `key`, `camera`, `layout`, `accent`, `max-peers`, `no-ui`, `ui="tiles"`, `auto-join`,
|
|
424
|
-
// `mono3d="off"`, `no-audio`, `no-self-view`, `
|
|
425
|
-
// `browser-url`, `debug`. Everything else is `el.options`, set before connect; options win.
|
|
614
|
+
// `mono3d="off"`, `no-audio`, `no-self-view`, `self-content-view`, `no-auto-converge`, `no-send-convergence`,
|
|
615
|
+
// `tile-aspect`, `invite-base`, `browser-url`, `debug`. Everything else is `el.options`, set before connect; options win.
|
|
426
616
|
|
|
427
617
|
export class DxrCallElement extends HTMLElement {
|
|
428
618
|
/** The mount seam: null = `mountCall` of `./call`; `./call/full` sets its own. */
|
package/camera.d.ts
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
// The public surface of this file is pinned by test/api-snapshot: a change here fails `npm test`
|
|
12
12
|
// until the snapshot is updated in the same change (`node tools/api-surface.mjs --update`).
|
|
13
13
|
|
|
14
|
-
import type { Inline3D, Inline3DUnsupported, TileHandle } from './index.js';
|
|
14
|
+
import type { Inline3D, Inline3DUnsupported, TileHandle, RectCoverOption } from './index.js';
|
|
15
15
|
import type { CursorShape, CursorShapeHit, CursorShapeOption, CursorSoftenOption } from './cursor-depth.js';
|
|
16
16
|
|
|
17
17
|
/** What the camera delivers: a side-by-side pair (left eye left) or one flat picture. */
|
|
@@ -21,8 +21,11 @@ export type CameraFormat = 'sbs' | 'mono';
|
|
|
21
21
|
* Which camera to open. `'auto'` (default): a stereo device when one is present — the DisplayXR
|
|
22
22
|
* Browser's "3D Camera" (recognised by its `displayxrStereo` track settings, never by its label),
|
|
23
23
|
* or a device delivering wider than 2.5:1 frames — else the default webcam. `'stereo'` prefers
|
|
24
|
-
* the pair and falls back to mono; `'mono'` never probes; a string is a `deviceId
|
|
25
|
-
* `
|
|
24
|
+
* the pair and falls back to mono; `'mono'` never probes; a string is a `deviceId`, or
|
|
25
|
+
* `'label:<text>'` for the first camera whose label contains the text (case-insensitive; for a URL
|
|
26
|
+
* or a config, where a per-origin deviceId is useless); a `MediaStream` is used as given, with
|
|
27
|
+
* {@link CameraOptions.format} declaring what it is. A `deviceId` or `label:` is explicit and
|
|
28
|
+
* opens an {@link CameraOptions.exclude excluded} device too.
|
|
26
29
|
*/
|
|
27
30
|
export type CameraPreference = 'auto' | 'stereo' | 'mono' | (string & {}) | MediaStream;
|
|
28
31
|
|
|
@@ -56,6 +59,20 @@ export interface CameraOptions {
|
|
|
56
59
|
* is then true. A throwing hook keeps the raw pair (`cam.rectifyError`).
|
|
57
60
|
*/
|
|
58
61
|
rectify?: (stream: MediaStream, info: { width: number; height: number; deviceId: string | null; label: string }) => MediaStream | Promise<MediaStream>;
|
|
62
|
+
/**
|
|
63
|
+
* Devices no probe may open. Built in, always (unless `false`): video inputs whose label marks a
|
|
64
|
+
* sensor rather than a camera — hand tracking, an eye tracker's or a "tracking camera"'s stream,
|
|
65
|
+
* a depth camera, an infrared / IR camera, and the Ultraleap / Leap Motion hand tracker, which
|
|
66
|
+
* enumerates under its bare brand name
|
|
67
|
+
* (`/\b(hand[- ]?track|tracking camera|eye[- ]?track|depth camera|ultraleap|leap motion|infrared|ir camera)/i`).
|
|
68
|
+
* Add your own: a string matches a label (case-insensitive substring) or a deviceId exactly; a
|
|
69
|
+
* RegExp tests the label. Applied on every probe path, the last-resort "try every device" one
|
|
70
|
+
* included; while an excluded device is listed the default camera is not opened blind (the first
|
|
71
|
+
* other camera is opened by id), and a default that turns out to be excluded is closed again.
|
|
72
|
+
* Left-out devices are listed in `skipped` with `excluded: true`. `false` turns the built-in
|
|
73
|
+
* list off. An explicit `deviceId` or `label:` (`prefer`) is your choice and is opened regardless.
|
|
74
|
+
*/
|
|
75
|
+
exclude?: ReadonlyArray<string | RegExp> | false;
|
|
59
76
|
/** Verbose console logging. */
|
|
60
77
|
debug?: boolean;
|
|
61
78
|
}
|
|
@@ -64,7 +81,18 @@ export interface CameraOptions {
|
|
|
64
81
|
export type CameraErrorCode = 'camera-busy' | 'permission-denied' | 'no-camera';
|
|
65
82
|
export interface CameraError extends Error {
|
|
66
83
|
code: CameraErrorCode;
|
|
67
|
-
skipped: ReadonlyArray<
|
|
84
|
+
skipped: ReadonlyArray<CameraSkip>;
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
/** A device {@link openCamera} tried and did not use: busy, refused, or left out (`excluded`, `error: 'excluded'`). */
|
|
88
|
+
export interface CameraSkip {
|
|
89
|
+
label: string;
|
|
90
|
+
error: string;
|
|
91
|
+
busy?: boolean;
|
|
92
|
+
denied?: boolean;
|
|
93
|
+
excluded?: boolean;
|
|
94
|
+
/** Why it was left out: `'excluded'` (a sensor, or your `exclude`), or `'app-share'` (a running DisplayXR app offered for 3D screen share — never a camera). */
|
|
95
|
+
reason?: 'excluded' | 'app-share';
|
|
68
96
|
}
|
|
69
97
|
|
|
70
98
|
export interface CameraEvents {
|
|
@@ -162,7 +190,7 @@ export interface StereoCamera {
|
|
|
162
190
|
/** True when this module opened the device (and `close()` stops it); false for a page-supplied stream. */
|
|
163
191
|
readonly owned: boolean;
|
|
164
192
|
/** Devices tried and not used (busy, denied, not a pair) — the clue a `'camera-busy'` report carries. */
|
|
165
|
-
readonly skipped: ReadonlyArray<
|
|
193
|
+
readonly skipped: ReadonlyArray<CameraSkip>;
|
|
166
194
|
readonly state: 'live' | 'ended' | 'closed';
|
|
167
195
|
/** A detached, playing `<video>` on the stream (never in the DOM); null without a document. */
|
|
168
196
|
readonly video: HTMLVideoElement | null;
|
|
@@ -214,8 +242,17 @@ export interface CameraViewOptions {
|
|
|
214
242
|
autoConverge?: boolean;
|
|
215
243
|
/** [-1, 1], + = push back: an offset on top of the convergence. Default 0. */
|
|
216
244
|
depth?: number;
|
|
217
|
-
/**
|
|
218
|
-
|
|
245
|
+
/**
|
|
246
|
+
* The tile's aspect (w/h) = the woven buffer's per-eye aspect. Default 16/9. `'fill'`: the
|
|
247
|
+
* canvas's own CSS box, read every frame (clamped to 0.3–4), for a page that sizes the tile.
|
|
248
|
+
*/
|
|
249
|
+
aspect?: number | 'fill';
|
|
250
|
+
/**
|
|
251
|
+
* The core's rect cover (`addImage`'s `rectCover`) on the woven registration: `'auto'`
|
|
252
|
+
* covers the tile across a later resize or move until it is woven again. Default: the core's
|
|
253
|
+
* (`'off'`).
|
|
254
|
+
*/
|
|
255
|
+
rectCover?: RectCoverOption;
|
|
219
256
|
/**
|
|
220
257
|
* `'depth'`: while the pointer is over the woven view and what is under it comes out of the
|
|
221
258
|
* screen, the normal arrow is drawn just in front of it (runtime ADR-046, measured off the pair's
|