@xmaxai/web-sdk 1.0.3 → 1.0.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,637 @@
1
+ <p align="center">
2
+ <img src="./docs/images/brand/xmax-sdk.png" alt="XmaxSDK — Realtime Interactive Video Generation" width="880">
3
+ </p>
4
+
5
+ <p align="center">
6
+ <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript"><img src="https://img.shields.io/badge/JavaScript-ES2020-F7DF1E" alt="JavaScript ES2020"></a>
7
+ <a href="https://platform.xmaxai.com/"><img src="https://img.shields.io/badge/Realtime-AI-FF9500" alt="Realtime AI"></a>
8
+ <a href="./LICENSE"><img src="https://img.shields.io/badge/License-MIT-4C9A2A" alt="MIT License"></a>
9
+ </p>
10
+
11
+ We introduce XmaxSDK, a JavaScript SDK designed for real-time interactive video generation via Xmax models. XmaxSDK implements an end-to-end pipeline covering media acquisition, video streaming, frame-by-frame generation, and on-device rendering, enabling developers to seamlessly integrate low-latency, high-fidelity video transformations into creative applications at a much lower cost than alternative solutions.
12
+
13
+ <p align="center"><img src="./docs/images/xlab/generation-demo.gif" alt="X-Lab realtime generation demo" width="33%" /><img src="./docs/images/xlab/index-demo.gif" alt="X-Lab index demo" width="33%" /><img src="./docs/images/xlab/storage-demo.gif" alt="X-Lab storage demo" width="33%" /></p>
14
+
15
+ <br>
16
+
17
+ ## What you can build with XmaxSDK
18
+
19
+ <table>
20
+ <tr>
21
+ <th width="24%" align="left">Realtime Use Case</th>
22
+ <th width="60%" align="left">Description</th>
23
+ <th width="16%" align="center">Demo</th>
24
+ </tr>
25
+ <tr>
26
+ <td rowspan="2" width="24%" valign="middle">
27
+ <strong>Character Swapping</strong>
28
+ </td>
29
+ <td width="60%" valign="middle">
30
+ Replace anyone in your live feed with a designated avatar in real-time.
31
+ </td>
32
+ <td rowspan="2" width="16%" align="center" valign="middle">
33
+ <a href="./docs/videos/use-cases/character-swapping.mp4">
34
+ <img src="./docs/images/use-cases/character-swapping-poster.png" alt="Play the Character Swapping demo" width="120">
35
+ <br>
36
+ <sub>▶ Play demo</sub>
37
+ </a>
38
+ </td>
39
+ </tr>
40
+ <tr>
41
+ <td width="60%" valign="middle">
42
+ <strong>Prompt:</strong> <code>视频中角色替换成参考图中角色</code>
43
+ <br><br>
44
+ <strong>Reference image:</strong> Select a clear image of the desired character with a clean background.
45
+ </td>
46
+ </tr>
47
+ <tr>
48
+ <td rowspan="2" width="24%" valign="middle">
49
+ <strong>Virtual Try-On</strong>
50
+ </td>
51
+ <td width="60%" valign="middle">
52
+ Seamlessly change outfits, preserving exact body shape, natural motion, and an
53
+ authentic fit.
54
+ </td>
55
+ <td rowspan="2" width="16%" align="center" valign="middle">
56
+ <a href="./docs/videos/use-cases/virtual-try-on.mp4">
57
+ <img src="./docs/images/use-cases/virtual-try-on-poster.png" alt="Play the Virtual Try-On demo" width="120">
58
+ <br>
59
+ <sub>▶ Play demo</sub>
60
+ </a>
61
+ </td>
62
+ </tr>
63
+ <tr>
64
+ <td width="60%" valign="middle">
65
+ <strong>Prompt:</strong> <code>视频中人物衣服替换成参考图中衣服</code>
66
+ <br><br>
67
+ <strong>Reference image:</strong> Select a clear image of the target outfit with a clean background.
68
+ </td>
69
+ </tr>
70
+ <tr>
71
+ <td rowspan="2" width="24%" valign="middle">
72
+ <strong>Video Restyling</strong>
73
+ </td>
74
+ <td width="60%" valign="middle">
75
+ Reimagine your world in any style with an immersive visual experience.
76
+ </td>
77
+ <td rowspan="2" width="16%" align="center" valign="middle">
78
+ <a href="./docs/videos/use-cases/video-restyling.mp4">
79
+ <img src="./docs/images/use-cases/video-restyling-poster.png" alt="Play the Video Restyling demo" width="120">
80
+ <br>
81
+ <sub>▶ Play demo</sub>
82
+ </a>
83
+ </td>
84
+ </tr>
85
+ <tr>
86
+ <td width="60%" valign="middle">
87
+ <strong>Prompt:</strong> <code>视频风格变为参考图指定的风格</code>
88
+ <br><br>
89
+ <strong>Reference image:</strong> Select an image that captures the artistic style you want to apply.
90
+ </td>
91
+ </tr>
92
+ <tr>
93
+ <td rowspan="2" width="24%" valign="middle">
94
+ <strong>AI Companions</strong>
95
+ </td>
96
+ <td width="60%" valign="middle">
97
+ Summon virtual characters into your live camera feed and interact with them
98
+ through gestures.
99
+ </td>
100
+ <td rowspan="2" width="16%" align="center" valign="middle">
101
+ <a href="./docs/videos/use-cases/ai-companions.mp4">
102
+ <img src="./docs/images/use-cases/ai-companions-poster.png" alt="Play the AI Companions demo" width="120">
103
+ <br>
104
+ <sub>▶ Play demo</sub>
105
+ </a>
106
+ </td>
107
+ </tr>
108
+ <tr>
109
+ <td width="60%" valign="middle">
110
+ <strong>Prompt:</strong> <code>指定角色在场景中互动</code>
111
+ <br><br>
112
+ <strong>Reference image:</strong> Select a clear image of the virtual character you want to summon with a clean background.
113
+ </td>
114
+ </tr>
115
+ <tr>
116
+ <td rowspan="2" width="24%" valign="middle">
117
+ <strong>Live Photo</strong>
118
+ </td>
119
+ <td width="60%" valign="middle">
120
+ Animate and control characters in your images simply by drawing motion
121
+ trajectories.
122
+ </td>
123
+ <td rowspan="2" width="16%" align="center" valign="middle">
124
+ <a href="./docs/videos/use-cases/live-photo.mp4">
125
+ <img src="./docs/images/use-cases/live-photo-poster.png" alt="Play the Live Photo demo" width="120">
126
+ <br>
127
+ <sub>▶ Play demo</sub>
128
+ </a>
129
+ </td>
130
+ </tr>
131
+ <tr>
132
+ <td width="60%" valign="middle">
133
+ <strong>Prompt:</strong> <code>让画面自然动起来</code>
134
+ <br><br>
135
+ <strong>Reference image:</strong> Use the input image as the reference
136
+ </td>
137
+ </tr>
138
+ </table>
139
+
140
+ <br>
141
+
142
+ ## Why XmaxSDK?
143
+
144
+ <table>
145
+ <thead>
146
+ <tr>
147
+ <th height="104" align="center" valign="middle">
148
+ <img src="./docs/images/why/low-latency.svg" alt="Low latency" width="36" height="36"><br>Low latency
149
+ </th>
150
+ <th height="104" align="center" valign="middle">
151
+ <img src="./docs/images/why/low-cost.svg" alt="Cost efficiency" width="36" height="36"><br>Cost efficiency
152
+ </th>
153
+ <th height="104" align="center" valign="middle">
154
+ <img src="./docs/images/why/high-fidelity.svg" alt="High fidelity" width="36" height="36"><br>High fidelity
155
+ </th>
156
+ </tr>
157
+ </thead>
158
+ <tbody>
159
+ <tr>
160
+ <td>End-to-end latency is measured in <img src="./docs/images/why/latency-highlight.svg" alt="hundreds of milliseconds" width="192" height="20" align="absmiddle">, ensuring that updates to generation conditions and interaction controls are reflected instantly.</td>
161
+ <td>Run on a <img src="./docs/images/why/gpu-highlight.svg" alt="single RTX 5090" width="126" height="20" align="absmiddle">, reducing inference costs by orders of magnitude versus datacenter GPUs like H100.</td>
162
+ <td>Our models support real-time generation at up to <img src="./docs/images/why/resolution-highlight.svg" alt="1080p" width="48" height="20" align="absmiddle">, delivering production-ready, high-quality video output.</td>
163
+ </tr>
164
+ </tbody>
165
+ </table>
166
+
167
+ <br>
168
+
169
+ ## Prerequisites
170
+
171
+ - A WebRTC-enabled browser running on HTTPS or localhost
172
+ - Node.js 18 or later, with pnpm or npm for development
173
+ - An Xmax API key
174
+
175
+ > [!WARNING]
176
+ > Never commit your Xmax API key to version control or embed a long-lived key in
177
+ > client-side bundles. Use short-lived temporary keys issued by your backend
178
+ > through the Xmax API. For step-by-step
179
+ > instructions, see [Authentication](https://platform.xmaxai.com/docs/authentication).
180
+
181
+ <br>
182
+
183
+ ## Installation
184
+
185
+ ```bash
186
+ npm install @xmaxai/web-sdk
187
+ ```
188
+
189
+ The package includes optional React components at `@xmaxai/web-sdk/react`.
190
+ React applications use their own React installation; non-React applications do not need React.
191
+
192
+ <br>
193
+
194
+ ## Quick Start
195
+
196
+ ### Configure permissions
197
+
198
+ Serve your application over HTTPS or localhost and allow camera access in the
199
+ browser. When embedding the application in an iframe, allow camera access:
200
+
201
+ ```html
202
+ <iframe src="https://your-app.example" allow="camera; microphone; autoplay"></iframe>
203
+ ```
204
+
205
+ Configure permissions to match your application's user experience. XmaxSDK
206
+ automatically prompts for camera access when creating the video stream and throws
207
+ an `XmaxError` if permission is denied or unavailable.
208
+
209
+ <br>
210
+
211
+ ### Generate and display video
212
+
213
+ The following JavaScript snippet creates a camera stream, starts real-time generation,
214
+ and binds the output to a video view. Run this within an async function in the browser,
215
+ after a user action such as clicking a Start button.
216
+
217
+ Choose a model with `RealtimeModel.x2_0` (`x2.0`) or
218
+ `RealtimeModel.x2_0_trtc` (`x2.0-trtc`). For Agora, use
219
+ `RealtimeModel.x2_0_agora` (`x2.0-agora`) with `RtcProvider.agora`. For VeRTC, use
220
+ `RealtimeModel.x2_1_preview` (`x2.1-preview`) with `RtcProvider.vertc`.
221
+
222
+ ```javascript
223
+ import {
224
+ CameraPosition,
225
+ RealtimeConfiguration,
226
+ RealtimeContext,
227
+ RealtimeModel,
228
+ RealtimeVideoFormat,
229
+ VideoContentMode,
230
+ XmaxClient,
231
+ XmaxConfiguration,
232
+ XmaxRealtimeVideoView,
233
+ } from "@xmaxai/web-sdk";
234
+
235
+ const client = new XmaxClient(
236
+ new XmaxConfiguration({ apiKey: "YOUR_TEMPORARY_XMAX_API_KEY" })
237
+ );
238
+
239
+ const realtime = client.createRealtimeManager(
240
+ new RealtimeConfiguration({ model: RealtimeModel.x2_0_trtc })
241
+ );
242
+
243
+ const localStream = await realtime.createLocalCameraStream({
244
+ videoFormat: new RealtimeVideoFormat({ width: 1024, height: 1920, fps: 30 }),
245
+ position: CameraPosition.front,
246
+ useMicrophone: false,
247
+ enableFrameValidation: true, // Optional; waits a fixed 200ms camera warmup before publishing; false skips it.
248
+ });
249
+
250
+ const videoView = new XmaxRealtimeVideoView({
251
+ localTrack: localStream.videoTrack,
252
+ videoContentMode: VideoContentMode.fill,
253
+ });
254
+
255
+ videoView.element.style.width = "100%";
256
+ videoView.element.style.height = "100%";
257
+ const container = document.getElementById("video-container");
258
+ if (!container) throw new Error("Missing #video-container element");
259
+ videoView.attach(container);
260
+
261
+ const remoteStream = await realtime.startGeneration({
262
+ localStream,
263
+ context: new RealtimeContext({
264
+ prompt: "视频中角色替换成参考图中角色",
265
+ referencePath: "https://platform.xmaxai.com/images/source/charx/chatx_image1.jpg",
266
+ }),
267
+ });
268
+
269
+ videoView.remoteTrack = remoteStream.videoTrack;
270
+ ```
271
+
272
+ Add a `video-container` element with an explicit width and height to your page.
273
+ The view displays a local camera preview until the first generated frame arrives.
274
+
275
+ Use `updateVideoFormat` after creating the local camera stream, including during
276
+ generation, to change upstream encoding without reconnecting or restarting the task:
277
+
278
+ ```javascript
279
+ await realtime.updateVideoFormat(new RealtimeVideoFormat({
280
+ width: 1024,
281
+ height: 1920,
282
+ fps: 20,
283
+ minimumBitrate: 1000,
284
+ maximumBitrate: 2000,
285
+ }));
286
+ ```
287
+
288
+ Pass a complete format; omitted bitrate and preference fields use SDK defaults.
289
+ The local track's `videoFormat` is updated on success and reused on reconnect.
290
+ Generation and remote frame interpolation retain the original format. Actual sent
291
+ resolution, frame rate, and bitrate depend on the device, browser, and network;
292
+ use the statistics callbacks to observe them. Await each operation before starting
293
+ another configuration or lifecycle operation.
294
+
295
+ Each model declares its supported RTC providers. The SDK selects the model's
296
+ default provider when `provider` is omitted. To use Agora, replace the manager
297
+ configuration above; camera capture, generation, rendering, and cleanup use the same APIs:
298
+
299
+ ```javascript
300
+ const realtime = client.createRealtimeManager(
301
+ new RealtimeConfiguration({
302
+ model: RealtimeModel.x2_0_agora,
303
+ })
304
+ );
305
+ ```
306
+
307
+ To use VeRTC, select the preview model:
308
+
309
+ ```javascript
310
+ const realtime = client.createRealtimeManager(
311
+ new RealtimeConfiguration({
312
+ model: RealtimeModel.x2_1_preview,
313
+ })
314
+ );
315
+ ```
316
+
317
+ The provider is fixed for each manager: `x2.0` and `x2.0-trtc` use
318
+ `RtcProvider.trtc`, `x2.0-agora` uses `RtcProvider.agora`, and
319
+ `x2.1-preview` uses `RtcProvider.vertc`.
320
+ Import `RtcProvider` to specify it explicitly; unsupported known-model/provider
321
+ combinations are rejected during configuration. Use the exported
322
+ `supportedRtcProviders(model)` to inspect model capabilities. The SDK does not read
323
+ `modelExtra.provider` from the backend; the resolved configuration determines how
324
+ RTC credentials are parsed.
325
+
326
+ The `x2.0-agora` and `x2.1-preview` models use
327
+ `https://dev.xmaxai.com/open/api/v1` in both environments.
328
+ The other built-in models and file uploads use the configured environment's API endpoint.
329
+
330
+ Custom realtime model names are also accepted without an SDK update:
331
+
332
+ ```javascript
333
+ const realtime = client.createRealtimeManager(
334
+ new RealtimeConfiguration({ model: "your-new-model" })
335
+ );
336
+ ```
337
+
338
+ The model name is sent unchanged; the server determines whether it is available.
339
+ Unknown models inherit the `x2.1-preview` defaults: VeRTC, the dev session API above
340
+ (in both client environments), a 1024 × 1920 camera at 30 fps, and the same
341
+ 1024 × 1920 / 1920 × 1024 input resolution buckets and media rules.
342
+ You may explicitly choose another supported `RtcProvider` for a custom model;
343
+ `supportedRtcProviders` lists all SDK adapters for custom models, with VeRTC first,
344
+ but this does not guarantee server-side support. `modelDisplayName` returns the
345
+ custom name unchanged. Empty/blank or non-string model names, invalid RTC providers,
346
+ and invalid video parameters are still rejected locally.
347
+
348
+ VeRTC uses `@volcengine/rtc` 4.69.3 and the fixed AppID
349
+ `69a177e226e9b90176a86b96`. Session credentials must contain this `rtc_app_id`,
350
+ `room_id`, and `room_token`; the RTC user is `user_id` (falling back to `userUid`),
351
+ and `bot_name` identifies the bot. Camera and microphone capture use VeRTC's SDK.
352
+ The preview model uses the same media rules as `x2.0-trtc`: 1024 × 1920 or
353
+ 1920 × 1024 input, with 30 fps by default.
354
+
355
+ VeRTC sends the existing room-event JSON as a single room text message, without
356
+ chunk envelopes. The adapter rejects messages above 64 KiB of UTF-8 data; vendor
357
+ send failures are returned to the caller. TRTC and Agora retain their chunking.
358
+ HTTP session heartbeats start after joining and publishing the local stream, with
359
+ the first request after 10 seconds. Changed tokens returned by heartbeat are
360
+ applied to Agora and VeRTC; expiry callbacks also request fresh credentials.
361
+ VeRTC restores media after token expiry without creating another business session
362
+ or restarting the generation task. HTTP heartbeats maintain the session and are
363
+ not treated as a client-side billing trigger.
364
+
365
+ `XmaxConfiguration.environment` selects the Agora region (`china` or `global`);
366
+ concurrent Agora managers must use the same environment. RTC modules load on demand.
367
+ Statistics not supplied by a provider are `undefined`, including Agora playback
368
+ buffer delay and separate uplink/downlink RTT. RTC end-to-end estimates do not
369
+ include AI generation time and use each provider's own measurement definition.
370
+
371
+ <br>
372
+
373
+ ### Using React
374
+
375
+ Use `XmaxRealtimeVideo` as your primary React view. Store the local and remote
376
+ tracks in component state, updating them dynamically as streams become available:
377
+
378
+ ```jsx
379
+ import { XmaxRealtimeVideo } from "@xmaxai/web-sdk/react";
380
+ import { VideoContentMode } from "@xmaxai/web-sdk";
381
+
382
+ <XmaxRealtimeVideo
383
+ localTrack={localTrack}
384
+ remoteTrack={remoteTrack}
385
+ videoContentMode={VideoContentMode.fill}
386
+ style={{ width: "100%", height: "100%" }}
387
+ />
388
+ ```
389
+
390
+ See the [React component](./packages/xmax-sdk/src/react/XmaxRealtimeVideo.tsx) for state binding and the
391
+ [example project](#example-project) for a complete implementation.
392
+
393
+ <br>
394
+
395
+ ### Listen for events
396
+
397
+ After creating `realtime`, register the listeners you need before creating the
398
+ input stream or starting generation.
399
+
400
+ | Listener | Purpose |
401
+ | --- | --- |
402
+ | `setStateListener` | Observe pipeline states and termination reasons during real-time generation. |
403
+ | `setLaunchTimingListener` | Monitor camera, connection, first-frame, and total launch timings. |
404
+ | `setLocalVideoStatisticsListener` | Monitor uplink video resolution, frame rate, and bitrate. |
405
+ | `setRemoteVideoStatisticsListener` | Monitor downlink video statistics, packet loss, playback buffer delay, RTT, and estimated RTC end-to-end delay. |
406
+
407
+ For example, monitor state changes and errors:
408
+
409
+ ```javascript
410
+ await realtime.setStateListener((state) => {
411
+ console.log("State:", state.connectionState);
412
+ if (state.reason?.kind === "failure") {
413
+ console.error("Error:", state.reason.error.code, state.reason.error.message);
414
+ }
415
+ });
416
+ ```
417
+
418
+ Handle errors thrown by async calls with `try/catch`. Failures that end the realtime
419
+ workflow are also available through `state.reason` after cleanup completes.
420
+
421
+ For camera input, bind the returned video track to a preview view. The SDK enters
422
+ `ready` after it has received a valid frame and the preview view is bound; observe
423
+ this through `setStateListener`.
424
+
425
+ <br>
426
+
427
+ ### Resource Cleanup
428
+
429
+ - **`disconnect()` — Stop Remote Generation**
430
+
431
+ Stops remote generation and cancels billing while keeping the local camera stream
432
+ and preview active. Use this when ending the online session but staying on the
433
+ current screen. You can start a new session later using the same local stream:
434
+
435
+ ```javascript
436
+ await realtime.disconnect();
437
+ ```
438
+
439
+ - **`close()` — Full Teardown & Release**
440
+
441
+ Ends the remote session, stops local media capture, and releases all engine
442
+ resources. Use this when leaving or dismissing the generation screen:
443
+
444
+ ```javascript
445
+ await realtime.close();
446
+ ```
447
+
448
+ > **Note:** These methods are alternatives, not sequential steps. When exiting a
449
+ > screen, call `close()` directly—there is no need to call `disconnect()` first.
450
+
451
+ <br>
452
+
453
+ > [!TIP]
454
+ > For complete usage examples, including camera input, reference images,
455
+ > and live statistics, see the [example project](./examples/xlab-react).
456
+
457
+ <br>
458
+
459
+ ### Generate from a local video
460
+
461
+ Use a manager configured for TRTC, Agora or VeRTC. The SDK decodes a local `File` or `Blob`, sends its video through Canvas and its audio through Web Audio, and receives the generated result over RTC. It does not upload the file to storage or request camera/microphone access.
462
+
463
+ ```ts
464
+ // Call directly from a file-selection or click handler to unlock browser playback.
465
+ const localStream = await realtime.createLocalVideoStream({
466
+ file,
467
+ videoFormat: new RealtimeVideoFormat({ width: 1920, height: 1024, fps: 30 }),
468
+ loop: true,
469
+ includeAudio: true,
470
+ });
471
+
472
+ // Optionally bind localStream.videoTrack to an SDK video view.
473
+ const remoteStream = await realtime.startGeneration({
474
+ localStream,
475
+ context: new RealtimeContext({ prompt: "Transform the scene into an animation" }),
476
+ });
477
+ ```
478
+
479
+ Preparation stops at the first frame. After the connection is established and the start signal is sent, the file plays from the beginning; playback does not wait for a remote frame. `loop` defaults to `true`, looping both file video and audio; set it to `false` to play once. The current protocol does not acknowledge server input readiness, so frame-perfect processing of the very beginning is not guaranteed.
480
+
481
+ For local files, requested dimensions automatically select the model resolution with the closest aspect ratio, preferring landscape on ties; exact matches are preserved. For example, `1280×720` selects `1920×1024`. Models without fixed resolutions retain their existing pixel-budget sizing rules. The file is scaled proportionally into the resulting Canvas, with black bars where needed, and the returned track's `videoFormat` reflects that target size. This does not change camera or network-video size validation. Local preview is always silent; `localAudioVolume` does not alter the file's uplink audio. File audio is enabled by default; a file without an audio track produces silence. Set `includeAudio: false` to omit the audio track entirely. Use `setRemoteAudioVolume` to hear the generated result (muted by default).
482
+
483
+ `disconnect()` pauses playback and retains the source. A new generation after reconnecting starts from the beginning; updating conditions during generation does not restart playback. With looping disabled, file completion holds the final frame and leaves the RTC connection open so remote tail frames can play. `close()` releases all file resources and tracks. `updateVideoFormat()` can adjust uplink encoding without changing the original model input format.
484
+
485
+ File codecs must be supported by the browser, and Canvas capture and Web Audio must be available. Keep the page foregrounded for stable frame delivery; browser background throttling can reduce the upload frame rate. Playback restrictions, decode errors and preparation timeouts are reported rather than silently sending an empty stream.
486
+
487
+ ### Generate from a network video
488
+
489
+ Pass an HTTP or HTTPS video URL to let the server read the source directly. The SDK joins the RTC room and receives generated video and audio without capturing or publishing local media.
490
+
491
+ ```ts
492
+ import { RealtimeContext, RealtimeVideoFormat, RealtimeVideoSampleMethod } from "@xmaxai/web-sdk";
493
+
494
+ const localStream = await realtime.createNetworkVideoStream({
495
+ url: "https://example.com/source.mp4",
496
+ videoFormat: new RealtimeVideoFormat({ width: 1920, height: 1024, fps: 30 }),
497
+ sampleMethod: RealtimeVideoSampleMethod.time,
498
+ onFinish: () => {
499
+ console.log("Server finished processing the video");
500
+ },
501
+ });
502
+
503
+ const remoteStream = await realtime.startGeneration({
504
+ localStream,
505
+ context: new RealtimeContext({ prompt: "Transform the scene into an animation" }),
506
+ });
507
+ ```
508
+
509
+ Use a video format supported by the selected model. Bind the returned tracks to SDK video views as usual. The optional local preview plays once, muted, independently of server processing; no Canvas or local upload is involved. Creating the source does not start generation. Sampling defaults to `time`; `fps` is also available.
510
+
511
+ The URL must be directly accessible to the server. Browser cookies and custom authorization headers are not forwarded. Local preview playback depends on browser codec support and page security policies; a preview failure does not stop server processing. The selected model's backend must support network video input.
512
+
513
+ `onFinish` is delivered once for the current task after the server's `video_stopped` message and generation readiness. It does not mean the last remote frame has finished playing, so the SDK does not automatically disconnect. Local preview completion does not trigger this callback. Remote audio follows `setRemoteAudioVolume` (muted by default).
514
+
515
+ `disconnect()` retains the source and preview for reconnection; `close()` releases them. Close the current source before creating another. `updateVideoFormat()` adjusts camera uplink encoding and is not available for a network video source.
516
+
517
+ ### Non-realtime video tasks
518
+
519
+ Use `client.createNonRealtimeManager()` to submit video-file processing tasks.
520
+ This is cloud processing, not on-device or offline execution. It uses the client's
521
+ API key and selects a dedicated task endpoint from `XmaxEnvironment`:
522
+
523
+ - `china`: `https://api.xmaxai.com/open/api/v1`
524
+ - `global`: `https://api.xmax.ai/open/api/v1`
525
+
526
+ Task requests append `/offline-task` and its subpaths to this base URL.
527
+ Realtime model endpoint overrides do not affect task requests; storage credentials
528
+ continue to use the environment's default API endpoint.
529
+
530
+ ```typescript
531
+ import { NonRealtimeQuality, NonRealtimeTaskError } from "@xmaxai/web-sdk";
532
+
533
+ const manager = client.createNonRealtimeManager();
534
+ const controller = new AbortController();
535
+
536
+ // videoFile is a File selected by the user.
537
+ const storedVideo = await client.createStorageService().uploadVideo({
538
+ data: videoFile,
539
+ fileName: videoFile.name,
540
+ onProgress: (loaded, total) => console.log(loaded, total),
541
+ });
542
+
543
+ const task = await manager.submitTask({
544
+ videoPath: storedVideo.url,
545
+ referencePath: referenceImageURL, // Optional; obtain via uploadImage().
546
+ prompt: "Change the character's clothing while preserving the motion.",
547
+ quality: NonRealtimeQuality.hd,
548
+ // Omit fps to retain the source frame rate, including fractional rates.
549
+ });
550
+
551
+ // Persist task.uid before waiting, so the task can be queried after a reload.
552
+ try {
553
+ const completed = await manager.waitForCompletion(task.uid, {
554
+ intervalMs: 2000,
555
+ timeoutMs: 30 * 60 * 1000, // Optional client-side deadline.
556
+ signal: controller.signal,
557
+ onTaskUpdated: (snapshot) => console.log(snapshot.status),
558
+ });
559
+ console.log(completed.result?.url);
560
+ } catch (error) {
561
+ if (error instanceof NonRealtimeTaskError) {
562
+ console.error(error.message, error.task);
563
+ } else {
564
+ throw error;
565
+ }
566
+ }
567
+ ```
568
+
569
+ - `submitTask()` returns immediately after acceptance and never automatically
570
+ retries. A timeout or invalid response may still mean the server created and
571
+ charged for the task; do not blindly resubmit.
572
+ - `waitForCompletion()` polls serially and retries transient query failures up to
573
+ three consecutive times by default, with backoff. A successful query resets the
574
+ retry count. Without `timeoutMs`, waiting continues until completion, failure, or
575
+ cancellation. `controller.abort()` stops local waiting and the in-flight query;
576
+ it does **not** cancel or refund the server task.
577
+ - `getTask(uid)` returns any task status, including `error`. Waiting for an `error`
578
+ task, or a completed task without a usable result, throws `NonRealtimeTaskError`
579
+ with the task snapshot attached.
580
+ - `getTasks([uid, ...])` queries up to 100 IDs. `listTasks({ pageNumber, pageSize,
581
+ status })` retrieves one page, newest first. Saved IDs can be queried or waited
582
+ on using a new manager instance.
583
+ - Upload local videos with `client.createStorageService().uploadVideo()` before
584
+ submitting a task. It shares the image upload options and progress callback,
585
+ uploads the original bytes via COS `putObject`, and returns a `StoredFile`.
586
+ It does not transcode or use multipart upload; COS simple upload supports files
587
+ up to [5GB](https://cloud.tencent.com/document/product/436/14113).
588
+ MIME inference supports MP4/M4V, MOV, WebM, AVI, MKV, and OGV; successful storage
589
+ does not guarantee that the task processor supports the file's codec or container.
590
+
591
+ ### Downloading videos
592
+
593
+ Call this from a user-initiated action, such as a download button:
594
+
595
+ ```typescript
596
+ await client.createStorageService().downloadVideo({
597
+ url: videoURL,
598
+ fileName: "video.mp4",
599
+ // signal: controller.signal, // Optional cancellation.
600
+ });
601
+ ```
602
+
603
+ The method fetches the complete file as a Blob and then requests browser saving.
604
+ The URL must be directly accessible and allow CORS for cross-origin requests;
605
+ the SDK does not send API credentials or cookies to it. This requires a browser
606
+ document and retains the complete file until the browser takes over. Resolution
607
+ means saving was requested, not that the user saved the file to disk.
608
+
609
+ ## Example Project
610
+
611
+ A complete example application featuring React is available in
612
+ [`examples/xlab-react`](./examples/xlab-react).
613
+ It demonstrates real-time generation using live camera feeds and reference images.
614
+
615
+ <br>
616
+
617
+ ## Dependencies
618
+
619
+ - <ins><strong>Tencent Cloud TRTC SDK for Web</strong></ins> enables low-latency, real-time audio and video communication.
620
+ - <ins><strong>Agora RTC SDK for Web</strong></ins> provides an alternative RTC transport for the Agora model.
621
+ - <ins><strong>Volcengine VeRTC SDK for Web</strong></ins> provides RTC transport and device capture for the X2.1 preview model.
622
+ - <ins><strong>Tencent Cloud COS SDK</strong></ins> handles media upload and download via object storage.
623
+
624
+ <br>
625
+
626
+ ## Contact us
627
+
628
+ For bug reports and feature requests, please open a
629
+ [GitHub Issue](../../issues). For integration
630
+ assistance and technical support, contact us at [sdk@xmax.ai](mailto:sdk@xmax.ai).
631
+
632
+ <br>
633
+
634
+ ## License
635
+
636
+ XmaxSDK Web is available under the terms of the [MIT License](./LICENSE).
637
+ Third-party dependencies and model weights retain their respective licenses.