@xmaxai/web-sdk 1.0.5 → 1.0.7

package/README.md

@@ -214,11 +214,6 @@ The following JavaScript snippet creates a camera stream, starts real-time gener
 and binds the output to a video view. Run this within an async function in the browser,
 after a user action such as clicking a Start button.
 
-Choose a model with `RealtimeModel.x2_0` (`x2.0`) or
-`RealtimeModel.x2_0_trtc` (`x2.0-trtc`). For Agora, use
-`RealtimeModel.x2_0_agora` (`x2.0-agora`) with `RtcProvider.agora`. For VeRTC, use
-`RealtimeModel.x2_1_preview` (`x2.1-preview`) with `RtcProvider.vertc`.
-
 ```javascript
 import {
   CameraPosition,
@@ -237,7 +232,7 @@ const client = new XmaxClient(
 );
 
 const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({ model: RealtimeModel.x2_0_trtc })
+  new RealtimeConfiguration({ model: RealtimeModel.x2_1_preview })
 );
 
 const localStream = await realtime.createLocalCameraStream({
@@ -272,11 +267,11 @@ videoView.remoteTrack = remoteStream.videoTrack;
 Add a `video-container` element with an explicit width and height to your page.
 The view displays a local camera preview until the first generated frame arrives.
 
-Use `updateVideoFormat` after creating the local camera stream, including during
+Use `updateUplinkVideoFormat` after creating the local camera stream, including during
 generation, to change upstream encoding without reconnecting or restarting the task:
 
 ```javascript
-await realtime.updateVideoFormat(new RealtimeVideoFormat({
+await realtime.updateUplinkVideoFormat(new RealtimeVideoFormat({
   width: 1024,
   height: 1920,
   fps: 20,
@@ -285,89 +280,6 @@ await realtime.updateVideoFormat(new RealtimeVideoFormat({
 }));
 ```
 
-Pass a complete format; omitted bitrate and preference fields use SDK defaults.
-The local track's `videoFormat` is updated on success and reused on reconnect.
-Generation and remote frame interpolation retain the original format. Actual sent
-resolution, frame rate, and bitrate depend on the device, browser, and network;
-use the statistics callbacks to observe them. Await each operation before starting
-another configuration or lifecycle operation.
-
-Each model declares its supported RTC providers. The SDK selects the model's
-default provider when `provider` is omitted. To use Agora, replace the manager
-configuration above; camera capture, generation, rendering, and cleanup use the same APIs:
-
-```javascript
-const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({
-    model: RealtimeModel.x2_0_agora,
-  })
-);
-```
-
-To use VeRTC, select the preview model:
-
-```javascript
-const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({
-    model: RealtimeModel.x2_1_preview,
-  })
-);
-```
-
-The provider is fixed for each manager: `x2.0` and `x2.0-trtc` use
-`RtcProvider.trtc`, `x2.0-agora` uses `RtcProvider.agora`, and
-`x2.1-preview` uses `RtcProvider.vertc`.
-Import `RtcProvider` to specify it explicitly; unsupported known-model/provider
-combinations are rejected during configuration. Use the exported
-`supportedRtcProviders(model)` to inspect model capabilities. The SDK does not read
-`modelExtra.provider` from the backend; the resolved configuration determines how
-RTC credentials are parsed.
-
-The `x2.0-agora` and `x2.1-preview` models use
-`https://dev.xmaxai.com/open/api/v1` in both environments.
-The other built-in models and file uploads use the configured environment's API endpoint.
-
-Custom realtime model names are also accepted without an SDK update:
-
-```javascript
-const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({ model: "your-new-model" })
-);
-```
-
-The model name is sent unchanged; the server determines whether it is available.
-Unknown models inherit the `x2.1-preview` defaults: VeRTC, the dev session API above
-(in both client environments), a 1024 × 1920 camera at 30 fps, and the same
-1024 × 1920 / 1920 × 1024 input resolution buckets and media rules.
-You may explicitly choose another supported `RtcProvider` for a custom model;
-`supportedRtcProviders` lists all SDK adapters for custom models, with VeRTC first,
-but this does not guarantee server-side support. `modelDisplayName` returns the
-custom name unchanged. Empty/blank or non-string model names, invalid RTC providers,
-and invalid video parameters are still rejected locally.
-
-VeRTC uses `@volcengine/rtc` 4.69.3 and the fixed AppID
-`69a177e226e9b90176a86b96`. Session credentials must contain this `rtc_app_id`,
-`room_id`, and `room_token`; the RTC user is `user_id` (falling back to `userUid`),
-and `bot_name` identifies the bot. Camera and microphone capture use VeRTC's SDK.
-The preview model uses the same media rules as `x2.0-trtc`: 1024 × 1920 or
-1920 × 1024 input, with 30 fps by default.
-
-VeRTC sends the existing room-event JSON as a single room text message, without
-chunk envelopes. The adapter rejects messages above 64 KiB of UTF-8 data; vendor
-send failures are returned to the caller. TRTC and Agora retain their chunking.
-HTTP session heartbeats start after joining and publishing the local stream, with
-the first request after 10 seconds. Changed tokens returned by heartbeat are
-applied to Agora and VeRTC; expiry callbacks also request fresh credentials.
-VeRTC restores media after token expiry without creating another business session
-or restarting the generation task. HTTP heartbeats maintain the session and are
-not treated as a client-side billing trigger.
-
-`XmaxConfiguration.environment` selects the Agora region (`china` or `global`);
-concurrent Agora managers must use the same environment. RTC modules load on demand.
-Statistics not supplied by a provider are `undefined`, including Agora playback
-buffer delay and separate uplink/downlink RTT. RTC end-to-end estimates do not
-include AI generation time and use each provider's own measurement definition.
-
 <br>
 
 ### Using React
@@ -422,6 +334,14 @@ For camera input, bind the returned video track to a preview view. The SDK enter
 `ready` after it has received a valid frame and the preview view is bound; observe
 this through `setStateListener`.
 
+At generation start, the SDK sends `params.mirror: true` for the front camera
+and `false` for the rear camera, local files, and network videos. The backend
+mirrors the input; generated results are not automatically mirrored in the
+browser. The front-camera local preview remains mirrored independently.
+This flag is sent only with `start`: switching cameras during an active generation
+does not update it. Disconnect and start generation again to apply the new camera
+position to the backend.
+
 <br>
 
 ### Resource Cleanup
@@ -456,156 +376,6 @@ this through `setStateListener`.
 
 <br>
 
-### Generate from a local video
-
-Use a manager configured for TRTC, Agora or VeRTC. The SDK decodes a local `File` or `Blob`, sends its video through Canvas and its audio through Web Audio, and receives the generated result over RTC. It does not upload the file to storage or request camera/microphone access.
-
-```ts
-// Call directly from a file-selection or click handler to unlock browser playback.
-const localStream = await realtime.createLocalVideoStream({
-  file,
-  videoFormat: new RealtimeVideoFormat({ width: 1920, height: 1024, fps: 30 }),
-  loop: true,
-  includeAudio: true,
-});
-
-// Optionally bind localStream.videoTrack to an SDK video view.
-const remoteStream = await realtime.startGeneration({
-  localStream,
-  context: new RealtimeContext({ prompt: "Transform the scene into an animation" }),
-});
-```
-
-Preparation stops at the first frame. After the connection is established and the start signal is sent, the file plays from the beginning; playback does not wait for a remote frame. `loop` defaults to `true`, looping both file video and audio; set it to `false` to play once. The current protocol does not acknowledge server input readiness, so frame-perfect processing of the very beginning is not guaranteed.
-
-For local files, requested dimensions automatically select the model resolution with the closest aspect ratio, preferring landscape on ties; exact matches are preserved. For example, `1280×720` selects `1920×1024`. Models without fixed resolutions retain their existing pixel-budget sizing rules. The file is scaled proportionally into the resulting Canvas, with black bars where needed, and the returned track's `videoFormat` reflects that target size. This does not change camera or network-video size validation. Local preview is always silent; `localAudioVolume` does not alter the file's uplink audio. File audio is enabled by default; a file without an audio track produces silence. Set `includeAudio: false` to omit the audio track entirely. Use `setRemoteAudioVolume` to hear the generated result (muted by default).
-
-`disconnect()` pauses playback and retains the source. A new generation after reconnecting starts from the beginning; updating conditions during generation does not restart playback. With looping disabled, file completion holds the final frame and leaves the RTC connection open so remote tail frames can play. `close()` releases all file resources and tracks. `updateVideoFormat()` can adjust uplink encoding without changing the original model input format.
-
-File codecs must be supported by the browser, and Canvas capture and Web Audio must be available. Keep the page foregrounded for stable frame delivery; browser background throttling can reduce the upload frame rate. Playback restrictions, decode errors and preparation timeouts are reported rather than silently sending an empty stream.
-
-### Generate from a network video
-
-Pass an HTTP or HTTPS video URL to let the server read the source directly. The SDK joins the RTC room and receives generated video and audio without capturing or publishing local media.
-
-```ts
-import { RealtimeContext, RealtimeVideoFormat, RealtimeVideoSampleMethod } from "@xmaxai/web-sdk";
-
-const localStream = await realtime.createNetworkVideoStream({
-  url: "https://example.com/source.mp4",
-  videoFormat: new RealtimeVideoFormat({ width: 1920, height: 1024, fps: 30 }),
-  sampleMethod: RealtimeVideoSampleMethod.time,
-  onFinish: () => {
-    console.log("Server finished processing the video");
-  },
-});
-
-const remoteStream = await realtime.startGeneration({
-  localStream,
-  context: new RealtimeContext({ prompt: "Transform the scene into an animation" }),
-});
-```
-
-Use a video format supported by the selected model. Bind the returned tracks to SDK video views as usual. The optional local preview plays once, muted, independently of server processing; no Canvas or local upload is involved. Creating the source does not start generation. Sampling defaults to `time`; `fps` is also available.
-
-The URL must be directly accessible to the server. Browser cookies and custom authorization headers are not forwarded. Local preview playback depends on browser codec support and page security policies; a preview failure does not stop server processing. The selected model's backend must support network video input.
-
-`onFinish` is delivered once for the current task after the server's `video_stopped` message and generation readiness. It does not mean the last remote frame has finished playing, so the SDK does not automatically disconnect. Local preview completion does not trigger this callback. Remote audio follows `setRemoteAudioVolume` (muted by default).
-
-`disconnect()` retains the source and preview for reconnection; `close()` releases them. Close the current source before creating another. `updateVideoFormat()` adjusts camera uplink encoding and is not available for a network video source.
-
-### Non-realtime video tasks
-
-Use `client.createNonRealtimeManager()` to submit video-file processing tasks.
-This is cloud processing, not on-device or offline execution. It uses the client's
-API key and selects a dedicated task endpoint from `XmaxEnvironment`:
-
-- `china`: `https://api.xmaxai.com/open/api/v1`
-- `global`: `https://api.xmax.ai/open/api/v1`
-
-Task requests append `/offline-task` and its subpaths to this base URL.
-Realtime model endpoint overrides do not affect task requests; storage credentials
-continue to use the environment's default API endpoint.
-
-```typescript
-import { NonRealtimeQuality, NonRealtimeTaskError } from "@xmaxai/web-sdk";
-
-const manager = client.createNonRealtimeManager();
-const controller = new AbortController();
-
-// videoFile is a File selected by the user.
-const storedVideo = await client.createStorageService().uploadVideo({
-  data: videoFile,
-  fileName: videoFile.name,
-  onProgress: (loaded, total) => console.log(loaded, total),
-});
-
-const task = await manager.submitTask({
-  videoPath: storedVideo.url,
-  referencePath: referenceImageURL, // Optional; obtain via uploadImage().
-  prompt: "Change the character's clothing while preserving the motion.",
-  quality: NonRealtimeQuality.hd,
-  // Omit fps to retain the source frame rate, including fractional rates.
-});
-
-// Persist task.uid before waiting, so the task can be queried after a reload.
-try {
-  const completed = await manager.waitForCompletion(task.uid, {
-    intervalMs: 2000,
-    timeoutMs: 30 * 60 * 1000, // Optional client-side deadline.
-    signal: controller.signal,
-    onTaskUpdated: (snapshot) => console.log(snapshot.status),
-  });
-  console.log(completed.result?.url);
-} catch (error) {
-  if (error instanceof NonRealtimeTaskError) {
-    console.error(error.message, error.task);
-  } else {
-    throw error;
-  }
-}
-```
-
-- `submitTask()` returns immediately after acceptance and never automatically
-  retries. A timeout or invalid response may still mean the server created and
-  charged for the task; do not blindly resubmit.
-- `waitForCompletion()` polls serially and retries transient query failures up to
-  three consecutive times by default, with backoff. A successful query resets the
-  retry count. Without `timeoutMs`, waiting continues until completion, failure, or
-  cancellation. `controller.abort()` stops local waiting and the in-flight query;
-  it does **not** cancel or refund the server task.
-- `getTask(uid)` returns any task status, including `error`. Waiting for an `error`
-  task, or a completed task without a usable result, throws `NonRealtimeTaskError`
-  with the task snapshot attached.
-- `getTasks([uid, ...])` queries up to 100 IDs. `listTasks({ pageNumber, pageSize,
-  status })` retrieves one page, newest first. Saved IDs can be queried or waited
-  on using a new manager instance.
-- Upload local videos with `client.createStorageService().uploadVideo()` before
-  submitting a task. It shares the image upload options and progress callback,
-  uploads the original bytes via COS `putObject`, and returns a `StoredFile`.
-  It does not transcode or use multipart upload; COS simple upload supports files
-  up to [5GB](https://cloud.tencent.com/document/product/436/14113).
-  MIME inference supports MP4/M4V, MOV, WebM, AVI, MKV, and OGV; successful storage
-  does not guarantee that the task processor supports the file's codec or container.
-
-### Downloading videos
-
-Call this from a user-initiated action, such as a download button:
-
-```typescript
-await client.createStorageService().downloadVideo({
-  url: videoURL,
-  fileName: "video.mp4",
-  // signal: controller.signal, // Optional cancellation.
-});
-```
-
-The method fetches the complete file as a Blob and then requests browser saving.
-The URL must be directly accessible and allow CORS for cross-origin requests;
-the SDK does not send API credentials or cookies to it. This requires a browser
-document and retains the complete file until the browser takes over. Resolution
-means saving was requested, not that the user saved the file to disk.
-
 ## Example Project
 
 A complete example application featuring React is available in
@@ -618,7 +388,7 @@ It demonstrates real-time generation using live camera feeds and reference image
 
 - <ins><strong>Tencent Cloud TRTC SDK for Web</strong></ins> enables low-latency, real-time audio and video communication.
 - <ins><strong>Agora RTC SDK for Web</strong></ins> provides an alternative RTC transport for the Agora model.
-- <ins><strong>Volcengine VeRTC SDK for Web</strong></ins> provides RTC transport and device capture for the X2.1 preview model.
+- <ins><strong>Volcengine VeRTC SDK for Web</strong></ins> enables real-time audio and video communication.
 - <ins><strong>Tencent Cloud COS SDK</strong></ins> handles media upload and download via object storage.
 
 <br>

package/dist/{VideoContentMode-BM97_H8V.d.cts → VideoRenderStatistics-C7zqzfuS.d.cts}

@@ -190,4 +190,26 @@ declare enum VideoContentMode {
     fill = "fill"
 }
 
-export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, RealtimeVideoEncoderPreference as b, type RealtimeVideoFormatInit as c };
+/**
+ * 单个视频视图最近采样窗口的呈现帧率，不代表屏幕实际刷新帧率。
+ */
+interface VideoRenderStatistics {
+    /**
+     * video 按浏览器 presentedFrames 增量计数；canvas 按实际呈现提交计数。
+     */
+    readonly source: "video" | "canvas";
+    /**
+     * 原帧与插值帧的总呈现帧率，单位 fps。
+     */
+    readonly frameRate: number;
+    /**
+     * 原帧呈现帧率，单位 fps。
+     */
+    readonly originalFrameRate: number;
+    /**
+     * 插值帧呈现帧率，单位 fps；未开启插帧时为 0。
+     */
+    readonly interpolatedFrameRate: number;
+}
+
+export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, type VideoRenderStatistics as b, RealtimeVideoEncoderPreference as c, type RealtimeVideoFormatInit as d };

package/dist/{VideoContentMode-BM97_H8V.d.ts → VideoRenderStatistics-C7zqzfuS.d.ts}

@@ -190,4 +190,26 @@ declare enum VideoContentMode {
     fill = "fill"
 }
 
-export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, RealtimeVideoEncoderPreference as b, type RealtimeVideoFormatInit as c };
+/**
+ * 单个视频视图最近采样窗口的呈现帧率，不代表屏幕实际刷新帧率。
+ */
+interface VideoRenderStatistics {
+    /**
+     * video 按浏览器 presentedFrames 增量计数；canvas 按实际呈现提交计数。
+     */
+    readonly source: "video" | "canvas";
+    /**
+     * 原帧与插值帧的总呈现帧率，单位 fps。
+     */
+    readonly frameRate: number;
+    /**
+     * 原帧呈现帧率，单位 fps。
+     */
+    readonly originalFrameRate: number;
+    /**
+     * 插值帧呈现帧率，单位 fps；未开启插帧时为 0。
+     */
+    readonly interpolatedFrameRate: number;
+}
+
+export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, type VideoRenderStatistics as b, RealtimeVideoEncoderPreference as c, type RealtimeVideoFormatInit as d };