@xmaxai/web-sdk 1.0.6 → 1.0.8

package/README.md

@@ -214,9 +214,6 @@ The following JavaScript snippet creates a camera stream, starts real-time gener
 and binds the output to a video view. Run this within an async function in the browser,
 after a user action such as clicking a Start button.
 
-For Agora, use `RealtimeModel.x2_1_preview` (`x2.1-preview`). For TRTC, use
-`RealtimeModel.x2_0_trtc` (`x2.0-trtc`). `RealtimeModel.x2_0` (`x2.0`) uses VeRTC.
-
 ```javascript
 import {
   CameraPosition,
@@ -235,7 +232,7 @@ const client = new XmaxClient(
 );
 
 const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({ model: RealtimeModel.x2_0_trtc })
+  new RealtimeConfiguration({ model: RealtimeModel.x2_1_preview })
 );
 
 const localStream = await realtime.createLocalCameraStream({
@@ -283,141 +280,6 @@ await realtime.updateUplinkVideoFormat(new RealtimeVideoFormat({
 }));
 ```
 
-Pass a complete format; omitted bitrate and preference fields use SDK defaults.
-The local track's `videoFormat` is updated on success and reused on reconnect.
-Generation and remote frame interpolation retain the original format. Actual sent
-resolution, frame rate, and bitrate depend on the device, browser, and network;
-use the statistics callbacks to observe them. Await each operation before starting
-another configuration or lifecycle operation.
-
-For a frame-rate-only experiment after connecting a local camera or file stream:
-
-```javascript
-await realtime.updateUplinkFrameRate(24);
-```
-
-This preserves the last successfully applied width, height, resolved bitrate limits,
-and encoding preference. It does not recalculate default bitrates for the new frame
-rate or copy measured send statistics into the configuration. The complete encoder
-configuration is still resubmitted, so preservation of RTC's internal adaptation
-state is not guaranteed. Validate actual send statistics under weak networks.
-
-**Agora camera sender-only experiment:** camera capture stays at its initial settings.
-Camera updates use `RTCRtpSender.setParameters()` (`scaleResolutionDownBy`,
-`maxFramerate`, `maxBitrate` in bps), not Agora's `setEncoderConfiguration()`.
-Only the maximum bitrate is applied; the standard sender API cannot enforce the
-configured minimum bitrate. Pre-publish updates are queued and applied immediately
-after publication, so the very first transmitted frames may use the initial profile.
-Scaling uses the actual captured dimensions, preserves aspect ratio, and does not upscale.
-Sender replacement/rejoin reapplies the latest target. Unsupported or silently ignored
-parameters are reported as errors, without falling back to camera constraints.
-Read-back confirms accepted parameters, not actual frame dimensions or persistent
-ownership against Agora's later adjustments; validate Chrome/Safari using RTC statistics.
-Search `Agora Sender Parameters` for accepted limits and `Agora RTC Statistics` for
-capture/send dimensions. File-video encoding and other RTC providers are unchanged.
-
-Agora uplink and downlink adaptation are always enabled; no configuration flag or manual
-selector is needed. Other RTC providers are not yet integrated. After generation
-starts, three consecutive poor uplink quality samples lower one tier, and five good
-samples probe an upgrade. Five tiers jointly change resolution and frame rate:
-1024×1920/30, 768×1440/24, 640×1200/24, 512×960/20, and 384×720/16.
-Landscape dimensions are swapped; custom sources scale proportionally from their
-initial or most recently manually applied format, with even dimensions and FPS
-capped by that baseline. Automatic updates preserve this baseline across reconnects.
-Default bitrate limits are recalculated for each tier; explicit limits and encoding
-preference are preserved. Uplink changes do not alter generation or result-stream sizes.
-Downlink independently uses the same quality thresholds and five resolution tiers,
-relative to the generation's original dimensions. It sends only `change_target_size`
-with `[width, height]`, without changing FPS, bitrate limits, or model inference.
-The protocol has no application ACK: a successful send records a requested tier,
-not a confirmed output size; use remote RTC statistics to verify actual dimensions.
-New Agora local tracks read the last applied uplink tier before publishing
-(the camera sender-only experiment applies it immediately after publication);
-new generation tasks restore the last requested downlink tier via `start.params.target_size`.
-Each direction falls back to L2 when no valid history is available. History is stored
-in `localStorage`, separately by model and direction, with no expiry or stability window
-in this first version. Storage failures never block generation. The original L1 baseline
-is retained for recovery; reconnecting the same local track keeps its in-memory uplink tier.
-Downlink changes only resolution, not FPS. Interpolation follows the requested dimensions,
-showing raw video while frame sizes do not match. See
-[the strategy and weak-network test steps](docs/realtime-adaptive-quality.md).
-
-Each model declares its supported RTC providers. The SDK selects the model's
-default provider when `provider` is omitted. To use Agora, replace the manager
-configuration above; camera capture, generation, rendering, and cleanup use the same APIs:
-
-```javascript
-const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({
-    model: RealtimeModel.x2_1_preview,
-  })
-);
-```
-
-To use VeRTC, select `x2.0`:
-
-```javascript
-const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({
-    model: RealtimeModel.x2_0,
-  })
-);
-```
-
-The provider is fixed for each manager: `x2.0-trtc` uses `RtcProvider.trtc`,
-`x2.1-preview` uses `RtcProvider.agora`, and `x2.0` uses `RtcProvider.vertc`.
-Import `RtcProvider` to specify it explicitly; unsupported known-model/provider
-combinations are rejected during configuration. Use the exported
-`supportedRtcProviders(model)` to inspect model capabilities. The SDK does not read
-`modelExtra.provider` from the backend; the resolved configuration determines how
-RTC credentials are parsed.
-
-The `x2.1-preview` model uses `https://cloud.xmax.22duck.cn/open/api/v1` in both environments.
-The other built-in models and file uploads use the configured
-environment's production API endpoint: `https://cloud.xmax.22duck.cn/open/api/v1`
-for China and `https://api.xmax.cloud/open/api/v1` for Global.
-
-Custom realtime model names are also accepted without an SDK update:
-
-```javascript
-const realtime = client.createRealtimeManager(
-  new RealtimeConfiguration({ model: "your-new-model" })
-);
-```
-
-The model name is sent unchanged; the server determines whether it is available.
-Unknown models inherit the `x2.1-preview` defaults: Agora, the production session API above
-(in both client environments), a 1024 × 1920 camera at 30 fps, and the same
-1024 × 1920 / 1920 × 1024 input resolution buckets and media rules.
-You may explicitly choose another supported `RtcProvider` for a custom model;
-`supportedRtcProviders` lists all SDK adapters for custom models, with Agora first,
-but this does not guarantee server-side support. `modelDisplayName` returns the
-custom name unchanged. Empty/blank or non-string model names, invalid RTC providers,
-and invalid video parameters are still rejected locally.
-
-VeRTC uses `@volcengine/rtc` 4.69.3 and the fixed AppID
-`69a177e226e9b90176a86b96`. Session credentials must contain this `rtc_app_id`,
-`room_id`, and `room_token`; the RTC user is `user_id` (falling back to `userUid`),
-and `bot_name` identifies the bot. Camera and microphone capture use VeRTC's SDK.
-The preview model uses the same media rules as `x2.0-trtc`: 1024 × 1920 or
-1920 × 1024 input, with 30 fps by default.
-
-VeRTC sends the existing room-event JSON as a single room text message, without
-chunk envelopes. The adapter rejects messages above 64 KiB of UTF-8 data; vendor
-send failures are returned to the caller. TRTC and Agora retain their chunking.
-HTTP session heartbeats start after joining and publishing the local stream, with
-the first request after 10 seconds. Changed tokens returned by heartbeat are
-applied to Agora and VeRTC; expiry callbacks also request fresh credentials.
-VeRTC restores media after token expiry without creating another business session
-or restarting the generation task. HTTP heartbeats maintain the session and are
-not treated as a client-side billing trigger.
-
-`XmaxConfiguration.environment` selects the Agora region (`china` or `global`);
-concurrent Agora managers must use the same environment. RTC modules load on demand.
-Statistics not supplied by a provider are `undefined`, including Agora playback
-buffer delay and separate uplink/downlink RTT. RTC end-to-end estimates do not
-include AI generation time and use each provider's own measurement definition.
-
 <br>
 
 ### Using React
@@ -514,156 +376,6 @@ position to the backend.
 
 <br>
 
-### Generate from a local video
-
-Use a manager configured for TRTC, Agora or VeRTC. The SDK decodes a local `File` or `Blob`, sends its video through Canvas and its audio through Web Audio, and receives the generated result over RTC. It does not upload the file to storage or request camera/microphone access.
-
-```ts
-// Call directly from a file-selection or click handler to unlock browser playback.
-const localStream = await realtime.createLocalVideoStream({
-  file,
-  videoFormat: new RealtimeVideoFormat({ width: 1920, height: 1024, fps: 30 }),
-  loop: true,
-  includeAudio: true,
-});
-
-// Optionally bind localStream.videoTrack to an SDK video view.
-const remoteStream = await realtime.startGeneration({
-  localStream,
-  context: new RealtimeContext({ prompt: "Transform the scene into an animation" }),
-});
-```
-
-Preparation stops at the first frame. After the connection is established and the start signal is sent, the file plays from the beginning; playback does not wait for a remote frame. `loop` defaults to `true`, looping both file video and audio; set it to `false` to play once. The current protocol does not acknowledge server input readiness, so frame-perfect processing of the very beginning is not guaranteed.
-
-For local files, requested dimensions automatically select the model resolution with the closest aspect ratio, preferring landscape on ties; exact matches are preserved. For example, `1280×720` selects `1920×1024`. Models without fixed resolutions retain their existing pixel-budget sizing rules. The file is scaled proportionally into the resulting Canvas, with black bars where needed, and the returned track's `videoFormat` reflects that target size. This does not change camera or network-video size validation. Local preview is always silent; `localAudioVolume` does not alter the file's uplink audio. File audio is enabled by default; a file without an audio track produces silence. Set `includeAudio: false` to omit the audio track entirely. Use `setRemoteAudioVolume` to hear the generated result (muted by default).
-
-`disconnect()` pauses playback and retains the source. A new generation after reconnecting starts from the beginning; updating conditions during generation does not restart playback. With looping disabled, file completion holds the final frame and leaves the RTC connection open so remote tail frames can play. `close()` releases all file resources and tracks. `updateUplinkVideoFormat()` can adjust uplink encoding without changing the original model input format.
-
-File codecs must be supported by the browser, and Canvas capture and Web Audio must be available. Keep the page foregrounded for stable frame delivery; browser background throttling can reduce the upload frame rate. Playback restrictions, decode errors and preparation timeouts are reported rather than silently sending an empty stream.
-
-### Generate from a network video
-
-Pass an HTTP or HTTPS video URL to let the server read the source directly. The SDK joins the RTC room and receives generated video and audio without capturing or publishing local media.
-
-```ts
-import { RealtimeContext, RealtimeVideoFormat, RealtimeVideoSampleMethod } from "@xmaxai/web-sdk";
-
-const localStream = await realtime.createNetworkVideoStream({
-  url: "https://example.com/source.mp4",
-  videoFormat: new RealtimeVideoFormat({ width: 1920, height: 1024, fps: 30 }),
-  sampleMethod: RealtimeVideoSampleMethod.time,
-  onFinish: () => {
-    console.log("Server finished processing the video");
-  },
-});
-
-const remoteStream = await realtime.startGeneration({
-  localStream,
-  context: new RealtimeContext({ prompt: "Transform the scene into an animation" }),
-});
-```
-
-Use a video format supported by the selected model. Bind the returned tracks to SDK video views as usual. The optional local preview plays once, muted, independently of server processing; no Canvas or local upload is involved. Creating the source does not start generation. Sampling defaults to `time`; `fps` is also available.
-
-The URL must be directly accessible to the server. Browser cookies and custom authorization headers are not forwarded. Local preview playback depends on browser codec support and page security policies; a preview failure does not stop server processing. The selected model's backend must support network video input.
-
-`onFinish` is delivered once for the current task after the server's `video_stopped` message and generation readiness. It does not mean the last remote frame has finished playing, so the SDK does not automatically disconnect. Local preview completion does not trigger this callback. Remote audio follows `setRemoteAudioVolume` (muted by default).
-
-`disconnect()` retains the source and preview for reconnection; `close()` releases them. Close the current source before creating another. `updateUplinkVideoFormat()` adjusts camera uplink encoding and is not available for a network video source.
-
-### Non-realtime video tasks
-
-Use `client.createNonRealtimeManager()` to submit video-file processing tasks.
-This is cloud processing, not on-device or offline execution. It uses the client's
-API key and selects a dedicated task endpoint from `XmaxEnvironment`:
-
-- `china`: `https://api.xmaxai.com/open/api/v1`
-- `global`: `https://api.xmax.ai/open/api/v1`
-
-Task requests append `/offline-task` and its subpaths to this base URL.
-Realtime model endpoint overrides do not affect task requests; storage credentials
-continue to use the environment's default API endpoint.
-
-```typescript
-import { NonRealtimeQuality, NonRealtimeTaskError } from "@xmaxai/web-sdk";
-
-const manager = client.createNonRealtimeManager();
-const controller = new AbortController();
-
-// videoFile is a File selected by the user.
-const storedVideo = await client.createStorageService().uploadVideo({
-  data: videoFile,
-  fileName: videoFile.name,
-  onProgress: (loaded, total) => console.log(loaded, total),
-});
-
-const task = await manager.submitTask({
-  videoPath: storedVideo.url,
-  referencePath: referenceImageURL, // Optional; obtain via uploadImage().
-  prompt: "Change the character's clothing while preserving the motion.",
-  quality: NonRealtimeQuality.hd,
-  // Omit fps to retain the source frame rate, including fractional rates.
-});
-
-// Persist task.uid before waiting, so the task can be queried after a reload.
-try {
-  const completed = await manager.waitForCompletion(task.uid, {
-    intervalMs: 2000,
-    timeoutMs: 30 * 60 * 1000, // Optional client-side deadline.
-    signal: controller.signal,
-    onTaskUpdated: (snapshot) => console.log(snapshot.status),
-  });
-  console.log(completed.result?.url);
-} catch (error) {
-  if (error instanceof NonRealtimeTaskError) {
-    console.error(error.message, error.task);
-  } else {
-    throw error;
-  }
-}
-```
-
-- `submitTask()` returns immediately after acceptance and never automatically
-  retries. A timeout or invalid response may still mean the server created and
-  charged for the task; do not blindly resubmit.
-- `waitForCompletion()` polls serially and retries transient query failures up to
-  three consecutive times by default, with backoff. A successful query resets the
-  retry count. Without `timeoutMs`, waiting continues until completion, failure, or
-  cancellation. `controller.abort()` stops local waiting and the in-flight query;
-  it does **not** cancel or refund the server task.
-- `getTask(uid)` returns any task status, including `error`. Waiting for an `error`
-  task, or a completed task without a usable result, throws `NonRealtimeTaskError`
-  with the task snapshot attached.
-- `getTasks([uid, ...])` queries up to 100 IDs. `listTasks({ pageNumber, pageSize,
-  status })` retrieves one page, newest first. Saved IDs can be queried or waited
-  on using a new manager instance.
-- Upload local videos with `client.createStorageService().uploadVideo()` before
-  submitting a task. It shares the image upload options and progress callback,
-  uploads the original bytes via COS `putObject`, and returns a `StoredFile`.
-  It does not transcode or use multipart upload; COS simple upload supports files
-  up to [5GB](https://cloud.tencent.com/document/product/436/14113).
-  MIME inference supports MP4/M4V, MOV, WebM, AVI, MKV, and OGV; successful storage
-  does not guarantee that the task processor supports the file's codec or container.
-
-### Downloading videos
-
-Call this from a user-initiated action, such as a download button:
-
-```typescript
-await client.createStorageService().downloadVideo({
-  url: videoURL,
-  fileName: "video.mp4",
-  // signal: controller.signal, // Optional cancellation.
-});
-```
-
-The method fetches the complete file as a Blob and then requests browser saving.
-The URL must be directly accessible and allow CORS for cross-origin requests;
-the SDK does not send API credentials or cookies to it. This requires a browser
-document and retains the complete file until the browser takes over. Resolution
-means saving was requested, not that the user saved the file to disk.
-
 ## Example Project
 
 A complete example application featuring React is available in
@@ -676,7 +388,7 @@ It demonstrates real-time generation using live camera feeds and reference image
 
 - <ins><strong>Tencent Cloud TRTC SDK for Web</strong></ins> enables low-latency, real-time audio and video communication.
 - <ins><strong>Agora RTC SDK for Web</strong></ins> provides an alternative RTC transport for the Agora model.
-- <ins><strong>Volcengine VeRTC SDK for Web</strong></ins> provides RTC transport and device capture for the X2.1 preview model.
+- <ins><strong>Volcengine VeRTC SDK for Web</strong></ins> enables real-time audio and video communication.
 - <ins><strong>Tencent Cloud COS SDK</strong></ins> handles media upload and download via object storage.
 
 <br>