@xmaxai/web-sdk 1.0.5 → 1.0.6

package/README.md

@@ -214,10 +214,8 @@ The following JavaScript snippet creates a camera stream, starts real-time gener
 and binds the output to a video view. Run this within an async function in the browser,
 after a user action such as clicking a Start button.
 
-Choose a model with `RealtimeModel.x2_0` (`x2.0`) or
-`RealtimeModel.x2_0_trtc` (`x2.0-trtc`). For Agora, use
-`RealtimeModel.x2_0_agora` (`x2.0-agora`) with `RtcProvider.agora`. For VeRTC, use
-`RealtimeModel.x2_1_preview` (`x2.1-preview`) with `RtcProvider.vertc`.
+For Agora, use `RealtimeModel.x2_1_preview` (`x2.1-preview`). For TRTC, use
+`RealtimeModel.x2_0_trtc` (`x2.0-trtc`). `RealtimeModel.x2_0` (`x2.0`) uses VeRTC.
 
 ```javascript
 import {
@@ -272,11 +270,11 @@ videoView.remoteTrack = remoteStream.videoTrack;
 Add a `video-container` element with an explicit width and height to your page.
 The view displays a local camera preview until the first generated frame arrives.
 
-Use `updateVideoFormat` after creating the local camera stream, including during
+Use `updateUplinkVideoFormat` after creating the local camera stream, including during
 generation, to change upstream encoding without reconnecting or restarting the task:
 
 ```javascript
-await realtime.updateVideoFormat(new RealtimeVideoFormat({
+await realtime.updateUplinkVideoFormat(new RealtimeVideoFormat({
   width: 1024,
   height: 1920,
   fps: 20,
@@ -292,6 +290,58 @@ resolution, frame rate, and bitrate depend on the device, browser, and network;
 use the statistics callbacks to observe them. Await each operation before starting
 another configuration or lifecycle operation.
 
+For a frame-rate-only experiment after connecting a local camera or file stream:
+
+```javascript
+await realtime.updateUplinkFrameRate(24);
+```
+
+This preserves the last successfully applied width, height, resolved bitrate limits,
+and encoding preference. It does not recalculate default bitrates for the new frame
+rate or copy measured send statistics into the configuration. The complete encoder
+configuration is still resubmitted, so preservation of RTC's internal adaptation
+state is not guaranteed. Validate actual send statistics under weak networks.
+
+**Agora camera sender-only experiment:** camera capture stays at its initial settings.
+Camera updates use `RTCRtpSender.setParameters()` (`scaleResolutionDownBy`,
+`maxFramerate`, `maxBitrate` in bps), not Agora's `setEncoderConfiguration()`.
+Only the maximum bitrate is applied; the standard sender API cannot enforce the
+configured minimum bitrate. Pre-publish updates are queued and applied immediately
+after publication, so the very first transmitted frames may use the initial profile.
+Scaling uses the actual captured dimensions, preserves aspect ratio, and does not upscale.
+Sender replacement/rejoin reapplies the latest target. Unsupported or silently ignored
+parameters are reported as errors, without falling back to camera constraints.
+Read-back confirms accepted parameters, not actual frame dimensions or persistent
+ownership against Agora's later adjustments; validate Chrome/Safari using RTC statistics.
+Search `Agora Sender Parameters` for accepted limits and `Agora RTC Statistics` for
+capture/send dimensions. File-video encoding and other RTC providers are unchanged.
+
+Agora uplink and downlink adaptation are always enabled; no configuration flag or manual
+selector is needed. Other RTC providers are not yet integrated. After generation
+starts, three consecutive poor uplink quality samples lower one tier, and five good
+samples probe an upgrade. Five tiers jointly change resolution and frame rate:
+1024×1920/30, 768×1440/24, 640×1200/24, 512×960/20, and 384×720/16.
+Landscape dimensions are swapped; custom sources scale proportionally from their
+initial or most recently manually applied format, with even dimensions and FPS
+capped by that baseline. Automatic updates preserve this baseline across reconnects.
+Default bitrate limits are recalculated for each tier; explicit limits and encoding
+preference are preserved. Uplink changes do not alter generation or result-stream sizes.
+Downlink independently uses the same quality thresholds and five resolution tiers,
+relative to the generation's original dimensions. It sends only `change_target_size`
+with `[width, height]`, without changing FPS, bitrate limits, or model inference.
+The protocol has no application ACK: a successful send records a requested tier,
+not a confirmed output size; use remote RTC statistics to verify actual dimensions.
+New Agora local tracks read the last applied uplink tier before publishing
+(the camera sender-only experiment applies it immediately after publication);
+new generation tasks restore the last requested downlink tier via `start.params.target_size`.
+Each direction falls back to L2 when no valid history is available. History is stored
+in `localStorage`, separately by model and direction, with no expiry or stability window
+in this first version. Storage failures never block generation. The original L1 baseline
+is retained for recovery; reconnecting the same local track keeps its in-memory uplink tier.
+Downlink changes only resolution, not FPS. Interpolation follows the requested dimensions,
+showing raw video while frame sizes do not match. See
+[the strategy and weak-network test steps](docs/realtime-adaptive-quality.md).
+
 Each model declares its supported RTC providers. The SDK selects the model's
 default provider when `provider` is omitted. To use Agora, replace the manager
 configuration above; camera capture, generation, rendering, and cleanup use the same APIs:
@@ -299,33 +349,33 @@ configuration above; camera capture, generation, rendering, and cleanup use the
 ```javascript
 const realtime = client.createRealtimeManager(
   new RealtimeConfiguration({
-    model: RealtimeModel.x2_0_agora,
+    model: RealtimeModel.x2_1_preview,
   })
 );
 ```
 
-To use VeRTC, select the preview model:
+To use VeRTC, select `x2.0`:
 
 ```javascript
 const realtime = client.createRealtimeManager(
   new RealtimeConfiguration({
-    model: RealtimeModel.x2_1_preview,
+    model: RealtimeModel.x2_0,
   })
 );
 ```
 
-The provider is fixed for each manager: `x2.0` and `x2.0-trtc` use
-`RtcProvider.trtc`, `x2.0-agora` uses `RtcProvider.agora`, and
-`x2.1-preview` uses `RtcProvider.vertc`.
+The provider is fixed for each manager: `x2.0-trtc` uses `RtcProvider.trtc`,
+`x2.1-preview` uses `RtcProvider.agora`, and `x2.0` uses `RtcProvider.vertc`.
 Import `RtcProvider` to specify it explicitly; unsupported known-model/provider
 combinations are rejected during configuration. Use the exported
 `supportedRtcProviders(model)` to inspect model capabilities. The SDK does not read
 `modelExtra.provider` from the backend; the resolved configuration determines how
 RTC credentials are parsed.
 
-The `x2.0-agora` and `x2.1-preview` models use
-`https://dev.xmaxai.com/open/api/v1` in both environments.
-The other built-in models and file uploads use the configured environment's API endpoint.
+The `x2.1-preview` model uses `https://cloud.xmax.22duck.cn/open/api/v1` in both environments.
+The other built-in models and file uploads use the configured
+environment's production API endpoint: `https://cloud.xmax.22duck.cn/open/api/v1`
+for China and `https://api.xmax.cloud/open/api/v1` for Global.
 
 Custom realtime model names are also accepted without an SDK update:
 
@@ -336,11 +386,11 @@ const realtime = client.createRealtimeManager(
 ```
 
 The model name is sent unchanged; the server determines whether it is available.
-Unknown models inherit the `x2.1-preview` defaults: VeRTC, the dev session API above
+Unknown models inherit the `x2.1-preview` defaults: Agora, the production session API above
 (in both client environments), a 1024 × 1920 camera at 30 fps, and the same
 1024 × 1920 / 1920 × 1024 input resolution buckets and media rules.
 You may explicitly choose another supported `RtcProvider` for a custom model;
-`supportedRtcProviders` lists all SDK adapters for custom models, with VeRTC first,
+`supportedRtcProviders` lists all SDK adapters for custom models, with Agora first,
 but this does not guarantee server-side support. `modelDisplayName` returns the
 custom name unchanged. Empty/blank or non-string model names, invalid RTC providers,
 and invalid video parameters are still rejected locally.
@@ -422,6 +472,14 @@ For camera input, bind the returned video track to a preview view. The SDK enter
 `ready` after it has received a valid frame and the preview view is bound; observe
 this through `setStateListener`.
 
+At generation start, the SDK sends `params.mirror: true` for the front camera
+and `false` for the rear camera, local files, and network videos. The backend
+mirrors the input; generated results are not automatically mirrored in the
+browser. The front-camera local preview remains mirrored independently.
+This flag is sent only with `start`: switching cameras during an active generation
+does not update it. Disconnect and start generation again to apply the new camera
+position to the backend.
+
 <br>
 
 ### Resource Cleanup
@@ -480,7 +538,7 @@ Preparation stops at the first frame. After the connection is established and th
 
 For local files, requested dimensions automatically select the model resolution with the closest aspect ratio, preferring landscape on ties; exact matches are preserved. For example, `1280×720` selects `1920×1024`. Models without fixed resolutions retain their existing pixel-budget sizing rules. The file is scaled proportionally into the resulting Canvas, with black bars where needed, and the returned track's `videoFormat` reflects that target size. This does not change camera or network-video size validation. Local preview is always silent; `localAudioVolume` does not alter the file's uplink audio. File audio is enabled by default; a file without an audio track produces silence. Set `includeAudio: false` to omit the audio track entirely. Use `setRemoteAudioVolume` to hear the generated result (muted by default).
 
-`disconnect()` pauses playback and retains the source. A new generation after reconnecting starts from the beginning; updating conditions during generation does not restart playback. With looping disabled, file completion holds the final frame and leaves the RTC connection open so remote tail frames can play. `close()` releases all file resources and tracks. `updateVideoFormat()` can adjust uplink encoding without changing the original model input format.
+`disconnect()` pauses playback and retains the source. A new generation after reconnecting starts from the beginning; updating conditions during generation does not restart playback. With looping disabled, file completion holds the final frame and leaves the RTC connection open so remote tail frames can play. `close()` releases all file resources and tracks. `updateUplinkVideoFormat()` can adjust uplink encoding without changing the original model input format.
 
 File codecs must be supported by the browser, and Canvas capture and Web Audio must be available. Keep the page foregrounded for stable frame delivery; browser background throttling can reduce the upload frame rate. Playback restrictions, decode errors and preparation timeouts are reported rather than silently sending an empty stream.
 
@@ -512,7 +570,7 @@ The URL must be directly accessible to the server. Browser cookies and custom au
 
 `onFinish` is delivered once for the current task after the server's `video_stopped` message and generation readiness. It does not mean the last remote frame has finished playing, so the SDK does not automatically disconnect. Local preview completion does not trigger this callback. Remote audio follows `setRemoteAudioVolume` (muted by default).
 
-`disconnect()` retains the source and preview for reconnection; `close()` releases them. Close the current source before creating another. `updateVideoFormat()` adjusts camera uplink encoding and is not available for a network video source.
+`disconnect()` retains the source and preview for reconnection; `close()` releases them. Close the current source before creating another. `updateUplinkVideoFormat()` adjusts camera uplink encoding and is not available for a network video source.
 
 ### Non-realtime video tasks
 

package/dist/{VideoContentMode-BM97_H8V.d.cts → VideoRenderStatistics-C7zqzfuS.d.cts}

@@ -190,4 +190,26 @@ declare enum VideoContentMode {
     fill = "fill"
 }
 
-export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, RealtimeVideoEncoderPreference as b, type RealtimeVideoFormatInit as c };
+/**
+ * 单个视频视图最近采样窗口的呈现帧率，不代表屏幕实际刷新帧率。
+ */
+interface VideoRenderStatistics {
+    /**
+     * video 按浏览器 presentedFrames 增量计数；canvas 按实际呈现提交计数。
+     */
+    readonly source: "video" | "canvas";
+    /**
+     * 原帧与插值帧的总呈现帧率，单位 fps。
+     */
+    readonly frameRate: number;
+    /**
+     * 原帧呈现帧率，单位 fps。
+     */
+    readonly originalFrameRate: number;
+    /**
+     * 插值帧呈现帧率，单位 fps；未开启插帧时为 0。
+     */
+    readonly interpolatedFrameRate: number;
+}
+
+export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, type VideoRenderStatistics as b, RealtimeVideoEncoderPreference as c, type RealtimeVideoFormatInit as d };

package/dist/{VideoContentMode-BM97_H8V.d.ts → VideoRenderStatistics-C7zqzfuS.d.ts}

@@ -190,4 +190,26 @@ declare enum VideoContentMode {
     fill = "fill"
 }
 
-export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, RealtimeVideoEncoderPreference as b, type RealtimeVideoFormatInit as c };
+/**
+ * 单个视频视图最近采样窗口的呈现帧率，不代表屏幕实际刷新帧率。
+ */
+interface VideoRenderStatistics {
+    /**
+     * video 按浏览器 presentedFrames 增量计数；canvas 按实际呈现提交计数。
+     */
+    readonly source: "video" | "canvas";
+    /**
+     * 原帧与插值帧的总呈现帧率，单位 fps。
+     */
+    readonly frameRate: number;
+    /**
+     * 原帧呈现帧率，单位 fps。
+     */
+    readonly originalFrameRate: number;
+    /**
+     * 插值帧呈现帧率，单位 fps；未开启插帧时为 0。
+     */
+    readonly interpolatedFrameRate: number;
+}
+
+export { CameraPosition as C, RealtimeVideoFormat as R, VideoContentMode as V, RealtimeVideoTrack as a, type VideoRenderStatistics as b, RealtimeVideoEncoderPreference as c, type RealtimeVideoFormatInit as d };