@spatialwalk/avatarkit 1.0.0-beta.1 → 1.0.0-beta.3

package/README.md

@@ -1,25 +1,25 @@
 # SPAvatarKit SDK
 
-基于 3D Gaussian Splatting 的实时虚拟人物头像渲染 SDK，支持音频驱动的动画渲染和高质量 3D 渲染。
+Real-time virtual avatar rendering SDK based on 3D Gaussian Splatting, supporting audio-driven animation rendering and high-quality 3D rendering.
 
-## 🚀 特性
+## 🚀 Features
 
-- **3D Gaussian Splatting 渲染** - 基于最新的点云渲染技术，提供高质量的 3D 虚拟人物
-- **音频驱动的实时动画渲染** - 用户提供音频数据，SDK 负责接收动画数据并渲染
-- **WebGPU/WebGL 双渲染后端** - 自动选择最佳渲染后端，确保兼容性
-- **WASM 高性能计算** - 使用 C++ 编译的 WebAssembly 模块进行几何计算
-- **TypeScript 支持** - 完整的类型定义和智能提示
-- **模块化架构** - 清晰的组件分离，易于集成和扩展
+- **3D Gaussian Splatting Rendering** - Based on the latest point cloud rendering technology, providing high-quality 3D virtual avatars
+- **Audio-Driven Real-Time Animation Rendering** - Users provide audio data, SDK handles receiving animation data and rendering
+- **WebGPU/WebGL Dual Rendering Backend** - Automatically selects the best rendering backend for compatibility
+- **WASM High-Performance Computing** - Uses C++ compiled WebAssembly modules for geometric calculations
+- **TypeScript Support** - Complete type definitions and IntelliSense
+- **Modular Architecture** - Clear component separation, easy to integrate and extend
 
-## 📦 安装
+## 📦 Installation
 
 ```bash
 npm install @spatialwalk/avatarkit
 ```
 
-## 🎯 快速开始
+## 🎯 Quick Start
 
-### 基础使用
+### Basic Usage
 
 ```typescript
 import { 
@@ -30,150 +30,173 @@ import {
   Environment 
 } from '@spatialwalk/avatarkit'
 
-// 1. 初始化 SDK
+// 1. Initialize SDK
 const configuration: Configuration = {
   environment: Environment.test,
 }
 
 await AvatarKit.initialize('your-app-id', configuration)
 
-// 设置 sessionToken（如果需要，单独调用）
+// Set sessionToken (if needed, call separately)
 // AvatarKit.setSessionToken('your-session-token')
 
-// 2. 加载角色
+// 2. Load character
 const avatarManager = new AvatarManager()
 const avatar = await avatarManager.load('character-id', (progress) => {
   console.log(`Loading progress: ${progress.progress}%`)
 })
 
-// 3. 创建视图（自动创建 Canvas 和 AvatarController）
+// 3. Create view (automatically creates Canvas and AvatarController)
 const container = document.getElementById('avatar-container')
 const avatarView = new AvatarView(avatar, container)
 
-// 4. 启动实时通信
+// 4. Start real-time communication
 await avatarView.avatarController.start()
 
-// 5. 发送音频数据
-// 如果音频是 Uint8Array，可以使用 slice().buffer 转换为 ArrayBuffer
-const audioUint8 = new Uint8Array(1024) // 示例：音频数据
-const audioData = audioUint8.slice().buffer // 简化的转换方式，适用于 ArrayBuffer 和 SharedArrayBuffer
-avatarView.avatarController.send(audioData, false) // 发送音频数据，积累到一定量后会自动开始播放
-avatarView.avatarController.send(audioData, true) // end=true 表示立即返回动画数据，不再积累
+// 5. Send audio data
+// If audio is Uint8Array, you can use slice().buffer to convert to ArrayBuffer
+const audioUint8 = new Uint8Array(1024) // Example: audio data
+const audioData = audioUint8.slice().buffer // Simplified conversion, works for ArrayBuffer and SharedArrayBuffer
+avatarView.avatarController.send(audioData, false) // Send audio data, will automatically start playing after accumulating enough data
+avatarView.avatarController.send(audioData, true) // end=true means immediately return animation data, no longer accumulating
 ```
 
-### 完整示例
+### Complete Example
 
-查看 GitHub 仓库中的示例代码了解完整的使用流程。
+Check the example code in the GitHub repository for the complete usage flow.
 
-## 🏗️ 架构概览
+**Example Project:** [Avatarkit-web-demo](https://github.com/spatialwalk/Avatarkit-web-demo)
 
-### 核心组件
+This repository contains complete examples for Vanilla JS, Vue 3, and React, demonstrating how to integrate and use SPAvatarKit SDK in different frameworks.
 
-- **AvatarKit** - SDK 初始化和管理
-- **AvatarManager** - 角色资源加载和管理
-- **AvatarView** - 3D 渲染视图（内部包含 AvatarController）
-- **AvatarController** - 实时通信和数据处理
-- **AvatarCoreAdapter** - WASM 模块适配器
+## 🏗️ Architecture Overview
 
-### 数据流
+### Core Components
+
+- **AvatarKit** - SDK initialization and management
+- **AvatarManager** - Character resource loading and management
+- **AvatarView** - 3D rendering view (internally contains AvatarController)
+- **AvatarController** - Real-time communication and data processing
+- **AvatarCoreAdapter** - WASM module adapter
+
+### Data Flow
 
 ```
-用户音频输入（16kHz mono PCM） → AvatarController → WebSocket → 后台处理
+User audio input (16kHz mono PCM) → AvatarController → WebSocket → Backend processing
                                               ↓
-后台返回动画数据（FLAME 关键帧） → AvatarController → AnimationPlayer
+Backend returns animation data (FLAME keyframes) → AvatarController → AnimationPlayer
                                               ↓
-FLAME 参数 → AvatarCore.computeFrameFlatFromParams() → Splat 数据
+FLAME parameters → AvatarCore.computeFrameFlatFromParams() → Splat data
                                               ↓
-Splat 数据 → RenderSystem → WebGPU/WebGL → Canvas 渲染
+Splat data → RenderSystem → WebGPU/WebGL → Canvas rendering
 ```
 
-**注意：** 用户需要自己提供音频数据（16kHz mono PCM），SDK 负责接收动画数据并渲染。
+**Note:** Users need to provide audio data themselves (16kHz mono PCM), SDK handles receiving animation data and rendering.
 
-## 📚 API 参考
+## 📚 API Reference
 
 ### AvatarKit
 
-SDK 的核心管理类，负责初始化和全局配置。
+The core management class of the SDK, responsible for initialization and global configuration.
 
 ```typescript
-// 初始化 SDK
+// Initialize SDK
 await AvatarKit.initialize(appId: string, configuration: Configuration)
 
-// 检查初始化状态
+// Check initialization status
 const isInitialized = AvatarKit.isInitialized
 
-// 清理资源（不再使用时必须调用）
+// Cleanup resources (must be called when no longer in use)
 AvatarKit.cleanup()
 ```
 
 ### AvatarManager
 
-角色资源管理器，负责下载、缓存和加载角色数据。
+Character resource manager, responsible for downloading, caching, and loading character data.
 
 ```typescript
 const manager = new AvatarManager()
 
-// 加载角色
+// Load character
 const avatar = await manager.load(
   characterId: string, 
   onProgress?: (progress: LoadProgressInfo) => void
 )
 
-// 清理缓存
+// Clear cache
 manager.clearCache()
 ```
 
 ### AvatarView
 
-3D 渲染视图，内部自动创建和管理 AvatarController。
+3D rendering view, internally automatically creates and manages AvatarController.
+
+**⚠️ Important Limitation:** Currently, the SDK only supports one AvatarView instance at a time. If you need to switch characters, you must first call the `dispose()` method to clean up the current AvatarView, then create a new instance.
 
 ```typescript
-// 创建视图（Canvas 会自动添加到容器中）
+// Create view (Canvas is automatically added to container)
 const avatarView = new AvatarView(avatar: Avatar, container?: HTMLElement)
 
-// 获取 Canvas 元素
+// Get Canvas element
 const canvas = avatarView.getCanvas()
 
-// 设置背景
+// Set background
 avatarView.setBackgroundImage('path/to/image.jpg')
 avatarView.setBackgroundOpaque(true)
 
-// 更新相机配置
+// Update camera configuration
 avatarView.updateCameraConfig(cameraConfig: CameraConfig)
 
-// 清理资源
+// Cleanup resources (must be called before switching characters)
 avatarView.dispose()
 ```
 
+**Character Switching Example:**
+
+```typescript
+// Before switching characters, must clean up old AvatarView first
+if (currentAvatarView) {
+  currentAvatarView.dispose()
+  currentAvatarView = null
+}
+
+// Load new character
+const newAvatar = await avatarManager.load('new-character-id')
+
+// Create new AvatarView
+currentAvatarView = new AvatarView(newAvatar, container)
+await currentAvatarView.avatarController.start()
+```
+
 ### AvatarController
 
-实时通信控制器，处理 WebSocket 连接和动画数据。
+Real-time communication controller, handles WebSocket connections and animation data.
 
 ```typescript
-// 启动连接
+// Start connection
 await avatarView.avatarController.start()
 
-// 发送音频数据
+// Send audio data
 avatarView.avatarController.send(audioData: ArrayBuffer, end: boolean)
-// audioData: 音频数据（ArrayBuffer 格式）
-// end: false（默认）- 正常发送音频数据，服务端会积累音频数据，积累到一定量后会自动返回动画数据并开始同步播放动画和音频
-// end: true - 立即返回动画数据，不再积累，用于结束当前对话或需要立即响应的场景
+// audioData: Audio data (ArrayBuffer format)
+// end: false (default) - Normal audio data sending, server will accumulate audio data, automatically returns animation data and starts synchronized playback of animation and audio after accumulating enough data
+// end: true - Immediately return animation data, no longer accumulating, used for ending current conversation or scenarios requiring immediate response
 
-// 打断对话
+// Interrupt conversation
 avatarView.avatarController.interrupt()
 
-// 关闭连接
+// Close connection
 avatarView.avatarController.close()
 
-// 设置事件回调
+// Set event callbacks
 avatarView.avatarController.onConnectionState = (state: ConnectionState) => {}
 avatarView.avatarController.onAvatarState = (state: AvatarState) => {}
 avatarView.avatarController.onError = (error: Error) => {}
 
-// 注意：不支持 sendText() 方法，调用会抛出错误
+// Note: sendText() method is not supported, calling it will throw an error
 ```
 
-## 🔧 配置
+## 🔧 Configuration
 
 ### Configuration
 
@@ -183,14 +206,15 @@ interface Configuration {
 }
 ```
 
-**说明：**
-- `environment`: 指定环境（cn/us/test），SDK 会根据环境自动使用对应的 API 地址和 WebSocket 地址
-- `sessionToken`: 通过 `AvatarKit.setSessionToken()` 单独设置，而不是在 Configuration 中
+**Description:**
+- `environment`: Specifies the environment (cn/us/test), SDK will automatically use the corresponding API address and WebSocket address based on the environment
+- `sessionToken`: Set separately via `AvatarKit.setSessionToken()`, not in Configuration
 
+```typescript
 enum Environment {
-  cn = 'cn',    // 中国区
-  us = 'us',    // 美国区
-  test = 'test' // 测试环境
+  cn = 'cn',    // China region
+  us = 'us',    // US region
+  test = 'test' // Test environment
 }
 ```
 
@@ -198,17 +222,17 @@ enum Environment {
 
 ```typescript
 interface CameraConfig {
-  position: [number, number, number]  // 相机位置
-  target: [number, number, number]    // 相机目标
-  fov: number                         // 视野角度
-  near: number                        // 近裁剪面
-  far: number                         // 远裁剪面
-  up?: [number, number, number]       // 上方向
-  aspect?: number                     // 宽高比
+  position: [number, number, number]  // Camera position
+  target: [number, number, number]    // Camera target
+  fov: number                         // Field of view angle
+  near: number                        // Near clipping plane
+  far: number                         // Far clipping plane
+  up?: [number, number, number]       // Up direction
+  aspect?: number                     // Aspect ratio
 }
 ```
 
-## 📊 状态管理
+## 📊 State Management
 
 ### ConnectionState
 
@@ -225,77 +249,77 @@ enum ConnectionState {
 
 ```typescript
 enum AvatarState {
-  idle = 'idle',      // 空闲状态，呈现呼吸态
-  active = 'active',  // 活跃中，等待可播放内容
-  playing = 'playing' // 播放中
+  idle = 'idle',      // Idle state, showing breathing animation
+  active = 'active',  // Active, waiting for playable content
+  playing = 'playing' // Playing
 }
 ```
 
-## 🎨 渲染系统
+## 🎨 Rendering System
 
-SDK 支持两种渲染后端：
+The SDK supports two rendering backends:
 
-- **WebGPU** - 现代浏览器的高性能渲染
-- **WebGL** - 兼容性更好的传统渲染
+- **WebGPU** - High-performance rendering for modern browsers
+- **WebGL** - Better compatibility traditional rendering
 
-渲染系统会自动选择最佳的后端，无需手动配置。
+The rendering system automatically selects the best backend, no manual configuration needed.
 
-## 🔍 调试和监控
+## 🔍 Debugging and Monitoring
 
-### 日志系统
+### Logging System
 
-SDK 内置了完整的日志系统，支持不同级别的日志输出：
+The SDK has a built-in complete logging system, supporting different levels of log output:
 
 ```typescript
 import { logger } from '@spatialwalk/avatarkit'
 
-// 设置日志级别
+// Set log level
 logger.setLevel('verbose') // 'basic' | 'verbose'
 
-// 手动日志输出
+// Manual log output
 logger.log('Info message')
 logger.warn('Warning message')
 logger.error('Error message')
 ```
 
-### 性能监控
+### Performance Monitoring
 
-SDK 提供了性能监控接口，可以监控渲染性能：
+The SDK provides performance monitoring interfaces to monitor rendering performance:
 
 ```typescript
-// 获取渲染性能统计
+// Get rendering performance statistics
 const stats = avatarView.getPerformanceStats()
 
 if (stats) {
-  console.log(`渲染耗时: ${stats.renderTime.toFixed(2)}ms`)
-  console.log(`排序耗时: ${stats.sortTime.toFixed(2)}ms`)
-  console.log(`渲染后端: ${stats.backend}`)
+  console.log(`Render time: ${stats.renderTime.toFixed(2)}ms`)
+  console.log(`Sort time: ${stats.sortTime.toFixed(2)}ms`)
+  console.log(`Rendering backend: ${stats.backend}`)
   
-  // 计算帧率
+  // Calculate frame rate
   const fps = 1000 / stats.renderTime
-  console.log(`帧率: ${fps.toFixed(2)} FPS`)
+  console.log(`Frame rate: ${fps.toFixed(2)} FPS`)
 }
 
-// 定期监控性能
+// Regular performance monitoring
 setInterval(() => {
   const stats = avatarView.getPerformanceStats()
   if (stats) {
-    // 发送到监控服务或显示在 UI 上
+    // Send to monitoring service or display on UI
     console.log('Performance:', stats)
   }
 }, 1000)
 ```
 
-**性能统计说明**：
-- `renderTime`: 总渲染耗时（毫秒），包含排序和 GPU 渲染
-- `sortTime`: 排序耗时（毫秒），使用 Radix Sort 算法对点云进行深度排序
-- `backend`: 当前使用的渲染后端（`'webgpu'` | `'webgl'` | `null`）
+**Performance Statistics Description:**
+- `renderTime`: Total rendering time (milliseconds), includes sorting and GPU rendering
+- `sortTime`: Sorting time (milliseconds), uses Radix Sort algorithm to depth-sort point cloud
+- `backend`: Currently used rendering backend (`'webgpu'` | `'webgl'` | `null`)
 
-## 🚨 错误处理
+## 🚨 Error Handling
 
 ### SPAvatarError
 
-SDK 使用自定义错误类型，提供更详细的错误信息：
+The SDK uses custom error types, providing more detailed error information:
 
 ```typescript
 import { SPAvatarError } from '@spatialwalk/avatarkit'
@@ -311,65 +335,70 @@ try {
 }
 ```
 
-### 错误回调
+### Error Callbacks
 
 ```typescript
 avatarView.avatarController.onError = (error: Error) => {
   console.error('AvatarController error:', error)
-  // 处理错误，比如重连、用户提示等
+  // Handle error, such as reconnection, user notification, etc.
 }
 ```
 
-## 🔄 资源管理
+## 🔄 Resource Management
 
-### 生命周期管理
+### Lifecycle Management
 
 ```typescript
-// 初始化
+// Initialize
 const avatarView = new AvatarView(avatar, container)
 await avatarView.avatarController.start()
 
-// 使用
+// Use
 avatarView.avatarController.send(audioData, false)
 
-// 清理
-avatarView.dispose() // 自动清理所有资源
+// Cleanup (must be called before switching characters)
+avatarView.dispose() // Automatically cleans up all resources
 ```
 
-### 内存优化
+**⚠️ Important Notes:** 
+- SDK currently only supports one AvatarView instance at a time
+- When switching characters, must first call `dispose()` to clean up old AvatarView, then create new instance
+- Not properly cleaning up may cause resource leaks and rendering errors
+
+### Memory Optimization
 
-- SDK 自动管理 WASM 内存分配
-- 支持角色和动画资源的动态加载/卸载
-- 提供内存使用监控接口
+- SDK automatically manages WASM memory allocation
+- Supports dynamic loading/unloading of character and animation resources
+- Provides memory usage monitoring interface
 
-### 音频数据发送
+### Audio Data Sending
 
-`send()` 方法接收 `ArrayBuffer` 格式的音频数据：
+The `send()` method receives audio data in `ArrayBuffer` format:
 
-**使用说明：**
-- `audioData`: 音频数据（ArrayBuffer 格式）
-- `end=false`（默认）- 正常发送音频数据，服务端会积累音频数据，积累到一定量后会自动返回动画数据并开始同步播放动画和音频
-- `end=true` - 立即返回动画数据，不再积累，用于结束当前对话或需要立即响应的场景
-- **重要**：不需要等待 `end=true` 才开始播放，积累到一定音频数据后就会自动开始播放
+**Usage:**
+- `audioData`: Audio data (ArrayBuffer format)
+- `end=false` (default) - Normal audio data sending, server will accumulate audio data, automatically returns animation data and starts synchronized playback of animation and audio after accumulating enough data
+- `end=true` - Immediately return animation data, no longer accumulating, used for ending current conversation or scenarios requiring immediate response
+- **Important**: No need to wait for `end=true` to start playing, it will automatically start playing after accumulating enough audio data
 
-## 🌐 浏览器兼容性
+## 🌐 Browser Compatibility
 
-- **Chrome/Edge** 90+ (推荐 WebGPU)
+- **Chrome/Edge** 90+ (WebGPU recommended)
 - **Firefox** 90+ (WebGL)
 - **Safari** 14+ (WebGL)
-- **移动端** iOS 14+, Android 8+
+- **Mobile** iOS 14+, Android 8+
 
-## 📝 许可证
+## 📝 License
 
 MIT License
 
-## 🤝 贡献
+## 🤝 Contributing
 
-欢迎提交 Issue 和 Pull Request！
+Issues and Pull Requests are welcome!
 
-## 📞 支持
+## 📞 Support
 
-如有问题，请联系：
-- 邮箱：support@spavatar.com
-- 文档：https://docs.spavatar.com
-- GitHub：https://github.com/spavatar/sdk
+For questions, please contact:
+- Email: support@spavatar.com
+- Documentation: https://docs.spavatar.com
+- GitHub: https://github.com/spavatar/sdk

package/dist/{StreamingAudioPlayer-C2TfYsO8.js → StreamingAudioPlayer-BeLlDiwE.js}

@@ -1,7 +1,7 @@
 var c = Object.defineProperty;
 var g = (h, t, e) => t in h ? c(h, t, { enumerable: !0, configurable: !0, writable: !0, value: e }) : h[t] = e;
 var s = (h, t, e) => g(h, typeof t != "symbol" ? t + "" : t, e);
-import { l as r } from "./index-DwhR9l52.js";
+import { l } from "./index-NmYXWJnL.js";
 class k {
   constructor(t) {
     // AudioContext is managed internally
@@ -48,7 +48,7 @@ class k {
    */
   addChunk(t, e = !1) {
     if (!this.audioContext) {
-      r.error("AudioContext not initialized");
+      l.error("AudioContext not initialized");
       return;
     }
     this.audioChunks.push({ data: t, isLast: e }), this.log(`Added chunk ${this.audioChunks.length}`, {
@@ -117,9 +117,9 @@ class k {
       this.scheduledChunks++;
       return;
     }
-    const l = e.data, d = e.isLast, n = this.pcmToAudioBuffer(l);
+    const r = e.data, d = e.isLast, n = this.pcmToAudioBuffer(r);
     if (!n) {
-      r.error("Failed to create AudioBuffer from PCM data");
+      l.error("Failed to create AudioBuffer from PCM data");
       return;
     }
     try {
@@ -135,7 +135,7 @@ class k {
         activeSources: this.activeSources.length
       });
     } catch (i) {
-      r.errorWithError("Failed to schedule audio chunk:", i);
+      l.errorWithError("Failed to schedule audio chunk:", i);
     }
   }
   /**
@@ -155,7 +155,7 @@ class k {
         a.getChannelData(o).fill(0);
       return a;
     }
-    const e = new Uint8Array(t), l = new Int16Array(e.buffer, 0, e.length / 2), d = l.length / this.channelCount, n = this.audioContext.createBuffer(
+    const e = new Uint8Array(t), r = new Int16Array(e.buffer, 0, e.length / 2), d = r.length / this.channelCount, n = this.audioContext.createBuffer(
       this.channelCount,
       d,
       this.sampleRate
@@ -164,7 +164,7 @@ class k {
       const u = n.getChannelData(i);
       for (let a = 0; a < d; a++) {
         const o = a * this.channelCount + i;
-        u[a] = l[o] / 32768;
+        u[a] = r[o] / 32768;
       }
     }
     return n;
@@ -205,12 +205,7 @@ class k {
    */
   stop() {
     if (this.audioContext) {
-      this.log("[StreamingAudioPlayer] Stopping playback", {
-        isPlaying: this.isPlaying,
-        audioChunks: this.audioChunks.length,
-        scheduledChunks: this.scheduledChunks,
-        activeSources: this.activeSources.length
-      }), this.isPlaying = !1, this.isPaused = !1, this.sessionStartTime = 0, this.scheduledTime = 0;
+      this.isPlaying = !1, this.isPaused = !1, this.sessionStartTime = 0, this.scheduledTime = 0;
       for (const t of this.activeSources) {
         t.onended = null;
         try {
@@ -284,10 +279,10 @@ class k {
    * Debug logging
    */
   log(t, e) {
-    this.debug && r.log(`[StreamingAudioPlayer] ${t}`, e || "");
+    this.debug && l.log(`[StreamingAudioPlayer] ${t}`, e || "");
   }
 }
 export {
   k as StreamingAudioPlayer
 };
-//# sourceMappingURL=StreamingAudioPlayer-C2TfYsO8.js.map
+//# sourceMappingURL=StreamingAudioPlayer-BeLlDiwE.js.map