@spatialwalk/avatarkit 1.0.0-beta.2 → 1.0.0-beta.21

package/CHANGELOG.md

@@ -0,0 +1,340 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0-beta.21] - 2025-01-25
+
+### ✨ New Features
+- **SDK Mode Connection Timeout Fallback** - Enhanced fallback mechanism for SDK mode:
+  - If WebSocket connection fails to establish within 15 seconds, the SDK automatically enters fallback mode
+  - In fallback mode, audio data can still be sent and will play normally, even though no animation data will be received
+  - Connection failures also trigger this fallback mechanism
+  - `ConnectionState.connecting` and `ConnectionState.failed` events are emitted to notify external applications
+  - Concurrent connection attempts are prevented to avoid race conditions
+
+### 🔧 Improvements
+- Improved connection state management: `isConnected` now accurately reflects actual connection status (false in fallback mode)
+- Added `canSend()` method in `NetworkLayer` to check if audio can be sent (including fallback mode)
+- Better separation of concerns: connection state checks are now handled by `NetworkLayer`, while playback logic remains in `AvatarController`
+
+## [1.0.0-beta.20] - 2025-01-25
+
+### 🔧 Improvements
+- Removed 1-second audio buffer requirement for auto-starting playback in Host mode
+
+## [1.0.0-beta.19] - 2025-01-25
+
+### 📚 Documentation
+- **Fixed misleading documentation** - Corrected Host mode API documentation to accurately reflect that `yieldAudioData()` can be called directly to start a new session without requiring `playback()` first
+  - Removed misleading "after playback() is called" comments
+  - Clarified two ways to start a session: using `playback()` for existing data, or `yieldAudioData()` directly for streaming
+  - Updated all examples and workflow descriptions to reflect correct API usage
+
+## [1.0.0-beta.18] - 2025-01-25
+
+### 🔧 API Changes
+- **Renamed `reqId` to `conversationId`** - Updated terminology for better clarity
+  - All methods and parameters that used `reqId` now use `conversationId`
+  - `getCurrentReqId()` → `getCurrentConversationId()`
+  - `generateReqId()` → `generateConversationId()`
+  - Updated all event logs and documentation to use `conversationId`
+  - Note: Protobuf protocol still uses `reqId` field name internally, but SDK API uses `conversationId`
+
+### 📚 Documentation
+- Enhanced Host mode documentation to clearly emphasize the workflow: send audio data first to get conversationId, then use that conversationId to send animation data
+- Updated Host Mode Example and Host Mode Flow sections with clearer step-by-step instructions
+
+## [1.0.0-beta.17] - 2025-01-24
+
+### ✨ New Features
+- **Audio-Only Fallback Mechanism** - SDK now includes automatic fallback to audio-only playback when animation data is unavailable
+  - SDK mode: Automatically enters audio-only mode when server returns an error
+  - Host mode: Automatically enters audio-only mode when empty animation data is provided
+  - Once in audio-only mode, subsequent animation data for that session is ignored
+  - Fallback mode is interruptible, just like normal playback mode
+
+### 🔧 API Changes
+- **Playback Mode Configuration** - Moved playback mode configuration from `AvatarView` constructor to `AvatarKit.initialize()`
+  - Playback mode is now determined by `drivingServiceMode` in `AvatarKit.initialize()` configuration
+  - `AvatarView` constructor now only requires `avatar` and `container` parameters
+  - Removed `AvatarViewOptions` interface
+  - `container` parameter is now required (no longer optional)
+- **Method Renames** - Renamed methods in `AvatarController` for Host mode to better reflect their purpose
+  - `play()` → `playback()`: Renamed to better reflect that the method is used for playback of existing data (replay mode)
+    - Old API: `avatarController.play(initialAudioChunks, initialKeyframes)`
+    - New API: `avatarController.playback(initialAudioChunks, initialKeyframes)`
+  - `sendAudioChunk()` → `yieldAudioData()`: Renamed to better reflect that the method yields/streams audio data
+    - Old API: `avatarController.sendAudioChunk(data, isLast)`
+    - New API: `avatarController.yieldAudioData(data, isLast)`
+  - `sendKeyframes()` → `yieldFramesData()`: Renamed to better reflect that the method yields/streams animation keyframes
+    - Old API: `avatarController.sendKeyframes(keyframes, reqId)`
+    - New API: `avatarController.yieldFramesData(keyframes, conversationId)`
+
+### 🔧 Improvements
+- Extended transition animation duration from 200ms to 400ms for smoother end-of-playback transitions
+
+### 📚 Documentation
+- Updated README.md to use "SDK mode" and "Host mode" terminology instead of "Network mode" and "External data mode"
+- Added fallback mechanism documentation
+- Updated API reference to reflect new constructor signature
+
+## [1.0.0-beta.16] - 2025-11-21
+
+### ✨ New Features
+- **Multi-AvatarView Support** - SDK now supports multiple `AvatarView` instances simultaneously
+  - Each `AvatarView` instance can manage its own character independently
+  - Multiple characters can be displayed and controlled at the same time
+  - Each instance maintains its own rendering context, playback state, and network connection
+  - Removed the previous limitation of "only one AvatarView instance at a time"
+
+### 📚 Documentation
+- Updated README.md to reflect multi-character support capabilities
+- Added multi-character usage examples
+- Removed outdated limitation notes about single AvatarView instance
+
+---
+
+## [1.0.0-beta.15] - 2025-01-20
+
+### ✨ New Features
+- Added authentication support for API requests
+  - `appId` is now automatically included in HTTP headers (`X-App-Id`) and WebSocket URL query parameters
+  - `sessionToken` is now automatically included in HTTP headers (`Authorization: Bearer {token}`) and WebSocket URL query parameters
+  - All network requests (character loading, resource downloads, WebSocket connections) now include authentication credentials
+
+### 🔧 Improvements
+- Fixed canvas resize handling for different aspect ratios
+  - Projection matrix is now correctly updated when canvas aspect ratio changes
+  - Rendering system properly handles portrait (9:16) and landscape (16:9) aspect ratios
+  - Canvas automatically adapts to container size changes via ResizeObserver
+
+### 🐛 Bug Fixes
+- Fixed `RenderSystem.handleResize()` to update projection matrix when canvas size changes
+  - Previously only updated aspect ratio, now also updates projection matrix for correct rendering
+
+---
+
+## [1.0.0-beta.14] - 2025-11-18
+
+### ✨ New Features
+- Added pause and resume functionality for avatar playback
+  - New `pause()` method to pause audio and animation playback
+  - New `resume()` method to resume playback from paused state
+  - Added `paused` state to `AvatarState` enum
+  - State changes are communicated via `onAvatarState` callback
+  - Works in both network mode and external data mode
+
+### 🔧 Improvements
+- Canvas now uses container's full dimensions instead of fixed 1:1 aspect ratio
+  - Canvas automatically adapts to container size (width and height)
+  - External developers can control canvas aspect ratio by setting container dimensions
+  - Rendering system correctly handles any aspect ratio without distortion
+
+---
+
+## [1.0.0-beta.13] - 2025-11-17
+
+### 🔒 API Improvements
+- Reduced console noise by setting default log level to Warning
+  - Logger now only outputs warnings and errors by default
+  - Set `VITE_DEBUG_LOG=true` to enable all logs
+  - Simplified logger configuration using `setGlobalLogLevel(LogLevel.Warning)`
+
+### 🐛 Bug Fixes
+- Fixed SDK config loader warning in non-debug mode
+  - Config loading failures are now silently handled in production
+  - Warnings only shown when `VITE_DEBUG_LOG=true`
+
+---
+
+## [1.0.0-beta.12] - 2025-11-17
+
+### 🔒 API Improvements
+- Marked performance monitoring APIs as internal
+  - `RenderPerformanceStats` interface and `getPerformanceStats()` method are now marked with `@internal` tag
+  - These APIs are not exposed in the public TypeScript definitions
+  - Reverted performance monitoring implementation to simpler version (only `renderTime` and `sortTime`)
+
+---
+
+## [1.0.0-beta.11] - 2025-11-17
+
+### 📚 Documentation
+- Updated demo repository link in README.md
+  - Changed example project link from `Avatarkit-web-demo` to `AvatarKit-Web-Demo` to match the new repository name
+
+---
+
+## [1.0.0-beta.10] - 2025-11-16
+
+### 🔒 API Improvements
+- Cleaned up public API to hide internal implementation details
+  - Marked internal methods with `@internal` JSDoc tag to exclude them from TypeScript declarations
+  - Removed unused environment configuration options (`realtimeApiBaseUrl`, `realtimeWsUrl`)
+  - Added `appId` getter to `AvatarKit` for accessing initialized app ID
+  - Internal methods like `getEnvironmentConfig`, `logEvent`, `getCanvas`, `getCameraConfig`, `updateCameraConfig` are now properly hidden from public API
+
+### 🐛 Bug Fixes
+- Fixed memory leaks when disposing AvatarView instances
+  - Improved resource cleanup to prevent memory leaks when switching or disposing avatar instances
+  - Properly clears all callbacks, event listeners, and large data buffers on disposal
+
+---
+## [1.0.0-beta.9] - 2025-11-15
+
+### 🐛 Bug Fixes
+- Fixed memory leaks when disposing AvatarView instances
+  - Improved resource cleanup to prevent memory leaks when switching or disposing avatar instances
+  - Properly clears all callbacks, event listeners, and large data buffers on disposal
+
+---
+
+## [1.0.0-beta.8] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed audio-animation synchronization issue after normal playback completion
+  - Playback state is now properly cleaned up when audio finishes normally, preventing synchronization issues in subsequent conversations
+
+---
+
+## [1.0.0-beta.7] - 2025-11-14
+
+### 🔄 Refactoring
+- Improved rendering state management for better stability and maintainability
+
+### 🐛 Bug Fixes
+- Fixed interrupt not generating transition animation when interrupting during playback
+- Fixed transition animation being regenerated and reset in external data mode when sending keyframes after playback starts
+
+---
+
+## [1.0.0-beta.6] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed transition animation being regenerated and reset in external data mode when sending keyframes after playback starts, causing the transition to play twice
+
+---
+
+## [1.0.0-beta.5] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed missing `AvatarPlaybackMode` enum export in published package
+
+---
+
+## [1.0.0-beta.4] - 2025-11-14
+
+### 🔄 Updates
+- Updated WASM module (avatar_core_wasm.wasm)
+
+---
+
+## [1.0.0-beta.3] - 2025-11-13
+
+### 🎉 Major Changes
+
+#### Architecture Refactoring - Three-Layer Architecture
+- **Rendering Layer (AvatarView)**: Pure rendering responsibility, receives rendering instructions from the playback layer
+- **Playback Layer (AvatarController)**: Unified data controller responsible for audio playback and animation rendering synchronization
+- **Network Layer (NetworkLayer)**: Optional network communication layer, automatically composed only in network mode
+
+#### Dual Playback Mode Support
+- **Network Mode**: 
+  - Send audio data to server via WebSocket
+  - Automatically receive and play animation data returned from server
+  - Use `controller.start()` and `controller.send()` methods
+  
+- **External Data Mode**:
+  - External components fully control audio and animation data acquisition
+  - SDK only responsible for synchronized playback of externally provided data
+  - Use `controller.playback()`, `controller.yieldAudioData()` and `controller.yieldFramesData()` methods
+
+### ✨ New Features
+
+- **Transition Animation Optimization**: 
+  - Transition animation duration adjusted to 200ms
+  - Improved audio-animation synchronization logic, ensuring audio and animation start playing together after transition animation ends
+  - Optimized transition frame generation and alignment logic
+
+- **Camera Configuration Enhancement**:
+  - Default transparent background, background rendering functionality removed
+  - Camera configuration aligned with iOS (Reversed-Z projection, near=0.01, far=100)
+  - Support dynamic camera configuration loading from characterSettings and camera.json
+
+- **Audio Sample Rate Configuration**:
+  - Updated default audio sample rate to 16kHz (previously 24kHz) to match backend requirements
+  - Audio format requirement: 16kHz mono PCM16
+
+- **Telemetry Migration**:
+  - Migrated from PostHog to Tencent Cloud CLS (Cloud Log Service)
+  - Support environment-based dynamic configuration (cn/test/us)
+  - Support both Token and SecretId/SecretKey authentication methods
+  - Aligned with iOS SDK configuration
+
+### 🔧 API Changes
+
+#### Breaking Changes
+
+- **AvatarView Constructor**:
+  ```typescript
+  // Old API
+  new AvatarView(avatar, container)
+  
+  // New API
+  new AvatarView(avatar, {
+    container: container,
+    playbackMode: AvatarPlaybackMode.network // or AvatarPlaybackMode.external
+  })
+  ```
+
+- **AvatarController Methods**:
+  - Network Mode: `start()` / `close()` / `send()`
+  - External Data Mode: `play()` / `sendAudioChunk()` / `sendKeyframes()`
+  - Common Methods: `interrupt()` / `clear()` / `dispose()`
+
+- **Removed Properties**:
+  - `realtimeStartTime` property removed (no longer needed)
+
+### 🐛 Bug Fixes
+
+- Fixed audio-animation desynchronization during transition animation
+- Fixed animation freezing issue in external data mode
+- Fixed audio playback speed abnormality caused by sample rate mismatch
+- Fixed camera configuration priority logic
+- Fixed WebSocket connection state callback timing issues
+
+### 📚 Documentation
+
+- Updated README.md with detailed explanation of three-layer architecture and two playback modes
+- Added audio format requirements documentation (16kHz mono PCM16)
+- Updated API reference documentation and code examples
+- Added architecture diagrams and flowcharts
+
+### 🧪 Testing
+
+- Updated unit tests to cover new architecture
+- Added unit tests for external data mode
+- Updated E2E tests to use new API
+- All tests passing (69 unit tests + 24 E2E tests)
+
+### 🔄 Internal Changes
+
+- Refactored `AvatarController` into unified playback layer
+- Extracted `NetworkLayer` as independent component
+- Optimized animation playback loop and audio-animation synchronization logic
+- Updated WASM module (avatar_core_wasm.wasm)
+- Improved error handling and logging
+
+### 📦 Dependencies
+
+- No new dependencies added
+- Maintained existing dependency versions
+
+---
+
+## [1.0.0-beta.2] - Previous Version
+
+(Previous version changelog entries...)