@spatialwalk/avatarkit 1.0.0-beta.3 → 1.0.0-beta.30

package/CHANGELOG.md

@@ -0,0 +1,436 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0-beta.30] - 2025-12-15
+
+### 🐛 Bugfix
+- **Template Resources Duplicate Download** - Fixed issue where template resources were being re-downloaded for each new character load. Template resources are now only loaded once during SDK initialization and reused for all characters.
+
+## [1.0.0-beta.29] - 2025-12-15
+
+### 🔄 Breaking Changes
+- **Environment Enum** - Removed `Environment.test`. Only `Environment.cn` and `Environment.intl` are now supported. Environment configuration must be explicitly provided.
+- **Log Level Default** - Default log level changed from `LogLevel.all` to `LogLevel.off`. Set `logLevel: LogLevel.all` in configuration to enable all logs.
+
+### ✨ New Features
+- **Avatar Caching** - Added avatar caching methods in `AvatarManager`:
+  - `retrieve(id: string): Avatar | undefined` - Get cached avatar by ID
+  - `clear(id: string): void` - Clear cached avatar for specific ID
+  - `clearAll(): void` - Clear all cached avatars
+  - `clearCache()` is now deprecated, use `clearAll()` instead
+- **Background Image Support** - Added background image control in `AvatarView`:
+  - `isOpaque: boolean` - Getter/setter to control canvas background transparency
+  - `setBackgroundImage(image: HTMLImageElement | string | null): void` - Set or remove background image
+
+### 🔧 Improvements
+- **Avatar Version Checking** - Avatar cache now automatically checks version and reloads if cached avatar version differs from latest metadata
+- **Concurrent Loading** - Multiple concurrent `load()` calls for the same avatar ID now reuse the same loading promise
+
+## [1.0.0-beta.28] - 2025-12-08
+
+### 🔄 Breaking Changes
+- **Class Renamed** - `AvatarKit` class has been renamed to `AvatarSDK` for better consistency
+  - Update all imports: `import { AvatarKit } from '@spatialwalk/avatarkit'` → `import { AvatarSDK } from '@spatialwalk/avatarkit'`
+  - Update all API calls: `AvatarKit.initialize()` → `AvatarSDK.initialize()`
+  - All static methods and properties remain the same, only the class name has changed
+
+### 🔧 Improvements
+- **Rendering Optimizations** - Added fragment shader discard optimization to improve performance and edge quality
+- **Shader Alignment** - Removed view matrix transpose to align with Android SDK implementation
+- **Resource Cleanup** - Removed unused `frameMono` and `audioMono` resources from SDK and tests
+
+## [1.0.0-beta.27] - 2025-12-04
+
+### 🐛 Bugfix
+
+## [1.0.0-beta.26] - 2025-12-03
+
+### ✨ New Features
+- **Log Level Configuration** - Added `logLevel` configuration option to control SDK logging verbosity
+  - `LogLevel.off` - Disable all logs
+  - `LogLevel.error` - Only error logs
+  - `LogLevel.warning` - Warning and error logs
+  - `LogLevel.all` - All logs (default)
+- **Standardized CLS Logging Events** - All CLS logging events now use standardized field names for better querying
+  - Standard events: `sdk_initialized`, `send_audio_failed`, `message_error`, `heartbeat_failed`, `service_restarted`, `session_token_invalid`, `session_token_expired`, `yield_animation_failed`, `fetch_avatar_metadata_failed`, `download_avatar_assets_failed`
+  - Standard fields: `req_id`, `con_id`, `avatar_id`, `description`
+- **End-to-End Latency Tracking** - Added `driving_latency` event to track latency metrics for both SDK and Host modes
+  - Tracks: `start`, `tap2`, `tap4`, `end`, `first_frame` timestamps
+  - Automatically reported at key moments during audio/animation processing
+
+### 🔧 Improvements
+- **CLS Configuration Alignment** - Updated CLS endpoints and Topic IDs to match iOS SDK configuration
+- **Default API Endpoints** - Fixed default API endpoint for `intl` environment to `api.intl.spatialwalk.cloud`
+- **Error Code Enum** - Added new error codes: `appIDUnrecognized`, `sessionTokenInvalid`, `sessionTokenExpired`, `failedToFetchAvatarMetadata`, `failedToDownloadAvatarAssets`
+
+### 🗑️ Removed
+- **Redundant Log Events** - Removed unnecessary log events: `playback_paused`, `playback_resumed`, `reconnecting`, `connect_success`, `server_error`, `decode_failed`
+
+## [1.0.0-beta.25] - 2025-12-03
+
+### 🔧 Improvements
+- **Removed Audio Buffer Limit** - Removed the 100 chunk limit on pending audio buffers to allow unlimited audio data buffering
+
+## [1.0.0-beta.24] - 2025-11-26
+
+### 🔧 API Changes
+- **Backend Endpoints Updated** - Backend API endpoints have been updated for all environments
+
+## [1.0.0-beta.23] - 2025-11-26
+
+### 🔧 API Changes
+- **Breaking Change** - `playback()` method is no longer supported and has been removed from public API
+
+## [1.0.0-beta.22] - 2025-11-26
+
+### 🔧 API Changes
+- **State Callback Renamed** - `onAvatarState` has been renamed to `onConversationState` for better clarity
+  - The callback now uses `ConversationState` enum with states: `idle` and `playing`
+- **Environment Enum Updated** - `Environment.us` has been renamed to `Environment.intl` for better internationalization support
+  - All references to `Environment.us` should be updated to `Environment.intl`
+  - Remote config endpoints now use `intl` instead of `us`
+
+### ✨ New Features
+- **Volume Control** - Added volume control API for audio playback
+  - `setVolume(volume: number)` - Set audio volume (0.0 to 1.0)
+  - `getVolume(): number` - Get current audio volume
+  - Volume control only affects the avatar's audio player, not system volume
+  - Volume changes take effect immediately, including for currently playing audio
+
+## [1.0.0-beta.21] - 2025-11-25
+
+### ✨ New Features
+- **SDK Mode Connection Timeout Fallback** - Enhanced fallback mechanism for SDK mode:
+  - If WebSocket connection fails to establish within 15 seconds, the SDK automatically enters fallback mode
+  - In fallback mode, audio data can still be sent and will play normally, even though no animation data will be received
+  - Connection failures also trigger this fallback mechanism
+  - `ConnectionState.connecting` and `ConnectionState.failed` events are emitted to notify external applications
+  - Concurrent connection attempts are prevented to avoid race conditions
+
+### 🔧 Improvements
+- Improved connection state management: `isConnected` now accurately reflects actual connection status (false in fallback mode)
+- Added `canSend()` method in `NetworkLayer` to check if audio can be sent (including fallback mode)
+- Better separation of concerns: connection state checks are now handled by `NetworkLayer`, while playback logic remains in `AvatarController`
+
+## [1.0.0-beta.20] - 2025-11-25
+
+### 🔧 Improvements
+- Removed 1-second audio buffer requirement for auto-starting playback in Host mode
+
+## [1.0.0-beta.19] - 2025-11-25
+
+### 📚 Documentation
+- **Fixed misleading documentation** - Corrected Host mode API documentation to accurately reflect that `yieldAudioData()` can be called directly to start a new session without requiring `playback()` first
+  - Removed misleading "after playback() is called" comments
+  - Clarified two ways to start a session: using `playback()` for existing data, or `yieldAudioData()` directly for streaming
+  - Updated all examples and workflow descriptions to reflect correct API usage
+
+## [1.0.0-beta.18] - 2025-11-25
+
+### 🔧 API Changes
+- **Renamed `reqId` to `conversationId`** - Updated terminology for better clarity
+  - All methods and parameters that used `reqId` now use `conversationId`
+  - `getCurrentReqId()` → `getCurrentConversationId()`
+  - `generateReqId()` → `generateConversationId()`
+  - Updated all event logs and documentation to use `conversationId`
+  - Note: Protobuf protocol still uses `reqId` field name internally, but SDK API uses `conversationId`
+
+### 📚 Documentation
+- Enhanced Host mode documentation to clearly emphasize the workflow: send audio data first to get conversationId, then use that conversationId to send animation data
+- Updated Host Mode Example and Host Mode Flow sections with clearer step-by-step instructions
+
+## [1.0.0-beta.17] - 2025-11-24
+
+### ✨ New Features
+- **Audio-Only Fallback Mechanism** - SDK now includes automatic fallback to audio-only playback when animation data is unavailable
+  - SDK mode: Automatically enters audio-only mode when server returns an error
+  - Host mode: Automatically enters audio-only mode when empty animation data is provided
+  - Once in audio-only mode, subsequent animation data for that session is ignored
+  - Fallback mode is interruptible, just like normal playback mode
+
+### 🔧 API Changes
+- **Playback Mode Configuration** - Moved playback mode configuration from `AvatarView` constructor to `AvatarSDK.initialize()`
+  - Playback mode is now determined by `drivingServiceMode` in `AvatarSDK.initialize()` configuration
+  - `AvatarView` constructor now only requires `avatar` and `container` parameters
+  - Removed `AvatarViewOptions` interface
+  - `container` parameter is now required (no longer optional)
+- **Method Renames** - Renamed methods in `AvatarController` for Host mode to better reflect their purpose
+  - `play()` → `playback()`: Renamed to better reflect that the method is used for playback of existing data (replay mode)
+    - Old API: `avatarController.play(initialAudioChunks, initialKeyframes)`
+    - New API: `avatarController.playback(initialAudioChunks, initialKeyframes)`
+  - `sendAudioChunk()` → `yieldAudioData()`: Renamed to better reflect that the method yields/streams audio data
+    - Old API: `avatarController.sendAudioChunk(data, isLast)`
+    - New API: `avatarController.yieldAudioData(data, isLast)`
+  - `sendKeyframes()` → `yieldFramesData()`: Renamed to better reflect that the method yields/streams animation keyframes
+    - Old API: `avatarController.sendKeyframes(keyframes, reqId)`
+    - New API: `avatarController.yieldFramesData(keyframes, conversationId)`
+
+### 🔧 Improvements
+- Extended transition animation duration from 200ms to 400ms for smoother end-of-playback transitions
+
+### 📚 Documentation
+- Updated README.md to use "SDK mode" and "Host mode" terminology instead of "Network mode" and "External data mode"
+- Added fallback mechanism documentation
+- Updated API reference to reflect new constructor signature
+
+## [1.0.0-beta.16] - 2025-11-21
+
+### ✨ New Features
+- **Multi-AvatarView Support** - SDK now supports multiple `AvatarView` instances simultaneously
+  - Each `AvatarView` instance can manage its own character independently
+  - Multiple characters can be displayed and controlled at the same time
+  - Each instance maintains its own rendering context, playback state, and network connection
+  - Removed the previous limitation of "only one AvatarView instance at a time"
+
+### 📚 Documentation
+- Updated README.md to reflect multi-character support capabilities
+- Added multi-character usage examples
+- Removed outdated limitation notes about single AvatarView instance
+
+---
+
+## [1.0.0-beta.15] - 2025-11-20
+
+### ✨ New Features
+- Added authentication support for API requests
+  - `appId` is now automatically included in HTTP headers (`X-App-Id`) and WebSocket URL query parameters
+  - `sessionToken` is now automatically included in HTTP headers (`Authorization: Bearer {token}`) and WebSocket URL query parameters
+  - All network requests (character loading, resource downloads, WebSocket connections) now include authentication credentials
+
+### 🔧 Improvements
+- Fixed canvas resize handling for different aspect ratios
+  - Projection matrix is now correctly updated when canvas aspect ratio changes
+  - Rendering system properly handles portrait (9:16) and landscape (16:9) aspect ratios
+  - Canvas automatically adapts to container size changes via ResizeObserver
+
+### 🐛 Bug Fixes
+- Fixed `RenderSystem.handleResize()` to update projection matrix when canvas size changes
+  - Previously only updated aspect ratio, now also updates projection matrix for correct rendering
+
+---
+
+## [1.0.0-beta.14] - 2025-11-18
+
+### ✨ New Features
+- Added pause and resume functionality for avatar playback
+  - New `pause()` method to pause audio and animation playback
+  - New `resume()` method to resume playback from paused state
+  - Added `paused` state to `AvatarState` enum
+  - State changes are communicated via `onAvatarState` callback
+  - Works in both network mode and external data mode
+
+### 🔧 Improvements
+- Canvas now uses container's full dimensions instead of fixed 1:1 aspect ratio
+  - Canvas automatically adapts to container size (width and height)
+  - External developers can control canvas aspect ratio by setting container dimensions
+  - Rendering system correctly handles any aspect ratio without distortion
+
+---
+
+## [1.0.0-beta.13] - 2025-11-17
+
+### 🔒 API Improvements
+- Reduced console noise by setting default log level to Warning
+  - Logger now only outputs warnings and errors by default
+  - Set `VITE_DEBUG_LOG=true` to enable all logs
+  - Simplified logger configuration using `setGlobalLogLevel(LogLevel.Warning)`
+
+### 🐛 Bug Fixes
+- Fixed SDK config loader warning in non-debug mode
+  - Config loading failures are now silently handled in production
+  - Warnings only shown when `VITE_DEBUG_LOG=true`
+
+---
+
+## [1.0.0-beta.12] - 2025-11-17
+
+### 🔒 API Improvements
+- Marked performance monitoring APIs as internal
+  - `RenderPerformanceStats` interface and `getPerformanceStats()` method are now marked with `@internal` tag
+  - These APIs are not exposed in the public TypeScript definitions
+  - Reverted performance monitoring implementation to simpler version (only `renderTime` and `sortTime`)
+
+---
+
+## [1.0.0-beta.11] - 2025-11-17
+
+### 📚 Documentation
+- Updated demo repository link in README.md
+  - Changed example project link from `Avatarkit-web-demo` to `AvatarSDK-Web-Demo` to match the new repository name
+
+---
+
+## [1.0.0-beta.10] - 2025-11-16
+
+### 🔒 API Improvements
+- Cleaned up public API to hide internal implementation details
+  - Marked internal methods with `@internal` JSDoc tag to exclude them from TypeScript declarations
+  - Removed unused environment configuration options (`realtimeApiBaseUrl`, `realtimeWsUrl`)
+  - Added `appId` getter to `AvatarSDK` for accessing initialized app ID
+  - Internal methods like `getEnvironmentConfig`, `logEvent`, `getCanvas`, `getCameraConfig`, `updateCameraConfig` are now properly hidden from public API
+
+### 🐛 Bug Fixes
+- Fixed memory leaks when disposing AvatarView instances
+  - Improved resource cleanup to prevent memory leaks when switching or disposing avatar instances
+  - Properly clears all callbacks, event listeners, and large data buffers on disposal
+
+---
+## [1.0.0-beta.9] - 2025-11-15
+
+### 🐛 Bug Fixes
+- Fixed memory leaks when disposing AvatarView instances
+  - Improved resource cleanup to prevent memory leaks when switching or disposing avatar instances
+  - Properly clears all callbacks, event listeners, and large data buffers on disposal
+
+---
+
+## [1.0.0-beta.8] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed audio-animation synchronization issue after normal playback completion
+  - Playback state is now properly cleaned up when audio finishes normally, preventing synchronization issues in subsequent conversations
+
+---
+
+## [1.0.0-beta.7] - 2025-11-14
+
+### 🔄 Refactoring
+- Improved rendering state management for better stability and maintainability
+
+### 🐛 Bug Fixes
+- Fixed interrupt not generating transition animation when interrupting during playback
+- Fixed transition animation being regenerated and reset in external data mode when sending keyframes after playback starts
+
+---
+
+## [1.0.0-beta.6] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed transition animation being regenerated and reset in external data mode when sending keyframes after playback starts, causing the transition to play twice
+
+---
+
+## [1.0.0-beta.5] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed missing `DrivingServiceMode` enum export in published package
+
+---
+
+## [1.0.0-beta.4] - 2025-11-14
+
+### 🔄 Updates
+- Updated WASM module (avatar_core_wasm.wasm)
+
+---
+
+## [1.0.0-beta.3] - 2025-11-13
+
+### 🎉 Major Changes
+
+#### Architecture Refactoring - Three-Layer Architecture
+- **Rendering Layer (AvatarView)**: Pure rendering responsibility, receives rendering instructions from the playback layer
+- **Playback Layer (AvatarController)**: Unified data controller responsible for audio playback and animation rendering synchronization
+- **Network Layer (NetworkLayer)**: Optional network communication layer, automatically composed only in network mode
+
+#### Dual Playback Mode Support
+- **Network Mode**: 
+  - Send audio data to server via WebSocket
+  - Automatically receive and play animation data returned from server
+  - Use `controller.start()` and `controller.send()` methods
+  
+- **External Data Mode**:
+  - External components fully control audio and animation data acquisition
+  - SDK only responsible for synchronized playback of externally provided data
+  - Use `controller.playback()`, `controller.yieldAudioData()` and `controller.yieldFramesData()` methods
+
+### ✨ New Features
+
+- **Transition Animation Optimization**: 
+  - Transition animation duration adjusted to 200ms
+  - Improved audio-animation synchronization logic, ensuring audio and animation start playing together after transition animation ends
+  - Optimized transition frame generation and alignment logic
+
+- **Camera Configuration Enhancement**:
+  - Default transparent background, background rendering functionality removed
+  - Camera configuration aligned with iOS (Reversed-Z projection, near=0.01, far=100)
+  - Support dynamic camera configuration loading from characterSettings and camera.json
+
+- **Audio Sample Rate Configuration**:
+  - Updated default audio sample rate to 16kHz (previously 24kHz) to match backend requirements
+  - Audio format requirement: 16kHz mono PCM16
+
+- **Telemetry Migration**:
+  - Migrated from PostHog to Tencent Cloud CLS (Cloud Log Service)
+  - Support environment-based dynamic configuration (cn/test/us)
+  - Support both Token and SecretId/SecretKey authentication methods
+  - Aligned with iOS SDK configuration
+
+### 🔧 API Changes
+
+#### Breaking Changes
+
+- **AvatarView Constructor**:
+  ```typescript
+  // Old API
+  new AvatarView(avatar, container)
+  
+  // New API
+  new AvatarView(avatar, {
+    container: container,
+    playbackMode: DrivingServiceMode.sdk // or DrivingServiceMode.host
+  })
+  ```
+
+- **AvatarController Methods**:
+  - Network Mode: `start()` / `close()` / `send()`
+  - External Data Mode: `play()` / `sendAudioChunk()` / `sendKeyframes()`
+  - Common Methods: `interrupt()` / `clear()` / `dispose()`
+
+- **Removed Properties**:
+  - `realtimeStartTime` property removed (no longer needed)
+
+### 🐛 Bug Fixes
+
+- Fixed audio-animation desynchronization during transition animation
+- Fixed animation freezing issue in external data mode
+- Fixed audio playback speed abnormality caused by sample rate mismatch
+- Fixed camera configuration priority logic
+- Fixed WebSocket connection state callback timing issues
+
+### 📚 Documentation
+
+- Updated README.md with detailed explanation of three-layer architecture and two playback modes
+- Added audio format requirements documentation (16kHz mono PCM16)
+- Updated API reference documentation and code examples
+- Added architecture diagrams and flowcharts
+
+### 🧪 Testing
+
+- Updated unit tests to cover new architecture
+- Added unit tests for external data mode
+- Updated E2E tests to use new API
+- All tests passing (69 unit tests + 24 E2E tests)
+
+### 🔄 Internal Changes
+
+- Refactored `AvatarController` into unified playback layer
+- Extracted `NetworkLayer` as independent component
+- Optimized animation playback loop and audio-animation synchronization logic
+- Updated WASM module (avatar_core_wasm.wasm)
+- Improved error handling and logging
+
+### 📦 Dependencies
+
+- No new dependencies added
+- Maintained existing dependency versions
+
+---
+
+## [1.0.0-beta.2] - Previous Version
+
+(Previous version changelog entries...)