@spatialwalk/avatarkit 1.0.0-beta.5 → 1.0.0-beta.51

package/CHANGELOG.md

@@ -5,10 +5,454 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.0.0-beta.51] - 2025-01-05
+
+### 🐛 Bugfix
+- **Character Settings Update** - Fixed issue where cached Avatar instances would not update `characterSettings` in resources when `characterMeta` was updated. Now `resources.characterSettings` is synchronized when `characterMeta` changes, ensuring new `AvatarView` instances use the latest configuration.
+
+### ✨ New Features
+- **Post-Processing Config Enhancement** - Added `translation` and `eyePose` fields to `PostProcessingConfig` interface:
+  - `translation`: Position offset (x, y, z)
+  - `eyePose`: Eye pose offset with bias and weights (x, y, z, weightX, weightY, weightZ), applied to both left and right eyes
+
+## [1.0.0-beta.50] - 2025-01-05
+
+### 🔧 Performance Improvements
+- **Removed CORS Preflight for Meta API** - Removed authentication headers (`X-App-Id` and `Authorization`) and unnecessary `Content-Type` header from character metadata API requests. This eliminates CORS preflight requests for simple GET requests, improving loading performance.
+
+## [1.0.0-beta.49] - 2025-01-05
+
+### 🐛 Bugfix
+- **Template Cache Version** - Fixed issue where template cache would be incorrectly cleared when switching between different SDK versions. Now uses independent template resource version (1.0.0) instead of SDK version, allowing different SDK versions to share the same template cache when template resources are identical.
+
+## [1.0.0-beta.48] - 2025-01-05
+
+### ✨ New Features
+- **PWA Cache Management** - Added automatic PWA cache management for character and template resources to improve loading performance
+
+### 🔧 Performance Improvements
+- **Cache Hit Rate Metrics** - Resource downloads now report cache status for analytics
+
+## [1.0.0-beta.47] - 2025-12-29
+
+### 🐛 Bugfix
+- **Avatar Meta Update** - Fixed issue where cached Avatar instances would return stale character metadata even when the latest metadata was fetched. Now the character metadata is always updated to the latest version when loading an avatar, even if the version number hasn't changed.
+
+## [1.0.0-beta.46] - 2025-12-29
+
+### 🐛 Bugfix
+- Expose `getCharacterMeta()` method in `Avatar` class as public API
+
+## [1.0.0-beta.45] - 2025-12-29
+
+### ✨ New Features
+- **Immediate Rendering on Pause** - When playback is paused, updating post-processing parameters or camera configuration will immediately re-render the current frame:
+  - Post-processing parameters (rotation, neck pose, jaw pose, eye blink, etc.) now apply immediately when updated during pause
+  - Camera configuration (position, target, FOV, etc.) now applies immediately when updated during pause
+  - During normal playback, these updates still apply on the next frame as before
+
+### 🔄 API Changes
+
+### 🔄 API Changes
+- **AvatarView Ready Promise Replaced with Callback** - Replaced `ready: Promise<void>` property with `onFirstRendering?: () => void` callback for better async handling:
+  - Removed `ready` Promise property
+  - Added `onFirstRendering` optional callback that is called when the first frame is rendered
+  - Callback-based approach is more idiomatic and easier to use than Promise-based approach
+
+### 🗑️ Removed
+- **Removed unused `seek()` method** - Removed unused and buggy `seek()` method from `StreamingAudioPlayer`
+
+## [1.0.0-beta.44] - 2025-12-29
+
+### 🔧 Performance Improvements
+- **Removed unnecessary headers from static resource requests** - Removed `X-App-Id` and `Authorization` headers from static resource downloads (shape.pb, point_cloud.ply, camera.json, etc.) to avoid CORS preflight requests and improve loading performance
+
+## [1.0.0-beta.43] - 2025-12-29
+
+### 🔧 Improvements
+- Remove some unnecessary telemetry reporting
+
+## [1.0.0-beta.42] - 2025-12-29
+
+### ✨ New Features
+- **PostHog Telemetry Configuration** - PostHog telemetry reporting now supports separate configuration for domestic (CN) and international (INTL) environments:
+  - Separate PostHog hosts and API keys for CN and INTL environments
+  - Compression disabled for domestic self-hosted instance, enabled for international official PostHog
+  - Localhost domain filtering to avoid test data pollution
+
+## [1.0.0-beta.41] - 2025-12-29
+
+### 🔄 Breaking Changes
+- **Background Image Support Removed** - Removed background image functionality from `AvatarView`:
+  - Removed `isOpaque` getter/setter property
+  - Removed `setBackgroundImage()` method
+  - Removed background image loading from `AvatarDownloader`
+  - Canvas now always uses transparent background
+
+## [1.0.0-beta.40] - 2025-12-27
+
+### 🔧 API Improvements
+- Extended post-processing parameter configuration interface
+
+## [1.0.0-beta.39] - 2025-12-27
+
+### 🔧 Improvements
+- Filter localhost domain from telemetry reporting to avoid test data pollution
+
+## [1.0.0-beta.38] - 2025-12-27
+
+### 🔄 Breaking Changes
+- Migrated telemetry reporting from Tencent Cloud CLS to PostHog
+
+### ✨ New Features
+- Implemented download queue for character resource loading
+
+## [1.0.0-beta.37] - 2025-12-24
+
+### 🐛 Bugfix
+- Various bug fixes and improvements
+
+## [1.0.0-beta.36] - 2025-12-20
+
+### ✨ New Features
+- **Audio Sample Rate Configuration** - Added configurable audio sample rate support in SDK initialization
+  - Added `audioFormat` configuration option with `channelCount` (fixed to 1) and `sampleRate` (supports: 8000, 16000, 22050, 24000, 32000, 44100, 48000 Hz, default: 16000)
+  - Audio recording and playback now automatically use the configured sample rate
+  - Vanilla demo updated with sample rate dropdown selector for easy testing
+
+## [1.0.0-beta.35] - 2025-12-16
+
+### 🔄 API Changes
+- **Avatar Transform API** - Changed `setTransform` method to `transform` getter/setter property for more idiomatic JavaScript API. Now supports both reading and setting transform values.
+
+## [1.0.0-beta.34] - 2025-12-16
+
+### ✨ New Features
+- **Avatar Transform API** - Added `setTransform` method to `AvatarView` for controlling avatar position and scale within the canvas. Supports normalized coordinates (-1 to 1) for position and scale factor.
+
+## [1.0.0-beta.32] - 2025-12-16
+
+### 🔄 Breaking Changes
+- **Host Mode Interface** - Updated `yieldFramesData` method to accept binary data array instead of single binary data. The method now processes multiple protobuf-encoded Message objects in a single call.
+
+## [1.0.0-beta.31] - 2025-12-16
+
+### 🐛 Bugfix
+- **Environment CORS Issue** - Fixed CORS issues when requesting configuration and character data APIs. SDK now provides default configuration fallback when config requests fail, ensuring smooth operation across different environments.
+
+## [1.0.0-beta.30] - 2025-12-15
+
+### 🐛 Bugfix
+- **Template Resources Duplicate Download** - Fixed issue where template resources were being re-downloaded for each new character load. Template resources are now only loaded once during SDK initialization and reused for all characters.
+
+## [1.0.0-beta.29] - 2025-12-15
+
+### 🔄 Breaking Changes
+- **Environment Enum** - Removed `Environment.test`. Only `Environment.cn` and `Environment.intl` are now supported. Environment configuration must be explicitly provided.
+- **Log Level Default** - Default log level changed from `LogLevel.all` to `LogLevel.off`. Set `logLevel: LogLevel.all` in configuration to enable all logs.
+
+### ✨ New Features
+- **Avatar Caching** - Added avatar caching methods in `AvatarManager`:
+  - `retrieve(id: string): Avatar | undefined` - Get cached avatar by ID
+  - `clear(id: string): void` - Clear cached avatar for specific ID
+  - `clearAll(): void` - Clear all cached avatars
+  - `clearCache()` is now deprecated, use `clearAll()` instead
+- **Background Image Support** - Added background image control in `AvatarView`:
+  - `isOpaque: boolean` - Getter/setter to control canvas background transparency
+  - `setBackgroundImage(image: HTMLImageElement | string | null): void` - Set or remove background image
+
+### 🔧 Improvements
+- **Avatar Version Checking** - Avatar cache now automatically checks version and reloads if cached avatar version differs from latest metadata
+- **Concurrent Loading** - Multiple concurrent `load()` calls for the same avatar ID now reuse the same loading promise
+
+## [1.0.0-beta.28] - 2025-12-08
+
+### 🔄 Breaking Changes
+- **Class Renamed** - `AvatarKit` class has been renamed to `AvatarSDK` for better consistency
+  - Update all imports: `import { AvatarKit } from '@spatialwalk/avatarkit'` → `import { AvatarSDK } from '@spatialwalk/avatarkit'`
+  - Update all API calls: `AvatarKit.initialize()` → `AvatarSDK.initialize()`
+  - All static methods and properties remain the same, only the class name has changed
+
+### 🔧 Improvements
+- **Rendering Optimizations** - Added fragment shader discard optimization to improve performance and edge quality
+- **Shader Alignment** - Removed view matrix transpose to align with Android SDK implementation
+- **Resource Cleanup** - Removed unused `frameMono` and `audioMono` resources from SDK and tests
+
+## [1.0.0-beta.27] - 2025-12-04
+
+### 🐛 Bugfix
+
+## [1.0.0-beta.26] - 2025-12-03
+
+### ✨ New Features
+- **Log Level Configuration** - Added `logLevel` configuration option to control SDK logging verbosity
+  - `LogLevel.off` - Disable all logs
+  - `LogLevel.error` - Only error logs
+  - `LogLevel.warning` - Warning and error logs
+  - `LogLevel.all` - All logs (default)
+- **Standardized CLS Logging Events** - All CLS logging events now use standardized field names for better querying
+  - Standard events: `sdk_initialized`, `send_audio_failed`, `message_error`, `heartbeat_failed`, `service_restarted`, `session_token_invalid`, `session_token_expired`, `yield_animation_failed`, `fetch_avatar_metadata_failed`, `download_avatar_assets_failed`
+  - Standard fields: `req_id`, `con_id`, `avatar_id`, `description`
+- **End-to-End Latency Tracking** - Added `driving_latency` event to track latency metrics for both SDK and Host modes
+  - Tracks: `start`, `tap2`, `tap4`, `end`, `first_frame` timestamps
+  - Automatically reported at key moments during audio/animation processing
+
+### 🔧 Improvements
+- **CLS Configuration Alignment** - Updated CLS endpoints and Topic IDs to match iOS SDK configuration
+- **Default API Endpoints** - Fixed default API endpoint for `intl` environment to `api.intl.spatialwalk.cloud`
+- **Error Code Enum** - Added new error codes: `appIDUnrecognized`, `sessionTokenInvalid`, `sessionTokenExpired`, `failedToFetchAvatarMetadata`, `failedToDownloadAvatarAssets`
+
+### 🗑️ Removed
+- **Redundant Log Events** - Removed unnecessary log events: `playback_paused`, `playback_resumed`, `reconnecting`, `connect_success`, `server_error`, `decode_failed`
+
+## [1.0.0-beta.25] - 2025-12-03
+
+### 🔧 Improvements
+- **Removed Audio Buffer Limit** - Removed the 100 chunk limit on pending audio buffers to allow unlimited audio data buffering
+
+## [1.0.0-beta.24] - 2025-11-26
+
+### 🔧 API Changes
+- **Backend Endpoints Updated** - Backend API endpoints have been updated for all environments
+
+## [1.0.0-beta.23] - 2025-11-26
+
+### 🔧 API Changes
+- **Breaking Change** - `playback()` method is no longer supported and has been removed from public API
+
+## [1.0.0-beta.22] - 2025-11-26
+
+### 🔧 API Changes
+- **State Callback Renamed** - `onAvatarState` has been renamed to `onConversationState` for better clarity
+  - The callback now uses `ConversationState` enum with states: `idle` and `playing`
+- **Environment Enum Updated** - `Environment.us` has been renamed to `Environment.intl` for better internationalization support
+  - All references to `Environment.us` should be updated to `Environment.intl`
+  - Remote config endpoints now use `intl` instead of `us`
+
+### ✨ New Features
+- **Volume Control** - Added volume control API for audio playback
+  - `setVolume(volume: number)` - Set audio volume (0.0 to 1.0)
+  - `getVolume(): number` - Get current audio volume
+  - Volume control only affects the avatar's audio player, not system volume
+  - Volume changes take effect immediately, including for currently playing audio
+
+## [1.0.0-beta.21] - 2025-11-25
+
+### ✨ New Features
+- **SDK Mode Connection Timeout Fallback** - Enhanced fallback mechanism for SDK mode:
+  - If WebSocket connection fails to establish within 15 seconds, the SDK automatically enters fallback mode
+  - In fallback mode, audio data can still be sent and will play normally, even though no animation data will be received
+  - Connection failures also trigger this fallback mechanism
+  - `ConnectionState.connecting` and `ConnectionState.failed` events are emitted to notify external applications
+  - Concurrent connection attempts are prevented to avoid race conditions
+
+### 🔧 Improvements
+- Improved connection state management: `isConnected` now accurately reflects actual connection status (false in fallback mode)
+- Added `canSend()` method in `NetworkLayer` to check if audio can be sent (including fallback mode)
+- Better separation of concerns: connection state checks are now handled by `NetworkLayer`, while playback logic remains in `AvatarController`
+
+## [1.0.0-beta.20] - 2025-11-25
+
+### 🔧 Improvements
+- Removed 1-second audio buffer requirement for auto-starting playback in Host mode
+
+## [1.0.0-beta.19] - 2025-11-25
+
+### 📚 Documentation
+- **Fixed misleading documentation** - Corrected Host mode API documentation to accurately reflect that `yieldAudioData()` can be called directly to start a new session without requiring `playback()` first
+  - Removed misleading "after playback() is called" comments
+  - Clarified two ways to start a session: using `playback()` for existing data, or `yieldAudioData()` directly for streaming
+  - Updated all examples and workflow descriptions to reflect correct API usage
+
+## [1.0.0-beta.18] - 2025-11-25
+
+### 🔧 API Changes
+- **Renamed `reqId` to `conversationId`** - Updated terminology for better clarity
+  - All methods and parameters that used `reqId` now use `conversationId`
+  - `getCurrentReqId()` → `getCurrentConversationId()`
+  - `generateReqId()` → `generateConversationId()`
+  - Updated all event logs and documentation to use `conversationId`
+  - Note: Protobuf protocol still uses `reqId` field name internally, but SDK API uses `conversationId`
+
+### 📚 Documentation
+- Enhanced Host mode documentation to clearly emphasize the workflow: send audio data first to get conversationId, then use that conversationId to send animation data
+- Updated Host Mode Example and Host Mode Flow sections with clearer step-by-step instructions
+
+## [1.0.0-beta.17] - 2025-11-24
+
+### ✨ New Features
+- **Audio-Only Fallback Mechanism** - SDK now includes automatic fallback to audio-only playback when animation data is unavailable
+  - SDK mode: Automatically enters audio-only mode when server returns an error
+  - Host mode: Automatically enters audio-only mode when empty animation data is provided
+  - Once in audio-only mode, subsequent animation data for that session is ignored
+  - Fallback mode is interruptible, just like normal playback mode
+
+### 🔧 API Changes
+- **Playback Mode Configuration** - Moved playback mode configuration from `AvatarView` constructor to `AvatarSDK.initialize()`
+  - Playback mode is now determined by `drivingServiceMode` in `AvatarSDK.initialize()` configuration
+  - `AvatarView` constructor now only requires `avatar` and `container` parameters
+  - Removed `AvatarViewOptions` interface
+  - `container` parameter is now required (no longer optional)
+- **Method Renames** - Renamed methods in `AvatarController` for Host mode to better reflect their purpose
+  - `play()` → `playback()`: Renamed to better reflect that the method is used for playback of existing data (replay mode)
+    - Old API: `avatarController.play(initialAudioChunks, initialKeyframes)`
+    - New API: `avatarController.playback(initialAudioChunks, initialKeyframes)`
+  - `sendAudioChunk()` → `yieldAudioData()`: Renamed to better reflect that the method yields/streams audio data
+    - Old API: `avatarController.sendAudioChunk(data, isLast)`
+    - New API: `avatarController.yieldAudioData(data, isLast)`
+  - `sendKeyframes()` → `yieldFramesData()`: Renamed to better reflect that the method yields/streams animation keyframes
+    - Old API: `avatarController.sendKeyframes(keyframes, reqId)`
+    - New API: `avatarController.yieldFramesData(keyframes, conversationId)`
+
+### 🔧 Improvements
+- Extended transition animation duration from 200ms to 400ms for smoother end-of-playback transitions
+
+### 📚 Documentation
+- Updated README.md to use "SDK mode" and "Host mode" terminology instead of "Network mode" and "External data mode"
+- Added fallback mechanism documentation
+- Updated API reference to reflect new constructor signature
+
+## [1.0.0-beta.16] - 2025-11-21
+
+### ✨ New Features
+- **Multi-AvatarView Support** - SDK now supports multiple `AvatarView` instances simultaneously
+  - Each `AvatarView` instance can manage its own character independently
+  - Multiple characters can be displayed and controlled at the same time
+  - Each instance maintains its own rendering context, playback state, and network connection
+  - Removed the previous limitation of "only one AvatarView instance at a time"
+
+### 📚 Documentation
+- Updated README.md to reflect multi-character support capabilities
+- Added multi-character usage examples
+- Removed outdated limitation notes about single AvatarView instance
+
+---
+
+## [1.0.0-beta.15] - 2025-11-20
+
+### ✨ New Features
+- Added authentication support for API requests
+  - `appId` is now automatically included in HTTP headers (`X-App-Id`) and WebSocket URL query parameters
+  - `sessionToken` is now automatically included in HTTP headers (`Authorization: Bearer {token}`) and WebSocket URL query parameters
+  - All network requests (character loading, resource downloads, WebSocket connections) now include authentication credentials
+
+### 🔧 Improvements
+- Fixed canvas resize handling for different aspect ratios
+  - Projection matrix is now correctly updated when canvas aspect ratio changes
+  - Rendering system properly handles portrait (9:16) and landscape (16:9) aspect ratios
+  - Canvas automatically adapts to container size changes via ResizeObserver
+
+### 🐛 Bug Fixes
+- Fixed `RenderSystem.handleResize()` to update projection matrix when canvas size changes
+  - Previously only updated aspect ratio, now also updates projection matrix for correct rendering
+
+---
+
+## [1.0.0-beta.14] - 2025-11-18
+
+### ✨ New Features
+- Added pause and resume functionality for avatar playback
+  - New `pause()` method to pause audio and animation playback
+  - New `resume()` method to resume playback from paused state
+  - Added `paused` state to `AvatarState` enum
+  - State changes are communicated via `onAvatarState` callback
+  - Works in both network mode and external data mode
+
+### 🔧 Improvements
+- Canvas now uses container's full dimensions instead of fixed 1:1 aspect ratio
+  - Canvas automatically adapts to container size (width and height)
+  - External developers can control canvas aspect ratio by setting container dimensions
+  - Rendering system correctly handles any aspect ratio without distortion
+
+---
+
+## [1.0.0-beta.13] - 2025-11-17
+
+### 🔒 API Improvements
+- Reduced console noise by setting default log level to Warning
+  - Logger now only outputs warnings and errors by default
+  - Set `VITE_DEBUG_LOG=true` to enable all logs
+  - Simplified logger configuration using `setGlobalLogLevel(LogLevel.Warning)`
+
+### 🐛 Bug Fixes
+- Fixed SDK config loader warning in non-debug mode
+  - Config loading failures are now silently handled in production
+  - Warnings only shown when `VITE_DEBUG_LOG=true`
+
+---
+
+## [1.0.0-beta.12] - 2025-11-17
+
+### 🔒 API Improvements
+- Marked performance monitoring APIs as internal
+  - `RenderPerformanceStats` interface and `getPerformanceStats()` method are now marked with `@internal` tag
+  - These APIs are not exposed in the public TypeScript definitions
+  - Reverted performance monitoring implementation to simpler version (only `renderTime` and `sortTime`)
+
+---
+
+## [1.0.0-beta.11] - 2025-11-17
+
+### 📚 Documentation
+- Updated demo repository link in README.md
+  - Changed example project link from `Avatarkit-web-demo` to `AvatarSDK-Web-Demo` to match the new repository name
+
+---
+
+## [1.0.0-beta.10] - 2025-11-16
+
+### 🔒 API Improvements
+- Cleaned up public API to hide internal implementation details
+  - Marked internal methods with `@internal` JSDoc tag to exclude them from TypeScript declarations
+  - Removed unused environment configuration options (`realtimeApiBaseUrl`, `realtimeWsUrl`)
+  - Added `appId` getter to `AvatarSDK` for accessing initialized app ID
+  - Internal methods like `getEnvironmentConfig`, `logEvent`, `getCanvas`, `getCameraConfig`, `updateCameraConfig` are now properly hidden from public API
+
+### 🐛 Bug Fixes
+- Fixed memory leaks when disposing AvatarView instances
+  - Improved resource cleanup to prevent memory leaks when switching or disposing avatar instances
+  - Properly clears all callbacks, event listeners, and large data buffers on disposal
+
+---
+## [1.0.0-beta.9] - 2025-11-15
+
+### 🐛 Bug Fixes
+- Fixed memory leaks when disposing AvatarView instances
+  - Improved resource cleanup to prevent memory leaks when switching or disposing avatar instances
+  - Properly clears all callbacks, event listeners, and large data buffers on disposal
+
+---
+
+## [1.0.0-beta.8] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed audio-animation synchronization issue after normal playback completion
+  - Playback state is now properly cleaned up when audio finishes normally, preventing synchronization issues in subsequent conversations
+
+---
+
+## [1.0.0-beta.7] - 2025-11-14
+
+### 🔄 Refactoring
+- Improved rendering state management for better stability and maintainability
+
+### 🐛 Bug Fixes
+- Fixed interrupt not generating transition animation when interrupting during playback
+- Fixed transition animation being regenerated and reset in external data mode when sending keyframes after playback starts
+
+---
+
+## [1.0.0-beta.6] - 2025-11-14
+
+### 🐛 Bug Fixes
+- Fixed transition animation being regenerated and reset in external data mode when sending keyframes after playback starts, causing the transition to play twice
+
+---
+
 ## [1.0.0-beta.5] - 2025-11-14
 
 ### 🐛 Bug Fixes
-- Fixed missing `AvatarPlaybackMode` enum export in published package
+- Fixed missing `DrivingServiceMode` enum export in published package
 
 ---
 
@@ -37,7 +481,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **External Data Mode**:
   - External components fully control audio and animation data acquisition
   - SDK only responsible for synchronized playback of externally provided data
-  - Use `controller.play()`, `controller.sendAudioChunk()` and `controller.sendKeyframes()` methods
+  - Use `controller.playback()`, `controller.yieldAudioData()` and `controller.yieldFramesData()` methods
 
 ### ✨ New Features
 
@@ -73,7 +517,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   // New API
   new AvatarView(avatar, {
     container: container,
-    playbackMode: AvatarPlaybackMode.network // or AvatarPlaybackMode.external
+    playbackMode: DrivingServiceMode.sdk // or DrivingServiceMode.host
   })
   ```