@myned-ai/gsplat-flame-avatar-renderer 1.0.5 → 1.0.7

package/README.md

@@ -4,16 +4,19 @@
 
 A specialized Gaussian Splatting library for rendering animated 3D avatars with FLAME parametric head model support, LAM (Large Avatar Model) head avatars, and ARKit blendshape compatibility.
 
+---
+
 ## Features
 
-- 🎭 **52 ARKit Blendshapes** — Complete facial expression control
-- 🔥 **FLAME Model Support** — Full integration with FLAME parametric head model
-- 🎮 **Animation State Machine** — Built-in states (Idle, Listening, Thinking, Responding)
-- ⚡ **GPU Optimized** — WebGL-based rendering with WebAssembly sorting
-- 🎯 **Three.js Integration** — Works seamlessly with Three.js r150+
-- 🧠 **LAM Head Avatars** — Support for Large Avatar Model based head avatars
-- 📦 **ZIP Asset Loading** — Load compressed avatar assets directly
-- 🔄 **Real-time Animation** — Smooth blendshape interpolation at 30fps
+- **52 ARKit Blendshapes** — Complete facial expression control
+- **FLAME Model Support** — Full integration with FLAME parametric head model
+- **Animation State Machine** — Built-in states (Idle, Listening, Thinking, Responding)
+- **GPU Optimized** — WebGL-based rendering with WebAssembly sorting
+- **Three.js Integration** — Works seamlessly with Three.js
+- **LAM Head Avatars** — Support for Large Avatar Model based head avatars
+- **ZIP Asset Loading** — Load compressed avatar assets directly
+- **Real-time Animation** — Smooth blendshape interpolation at 30fps
+- **Conditional Iris Occlusion Fix** — Optional iris fade during eye blinks via iris_occlusion.json
 
 ---
 
@@ -32,7 +35,8 @@ import { GaussianSplatRenderer } from 'gsplat-flame-avatar-renderer';
 
 const container = document.getElementById('avatar-container');
 
-const renderer = await GaussianSplatRenderer.getInstance(
+// Create a new renderer instance (v1.0.6+)
+const renderer = await GaussianSplatRenderer.create(
   container,
   './path/to/avatar.zip',
   {
@@ -46,6 +50,9 @@ const renderer = await GaussianSplatRenderer.getInstance(
     })
   }
 );
+
+// Don't forget to clean up when done!
+// renderer.dispose();
 ```
 
 ---
@@ -56,9 +63,9 @@ const renderer = await GaussianSplatRenderer.getInstance(
 
 The main class for rendering Gaussian splat avatars.
 
-#### `GaussianSplatRenderer.getInstance(container, assetPath, options)`
+#### `GaussianSplatRenderer.create(container, assetPath, options)`
 
-Creates or returns a singleton renderer instance.
+Creates a new renderer instance with proper resource isolation.
 
 | Parameter | Type | Description |
 |-----------|------|-------------|
@@ -76,6 +83,65 @@ Creates or returns a singleton renderer instance.
 | `loadProgress` | `(progress: number) => void` | Loading progress callback |
 | `downloadProgress` | `(progress: number) => void` | Download progress callback |
 
+**Advanced Options:**
+
+Additional options for customizing avatar appearance and rendering:
+
+**Scene Reveal & Fade-In**
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `sceneRevealMode` | `number` | `0` (Default) | Controls how avatar appears: `0` = Default (fast for final, slow for progressive), `1` = Gradual (slow fade-in), `2` = Instant (immediate) |
+| `sceneFadeInRateMultiplier` | `number` | `1.0` | Speed multiplier for fade-in effect. Higher = faster appearance |
+
+**Camera & View Settings**
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `initialCameraPosition` | `Array<number>` | `[0, 10, 15]` | Starting camera position [x, y, z] |
+| `initialCameraRotation` | `Array<number>` | `[0, 0, 0]` | Starting camera rotation [x, y, z] in radians |
+| `initialCameraLookAt` | `Array<number>` | `[0, 0, 0]` | Camera focal point [x, y, z] |
+| `cameraUp` | `Array<number>` | `[0, 1, 0]` | Camera up vector [x, y, z] |
+
+**Rendering Quality**
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `antialiased` | `boolean` | `false` | Enable anti-aliasing to reduce artifacts at different resolutions |
+| `kernel2DSize` | `number` | `0.3` | Constant added to projected 2D splat scales (affects splat smoothness) |
+| `maxScreenSpaceSplatSize` | `number` | `1024` | Maximum screen-space splat size in pixels |
+| `halfPrecisionCovariancesOnGPU` | `boolean` | `false` | Use 16-bit float for GPU covariances (better performance, slight quality loss) |
+| `ignoreDevicePixelRatio` | `boolean` | `false` | Force device pixel ratio to 1 for better performance on high-DPI displays |
+
+#### `renderer.dispose()`
+
+Cleans up all resources (WebGL context, textures, workers, event listeners). **Always call this when you're done with a renderer instance** to prevent memory leaks.
+
+```javascript
+// Clean up when done
+renderer.dispose();
+```
+
+#### Migration from v1.0.5
+
+If you're upgrading from v1.0.5 or earlier:
+
+```javascript
+// OLD (still works but deprecated)
+const renderer = await GaussianSplatRenderer.getInstance(container, path, options);
+
+// NEW (recommended)
+const renderer = await GaussianSplatRenderer.create(container, path, options);
+// ... use renderer ...
+renderer.dispose(); // Don't forget!
+```
+
+**Key Changes:**
+- Each `create()` call returns a **new independent instance** (no longer a singleton)
+- You can now have **multiple renderers** on the same page
+- Each instance has its own canvas element
+- **Must call `dispose()`** when finished to clean up resources
+
 ### Animation States
 
 The renderer supports the following states via `getChatState`:
@@ -112,12 +178,36 @@ The avatar ZIP file should contain:
 ```
 avatar.zip
 ├── avatar/
-│   ├── offset.ply          # Gaussian splat point cloud
-│   ├── animation.glb       # Animation clips
-│   ├── skin.glb            # Skinning/skeleton data
-│   └── vertex_order.json   # Vertex ordering data
+│   ├── offset.ply             # Gaussian splats point cloud (required)
+│   ├── animation.glb          # Animation clips (required)
+│   ├── skin.glb               # Skinning/skeleton data (required)
+│   ├── vertex_order.json      # Vertex ordering data (required)
+│   └── iris_occlusion.json    # Iris occlusion ranges (optional)
+```
+
+### Optional: Iris Occlusion
+
+To enable iris fade during eye blinks if the iris is still visible, include `iris_occlusion.json` in your avatar folder to apply a fix.
+
+An example of `iris_occlusion.json` :
+
+```json
+{
+  "right_iris": [
+    [4000, 4100],
+    [3930, 3970]
+  ],
+  "left_iris": [
+    [4470, 4605],
+    [18370, 18799]
+  ]
+}
 ```
 
+Each array `[start, end]` defines a range of Gaussian splat indices that belong to the iris. When eye blink blendshapes (eyeBlinkLeft, eyeBlinkRight) increase, these splats will smoothly fade out to prevent visual artifacts during eye closure.
+
+If `iris_occlusion.json` is not present, the renderer will work normally without iris occlusion.
+
 ---
 
 ## Architecture Overview
@@ -145,6 +235,7 @@ The library supports hybrid rendering where Gaussian Splat positions are deforme
 - **52 blendshapes**: Full ARKit compatibility for face tracking
 - **LBS Skinning**: Splat positions transformed via bone matrices and blend weights
 - **GPU-based**: All deformation computed in vertex shader via texture lookups
+- **Iris Occlusion**: Optional dynamic iris fade during eye blinks (configured per-avatar via JSON)
 
 ---
 
@@ -156,50 +247,87 @@ The library supports hybrid rendering where Gaussian Splat positions are deforme
 | `src/buffers/` | 5 | GPU buffer management (SplatBuffer, partitioning) |
 | `src/core/` | 6 | Rendering engine (Viewer, SplatMesh, SplatTree) |
 | `src/enums/` | 8 | Constants and enumerations |
+| `src/errors/` | 1 | Custom error classes (ValidationError, NetworkError, ParseError, etc.) |
 | `src/flame/` | 5 | FLAME model integration (FlameAnimator, textures) |
 | `src/loaders/` | 6 | PLY format loader (INRIA v1) |
-| `src/materials/` | 4 | WebGL shaders (SplatMaterial2D/3D) |
+| `src/materials/` | 4 | WebGL shaders (SplatMaterial2D/3D, conditional iris occlusion) |
 | `src/raycaster/` | 4 | Intersection testing |
 | `src/renderer/` | 4 | Application layer (GaussianSplatRenderer, AnimationManager) |
-| `src/utils/` | 3 | Shared utilities |
+| `src/utils/` | 8 | Shared utilities (validation, logging, object pooling, blob management) |
 | `src/worker/` | 2 | WebAssembly sorting worker |
 
 ### Key Components
 
 | Component | Purpose |
 |-----------|---------|
-| **GaussianSplatRenderer** | Main entry point, ZIP loading, render loop |
+| **GaussianSplatRenderer** | Main entry point, ZIP loading, render loop, asset management |
 | **Viewer** | Scene management, camera, render pipeline |
-| **SplatMesh** | GPU textures, instanced geometry |
+| **SplatMesh** | GPU textures, instanced geometry, iris config forwarding |
+| **SplatMaterial3D** | WebGL shader generation, conditional iris occlusion code |
 | **FlameAnimator** | Bone rotations, blendshape weights |
 | **FlameTextureManager** | GPU texture uploads for FLAME data |
 | **AnimationManager** | State machine (Idle, Listen, Think, Speak) |
-| **PlyLoader** | Loading PLY files from ZIP |
+| **PlyLoader** | Loading PLY files from ZIP with validation |
 | **SortWorker** | WebAssembly depth sorting |
+| **ValidationUtils** | Input validation (URLs, paths, ranges, types) |
+| **Logger** | Structured logging system |
+| **ObjectPool** | Object pooling for performance (Vector3, Quaternion) |
+| **BlobUrlManager** | Blob URL lifecycle management |
 
 ---
 
 ## Build Output
 
-Rollup produces ESM and CommonJS bundles in `dist/`:
+Rollup produces optimized ESM and CommonJS bundles in `dist/`:
+
+| File | Size | Format | Source Maps |
+|------|------|--------|-------------|
+| `gsplat-flame-avatar-renderer.esm.min.js`| 244 KB | ES Module (production, minified) | External (.map) |
+| `gsplat-flame-avatar-renderer.cjs.min.js`| 246 KB | CommonJS (production, minified) | External (.map) |
+| `gsplat-flame-avatar-renderer.esm.js` | 2.1 MB | ES Module (development, unminified) | Inline |
+| `gsplat-flame-avatar-renderer.cjs.js` | 2.1 MB | CommonJS (development, unminified) | Inline |
 
-| File | Format |
-|------|--------|
-| `gsplat-flame-avatar-renderer.esm.js` | ES Module (ESM) — for modern bundlers and browsers |
-| `gsplat-flame-avatar-renderer.cjs.js` | CommonJS (CJS) — for Node `require()` consumers |
+** The minified builds are used by default** when importing the package. Development builds are automatically selected when `NODE_ENV=development`.
 
-Builds include inline source maps. Type declarations are available at `dist/index.d.ts`, and the package publishes the `src/` folder for source inspection.
+Type declarations are available at `dist/index.d.ts`, and the package publishes the `src/` folder for source inspection.
+
+---
 
+## Browser Compatibility
+
+### Desktop
+| Browser | Version | WebGL 2 | SharedArrayBuffer | Status |
+|---------|---------|---------|-------------------|--------|
+| Chrome | 80+ | ✓ | ✓ (with headers) | Fully Supported |
+| Firefox | 75+ | ✓ | ✓ (with headers) | Fully Supported |
+| Safari | 14+ | ✓ | ✓ (macOS 11.3+) | Fully Supported |
+| Edge | 80+ | ✓ | ✓ (with headers) | Fully Supported |
+
+### Mobile
+| Browser | Version | Performance | Notes |
+|---------|---------|-------------|-------|
+| Safari iOS | 14.5+ | Good | Enable WebGL 2 in Settings |
+| Chrome Android | 80+ | Good | May require reduced splat count |
+| Samsung Internet | 13+ | Fair | Limited SharedArrayBuffer support |
+
+**Known Limitations:**
+- iOS Safari: WebGL context may be lost during backgrounding
+- Android: Performance varies significantly by device
+- Firefox Android: SharedArrayBuffer requires site isolation
 ---
 
-## Browser Support
+## Deployment Requirements
 
-- Chrome 80+
-- Firefox 75+
-- Safari 14+
-- Edge 80+
+### SharedArrayBuffer Configuration
+
+This library uses WebAssembly with SharedArrayBuffer for high-performance splat sorting. To deploy in production, your server **must** send the following HTTP headers:
+
+```
+Cross-Origin-Embedder-Policy: require-corp
+Cross-Origin-Opener-Policy: same-origin
+```
 
-**Requires WebGL 2.0 support.**
+Without these headers, SharedArrayBuffer will be disabled by the browser and the renderer will fail to initialize.
 
 ---