loomlarge 0.1.0 → 0.1.4

package/README.md

@@ -1,714 +1,847 @@
-![Latticework by Lovelace LOL](LoomLarge.png)
+# LoomLarge
 
-# LoomLarge (Latticework Animation Platform)
+A FACS-based morph and bone mapping library for controlling high-definition 3D characters in Three.js.
 
-**LoomLarge** is a next-generation interactive 3D character animation platform built on **Latticework**, featuring reactive state management with XState, facial animation control via ARKit FACS (Facial Action Coding System), and modular agency-based architecture for lip-sync, eye/head tracking, prosodic expression, and conversational AI.
+LoomLarge provides pre-built mappings that connect [Facial Action Coding System (FACS)](https://en.wikipedia.org/wiki/Facial_Action_Coding_System) Action Units to the morph targets and bone transforms found in Character Creator 4 (CC4) characters. Instead of manually figuring out which blend shapes correspond to which facial movements, you can simply say `setAU(12, 0.8)` and the library handles the rest.
 
 ---
 
 ## Table of Contents
-1. [Quick Start](#quick-start)
-2. [Core Concepts](#core-concepts)
-   - [Latticework Architecture](#latticework-architecture)
-   - [Composite Rotation System](#composite-rotation-system)
-   - [XState & Reactive Services](#xstate--reactive-services)
-3. [Installation](#installation)
-4. [Project Structure](#project-structure)
-5. [How It Works](#how-it-works)
-   - [Animation Service](#animation-service)
-   - [Eye & Head Tracking](#eye--head-tracking)
-   - [Lip-Sync Agency](#lip-sync-agency)
-   - [Prosodic Expression](#prosodic-expression)
-6. [Modules](#modules)
-7. [Development](#development)
-8. [Deployment](#deployment)
-9. [License & Acknowledgments](#license--acknowledgments)
+
+1. [Installation & Setup](#1-installation--setup)
+2. [Using Presets](#2-using-presets)
+3. [Extending & Custom Presets](#3-extending--custom-presets)
+4. [Action Unit Control](#4-action-unit-control)
+5. [Mix Weight System](#5-mix-weight-system)
+6. [Composite Rotation System](#6-composite-rotation-system)
+7. [Continuum Pairs](#7-continuum-pairs)
+8. [Direct Morph Control](#8-direct-morph-control)
+9. [Viseme System](#9-viseme-system)
+10. [Transition System](#10-transition-system)
+11. [Playback & State Control](#11-playback--state-control)
+12. [Hair Physics](#12-hair-physics)
 
 ---
 
-## Quick Start
+## 1. Installation & Setup
+
+### Install the package
 
 ```bash
-# Clone the repository
-git clone https://github.com/meekmachine/LoomLarge.git
-cd LoomLarge
+npm install loomlarge
+```
 
-# Install dependencies
-yarn install
+### Peer dependency
 
-# Start development server (Vite)
-yarn dev
+LoomLarge requires Three.js as a peer dependency:
+
+```bash
+npm install three
+```
 
-# Build for production
-yarn build
+### Basic setup
 
-# Deploy to GitHub Pages
-yarn deploy
+```typescript
+import * as THREE from 'three';
+import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
+import { LoomLargeThree, collectMorphMeshes, CC4_PRESET } from 'loomlarge';
+
+// 1. Create the LoomLarge controller with a preset
+const loom = new LoomLargeThree({ auMappings: CC4_PRESET });
+
+// 2. Set up your Three.js scene
+const scene = new THREE.Scene();
+const camera = new THREE.PerspectiveCamera(35, window.innerWidth / window.innerHeight, 0.1, 100);
+const renderer = new THREE.WebGLRenderer({ antialias: true });
+renderer.setSize(window.innerWidth, window.innerHeight);
+document.body.appendChild(renderer.domElement);
+
+// 3. Load your character model
+const loader = new GLTFLoader();
+loader.load('/character.glb', (gltf) => {
+  scene.add(gltf.scene);
+
+  // 4. Collect all meshes that have morph targets
+  const meshes = collectMorphMeshes(gltf.scene);
+
+  // 5. Initialize LoomLarge with the meshes and model
+  loom.onReady({ meshes, model: gltf.scene });
+});
+
+// 6. In your animation loop, call loom.update(deltaSeconds)
+// This drives all transitions and animations
 ```
 
-The development server will start at `http://localhost:5173` with hot module replacement enabled.
+### Quick start examples
 
----
+Once your character is loaded, you can control facial expressions immediately:
 
-## Core Concepts
+```typescript
+// Make the character smile
+loom.setAU(12, 0.8);
 
-### Latticework Architecture
+// Raise eyebrows
+loom.setAU(1, 0.6);
+loom.setAU(2, 0.6);
 
-- **Agency-Based Design**: Independent services (agencies) handle specialized tasks:
-  - **Animation Agency**: Core snippet scheduling and playback
-  - **Lip-Sync Agency**: Phoneme prediction and viseme animation
-  - **Prosodic Expression Agency**: Emotional head gestures and speech timing
-  - **Eye/Head Tracking Agency**: Gaze control with mouse, webcam, or manual modes
-  - **Conversation Agency**: Multi-modal conversational AI orchestration
+// Blink
+loom.setAU(45, 1.0);
 
-- **Immutable State**: Reactive state management ensures predictable updates and time-travel debugging
-- **XState Machines**: Declarative state machines replace callback hell and ad-hoc timers
+// Open jaw
+loom.setAU(26, 0.5);
 
-### Composite Rotation System
+// Turn head left
+loom.setAU(51, 0.4);
 
-The **Composite Rotation System** in EngineThree allows smooth blending of blendshapes (morphs) and bone rotations:
+// Look up
+loom.setAU(63, 0.6);
+```
 
-- **Continuum Values** (-1 to +1): Single value controls paired AUs (e.g., Head Left ↔ Right)
-- **Mix Weights** (0 to 1): Blend between 100% morph (0) and 100% bone (1)
-- **Unified Rotation State**: Prevents axis conflicts when multiple systems control the same bones
+Animate smoothly with transitions:
 
-Example:
 ```typescript
-// Eyes: -1 (look left) to +1 (look right)
-engine.applyEyeComposite(yaw, pitch);
+// Smile over 200ms
+await loom.transitionAU(12, 0.8, 200).promise;
 
-// Head: yaw/pitch/roll with mix control
-engine.applyHeadComposite(yaw, pitch, roll);
+// Then fade back to neutral
+await loom.transitionAU(12, 0, 300).promise;
 ```
 
-### XState & Reactive Services
+### The `collectMorphMeshes` helper
+
+This utility function traverses a Three.js scene and returns all meshes that have `morphTargetInfluences` (i.e., blend shapes). It's the recommended way to gather meshes for LoomLarge:
 
-- **XState 5.x**: Modern state machines with TypeScript support
-- **Service Pattern**: Each agency exposes a service with start/stop/update lifecycle
-- **Global Context**: Services registered in `ModulesContext` for cross-component access
+```typescript
+import { collectMorphMeshes } from 'loomlarge';
+
+const meshes = collectMorphMeshes(gltf.scene);
+// Returns: Array of THREE.Mesh objects with morph targets
+```
 
 ---
 
-## Installation
+## 2. Using Presets
 
-### Prerequisites
+Presets define how FACS Action Units map to your character's morph targets and bones. LoomLarge ships with `CC4_PRESET` for Character Creator 4 characters.
 
-- **Node.js** 18+ (LTS recommended)
-- **Yarn** 1.22+ or npm 8+
-- Modern browser with WebGL 2.0 support
+### What's in a preset?
 
-### Setup Steps
+```typescript
+import { CC4_PRESET } from 'loomlarge';
+
+// CC4_PRESET contains:
+{
+  auToMorphs: {
+    // AU number → array of morph target names
+    1: ['Brow_Raise_Inner_L', 'Brow_Raise_Inner_R'],
+    12: ['Mouth_Smile_L', 'Mouth_Smile_R'],
+    45: ['Eye_Blink_L', 'Eye_Blink_R'],
+    // ... 87 AUs total
+  },
+
+  auToBones: {
+    // AU number → array of bone bindings
+    51: [{ node: 'HEAD', channel: 'ry', scale: -1, maxDegrees: 30 }],
+    61: [{ node: 'EYE_L', channel: 'rz', scale: 1, maxDegrees: 25 }],
+    // ... 32 bone bindings
+  },
+
+  boneNodes: {
+    // Logical bone name → actual node name in skeleton
+    'HEAD': 'CC_Base_Head',
+    'JAW': 'CC_Base_JawRoot',
+    'EYE_L': 'CC_Base_L_Eye',
+    'EYE_R': 'CC_Base_R_Eye',
+    'TONGUE': 'CC_Base_Tongue01',
+  },
+
+  visemeKeys: [
+    // 15 viseme morph names for lip-sync
+    'V_EE', 'V_Er', 'V_IH', 'V_Ah', 'V_Oh',
+    'V_W_OO', 'V_S_Z', 'V_Ch_J', 'V_F_V', 'V_TH',
+    'V_T_L_D_N', 'V_B_M_P', 'V_K_G_H_NG', 'V_AE', 'V_R'
+  ],
 
-1. **Clone and install**:
-   ```bash
-   git clone https://github.com/meekmachine/LoomLarge.git
-   cd LoomLarge
-   yarn install
-   ```
+  morphToMesh: {
+    // Routes morph categories to specific meshes
+    'face': ['CC_Base_Body'],
+    'tongue': ['CC_Base_Tongue'],
+    'eye': ['CC_Base_EyeOcclusion_L', 'CC_Base_EyeOcclusion_R'],
+  },
+
+  auMixDefaults: {
+    // Default morph/bone blend weights (0 = morph, 1 = bone)
+    26: 0.5,  // Jaw drop: 50% morph, 50% bone
+    51: 0.7,  // Head turn: 70% bone
+  },
+
+  auInfo: {
+    // Metadata about each AU
+    '12': {
+      name: 'Lip Corner Puller',
+      muscularBasis: 'zygomaticus major',
+      faceArea: 'Lower',
+      facePart: 'Mouth',
+    },
+    // ...
+  }
+}
+```
 
-2. **Add your 3D model**:
-   - Place GLB file in `public/characters/`
-   - Update model path in `src/App.tsx`:
-     ```typescript
-     const glbSrc = import.meta.env.BASE_URL + "characters/your-model.glb";
-     ```
+### Passing a preset to LoomLarge
 
-3. **Configure API keys** (optional, for AI modules):
-   - Create `.env.local`:
-     ```
-     VITE_ANTHROPIC_API_KEY=sk-ant-...
-     ```
+```typescript
+import { LoomLargeThree, CC4_PRESET } from 'loomlarge';
 
-4. **Start development**:
-   ```bash
-   yarn dev
-   ```
+const loom = new LoomLargeThree({ auMappings: CC4_PRESET });
+```
 
 ---
 
-## Project Structure
-
-```
-LoomLarge/
-├── README.md                           # ← This file
-├── package.json
-├── vite.config.ts                      # Vite bundler config
-├── public/
-│   ├── characters/                     # GLB 3D models
-│   ├── animations/                     # Pre-baked animation JSON
-│   ├── models/                         # ML models (face-api.js)
-│   └── skyboxes/                       # Environment maps
-├── src/
-│   ├── App.tsx                         # Main React app entry
-│   ├── main.tsx                        # Vite entry point
-│   ├── engine/
-│   │   ├── EngineThree.ts              # Three.js engine (AU/morph control)
-│   │   ├── EngineWind.ts               # Wind physics for hair/cloth
-│   │   └── arkit/
-│   │       └── shapeDict.ts            # ARKit FACS AU → morph mappings
-│   ├── latticework/                    # Core agencies
-│   │   ├── animation/                  # Animation scheduler (XState)
-│   │   ├── lipsync/                    # Lip-sync phoneme predictor
-│   │   ├── prosodic/                   # Prosodic expression (head gestures)
-│   │   ├── eyeHeadTracking/            # Eye/head tracking service
-│   │   ├── conversation/               # Conversational AI orchestration
-│   │   └── transcription/              # Speech-to-text services
-│   ├── components/
-│   │   ├── au/                         # AU control UI components
-│   │   │   ├── AUSection.tsx           # AU sliders
-│   │   │   ├── VisemeSection.tsx       # Viseme controls
-│   │   │   ├── EyeHeadTrackingSection.tsx  # Eye/head tracking UI
-│   │   │   └── ContinuumSlider.tsx     # Bidirectional AU slider
-│   │   ├── SliderDrawer.tsx            # Main dockable UI drawer
-│   │   ├── PlaybackControls.tsx        # Animation playback controls
-│   │   ├── CurveEditor.tsx             # Visual curve editor
-│   │   └── ModulesMenu.tsx             # Module activation UI
-│   ├── modules/                        # Pluggable modules
-│   │   ├── aiChat/                     # AI chat module (Anthropic)
-│   │   ├── frenchQuiz/                 # French quiz demo module
-│   │   └── config.ts                   # Module registry
-│   ├── context/
-│   │   ├── threeContext.tsx            # Global engine/animation context
-│   │   └── ModulesContext.tsx          # Module state management
-│   ├── hooks/
-│   │   └── useWebcamEyeTracking.ts     # Webcam face tracking hook
-│   ├── scenes/
-│   │   └── CharacterGLBScene.tsx       # Three.js React scene
-│   └── utils/
-│       └── animationLoader.ts          # Load animation JSON files
-└── docs/                               # Documentation
-    ├── QUICK_START.md
-    ├── LIPSYNC_COMPLETE_GUIDE.md
-    ├── BACKEND_INTEGRATION.md
-    └── DEPLOYMENT.md
+## 3. Extending & Custom Presets
+
+### Extending an existing preset
+
+Use spread syntax to override specific mappings while keeping the rest:
+
+```typescript
+import { CC4_PRESET } from 'loomlarge';
+
+const MY_PRESET = {
+  ...CC4_PRESET,
+
+  // Override AU12 (smile) with custom morph names
+  auToMorphs: {
+    ...CC4_PRESET.auToMorphs,
+    12: ['MySmile_Left', 'MySmile_Right'],
+  },
+
+  // Add a new bone binding
+  auToBones: {
+    ...CC4_PRESET.auToBones,
+    99: [{ node: 'CUSTOM_BONE', channel: 'ry', scale: 1, maxDegrees: 45 }],
+  },
+
+  // Update bone node paths
+  boneNodes: {
+    ...CC4_PRESET.boneNodes,
+    'CUSTOM_BONE': 'MyRig_CustomBone',
+  },
+};
+
+const loom = new LoomLargeThree({ auMappings: MY_PRESET });
+```
+
+### Creating a preset from scratch
+
+```typescript
+import { AUMappingConfig } from 'loomlarge';
+
+const CUSTOM_PRESET: AUMappingConfig = {
+  auToMorphs: {
+    1: ['brow_inner_up_L', 'brow_inner_up_R'],
+    2: ['brow_outer_up_L', 'brow_outer_up_R'],
+    12: ['mouth_smile_L', 'mouth_smile_R'],
+    45: ['eye_blink_L', 'eye_blink_R'],
+  },
+
+  auToBones: {
+    51: [{ node: 'HEAD', channel: 'ry', scale: -1, maxDegrees: 30 }],
+    52: [{ node: 'HEAD', channel: 'ry', scale: 1, maxDegrees: 30 }],
+  },
+
+  boneNodes: {
+    'HEAD': 'head_bone',
+    'JAW': 'jaw_bone',
+  },
+
+  visemeKeys: ['aa', 'ee', 'ih', 'oh', 'oo'],
+
+  morphToMesh: {
+    'face': ['body_mesh'],
+  },
+};
+```
+
+### Changing presets at runtime
+
+```typescript
+// Switch to a different preset
+loom.setAUMappings(ANOTHER_PRESET);
+
+// Get current mappings
+const current = loom.getAUMappings();
 ```
 
 ---
 
-## How It Works
-
-### Animation Service
-
-The **Animation Service** (`latticework/animation/animationService.ts`) is the core scheduler:
-
-1. **Load Snippets**: JSON files defining AU/morph keyframe curves
-   ```typescript
-   const anim = createAnimationService(host);
-   anim.loadSnippet('smile', smileSnippetJSON);
-   ```
-
-2. **Schedule Playback**: Queue snippets with priority and duration
-   ```typescript
-   anim.schedule('smile', {
-     duration: 1000,
-     priority: 10,
-     loop: false
-   });
-   ```
-
-3. **XState Machine**: Manages snippet lifecycle (`idle` → `playing` → `paused`)
-   - Driven by central frame loop (`threeContext.tsx`)
-   - Handles overlapping snippets with priority-based blending
-
-4. **Host Interface**: Abstraction layer for AU/morph application
-   ```typescript
-   const host = {
-     applyAU: (id, value) => engine.setAU(id, value),
-     setMorph: (key, value) => engine.setMorph(key, value),
-     transitionAU: (id, value, duration) => engine.transitionAU(id, value, duration)
-   };
-   ```
-
-### Eye & Head Tracking
-
-The **Eye/Head Tracking Service** (`latticework/eyeHeadTracking/eyeHeadTrackingService.ts`) provides three tracking modes:
-
-1. **Manual Mode**: Direct slider control of gaze direction
-2. **Mouse Mode**: Character follows cursor with mirror behavior
-   - Mouse left → Character looks right (at user)
-   - Negative x coordinate for natural gaze
-3. **Webcam Mode**: Face tracking using BlazeFace model
-   - Real-time eye position detection
-   - Normalized coordinates (-1 to 1)
-
-**Key Features**:
-- **Composite Methods**: Uses `applyEyeComposite(yaw, pitch)` and `applyHeadComposite(yaw, pitch, roll)`
-- **Intensity Control**: Separate sliders for eye and head movement intensity
-- **Head Follow Eyes**: Optional delayed head movement matching eye gaze
-- **Global Service**: Created in App.tsx and shared via ModulesContext
-
-**Usage**:
-```typescript
-// Initialize service with engine reference
-const service = createEyeHeadTrackingService({
-  eyeTrackingEnabled: true,
-  headTrackingEnabled: true,
-  headFollowEyes: true,
-  eyeIntensity: 1.0,
-  headIntensity: 0.5,
-  engine: engine
+## 4. Action Unit Control
+
+Action Units are the core of FACS. Each AU represents a specific muscular movement of the face.
+
+### Setting an AU immediately
+
+```typescript
+// Set AU12 (smile) to 80% intensity
+loom.setAU(12, 0.8);
+
+// Set AU45 (blink) to full intensity
+loom.setAU(45, 1.0);
+
+// Set to 0 to deactivate
+loom.setAU(12, 0);
+```
+
+### Transitioning an AU over time
+
+```typescript
+// Animate AU12 to 0.8 over 200ms
+const handle = loom.transitionAU(12, 0.8, 200);
+
+// Wait for completion
+await handle.promise;
+
+// Or chain transitions
+loom.transitionAU(12, 1.0, 200).promise.then(() => {
+  loom.transitionAU(12, 0, 300);  // Fade out
 });
+```
 
-service.start();
-service.setMode('mouse'); // or 'webcam' or 'manual'
+### Getting the current AU value
 
-// Set gaze target manually
-service.setGazeTarget({ x: 0.5, y: -0.2, z: 0 });
+```typescript
+const smileAmount = loom.getAU(12);
+console.log(`Current smile: ${smileAmount}`);
 ```
 
-### Lip-Sync Agency
+### Asymmetric control with balance
 
-The **Lip-Sync Agency** (`latticework/lipsync/`) generates viseme animations from text:
+Many AUs have left and right variants (e.g., `Mouth_Smile_L` and `Mouth_Smile_R`). The `balance` parameter lets you control them independently:
 
-1. **Phoneme Prediction**: Enhanced predictor with coarticulation model
-   ```typescript
-   const predictor = new EnhancedPhonemePredictor();
-   const phonemes = predictor.predict('Hello world');
-   ```
+```typescript
+// Balance range: -1 (left only) to +1 (right only), 0 = both equal
 
-2. **Viseme Mapping**: Phonemes → ARKit visemes (AA, CH_J, DD, etc.)
-   - Timing based on phoneme duration and speech rate
-   - Coarticulation smoothing between adjacent phonemes
+// Smile on both sides equally
+loom.setAU(12, 0.8, 0);
 
-3. **Animation Snippet Generation**: Creates JSON snippets for animation service
-   ```typescript
-   const snippet = generateLipsyncSnippet(text, {
-     speechRate: 1.0,
-     intensity: 0.8,
-     style: 'relaxed' // or 'precise', 'theatrical', etc.
-   });
-   ```
+// Smile only on left side
+loom.setAU(12, 0.8, -1);
 
-4. **Integration**: Scheduled via animation service with high priority (30)
+// Smile only on right side
+loom.setAU(12, 0.8, 1);
 
-### Prosodic Expression
+// 70% left, 30% right
+loom.setAU(12, 0.8, -0.4);
+```
 
-The **Prosodic Expression Agency** (`latticework/prosodic/prosodicService.ts`) adds emotional head movements:
+### String-based side selection
 
-1. **XState Machine**: Models prosodic states (idle → analyzing → expressing)
-2. **Gesture Library**: Pre-defined head nods, tilts, and shakes
-   - Nod: Positive affirmation (head pitch down)
-   - Shake: Negation (head yaw side-to-side)
-   - Tilt: Curiosity/emphasis (head roll)
+You can also specify the side directly in the AU ID:
 
-3. **Emotion Mapping**: Text analysis triggers appropriate gestures
-   ```typescript
-   // Question → slight head tilt
-   // Exclamation → head nod emphasis
-   // Negation words → head shake
-   ```
+```typescript
+// These are equivalent:
+loom.setAU('12L', 0.8);    // Left side only
+loom.setAU(12, 0.8, -1);   // Left side only
 
-4. **Scheduling**: Prosodic snippets scheduled with medium priority (20)
+loom.setAU('12R', 0.8);    // Right side only
+loom.setAU(12, 0.8, 1);    // Right side only
+```
 
 ---
 
-## Modules
-
-LoomLarge supports pluggable modules for extended functionality:
-
-### AI Chat Module
-- **Description**: Real-time conversational AI using Anthropic Claude
-- **Location**: `src/modules/aiChat/`
-- **Features**:
-  - Streaming text-to-speech synthesis
-  - Lip-sync integration with prosodic expression
-  - Eye/head tracking during conversation
-  - WebSocket or LiveKit audio streaming
-
-**Activation**:
-```typescript
-// Via ModulesMenu UI or programmatically:
-import { AIChatApp } from './modules/aiChat';
-<AIChatApp animationManager={anim} />
-```
-
-### French Quiz Module
-- **Description**: Interactive language learning demo
-- **Location**: `src/modules/frenchQuiz/`
-- **Features**:
-  - Survey-style question flow
-  - Facial expressions tied to correct/incorrect answers
-  - Modal-based UI with progress tracking
-
-### Custom Modules
-
-Create your own modules by following this pattern:
-
-1. **Define module config** (`src/modules/config.ts`):
-   ```typescript
-   export default {
-     modules: [
-       {
-         name: 'My Module',
-         description: 'Custom module description',
-         component: './modules/myModule/index.tsx'
-       }
-     ]
-   };
-   ```
-
-2. **Create module component**:
-   ```typescript
-   // src/modules/myModule/index.tsx
-   import React from 'react';
-   import { useModulesContext } from '../../context/ModulesContext';
-
-   export default function MyModule({ animationManager }: any) {
-     const { eyeHeadTrackingService } = useModulesContext();
-
-     // Your module logic here
-     return <div>My Module UI</div>;
-   }
-   ```
+## 5. Mix Weight System
 
----
+Some AUs can be driven by both morph targets (blend shapes) AND bone rotations. The mix weight controls the blend between them.
 
-## Development
+### Why mix weights?
 
-### Running the Dev Server
+Take jaw opening (AU26) as an example:
+- **Morph-only (weight 0)**: Vertices deform to show open mouth, but jaw bone doesn't move
+- **Bone-only (weight 1)**: Jaw bone rotates down, but no soft tissue deformation
+- **Mixed (weight 0.5)**: Both contribute equally for realistic results
 
-```bash
-yarn dev
+### Setting mix weights
+
+```typescript
+// Get the default mix weight for AU26
+const weight = loom.getAUMixWeight(26);  // e.g., 0.5
+
+// Set to pure morph
+loom.setAUMixWeight(26, 0);
+
+// Set to pure bone
+loom.setAUMixWeight(26, 1);
+
+// Set to 70% bone, 30% morph
+loom.setAUMixWeight(26, 0.7);
+```
+
+### Which AUs support mixing?
+
+Only AUs that have both `auToMorphs` AND `auToBones` entries support mixing. Common examples:
+- AU26 (Jaw Drop)
+- AU27 (Mouth Stretch)
+- AU51-56 (Head movements)
+- AU61-64 (Eye movements)
+
+```typescript
+import { isMixedAU } from 'loomlarge';
+
+if (isMixedAU(26)) {
+  console.log('AU26 supports morph/bone mixing');
+}
 ```
 
-Access at `http://localhost:5173` with:
-- Hot module replacement (HMR)
-- Source maps for debugging
-- Console logging for all services
+---
 
-### Testing Animation Snippets
+## 6. Composite Rotation System
 
-Load test animations in the browser console:
+Bones like the head and eyes need multi-axis rotation (pitch, yaw, roll). The composite rotation system handles this automatically.
 
-```javascript
-// Global handles (auto-exposed in dev mode)
-window.engine   // EngineThree instance
-window.anim     // Animation service
+### How it works
 
-// Load and play a snippet
-anim.loadSnippet('test', {
-  duration: 2000,
-  keyframes: {
-    'AU_12': [[0, 0], [1000, 1], [2000, 0]], // Smile curve
-    'AU_6': [[0, 0], [1000, 0.8], [2000, 0]]  // Cheek raise
-  }
-});
-anim.schedule('test', { priority: 10 });
-anim.play();
+When you set an AU that affects a bone rotation, LoomLarge:
+1. Queues the rotation update in `pendingCompositeNodes`
+2. At the end of `update()`, calls `flushPendingComposites()`
+3. Applies all three axes (pitch, yaw, roll) together to prevent gimbal issues
+
+### Supported bones and their axes
+
+| Bone | Pitch (X) | Yaw (Y) | Roll (Z) |
+|------|-----------|---------|----------|
+| HEAD | AU53 (up) / AU54 (down) | AU51 (left) / AU52 (right) | AU55 (tilt left) / AU56 (tilt right) |
+| EYE_L | AU63 (up) / AU64 (down) | AU61 (left) / AU62 (right) | - |
+| EYE_R | AU63 (up) / AU64 (down) | AU61 (left) / AU62 (right) | - |
+| JAW | AU25-27 (open) | AU30 (left) / AU35 (right) | - |
+| TONGUE | AU37 (up) / AU38 (down) | AU39 (left) / AU40 (right) | AU41 / AU42 (tilt) |
+
+### Example: Moving the head
+
+```typescript
+// Turn head left 50%
+loom.setAU(51, 0.5);
+
+// Turn head right 50%
+loom.setAU(52, 0.5);
+
+// Tilt head up 30%
+loom.setAU(53, 0.3);
+
+// Combine: turn left AND tilt up
+loom.setAU(51, 0.5);
+loom.setAU(53, 0.3);
+// Both are applied together in a single composite rotation
 ```
 
-### Debugging Eye/Head Tracking
+### Example: Eye gaze
 
-The service includes comprehensive diagnostic logging:
+```typescript
+// Look left
+loom.setAU(61, 0.7);
 
-```javascript
-// Check current mode
-window.eyeHeadTrackingService?.getMode(); // 'manual' | 'mouse' | 'webcam'
+// Look right
+loom.setAU(62, 0.7);
 
-// Set gaze manually
-window.eyeHeadTrackingService?.setGazeTarget({ x: 0.5, y: -0.3, z: 0 });
+// Look up
+loom.setAU(63, 0.5);
 
-// Update configuration
-window.eyeHeadTrackingService?.updateConfig({
-  eyeIntensity: 1.0,
-  headIntensity: 0.7,
-  headFollowEyes: true
-});
+// Look down-right (combined)
+loom.setAU(62, 0.6);
+loom.setAU(64, 0.4);
 ```
 
-### TypeScript Type Checking
+---
 
-```bash
-yarn typecheck
+## 7. Continuum Pairs
+
+Continuum pairs are bidirectional AU pairs that represent opposite directions on the same axis. They're linked so that activating one should deactivate the other.
+
+### Pair mappings
+
+| Pair | Description |
+|------|-------------|
+| AU51 ↔ AU52 | Head turn left / right |
+| AU53 ↔ AU54 | Head up / down |
+| AU55 ↔ AU56 | Head tilt left / right |
+| AU61 ↔ AU62 | Eyes look left / right |
+| AU63 ↔ AU64 | Eyes look up / down |
+| AU30 ↔ AU35 | Jaw shift left / right |
+| AU37 ↔ AU38 | Tongue up / down |
+| AU39 ↔ AU40 | Tongue left / right |
+| AU73 ↔ AU74 | Tongue narrow / wide |
+| AU76 ↔ AU77 | Tongue tip up / down |
+
+### Working with pairs
+
+When using continuum pairs, set one AU from the pair and leave the other at 0:
+
+```typescript
+// Head looking left at 50%
+loom.setAU(51, 0.5);
+loom.setAU(52, 0);  // Right should be 0
+
+// Head looking right at 70%
+loom.setAU(51, 0);  // Left should be 0
+loom.setAU(52, 0.7);
 ```
 
-Runs `tsc --noEmit` to validate types without building.
+### The CONTINUUM_PAIRS_MAP
+
+You can access pair information programmatically:
+
+```typescript
+import { CONTINUUM_PAIRS_MAP } from 'loomlarge';
+
+const pair = CONTINUUM_PAIRS_MAP[51];
+// { pairId: 52, isNegative: true, axis: 'yaw', node: 'HEAD' }
+```
 
 ---
 
-## Deployment
+## 8. Direct Morph Control
 
-### GitHub Pages Deployment
+Sometimes you need to control morph targets directly by name, bypassing the AU system.
 
-The project is configured for automatic GitHub Pages deployment:
+### Setting a morph immediately
 
-```bash
-yarn deploy
+```typescript
+// Set a specific morph to 50%
+loom.setMorph('Mouth_Smile_L', 0.5);
+
+// Set on specific meshes only
+loom.setMorph('Mouth_Smile_L', 0.5, ['CC_Base_Body']);
 ```
 
-This script:
-1. Builds production bundle (`yarn build`)
-2. Deploys to `gh-pages` branch
-3. Publishes to `https://meekmachine.github.io/LoomLarge`
+### Transitioning a morph
 
-**Configuration** (`vite.config.ts`):
 ```typescript
-export default defineConfig({
-  base: '/LoomLarge/', // GitHub repo name
-  build: {
-    outDir: 'dist',
-    assetsDir: 'assets'
-  }
-});
-```
+// Animate morph over 200ms
+const handle = loom.transitionMorph('Mouth_Smile_L', 0.8, 200);
 
-### Custom Domain
+// With mesh targeting
+loom.transitionMorph('Eye_Blink_L', 1.0, 100, ['CC_Base_Body']);
 
-To use a custom domain:
+// Wait for completion
+await handle.promise;
+```
 
-1. Add `CNAME` file to `public/`:
-   ```
-   your-domain.com
-   ```
+### Reading current morph value
+
+```typescript
+const value = loom.getMorphValue('Mouth_Smile_L');
+```
 
-2. Configure DNS:
-   ```
-   A    185.199.108.153
-   A    185.199.109.153
-   A    185.199.110.153
-   A    185.199.111.153
-   CNAME www your-username.github.io
-   ```
+### Morph caching
 
-3. Deploy:
-   ```bash
-   yarn deploy
-   ```
+LoomLarge caches morph target lookups for performance. The first time you access a morph, it searches all meshes and caches the index. Subsequent accesses are O(1).
 
 ---
 
-## API Reference
+## 9. Viseme System
+
+Visemes are mouth shapes used for lip-sync. LoomLarge includes 15 visemes with automatic jaw coupling.
+
+### The 15 visemes
 
-### Animation Service API
+| Index | Key | Phoneme Example |
+|-------|-----|-----------------|
+| 0 | EE | "b**ee**" |
+| 1 | Er | "h**er**" |
+| 2 | IH | "s**i**t" |
+| 3 | Ah | "f**a**ther" |
+| 4 | Oh | "g**o**" |
+| 5 | W_OO | "t**oo**" |
+| 6 | S_Z | "**s**un, **z**oo" |
+| 7 | Ch_J | "**ch**ip, **j**ump" |
+| 8 | F_V | "**f**un, **v**an" |
+| 9 | TH | "**th**ink" |
+| 10 | T_L_D_N | "**t**op, **l**ip, **d**og, **n**o" |
+| 11 | B_M_P | "**b**at, **m**an, **p**op" |
+| 12 | K_G_H_NG | "**k**ite, **g**o, **h**at, si**ng**" |
+| 13 | AE | "c**a**t" |
+| 14 | R | "**r**ed" |
+
+### Setting a viseme
 
 ```typescript
-interface AnimationService {
-  loadSnippet(name: string, snippet: AnimationSnippet): void;
-  schedule(name: string, options?: ScheduleOptions): void;
-  play(): void;
-  pause(): void;
-  stop(): void;
-  setLoop(loop: boolean): void;
-  scrub(time: number): void;
-  step(deltaSeconds: number): void;
-  dispose(): void;
-}
+// Set viseme 3 (Ah) to full intensity
+loom.setViseme(3, 1.0);
+
+// With jaw scale (0-1, default 1)
+loom.setViseme(3, 1.0, 0.5);  // Half jaw opening
 ```
 
-### Eye/Head Tracking Service API
+### Transitioning visemes
 
 ```typescript
-interface EyeHeadTrackingService {
-  start(): void;
-  stop(): void;
-  setGazeTarget(target: GazeTarget): void;
-  setMode(mode: 'manual' | 'mouse' | 'webcam'): void;
-  getMode(): 'manual' | 'mouse' | 'webcam';
-  updateConfig(config: Partial<EyeHeadTrackingConfig>): void;
-  setSpeaking(isSpeaking: boolean): void;
-  setListening(isListening: boolean): void;
-  blink(): void;
-  dispose(): void;
-}
+// Animate to viseme over 80ms (typical for speech)
+const handle = loom.transitionViseme(3, 1.0, 80);
+
+// Disable jaw coupling
+loom.transitionViseme(3, 1.0, 80, 0);
 ```
 
-### EngineThree Composite Methods
+### Automatic jaw coupling
 
-```typescript
-class EngineThree {
-  // Eye composite rotation (yaw/pitch)
-  applyEyeComposite(yaw: number, pitch: number): void;
+Each viseme has a predefined jaw opening amount. When you set a viseme, the jaw automatically opens proportionally:
 
-  // Head composite rotation (yaw/pitch/roll)
-  applyHeadComposite(yaw: number, pitch: number, roll?: number): void;
+| Viseme | Jaw Amount |
+|--------|------------|
+| EE | 0.15 |
+| Ah | 0.70 |
+| Oh | 0.50 |
+| B_M_P | 0.20 |
 
-  // Get/Set AU mix weight (morph ↔ bone blend)
-  getAUMixWeight(auId: number): number | undefined;
-  setAUMixWeight(auId: number, mix: number): void;
+The `jawScale` parameter multiplies this amount:
+- `jawScale = 1.0`: Normal jaw opening
+- `jawScale = 0.5`: Half jaw opening
+- `jawScale = 0`: No jaw movement (viseme only)
 
-  // Direct AU control
-  setAU(id: number | string, value: number): void;
-  setMorph(key: string, value: number): void;
+### Lip-sync example
+
+```typescript
+async function speak(phonemes: number[]) {
+  for (const viseme of phonemes) {
+    // Clear previous viseme
+    for (let i = 0; i < 15; i++) loom.setViseme(i, 0);
 
-  // Smooth transitions
-  transitionAU(id: number | string, target: number, duration?: number): void;
-  transitionMorph(key: string, target: number, duration?: number): void;
+    // Transition to new viseme
+    await loom.transitionViseme(viseme, 1.0, 80).promise;
+
+    // Hold briefly
+    await new Promise(r => setTimeout(r, 100));
+  }
+
+  // Return to neutral
+  for (let i = 0; i < 15; i++) loom.setViseme(i, 0);
 }
+
+// "Hello" approximation
+speak([5, 0, 10, 4]);
 ```
 
 ---
 
-## Troubleshooting
+## 10. Transition System
 
-### Character Not Moving
+All animated changes in LoomLarge go through the transition system, which provides smooth interpolation with easing.
 
-1. **Check engine initialization**:
-   ```javascript
-   console.log(window.engine); // Should show EngineThree instance
-   ```
+### TransitionHandle
 
-2. **Verify service startup**:
-   ```javascript
-   console.log(window.anim?.getSnapshot?.().value); // Should show 'playing' or 'idle'
-   ```
+Every transition method returns a `TransitionHandle`:
 
-3. **Check eye/head tracking**:
-   ```javascript
-   window.eyeHeadTrackingService?.getState(); // Should show current gaze/status
-   ```
+```typescript
+interface TransitionHandle {
+  promise: Promise<void>;  // Resolves when transition completes
+  pause(): void;           // Pause this transition
+  resume(): void;          // Resume this transition
+  cancel(): void;          // Cancel immediately
+}
+```
 
-### Eye Tracking Direction Issues
+### Using handles
 
-- **Eyes looking wrong way**: Check coordinate negation in `eyeHeadTrackingService.ts:283`
-- **Head not following**: Verify `headFollowEyes` is enabled in config
-- **Intensity too low**: Increase `eyeIntensity` and `headIntensity` sliders
+```typescript
+// Start a transition
+const handle = loom.transitionAU(12, 1.0, 500);
 
-### Animation Not Loading
+// Pause it
+handle.pause();
 
-1. **Check JSON format**:
-   - Must have `duration` and `keyframes` fields
-   - AU keys must match ARKit spec (e.g., 'AU_12', not '12')
+// Resume later
+handle.resume();
 
-2. **Verify snippet name**:
-   ```javascript
-   anim.listSnippets(); // Show all loaded snippets
-   ```
+// Or cancel entirely
+handle.cancel();
 
-3. **Check console for errors**:
-   - Look for `[Animation]` prefixed logs
-   - Validate keyframe curve format: `[[time, value], ...]`
+// Wait for completion
+await handle.promise;
+```
 
-### TensorFlow.js Bundling Errors
+### Combining multiple transitions
 
-If you encounter errors related to TensorFlow.js modules (e.g., `@tensorflow/tfjs-core/dist/ops/ops_for_converter`), this is a **known issue** with TensorFlow.js 4.x and Vite.
+When you call `transitionAU`, it may create multiple internal transitions (one per morph target). The returned handle controls all of them:
 
-**The Problem:**
-- TensorFlow.js references internal module paths that don't actually exist in the npm package
-- Vite's esbuild optimizer cannot resolve these phantom paths
-- Results in 100+ bundling errors about missing exports and modules
+```typescript
+// AU12 might affect Mouth_Smile_L and Mouth_Smile_R
+const handle = loom.transitionAU(12, 1.0, 200);
 
-**Solution (Already Implemented):**
-LoomLarge loads TensorFlow.js and BlazeFace from **CDN** instead of npm packages:
+// Pausing the handle pauses both morph transitions
+handle.pause();
+```
+
+### Easing
+
+The default easing is `easeInOutQuad`. Custom easing can be provided when using the Animation system directly:
 
-```html
-<!-- index.html -->
-<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@4.22.0/dist/tf.min.js"></script>
-<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/blazeface@0.0.7/dist/blazeface.js"></script>
+```typescript
+// The AnimationThree class supports custom easing
+animation.addTransition(
+  'custom',
+  0,
+  1,
+  200,
+  (v) => console.log(v),
+  (t) => t * t  // Custom ease-in quadratic
+);
 ```
 
-The code uses global `blazeface` object with TypeScript declarations:
+### Active transition count
+
 ```typescript
-// useWebcamEyeTracking.ts
-declare const blazeface: any;
-const model = await blazeface.load();
+const count = loom.getActiveTransitionCount();
+console.log(`${count} transitions in progress`);
 ```
 
-Vite config excludes TensorFlow packages from optimization:
+### Clearing all transitions
+
 ```typescript
-// vite.config.ts
-optimizeDeps: {
-  exclude: [
-    '@tensorflow/tfjs',
-    '@tensorflow/tfjs-core',
-    '@tensorflow/tfjs-converter',
-    '@tensorflow/tfjs-backend-cpu',
-    '@tensorflow/tfjs-backend-webgl',
-    '@tensorflow-models/blazeface',
-  ],
-}
+// Cancel everything immediately
+loom.clearTransitions();
 ```
 
-**If you still see errors:**
-1. Ensure TensorFlow packages are NOT in `package.json` dependencies
-2. Clear Vite cache: `rm -rf node_modules/.vite`
-3. Restart dev server: `yarn dev`
+---
 
-See [src/hooks/README_useWebcamEyeTracking.md](src/hooks/README_useWebcamEyeTracking.md) for full documentation.
+## 11. Playback & State Control
 
-### Build Errors
+### Pausing and resuming
 
-```bash
-# Clear cache and rebuild
-rm -rf node_modules dist .vite
-yarn install
-yarn build
+```typescript
+// Pause all animation updates
+loom.pause();
+
+// Check pause state
+if (loom.getPaused()) {
+  console.log('Animation is paused');
+}
+
+// Resume
+loom.resume();
 ```
 
----
+When paused, `loom.update()` stops processing transitions, but you can still call `setAU()` for immediate changes.
 
-## Performance Optimization
+### Resetting to neutral
 
-### Animation Loop
+```typescript
+// Reset everything to rest state
+loom.resetToNeutral();
+```
 
-- Central frame loop runs at 60 FPS (`threeContext.tsx`)
-- Animation scheduler ticks every frame via `step(deltaTime)`
-- Morph/bone updates batched per frame
+This:
+- Clears all AU values to 0
+- Cancels all active transitions
+- Resets all morph targets to 0
+- Returns all bones to their original position/rotation
 
-### Composite Rotation Caching
+### Mesh visibility
 
-- Rotation state cached in `compositeRotationState` map
-- Only recalculates when values change
-- Avoids redundant Three.js object updates
+```typescript
+// Get list of all meshes
+const meshes = loom.getMeshList();
+// Returns: [{ name: 'CC_Base_Body', visible: true, morphCount: 80 }, ...]
 
-### Snippet Scheduling
+// Hide a mesh
+loom.setMeshVisible('CC_Base_Hair', false);
 
-- Priority-based scheduler prevents conflicts
-- Lower priority snippets paused when higher priority plays
-- Automatic cleanup of completed snippets
+// Show it again
+loom.setMeshVisible('CC_Base_Hair', true);
+```
 
----
+### Cleanup
 
-## Contributing
-
-We welcome contributions! Please follow these guidelines:
-
-1. **Fork and clone** the repository
-2. **Create a feature branch**: `git checkout -b feature/my-feature`
-3. **Follow code style**:
-   - TypeScript strict mode
-   - ESLint/Prettier for formatting
-   - Descriptive variable names
-4. **Test thoroughly**:
-   - Manual testing in dev mode
-   - TypeScript type checking (`yarn typecheck`)
-5. **Commit with descriptive messages**:
-   ```
-   feat: Add webcam eye tracking support
-   fix: Correct head yaw direction in mouse mode
-   docs: Update README with API reference
-   ```
-6. **Push and create PR** to `main` branch
+```typescript
+// When done, dispose of resources
+loom.dispose();
+```
 
 ---
 
-## License & Acknowledgments
+## 12. Hair Physics
 
-**© 2025 Jonathan Sutton Fields, Lovelace LOL**
-Licensed under the **Loom Large, Latticework copyleft license**
+LoomLarge includes an experimental hair physics system that simulates hair movement based on head motion.
 
-### Acknowledgments
+### Basic setup
 
-- **Three.js** – 3D rendering engine
-- **XState** – State machine library
-- **React** – UI framework
-- **Vite** – Lightning-fast bundler
-- **ARKit** – Facial Action Coding System specification
-- **BlazeFace** – Webcam face detection model
+```typescript
+import { HairPhysics } from 'loomlarge';
 
-### Related Projects
+const hair = new HairPhysics();
+```
 
-- **VISOS** – Predecessor architecture (object-oriented)
-- **eEVA Workbench** – Original survey/conversation platform
-- **Latticework** – Core agency framework
+### Updating in animation loop
 
----
+```typescript
+function animate() {
+  // Get current head state (from your tracking system or AU values)
+  const headState = {
+    yaw: 0,           // Head rotation in radians
+    pitch: 0,
+    roll: 0,
+    yawVelocity: 0.5, // Angular velocity
+    pitchVelocity: 0,
+  };
+
+  // Update hair physics
+  const hairMorphs = hair.update(deltaTime, headState);
+
+  // Apply hair morphs
+  for (const [morphName, value] of Object.entries(hairMorphs)) {
+    loom.setMorph(morphName, value);
+  }
+}
+```
+
+### Output morphs
+
+The physics system outputs 6 morph values:
 
-## Support
+| Morph | Description |
+|-------|-------------|
+| L_Hair_Left | Left side, swing left |
+| L_Hair_Right | Left side, swing right |
+| L_Hair_Front | Left side, swing forward |
+| R_Hair_Left | Right side, swing left |
+| R_Hair_Right | Right side, swing right |
+| R_Hair_Front | Right side, swing forward |
 
-- **Issues**: [GitHub Issues](https://github.com/meekmachine/LoomLarge/issues)
-- **Discussions**: [GitHub Discussions](https://github.com/meekmachine/LoomLarge/discussions)
-- **Email**: jonathan@lovelacelol.com
+### Physics forces
+
+The simulation models 5 forces:
+
+1. **Spring restoration** - Pulls hair back to rest position
+2. **Damping** - Air resistance prevents infinite oscillation
+3. **Gravity** - Hair swings based on head tilt
+4. **Inertia** - Hair lags behind head movement
+5. **Wind** - Optional oscillating wind force
+
+### Configuration
+
+```typescript
+const hair = new HairPhysics({
+  mass: 1.0,
+  stiffness: 50,
+  damping: 5,
+  gravity: 9.8,
+  headInfluence: 0.8,  // How much head movement affects hair
+  wind: {
+    strength: 0,
+    direction: { x: 1, y: 0, z: 0 },
+    turbulence: 0.2,
+    frequency: 1.0,
+  },
+});
+```
 
 ---
 
-**Built with ❤️ by the Latticework team**
+## Resources
+
+- [FACS on Wikipedia](https://en.wikipedia.org/wiki/Facial_Action_Coding_System)
+- [Paul Ekman Group - FACS](https://www.paulekman.com/facial-action-coding-system/)
+- [Character Creator 4](https://www.reallusion.com/character-creator/)
+- [Three.js Documentation](https://threejs.org/docs/)
+
+## License
+
+MIT License - see LICENSE file for details.