@viji-dev/core 0.1.0-alpha.1

package/README.md

@@ -0,0 +1,1169 @@
+# Viji Core Package (`@viji-dev/core`)
+
+**Universal execution engine for Viji Creative scenes**
+
+A powerful, secure, and feature-rich JavaScript/TypeScript library that provides the foundation for creative scene execution across all Viji platform contexts. The core offers identical IFrame + WebWorker execution with comprehensive parameter management, audio/video analysis, user interaction handling, and performance optimization.
+
+## 🚀 Features
+
+### ✅ **Core Execution Engine**
+- **Secure IFrame + WebWorker Architecture**: Complete isolation with controlled communication
+- **Multi-Instance Support**: Concurrent instances for main scenes and previews
+- **Automatic Resource Management**: Memory leak prevention and cleanup
+
+### ✅ **Parameter System**
+- **Declarative Parameter Definition**: Define parameters once with automatic UI generation
+- **Proxy-Based Access**: Fast parameter access in render loops
+- **Category-Based Organization**: Audio, video, interaction, and general parameters
+- **Real-time Validation**: Type safety and range checking
+- **Capability-Aware UI**: Parameters shown based on active features
+
+### ✅ **Audio Analysis**
+- **Real-time Audio Processing**: Volume, frequency analysis, and beat detection
+- **Custom Frequency Bands**: Bass, mid, treble, and custom band analysis
+- **Multiple Input Sources**: Microphone, audio files, and screen capture
+- **Audio-Reactive Scenes**: Make scenes respond to audio input
+
+### ✅ **Video Analysis**
+- **Real-time Video Processing**: Frame analysis in separate WebWorker
+- **Multiple Input Sources**: Camera, video files, and screen capture
+- **Video-Reactive Scenes**: Make scenes respond to video motion and brightness
+- **Frame Data Access**: Raw video frame data for custom analysis
+
+### ✅ **User Interaction**
+- **Mouse Tracking**: Position, buttons, movement, and scroll wheel
+- **Keyboard Input**: Key states, modifiers, and event handling
+- **Touch Support**: Multi-touch with gesture detection
+- **Canvas-Coordinate Mapping**: Accurate input positioning
+
+### ✅ **Performance Optimization**
+- **Configurable Frame Rates**: Full (60fps) or half (30fps) modes
+- **Resolution Scaling**: Fractional or explicit canvas dimensions
+- **Adaptive Performance**: Automatic optimization based on hardware
+- **Memory Management**: Efficient resource pooling and cleanup
+
+## 📦 Installation
+
+```bash
+npm install @viji-dev/core
+```
+
+## 🎯 Quick Start
+
+### Basic Scene Creation
+
+```typescript
+import { VijiCore } from '@viji-dev/core';
+
+// Artist scene code
+const sceneCode = `
+// Define parameters using helper functions
+const color = viji.color('#ff6b6b', {
+  label: 'Shape Color',
+  description: 'Color of the animated shape',
+  group: 'appearance'
+});
+
+const size = viji.slider(50, {
+  min: 10,
+  max: 150,
+  step: 5,
+  label: 'Shape Size',
+  description: 'Size of the animated shape',
+  group: 'appearance'
+});
+
+const speed = viji.slider(1.0, {
+  min: 0.1,
+  max: 3.0,
+  step: 0.1,
+  label: 'Animation Speed',
+  description: 'Speed of the animation',
+  group: 'animation'
+});
+
+// Main render function
+function render(viji) {
+  const ctx = viji.useContext('2d');
+  
+  // Clear canvas
+  ctx.fillStyle = '#2c3e50';
+  ctx.fillRect(0, 0, viji.width, viji.height);
+  
+  // Animated shape
+  const time = viji.time * speed.value;
+  const x = viji.width / 2 + Math.sin(time) * 100;
+  const y = viji.height / 2 + Math.cos(time) * 100;
+  
+  ctx.fillStyle = color.value;
+  ctx.beginPath();
+  ctx.arc(x, y, size.value / 2, 0, Math.PI * 2);
+  ctx.fill();
+}
+`;
+
+// Create core instance
+const core = new VijiCore({
+  hostContainer: document.getElementById('scene-container'),
+  sceneCode: sceneCode,
+  frameRateMode: 'full',
+  allowUserInteraction: true
+});
+
+// Initialize and start rendering
+await core.initialize();
+console.log('Scene is running!');
+```
+
+## 🔧 Integration API
+
+### Core Configuration
+
+The `VijiCoreConfig` interface defines all available configuration options:
+
+```typescript
+interface VijiCoreConfig {
+  // Required configuration
+  hostContainer: HTMLElement;           // Container element for the scene
+  sceneCode: string;                    // Artist JavaScript code with render function
+  
+  // Performance configuration
+  frameRateMode?: 'full' | 'half';      // 'full' = 60fps, 'half' = 30fps
+  autoOptimize?: boolean;               // Enable automatic performance optimization
+  
+  // Input streams
+  audioStream?: MediaStream;            // Audio input for analysis
+  videoStream?: MediaStream;            // Video input for analysis
+  
+  // Audio analysis configuration
+  analysisConfig?: {
+    fftSize?: number;                   // FFT size for frequency analysis (default: 2048)
+    smoothing?: number;                 // Smoothing factor 0-1 (default: 0.8)
+    frequencyBands?: FrequencyBand[];   // Custom frequency bands
+    beatDetection?: boolean;            // Enable beat detection
+    onsetDetection?: boolean;           // Enable onset detection
+  };
+
+  // Parameter system
+  parameters?: ParameterGroup[];        // Initial parameter values
+  
+  // Feature toggles
+  noInputs?: boolean;                   // Disable all input processing
+  allowUserInteraction?: boolean;       // Enable mouse/keyboard/touch events
+}
+```
+
+### Instance Management
+
+#### Creation and Initialization
+
+```typescript
+// Create core instance
+const core = new VijiCore({
+  hostContainer: document.getElementById('scene-container'),
+  sceneCode: sceneCode,
+  frameRateMode: 'full',
+  allowUserInteraction: true
+});
+
+// Initialize the core (required before use)
+await core.initialize();
+
+// Check if core is ready for operations
+  if (core.ready) {
+  console.log('Core is ready for use');
+}
+
+// Get current configuration
+const config = core.configuration;
+console.log('Current frame rate mode:', config.frameRateMode);
+```
+
+#### Performance Control
+
+```typescript
+// Frame rate control
+await core.setFullFrameRate();          // Set to 60fps mode
+await core.setHalfFrameRate();          // Set to 30fps mode
+
+// Resolution control
+await core.setResolution(0.75);         // Set to 75% of container size
+await core.setResolution(0.5);          // Set to 50% for performance
+await core.updateResolution();          // Auto-detect container size changes
+
+// Get performance statistics
+const stats = core.getStats();
+console.log('Current FPS:', stats.frameRate.effectiveRefreshRate);
+console.log('Canvas size:', stats.resolution);
+console.log('Scale factor:', stats.scale);
+console.log('Parameter count:', stats.parameterCount);
+```
+
+#### Debug and Development
+
+```typescript
+// Enable debug logging
+core.setDebugMode(true);
+
+// Check debug mode status
+const isDebugEnabled = core.getDebugMode();
+
+// Debug mode provides detailed logging for:
+// - Initialization process
+// - Communication between components
+// - Parameter system operations
+// - Audio/video stream processing
+// - Performance statistics
+```
+
+### Parameter Management
+
+The parameter system provides a powerful way to create interactive scenes with automatic UI generation.
+
+#### Parameter Definition and Access
+
+```typescript
+// Listen for parameter definitions from artist code
+core.onParametersDefined((groups) => {
+  console.log('Parameters available:', groups);
+  
+  // Each group contains:
+  // - groupName: string
+  // - category: 'audio' | 'video' | 'interaction' | 'general'
+  // - description: string
+  // - parameters: Record<string, ParameterDefinition>
+  
+  // Generate UI based on parameter groups
+  generateParameterUI(groups);
+});
+
+// Set individual parameter values
+await core.setParameter('color', '#ff0000');
+await core.setParameter('size', 75);
+await core.setParameter('enabled', true);
+
+// Set multiple parameters efficiently
+await core.setParameters({
+  'color': '#00ff00',
+  'size': 100,
+  'speed': 2.0,
+  'enabled': false
+});
+
+// Get current parameter values
+const values = core.getParameterValues();
+const color = core.getParameter('color');
+
+// Listen for parameter changes
+core.onParameterChange('size', (value) => {
+  console.log('Size parameter changed to:', value);
+});
+
+// Listen for parameter errors
+core.onParameterError((error) => {
+  console.error('Parameter error:', error.message);
+  console.error('Error code:', error.code);
+});
+```
+
+#### Capability-Aware Parameters
+
+```typescript
+// Get all parameter groups
+const allGroups = core.getParameterGroups();
+
+// Get parameter groups filtered by active capabilities
+const visibleGroups = core.getVisibleParameterGroups();
+
+// Check current capabilities
+const capabilities = core.getCapabilities();
+console.log('Audio available:', capabilities.hasAudio);
+console.log('Video available:', capabilities.hasVideo);
+console.log('Interaction enabled:', capabilities.hasInteraction);
+
+// Check if specific parameter category is active
+const isAudioActive = core.isCategoryActive('audio');
+const isVideoActive = core.isCategoryActive('video');
+
+// Parameters are automatically categorized:
+// - 'audio': Only shown when audio stream is connected
+// - 'video': Only shown when video stream is connected  
+// - 'interaction': Only shown when user interaction is enabled
+// - 'general': Always available
+```
+
+### Audio and Video Integration
+
+#### Audio Stream Management
+
+```typescript
+// Set audio stream for analysis
+const audioStream = await navigator.mediaDevices.getUserMedia({ 
+  audio: {
+    echoCancellation: false,
+    noiseSuppression: false,
+    autoGainControl: false
+  } 
+});
+await core.setAudioStream(audioStream);
+
+// Configure audio analysis
+await core.setAudioAnalysisConfig({
+  fftSize: 2048,        // Higher values = better frequency resolution
+  smoothing: 0.8        // 0 = no smoothing, 1 = maximum smoothing
+});
+
+// Get current audio stream
+const currentStream = core.getAudioStream();
+
+// Disconnect audio
+await core.setAudioStream(null);
+```
+
+#### Video Stream Management
+
+```typescript
+// Set video stream for analysis
+const videoStream = await navigator.mediaDevices.getUserMedia({ 
+  video: {
+    width: { ideal: 640 },
+    height: { ideal: 480 },
+    frameRate: { ideal: 30 }
+  } 
+});
+await core.setVideoStream(videoStream);
+
+// Video analysis includes:
+// - Real-time frame processing
+// - Frame data access for custom analysis
+// - Brightness and motion detection
+// - Custom computer vision processing
+
+// Disconnect video
+await core.setVideoStream(null);
+```
+
+#### Capability Change Monitoring
+
+```typescript
+// Listen for capability changes
+core.onCapabilitiesChange((capabilities) => {
+  console.log('Capabilities updated:', capabilities);
+  
+  // Update UI based on new capabilities
+  if (capabilities.hasAudio) {
+    showAudioControls();
+  } else {
+    hideAudioControls();
+  }
+  
+  if (capabilities.hasVideo) {
+    showVideoControls();
+  } else {
+    hideVideoControls();
+  }
+});
+```
+
+### Event Handling and Lifecycle
+
+#### Core Lifecycle Events
+
+```typescript
+// Core is ready for operations
+if (core.ready) {
+  // All systems initialized and running
+  console.log('Core is fully operational');
+}
+
+// Check if parameters are initialized
+if (core.parametersReady) {
+  // Parameter system is ready
+  console.log('Parameters are available');
+  }
+```
+
+#### Cleanup and Resource Management
+
+```typescript
+// Destroy instance and clean up all resources
+await core.destroy();
+
+// This automatically:
+// - Stops all rendering loops
+// - Disconnects audio/video streams
+// - Cleans up WebWorker and IFrame
+// - Releases all event listeners
+// - Clears parameter system
+// - Frees memory resources
+```
+
+## 🎨 Artist API
+
+The artist API provides a comprehensive set of tools for creating interactive, audio-reactive, and video-responsive scenes.
+
+### Canvas and Rendering
+
+```typescript
+function render(viji) {
+  // Get canvas contexts
+  const ctx = viji.useContext('2d');     // 2D rendering context
+  const gl = viji.useContext('webgl');   // WebGL rendering context
+  
+  // Canvas properties
+  viji.canvas;        // OffscreenCanvas object
+  viji.width;         // Canvas width in pixels
+  viji.height;        // Canvas height in pixels
+  viji.pixelRatio;    // Device pixel ratio for crisp rendering
+  
+  // Example: Draw a responsive circle
+  const centerX = viji.width / 2;
+  const centerY = viji.height / 2;
+  const radius = Math.min(viji.width, viji.height) * 0.1;
+  
+  ctx.fillStyle = '#ff6b6b';
+  ctx.beginPath();
+  ctx.arc(centerX, centerY, radius, 0, Math.PI * 2);
+  ctx.fill();
+}
+```
+
+### Timing Information
+
+The timing system provides FPS-independent timing data for smooth animations:
+
+```typescript
+function render(viji) {
+  // Timing data (FPS independent)
+  viji.time;          // Elapsed time in seconds since scene start
+  viji.deltaTime;     // Time since last frame in seconds
+  viji.frameCount;    // Total number of frames rendered
+  viji.fps;           // Current frames per second
+  
+  // Example: Smooth animation regardless of frame rate
+  const animationSpeed = 2.0; // rotations per second
+  const rotation = (viji.time * animationSpeed * Math.PI * 2) % (Math.PI * 2);
+  
+  ctx.save();
+  ctx.translate(viji.width / 2, viji.height / 2);
+  ctx.rotate(rotation);
+  ctx.fillRect(-25, -25, 50, 50);
+  ctx.restore();
+}
+```
+
+### Parameter System
+
+The parameter system allows artists to define interactive parameters that automatically generate UI controls.
+
+#### Parameter Definition
+
+```typescript
+// Define parameters (call once outside render loop)
+const color = viji.color('#ff6b6b', {
+  label: 'Primary Color',
+  description: 'Main color for shapes',
+  group: 'appearance',
+  category: 'general'
+});
+
+const size = viji.slider(50, {
+  min: 10,
+  max: 150,
+  step: 5,
+  label: 'Shape Size',
+  description: 'Size of shapes in pixels',
+  group: 'appearance',
+  category: 'general'
+});
+
+const speed = viji.slider(1.0, {
+  min: 0.1,
+  max: 3.0,
+  step: 0.1,
+  label: 'Animation Speed',
+  description: 'Speed of animation in rotations per second',
+  group: 'animation',
+  category: 'general'
+});
+
+const useAudio = viji.toggle(false, {
+  label: 'Audio Reactive',
+  description: 'Make shapes react to audio input',
+  group: 'audio',
+  category: 'audio'
+});
+
+const shapeType = viji.select('circle', {
+  options: ['circle', 'square', 'triangle', 'star'],
+  label: 'Shape Type',
+  description: 'Type of shape to draw',
+  group: 'appearance',
+  category: 'general'
+});
+
+const title = viji.text('My Scene', {
+  label: 'Scene Title',
+  description: 'Title displayed in the scene',
+  group: 'text',
+  category: 'general',
+  maxLength: 50
+});
+
+const particleCount = viji.number(5, {
+  min: 1,
+  max: 20,
+  step: 1,
+  label: 'Particle Count',
+  description: 'Number of particles to render',
+  group: 'animation',
+  category: 'general'
+});
+```
+
+#### Parameter Usage in Render Loop
+
+```typescript
+function render(viji) {
+  const ctx = viji.useContext('2d');
+  
+  // Fast parameter access (proxy-based)
+  ctx.fillStyle = color.value;           // Get current color value
+  const radius = size.value / 2;         // Get current size value
+  const animationSpeed = speed.value;    // Get current speed value
+  
+  // Clear canvas
+  ctx.fillStyle = '#2c3e50';
+  ctx.fillRect(0, 0, viji.width, viji.height);
+  
+  // Draw title
+  ctx.fillStyle = 'white';
+  ctx.font = '20px Arial';
+  ctx.textAlign = 'center';
+  ctx.fillText(title.value, viji.width / 2, 30);
+  
+  // Draw particles
+  for (let i = 0; i < particleCount.value; i++) {
+    const angle = (i / particleCount.value) * Math.PI * 2 + (viji.time * animationSpeed);
+    const x = viji.width / 2 + Math.cos(angle) * 100;
+    const y = viji.height / 2 + Math.sin(angle) * 100;
+    
+    ctx.fillStyle = color.value;
+    ctx.beginPath();
+    
+    switch (shapeType.value) {
+      case 'circle':
+        ctx.arc(x, y, radius, 0, Math.PI * 2);
+        break;
+      case 'square':
+        ctx.rect(x - radius, y - radius, radius * 2, radius * 2);
+        break;
+      case 'triangle':
+        ctx.moveTo(x, y - radius);
+        ctx.lineTo(x - radius, y + radius);
+        ctx.lineTo(x + radius, y + radius);
+        ctx.closePath();
+        break;
+    }
+    
+    ctx.fill();
+  }
+}
+```
+
+### Audio Analysis
+
+The audio system provides real-time analysis of audio input with comprehensive frequency and volume data.
+
+#### Audio API Overview
+
+```typescript
+function render(viji) {
+  const audio = viji.audio;
+  
+  if (audio.isConnected) {
+    // Volume analysis
+    const volume = audio.volume.rms;     // 0-1 RMS volume (true volume)
+    const peak = audio.volume.peak;      // 0-1 peak volume (maximum amplitude)
+    
+    // Frequency bands (0-1 values)
+    const bass = audio.bands.bass;       // 60-250 Hz
+    const mid = audio.bands.mid;         // 500-2000 Hz
+    const treble = audio.bands.treble;   // 2000-20000 Hz
+    
+    // Extended frequency bands
+    const subBass = audio.bands.subBass;     // 20-60 Hz
+    const lowMid = audio.bands.lowMid;       // 250-500 Hz
+    const highMid = audio.bands.highMid;     // 2000-4000 Hz
+    const presence = audio.bands.presence;   // 4000-6000 Hz
+    const brilliance = audio.bands.brilliance; // 6000-20000 Hz
+    
+    // Beat detection
+    if (audio.beat?.isKick) {
+      // Kick drum detected
+      console.log('Kick detected!');
+    }
+    
+    // Raw frequency data (0-255 values)
+    const frequencyData = audio.getFrequencyData();
+    
+    // Example: Audio-reactive animation
+    const scale = 1 + (volume * 2); // Scale based on volume
+    const hue = (bass * 360); // Color based on bass
+    
+    ctx.save();
+    ctx.translate(viji.width / 2, viji.height / 2);
+    ctx.scale(scale, scale);
+    ctx.fillStyle = `hsl(${hue}, 70%, 60%)`;
+    ctx.fillRect(-25, -25, 50, 50);
+    ctx.restore();
+  }
+}
+```
+
+#### Audio-Reactive Scene Example
+
+```typescript
+// Define audio-reactive parameters
+const audioReactive = viji.toggle(true, {
+  label: 'Audio Reactive',
+  description: 'Make shapes react to audio',
+  group: 'audio',
+  category: 'audio'
+});
+
+const volumeSensitivity = viji.slider(1.0, {
+  min: 0.1,
+  max: 5.0,
+  step: 0.1,
+  label: 'Volume Sensitivity',
+  description: 'How sensitive shapes are to volume',
+  group: 'audio',
+  category: 'audio'
+});
+
+const bassReactivity = viji.slider(1.0, {
+  min: 0,
+  max: 3.0,
+  step: 0.1,
+  label: 'Bass Reactivity',
+  description: 'How much shapes react to bass',
+  group: 'audio',
+  category: 'audio'
+});
+
+function render(viji) {
+  const ctx = viji.useContext('2d');
+  const audio = viji.audio;
+  
+  // Clear canvas
+  ctx.fillStyle = '#2c3e50';
+  ctx.fillRect(0, 0, viji.width, viji.height);
+  
+  if (audioReactive.value && audio.isConnected) {
+    // Audio-reactive animation
+    const volume = audio.volume.rms * volumeSensitivity.value;
+    const bass = audio.bands.bass * bassReactivity.value;
+    
+    // Scale based on volume
+    const scale = 1 + volume;
+    
+    // Color based on bass
+    const hue = 200 + (bass * 160); // Blue to purple range
+    
+    // Position based on frequency distribution
+    const x = viji.width * (audio.bands.mid + audio.bands.treble) / 2;
+    const y = viji.height * (1 - audio.bands.bass);
+    
+    ctx.save();
+    ctx.translate(x, y);
+    ctx.scale(scale, scale);
+    ctx.fillStyle = `hsl(${hue}, 80%, 60%)`;
+    ctx.beginPath();
+    ctx.arc(0, 0, 30, 0, Math.PI * 2);
+    ctx.fill();
+    ctx.restore();
+  }
+}
+```
+
+### Video Analysis
+
+The video system provides real-time video frame analysis with frame data access for custom processing.
+
+#### Video API Overview
+
+```typescript
+function render(viji) {
+  const video = viji.video;
+  
+  if (video.isConnected) {
+    // Video properties
+    const frameWidth = video.frameWidth;
+    const frameHeight = video.frameHeight;
+    const frameRate = video.frameRate;
+    
+    // Current video frame (OffscreenCanvas)
+    if (video.currentFrame) {
+      // Draw video frame as background
+      ctx.globalAlpha = 0.3;
+      ctx.drawImage(video.currentFrame, 0, 0, viji.width, viji.height);
+      ctx.globalAlpha = 1.0;
+    }
+    
+    // Frame data for custom analysis
+    const frameData = video.getFrameData();
+    
+    // Example: Custom video analysis
+    if (frameData) {
+      // Access raw pixel data for custom processing
+      const imageData = frameData.data;
+      const width = frameData.width;
+      const height = frameData.height;
+      
+      // Example: Calculate average brightness
+      let totalBrightness = 0;
+      for (let i = 0; i < imageData.length; i += 4) {
+        const r = imageData[i];
+        const g = imageData[i + 1];
+        const b = imageData[i + 2];
+        totalBrightness += (r + g + b) / 3;
+      }
+      const averageBrightness = totalBrightness / (imageData.length / 4);
+      
+      // Use brightness for effects
+      const brightness = averageBrightness / 255; // Normalize to 0-1
+      
+      // Create brightness-reactive animation
+      ctx.fillStyle = `rgba(255, 255, 255, ${brightness * 0.5})`;
+      ctx.fillRect(0, 0, viji.width, viji.height);
+    }
+  }
+}
+```
+
+#### Video-Reactive Scene Example
+
+```typescript
+// Define video-reactive parameters
+const videoReactive = viji.toggle(true, {
+  label: 'Video Reactive',
+  description: 'Make shapes react to video',
+  group: 'video',
+  category: 'video'
+});
+
+const motionSensitivity = viji.slider(1.0, {
+  min: 0.1,
+  max: 3.0,
+  step: 0.1,
+  label: 'Motion Sensitivity',
+  description: 'How sensitive shapes are to video changes',
+  group: 'video',
+  category: 'video'
+});
+
+function render(viji) {
+  const ctx = viji.useContext('2d');
+  const video = viji.video;
+  
+  if (videoReactive.value && video.isConnected) {
+    // Video-reactive animation using frame data
+    const frameData = video.getFrameData();
+    
+    if (frameData) {
+      // Simple motion detection (compare with previous frame)
+      // This is a basic example - you can implement more sophisticated analysis
+      const imageData = frameData.data;
+      let motionEnergy = 0;
+      
+      // Calculate motion energy (simplified)
+      for (let i = 0; i < imageData.length; i += 4) {
+        const brightness = (imageData[i] + imageData[i + 1] + imageData[i + 2]) / 3;
+        motionEnergy += brightness;
+      }
+      
+      const normalizedMotion = (motionEnergy / (imageData.length / 4)) / 255;
+      const scale = 1 + (normalizedMotion * motionSensitivity.value);
+      
+      // Create motion-reactive shapes
+      ctx.save();
+      ctx.translate(viji.width / 2, viji.height / 2);
+      ctx.scale(scale, scale);
+      ctx.fillStyle = `hsl(${normalizedMotion * 360}, 70%, 60%)`;
+      ctx.beginPath();
+      ctx.arc(0, 0, 30, 0, Math.PI * 2);
+      ctx.fill();
+      ctx.restore();
+    }
+  }
+}
+```
+
+### User Interaction
+
+The interaction system provides comprehensive support for mouse, keyboard, and touch input.
+
+#### Mouse Interaction
+
+```typescript
+function render(viji) {
+  const mouse = viji.mouse;
+  
+  // Mouse position (canvas coordinates)
+  if (mouse.isInCanvas) {
+    const x = mouse.x;                   // Current X coordinate
+    const y = mouse.y;                   // Current Y coordinate
+    
+    // Mouse movement
+    const deltaX = mouse.deltaX;         // X movement since last frame
+    const deltaY = mouse.deltaY;         // Y movement since last frame
+    const velocity = mouse.velocity;     // Smoothed velocity { x, y }
+    
+    // Mouse buttons
+    const isPressed = mouse.isPressed;   // Any button currently pressed
+    const leftButton = mouse.leftButton; // Left button state
+    const rightButton = mouse.rightButton; // Right button state
+    const middleButton = mouse.middleButton; // Middle button state
+    
+    // Frame-based events
+    const wasPressed = mouse.wasPressed; // Button was pressed this frame
+    const wasReleased = mouse.wasReleased; // Button was released this frame
+    const wasMoved = mouse.wasMoved;     // Mouse moved this frame
+    
+    // Scroll wheel
+    const wheelDelta = mouse.wheelDelta; // Combined wheel delta
+    const wheelX = mouse.wheelX;         // Horizontal wheel delta
+    const wheelY = mouse.wheelY;         // Vertical wheel delta
+    
+    // Example: Mouse-reactive animation
+    ctx.fillStyle = leftButton ? 'red' : 'blue';
+    ctx.beginPath();
+    ctx.arc(x, y, 20 + Math.abs(velocity.x + velocity.y), 0, Math.PI * 2);
+    ctx.fill();
+  }
+}
+```
+
+#### Keyboard Interaction
+
+```typescript
+function render(viji) {
+  const keyboard = viji.keyboard;
+  
+  // Key state queries
+  if (keyboard.isPressed('w')) {
+    // W key is currently pressed
+    console.log('W key is held down');
+  }
+  
+  if (keyboard.wasPressed('space')) {
+    // Space was pressed this frame
+    console.log('Space was pressed!');
+  }
+  
+  if (keyboard.wasReleased('escape')) {
+    // Escape was released this frame
+    console.log('Escape was released!');
+  }
+  
+  // Active key tracking
+  const activeKeys = keyboard.activeKeys;        // Set of currently pressed keys
+  const pressedThisFrame = keyboard.pressedThisFrame; // Set of keys pressed this frame
+  const releasedThisFrame = keyboard.releasedThisFrame; // Set of keys released this frame
+  
+  // Modifier keys
+  const shift = keyboard.shift;          // Shift key is held
+  const ctrl = keyboard.ctrl;            // Ctrl key is held
+  const alt = keyboard.alt;              // Alt key is held
+  const meta = keyboard.meta;            // Meta/Cmd key is held
+  
+  // Recent activity
+  const lastKeyPressed = keyboard.lastKeyPressed;   // Last key that was pressed
+  const lastKeyReleased = keyboard.lastKeyReleased; // Last key that was released
+  
+  // Example: Keyboard-controlled movement
+  let moveX = 0;
+  let moveY = 0;
+  
+  if (keyboard.isPressed('w') || keyboard.isPressed('W')) moveY -= 5;
+  if (keyboard.isPressed('s') || keyboard.isPressed('S')) moveY += 5;
+  if (keyboard.isPressed('a') || keyboard.isPressed('A')) moveX -= 5;
+  if (keyboard.isPressed('d') || keyboard.isPressed('D')) moveX += 5;
+  
+  // Apply movement
+  ctx.save();
+  ctx.translate(moveX, moveY);
+  ctx.fillStyle = 'green';
+  ctx.fillRect(0, 0, 50, 50);
+  ctx.restore();
+}
+```
+
+#### Touch Interaction
+
+```typescript
+function render(viji) {
+  const touches = viji.touches;
+  
+  // Touch points
+  for (const touch of touches.points) {
+    const x = touch.x;                   // Touch X coordinate
+    const y = touch.y;                   // Touch Y coordinate
+    const pressure = touch.pressure;     // Pressure (0-1)
+    const radius = touch.radius;         // Touch radius
+    const id = touch.id;                 // Unique touch ID
+    
+    // Movement
+    const deltaX = touch.deltaX;         // X movement since last frame
+    const deltaY = touch.deltaY;         // Y movement since last frame
+    const velocity = touch.velocity;     // Movement velocity { x, y }
+    
+    // Lifecycle
+    const isNew = touch.isNew;           // Touch started this frame
+    const isActive = touch.isActive;     // Touch is currently active
+    const isEnding = touch.isEnding;     // Touch ending this frame
+    
+    // Draw touch point
+    ctx.fillStyle = isNew ? 'red' : isEnding ? 'yellow' : 'blue';
+    ctx.beginPath();
+    ctx.arc(x, y, radius * 2, 0, Math.PI * 2);
+    ctx.fill();
+  }
+  
+  // Touch events
+  const started = touches.started;       // Touches that started this frame
+  const moved = touches.moved;           // Touches that moved this frame
+  const ended = touches.ended;           // Touches that ended this frame
+  
+  // Primary touch (first touch point)
+  const primary = touches.primary;       // Primary touch point or null
+  
+  // Touch gestures
+  const gestures = touches.gestures;
+  
+  if (gestures.isPinching) {
+    const scale = gestures.pinchScale;       // Current pinch scale
+    const delta = gestures.pinchDelta;       // Scale change since last frame
+    
+    // React to pinch gesture
+    ctx.save();
+    ctx.scale(scale, scale);
+    ctx.fillStyle = 'purple';
+    ctx.fillRect(0, 0, 100, 100);
+    ctx.restore();
+  }
+  
+  if (gestures.isRotating) {
+    const angle = gestures.rotationAngle;    // Current rotation angle
+    const delta = gestures.rotationDelta;    // Rotation change since last frame
+    
+    // React to rotation gesture
+    ctx.save();
+    ctx.rotate(angle);
+    ctx.fillStyle = 'orange';
+    ctx.fillRect(-25, -25, 50, 50);
+    ctx.restore();
+  }
+  
+  if (gestures.isPanning) {
+    const panDelta = gestures.panDelta;      // Pan movement { x, y }
+    
+    // React to pan gesture
+    ctx.save();
+    ctx.translate(panDelta.x, panDelta.y);
+    ctx.fillStyle = 'cyan';
+    ctx.fillRect(0, 0, 50, 50);
+    ctx.restore();
+  }
+  
+  if (gestures.isTapping) {
+    const tapCount = gestures.tapCount;      // Number of taps
+    const tapPosition = gestures.tapPosition; // { x, y } tap position
+    
+    // React to tap gesture
+    if (tapPosition) {
+      ctx.fillStyle = 'lime';
+      ctx.beginPath();
+      ctx.arc(tapPosition.x, tapPosition.y, 30, 0, Math.PI * 2);
+      ctx.fill();
+    }
+  }
+}
+```
+
+## 🏗️ Architecture
+
+### Security Model
+
+The core implements a comprehensive security model to ensure safe execution of artist code:
+
+- **IFrame Isolation**: Complete separation from host environment with sandboxed execution
+- **WebWorker Sandboxing**: Artist code runs with controlled API access only
+- **Blob URL Creation**: Secure worker and IFrame creation from blob URLs
+- **Resource Protection**: Memory leaks and errors cannot affect main application
+- **Controlled Communication**: Optimized message passing with validation
+
+### Performance Features
+
+The core provides extensive performance optimization capabilities:
+
+- **Configurable Frame Rates**: Full (60fps) or half (30fps) modes for performance tuning
+- **Resolution Scaling**: Fractional (0.1-1.0) or explicit canvas dimensions
+- **Adaptive Optimization**: Automatic performance tuning based on hardware capabilities
+- **Efficient Communication**: Optimized message passing between components
+- **Memory Management**: Automatic resource cleanup and memory leak prevention
+
+### Multi-Instance Support
+
+The core supports multiple concurrent instances for complex applications:
+
+```typescript
+// Main scene with full features
+const mainCore = new VijiCore({
+  hostContainer: document.getElementById('main-scene'),
+  resolution: { width: 1920, height: 1080 },
+  frameRateMode: 'full',
+  allowUserInteraction: true,
+  audioStream: sharedAudioStream,
+  videoStream: sharedVideoStream
+});
+
+// Preview instance with reduced features
+const previewCore = new VijiCore({
+  hostContainer: document.getElementById('preview'),
+  resolution: 0.25, // 25% resolution for performance
+  frameRateMode: 'half',
+  noInputs: true,
+  allowUserInteraction: false,
+  audioStream: sharedAudioStream // Shared efficiently across instances
+});
+
+// Thumbnail instance for gallery view
+const thumbnailCore = new VijiCore({
+  hostContainer: document.getElementById('thumbnail'),
+  resolution: 0.1, // 10% resolution
+  frameRateMode: 'half',
+  noInputs: true,
+  allowUserInteraction: false
+});
+
+// To change scenes, create a new instance and destroy the old one
+const newCore = new VijiCore({
+  hostContainer: document.getElementById('scene-host'),
+  sceneCode: newSceneCode,
+  audioStream: sharedAudioStream,
+  videoStream: sharedVideoStream
+});
+
+// Automatic comprehensive cleanup when destroyed
+await oldCore.destroy();
+```
+
+## 🔍 Error Handling
+
+The core provides comprehensive error handling with detailed error information:
+
+```typescript
+import { VijiCoreError } from '@viji-dev/core';
+
+try {
+  const core = new VijiCore(config);
+  await core.initialize();
+} catch (error) {
+  if (error instanceof VijiCoreError) {
+    console.error(`Core error [${error.code}]:`, error.message);
+    console.error('Error context:', error.context);
+    
+    // Handle specific error types
+    switch (error.code) {
+      case 'INVALID_CONFIG':
+        console.error('Configuration is invalid:', error.context);
+        break;
+      case 'INITIALIZATION_ERROR':
+        console.error('Failed to initialize core:', error.context);
+        break;
+      case 'CORE_NOT_READY':
+        console.error('Core is not ready for operations');
+        break;
+      case 'INSTANCE_DESTROYED':
+        console.error('Core instance has been destroyed');
+        break;
+      case 'PARAMETERS_NOT_INITIALIZED':
+        console.error('Parameter system not yet initialized');
+        break;
+      case 'UNKNOWN_PARAMETER':
+        console.error('Parameter not found:', error.context);
+        break;
+    }
+  } else {
+    console.error('Unexpected error:', error);
+  }
+}
+```
+
+**Common Error Codes:**
+- `INVALID_CONFIG` - Configuration validation failed
+- `INITIALIZATION_ERROR` - Failed to initialize core components
+- `CORE_NOT_READY` - Operation attempted before ready
+- `INSTANCE_DESTROYED` - Operation attempted after destroy
+- `PARAMETERS_NOT_INITIALIZED` - Parameters not yet available
+- `UNKNOWN_PARAMETER` - Parameter not found
+- `CONCURRENT_INITIALIZATION` - Multiple initialization attempts
+- `MANAGER_NOT_READY` - Internal component not available
+
+## 🧪 Development
+
+```bash
+# Install dependencies
+npm install
+
+# Build the package
+npm run build
+
+# Run tests
+npm test
+
+# Development build (watch mode)
+npm run dev
+
+# Type checking
+npm run type-check
+
+# Linting
+npm run lint
+```
+
+## 📚 Examples
+
+The package includes comprehensive examples in the `/example` directory:
+
+- **Basic Scene Creation**: Simple animated shapes with parameters
+- **Audio-Reactive Scenes**: Scenes that respond to audio input
+- **Video-Reactive Scenes**: Scenes that respond to video analysis
+- **Interactive Scenes**: Mouse, keyboard, and touch interaction
+- **Parameter System**: Complete parameter definition and UI generation
+- **Multi-Instance**: Multiple concurrent scene instances
+
+## 🎯 Use Cases
+
+### Platform Integration
+The core integrates seamlessly with the Viji platform, providing scene execution while the platform handles UI, user management, and social features.
+
+### SDK Development
+The core serves as the execution foundation for the Viji SDK, ensuring identical behavior between development and platform environments.
+
+### Standalone Applications
+Use the core directly in custom applications for creative scene rendering with full feature support.
+
+## 📄 License
+
+MIT License - see LICENSE file for details.
+
+## 🤝 Contributing
+
+The Viji Core package is part of the larger Viji Creative SDK ecosystem. For contribution guidelines and development setup, please refer to the main Viji project documentation.
+
+---
+
+**Viji Core** - Universal execution engine for creative scenes across all contexts.