@djangocfg/ui-nextjs 2.1.81 → 2.1.83

package/src/tools/AudioPlayer/@refactoring3/03-HYBRID-ARCHITECTURE-PROPOSAL.md

@@ -0,0 +1,769 @@
+# Hybrid Audio Player Architecture Proposal
+
+**Date:** 2025-12-30
+**Status:** Proposal
+**Goal:** Combine native HTML5 audio playback with Web Audio API visualization and reactive effects
+
+---
+
+## Executive Summary
+
+This proposal outlines a hybrid architecture that:
+1. Uses **native HTML5 `<audio>` element** for playback (eliminates crackling issues)
+2. Connects **Web Audio API AnalyserNode** to the audio element for real-time visualization
+3. Preserves all **reactive cover effects** (glow, orbs, spotlight, mesh)
+4. Supports **chunked/streaming audio** with proper Range request handling
+
+---
+
+## Problem Statement
+
+### Current Issues
+
+1. **WaveSurfer.js limitations with streaming:**
+   - Requires full file download before seek works properly
+   - Seek beyond buffered content (>2 minutes for 2MB chunks) breaks playback
+   - After failed seek, player state becomes corrupted
+
+2. **Audio crackling in pure Web Audio approach:**
+   - Direct AudioBufferSourceNode playback can cause crackling
+   - Manual buffer scheduling is complex and error-prone
+   - GC pauses during buffer transitions cause audible artifacts
+
+3. **Architecture complexity:**
+   - Current code has two separate players (WaveSurfer-based and Progressive)
+   - Duplication of state management and controls
+   - Reactive effects tightly coupled to WaveSurfer implementation
+
+---
+
+## Architecture Overview
+
+```
+                              HYBRID AUDIO PLAYER ARCHITECTURE
++===============================================================================+
+|                                                                               |
+|  +-------------------------------------------------------------------------+  |
+|  |                          HybridAudioProvider                            |  |
+|  |  (React Context - manages all state and coordinates components)         |  |
+|  +-------------------------------------------------------------------------+  |
+|                                      |                                        |
+|          +---------------------------+---------------------------+            |
+|          |                           |                           |            |
+|          v                           v                           v            |
+|  +---------------+          +----------------+          +------------------+  |
+|  |    PLAYBACK   |          |  VISUALIZATION |          |  REACTIVE FX     |  |
+|  |    LAYER      |          |     LAYER      |          |     LAYER        |  |
+|  +---------------+          +----------------+          +------------------+  |
+|  |               |          |                |          |                  |  |
+|  | HTML5 <audio> |   --->   | MediaElement   |   --->   | useAudioAnalysis |  |
+|  | (native)      |          | SourceNode     |          | (bass/mid/high)  |  |
+|  |               |          |       |        |          |        |         |  |
+|  | - Range req   |          |       v        |          |        v         |  |
+|  | - Buffering   |          | AnalyserNode   |          | AudioReactive    |  |
+|  | - Seek        |          |       |        |          | Cover            |  |
+|  | - No crackling|          |       v        |          | - GlowEffect     |  |
+|  +-------+-------+          | Destination    |          | - OrbsEffect     |  |
+|          |                  +-------+--------+          | - SpotlightEffect|  |
+|          |                          |                   | - MeshEffect     |  |
+|          v                          v                   +------------------+  |
+|  +-------+-------+          +-------+--------+                                |
+|  | useAudioState |          | useSharedWeb   |                                |
+|  | - currentTime |          | Audio          |                                |
+|  | - duration    |          | (singleton     |                                |
+|  | - buffered    |          |  context)      |                                |
+|  | - isPlaying   |          +----------------+                                |
+|  +---------------+                                                            |
+|                                                                               |
++===============================================================================+
+
+                            DATA FLOW DIAGRAM
+
+  +-----------+     +-----------+     +--------------+     +---------------+
+  |   Audio   |     |  Browser  |     |   Audio      |     | React State   |
+  |   Source  | --> |  Handles  | --> |   Element    | --> | Updates       |
+  |   (URL)   |     |  Range    |     |   Events     |     | (60fps)       |
+  +-----------+     |  Requests |     +--------------+     +---------------+
+                    +-----------+              |
+                                               v
+                    +--------------------------------------------------+
+                    |              Web Audio Graph                      |
+                    |                                                   |
+                    |  [MediaElementSource] --> [AnalyserNode] --> [Destination]
+                    |                                  |                |
+                    |                                  v                |
+                    |                        [getByteFrequencyData]     |
+                    |                                  |                |
+                    |                                  v                |
+                    |                        [AudioLevels State]        |
+                    |                        { bass, mid, high, overall }
+                    +--------------------------------------------------+
+```
+
+---
+
+## Key Components
+
+### 1. HybridAudioProvider (Context)
+
+**File:** `context/HybridAudioProvider.tsx`
+
+**Responsibilities:**
+- Manage audio element lifecycle
+- Create and maintain Web Audio graph
+- Provide unified state to all consumers
+- Handle playback controls
+- Coordinate visualization and effects
+
+```typescript
+interface HybridAudioContextState {
+  // Core instances
+  audioElement: HTMLAudioElement | null;
+  audioContext: AudioContext | null;
+  sourceNode: MediaElementAudioSourceNode | null;
+  analyserNode: AnalyserNode | null;
+
+  // Playback state
+  isReady: boolean;
+  isPlaying: boolean;
+  currentTime: number;
+  duration: number;
+  volume: number;
+  isMuted: boolean;
+  isLooping: boolean;
+  buffered: TimeRanges | null;
+
+  // Audio analysis (for reactive effects)
+  audioLevels: AudioLevels;
+
+  // Actions
+  play: () => Promise<void>;
+  pause: () => void;
+  togglePlay: () => void;
+  seek: (time: number) => void;
+  seekTo: (progress: number) => void;
+  skip: (seconds: number) => void;
+  setVolume: (vol: number) => void;
+  toggleMute: () => void;
+  toggleLoop: () => void;
+}
+```
+
+### 2. useHybridAudio (Core Hook)
+
+**File:** `hooks/useHybridAudio.ts`
+
+**Responsibilities:**
+- Create HTML5 audio element
+- Connect to Web Audio API for analysis only (not playback)
+- Extract frequency data for visualizations
+- Handle streaming/chunked audio sources
+
+```typescript
+interface UseHybridAudioOptions {
+  src: string;
+  autoPlay?: boolean;
+  initialVolume?: number;
+  loop?: boolean;
+  crossOrigin?: 'anonymous' | 'use-credentials';
+
+  // Callbacks
+  onPlay?: () => void;
+  onPause?: () => void;
+  onEnded?: () => void;
+  onTimeUpdate?: (time: number) => void;
+  onError?: (error: Error) => void;
+}
+
+interface UseHybridAudioReturn {
+  // Refs
+  audioRef: RefObject<HTMLAudioElement>;
+
+  // State
+  state: AudioState;
+
+  // Controls
+  controls: AudioControls;
+
+  // Web Audio for visualization
+  webAudio: {
+    context: AudioContext | null;
+    analyser: AnalyserNode | null;
+    createAnalyser: (options?: AnalyserOptions) => AnalyserNode | null;
+    disconnectAnalyser: (analyser: AnalyserNode) => void;
+  };
+}
+```
+
+### 3. useAudioAnalysis (Visualization Hook)
+
+**File:** `hooks/useAudioAnalysis.ts` (existing, minor updates)
+
+**Responsibilities:**
+- Read frequency data from AnalyserNode
+- Calculate bass, mid, high, and overall levels
+- Provide smoothed values for animations
+- Run at 60fps during playback
+
+```typescript
+// Input: analyser node from useHybridAudio
+// Output: AudioLevels { bass, mid, high, overall }
+
+// Frequency bands:
+// - Bass: 0-15% of spectrum (sub-bass to low-mids)
+// - Mids: 15-50% of spectrum (vocals, instruments)
+// - Highs: 50-100% of spectrum (cymbals, harmonics)
+```
+
+### 4. WaveformVisualization (Component)
+
+**File:** `components/WaveformVisualization.tsx`
+
+**Responsibilities:**
+- Real-time waveform rendering using AnalyserNode
+- Progress bar with buffered regions
+- Click-to-seek functionality
+- Responsive canvas rendering
+
+**Two Modes:**
+1. **Frequency Bars Mode:** Real-time frequency visualization (like current equalizer)
+2. **Waveform Mode:** Pre-computed peaks (using Progressive player's approach)
+
+```typescript
+interface WaveformVisualizationProps {
+  mode: 'frequency' | 'waveform';
+
+  // For waveform mode (pre-computed)
+  peaks?: number[];
+  loadedPercent?: number;
+
+  // Shared props
+  currentTime: number;
+  duration: number;
+  buffered: TimeRanges | null;
+  onSeek: (time: number) => void;
+
+  // Styling
+  style?: WaveformStyle;
+  className?: string;
+}
+```
+
+### 5. AudioReactiveCover (Existing Component)
+
+**File:** `components/ReactiveCover/AudioReactiveCover.tsx` (existing)
+
+**No changes needed** - already consumes `audioLevels` from context.
+
+Effects preserved:
+- **GlowEffect:** Multi-layer glow responding to bass/mid/high
+- **OrbsEffect:** Floating orbs with position/size based on frequencies
+- **SpotlightEffect:** Rotating spotlight with color intensity
+- **MeshEffect:** Morphing mesh gradients
+
+---
+
+## Web Audio Connection Architecture
+
+### The Key Insight
+
+HTML5 `<audio>` handles playback natively while Web Audio API provides analysis capabilities:
+
+```
++-------------------+
+|   HTML5 <audio>   |  <-- Browser handles all playback
+|   - src loading   |      - Range requests
+|   - buffering     |      - Decoding
+|   - playback      |      - No crackling
++--------+----------+
+         |
+         | createMediaElementSource()
+         | (connects once, never disconnect)
+         v
++--------+----------+
+| MediaElementSource|  <-- Bridge between HTML5 audio and Web Audio
++--------+----------+
+         |
+         | connect()
+         v
++--------+----------+
+|   AnalyserNode    |  <-- For visualization only
+|   - fftSize: 256  |      - getByteFrequencyData()
+|   - smoothing: 0.8|      - getByteTimeDomainData()
++--------+----------+
+         |
+         | connect()
+         v
++--------+----------+
+| AudioDestination  |  <-- Audio output (speakers)
++-------------------+
+
+IMPORTANT: Audio flows THROUGH analyser to destination
+           This preserves audio output while enabling analysis
+```
+
+### Code Implementation
+
+```typescript
+function connectWebAudio(audioElement: HTMLAudioElement): WebAudioGraph {
+  // Create or reuse AudioContext
+  const AudioContextClass = window.AudioContext ||
+    (window as any).webkitAudioContext;
+  const audioContext = new AudioContextClass();
+
+  // CRITICAL: Only create MediaElementSourceNode ONCE per audio element
+  // Attempting to create multiple sources throws InvalidStateError
+  const sourceNode = audioContext.createMediaElementSource(audioElement);
+
+  // Create analyser for visualization
+  const analyserNode = audioContext.createAnalyser();
+  analyserNode.fftSize = 256;
+  analyserNode.smoothingTimeConstant = 0.8;
+
+  // Connect the graph: source -> analyser -> destination
+  sourceNode.connect(analyserNode);
+  analyserNode.connect(audioContext.destination);
+
+  return { audioContext, sourceNode, analyserNode };
+}
+```
+
+### Handling AudioContext Suspension
+
+Browsers require user interaction before AudioContext can start:
+
+```typescript
+useEffect(() => {
+  const handleUserInteraction = async () => {
+    if (audioContext?.state === 'suspended') {
+      await audioContext.resume();
+    }
+  };
+
+  // Resume on first play
+  audioElement?.addEventListener('play', handleUserInteraction);
+
+  return () => {
+    audioElement?.removeEventListener('play', handleUserInteraction);
+  };
+}, [audioContext, audioElement]);
+```
+
+---
+
+## Streaming Audio Support
+
+### How HTML5 Audio Handles Streaming
+
+```
+Browser Request:          Server Response:
++------------------+      +------------------+
+| GET /audio.mp3   |      | 206 Partial      |
+| Range: bytes=0-  |  --> | Content-Range:   |
++------------------+      | bytes 0-2MB/10MB |
+                          +------------------+
+
+Seek Request:             Server Response:
++------------------+      +------------------+
+| GET /audio.mp3   |      | 206 Partial      |
+| Range: bytes=5MB-|  --> | Content-Range:   |
++------------------+      | bytes 5MB-7MB/   |
+                          +------------------+
+
+The browser automatically:
+1. Requests chunks as needed
+2. Buffers ahead for smooth playback
+3. Handles seek by requesting new ranges
+4. Reports buffered ranges via audio.buffered
+```
+
+### Visualizing Buffered Regions
+
+```typescript
+function renderBufferedRegions(
+  ctx: CanvasRenderingContext2D,
+  buffered: TimeRanges,
+  duration: number,
+  width: number,
+  height: number
+) {
+  ctx.fillStyle = 'rgba(255, 255, 255, 0.1)';
+
+  for (let i = 0; i < buffered.length; i++) {
+    const start = (buffered.start(i) / duration) * width;
+    const end = (buffered.end(i) / duration) * width;
+    ctx.fillRect(start, height - 2, end - start, 2);
+  }
+}
+```
+
+---
+
+## Implementation Steps
+
+### Phase 1: Core Infrastructure
+
+**Files to create:**
+- `hooks/useHybridAudio.ts`
+- `context/HybridAudioProvider.tsx`
+
+**Tasks:**
+1. Create `useHybridAudio` hook combining HTML5 audio + Web Audio API
+2. Build `HybridAudioProvider` context wrapper
+3. Integrate with existing `useSharedWebAudio` pattern (singleton source)
+4. Add comprehensive event handling (timeupdate, progress, seeking, etc.)
+
+### Phase 2: Visualization Integration
+
+**Files to modify:**
+- `hooks/useAudioAnalysis.ts` (minor updates)
+- `components/AudioEqualizer.tsx` (connect to new context)
+
+**Files to create:**
+- `components/HybridWaveform.tsx`
+
+**Tasks:**
+1. Update `useAudioAnalysis` to accept hybrid context
+2. Create `HybridWaveform` component with dual modes
+3. Integrate with existing canvas rendering from Progressive player
+
+### Phase 3: Player Component
+
+**Files to create:**
+- `components/HybridAudioPlayer.tsx`
+- `components/HybridSimplePlayer.tsx`
+
+**Tasks:**
+1. Build main `HybridAudioPlayer` component
+2. Create simplified `HybridSimplePlayer` wrapper
+3. Integrate all UI controls (play, pause, seek, volume, loop)
+4. Add keyboard shortcuts support
+
+### Phase 4: Reactive Cover Integration
+
+**Files to modify:**
+- `components/ReactiveCover/AudioReactiveCover.tsx` (minimal changes)
+
+**Tasks:**
+1. Verify reactive effects work with new context
+2. Test all effect variants (glow, orbs, spotlight, mesh)
+3. Ensure smooth 60fps animations
+
+### Phase 5: Migration & Testing
+
+**Tasks:**
+1. Create migration guide from WaveSurfer-based player
+2. Update all usages in application
+3. Performance testing with various audio formats/sizes
+4. Cross-browser testing (Chrome, Firefox, Safari, Edge)
+
+---
+
+## Component Hierarchy
+
+```
+HybridSimplePlayer
+|
++-- HybridAudioProvider (context)
+    |
+    +-- AudioReactiveCover (optional)
+    |   |
+    |   +-- GlowEffect / OrbsEffect / SpotlightEffect / MeshEffect
+    |   |
+    |   +-- Cover Art (children)
+    |
+    +-- HybridAudioPlayer
+        |
+        +-- HybridWaveform (canvas visualization)
+        |
+        +-- Timer (current / duration)
+        |
+        +-- Controls
+            |
+            +-- PlayPauseButton
+            +-- SkipButtons
+            +-- VolumeControl
+            +-- LoopToggle
+            +-- ShortcutsHelp
+```
+
+---
+
+## File Structure
+
+```
+AudioPlayer/
++-- @refactoring3/
+|   +-- 03-HYBRID-ARCHITECTURE-PROPOSAL.md  (this file)
+|
++-- context/
+|   +-- AudioProvider.tsx          (existing - WaveSurfer)
+|   +-- HybridAudioProvider.tsx    (NEW)
+|   +-- index.ts
+|
++-- hooks/
+|   +-- useSharedWebAudio.ts       (existing - reuse pattern)
+|   +-- useAudioAnalysis.ts        (existing - minor updates)
+|   +-- useHybridAudio.ts          (NEW)
+|   +-- useAudioHotkeys.ts         (existing - reuse)
+|   +-- index.ts
+|
++-- components/
+|   +-- AudioPlayer.tsx            (existing - WaveSurfer)
+|   +-- SimpleAudioPlayer.tsx      (existing - WaveSurfer)
+|   +-- HybridAudioPlayer.tsx      (NEW)
+|   +-- HybridSimplePlayer.tsx     (NEW)
+|   +-- HybridWaveform.tsx         (NEW)
+|   +-- AudioEqualizer.tsx         (existing)
+|   +-- ReactiveCover/             (existing - no changes)
+|   +-- index.ts
+|
++-- progressive/
+|   +-- ProgressiveAudioPlayer.tsx (existing - can be deprecated later)
+|   +-- useAudioElement.ts         (existing - reference for useHybridAudio)
+|   +-- WaveformCanvas.tsx         (existing - reuse for HybridWaveform)
+|   +-- ...
+|
++-- types/
+|   +-- audio.ts                   (extend with hybrid types)
+|   +-- index.ts
+|
++-- effects/
+|   +-- index.ts                   (existing - no changes)
+```
+
+---
+
+## API Design
+
+### Basic Usage
+
+```tsx
+import { HybridSimplePlayer } from '@djangocfg/ui-nextjs/tools/AudioPlayer';
+
+function MyComponent() {
+  return (
+    <HybridSimplePlayer
+      src="https://example.com/stream/audio.mp3"
+      title="Track Title"
+      artist="Artist Name"
+      coverArt="/path/to/cover.jpg"
+      reactiveCover
+      variant="spotlight"
+    />
+  );
+}
+```
+
+### Advanced Usage with Context
+
+```tsx
+import {
+  HybridAudioProvider,
+  HybridAudioPlayer,
+  AudioReactiveCover,
+  useHybridAudio
+} from '@djangocfg/ui-nextjs/tools/AudioPlayer';
+
+function CustomPlayer({ src }) {
+  const containerRef = useRef(null);
+
+  return (
+    <HybridAudioProvider src={src}>
+      <div className="flex gap-4">
+        <AudioReactiveCover variant="orbs">
+          <img src={coverUrl} alt="Cover" />
+        </AudioReactiveCover>
+
+        <HybridAudioPlayer
+          ref={containerRef}
+          showWaveform
+          showEqualizer={false}
+        />
+      </div>
+    </HybridAudioProvider>
+  );
+}
+```
+
+### Using Hooks Directly
+
+```tsx
+import { useHybridAudio, useAudioAnalysis } from '@djangocfg/ui-nextjs/tools/AudioPlayer';
+
+function CustomVisualization({ src }) {
+  const { audioRef, state, controls, webAudio } = useHybridAudio({ src });
+  const levels = useAudioAnalysis(webAudio, state.isPlaying);
+
+  return (
+    <div>
+      <audio ref={audioRef} />
+      <CustomCanvas levels={levels} />
+      <button onClick={controls.togglePlay}>
+        {state.isPlaying ? 'Pause' : 'Play'}
+      </button>
+    </div>
+  );
+}
+```
+
+---
+
+## Migration Path
+
+### From WaveSurfer-based Player
+
+```tsx
+// Before (WaveSurfer)
+<SimpleAudioPlayer
+  src={url}
+  showWaveform
+  reactiveCover
+  variant="spotlight"
+/>
+
+// After (Hybrid)
+<HybridSimplePlayer
+  src={url}
+  showWaveform
+  reactiveCover
+  variant="spotlight"
+/>
+```
+
+### From Progressive Player
+
+```tsx
+// Before (Progressive)
+<ProgressiveAudioPlayer
+  src={streamUrl}
+  showWaveform
+/>
+
+// After (Hybrid)
+<HybridSimplePlayer
+  src={streamUrl}
+  showWaveform
+  waveformMode="frequency"  // or "waveform" for pre-computed peaks
+/>
+```
+
+---
+
+## Performance Considerations
+
+### Memory Usage
+
+| Component | Memory Impact |
+|-----------|--------------|
+| HTML5 Audio | Browser-managed buffering (~5-20MB depending on format) |
+| Web Audio Context | ~2-4KB per context |
+| AnalyserNode | ~1KB + fftSize bytes for data arrays |
+| Canvas | width * height * 4 bytes (RGBA) |
+| Reactive Effects | Minimal (CSS transforms) |
+
+### CPU Usage
+
+| Operation | Frequency | Cost |
+|-----------|-----------|------|
+| Audio decoding | On-demand | Browser-optimized |
+| getByteFrequencyData | 60fps during playback | ~0.1ms |
+| Canvas render | 60fps during playback | ~1-2ms |
+| React state updates | 60fps during playback | ~0.5ms |
+| Effect calculations | 60fps during playback | ~0.2ms |
+
+### Optimization Tips
+
+1. **Throttle state updates:** Use `requestAnimationFrame` for visualization updates
+2. **Avoid re-renders:** Memoize effect calculations, use refs for animation data
+3. **Canvas optimization:** Use `willReadFrequently: true` for 2D context
+4. **Web Audio:** Reuse AudioContext (singleton pattern)
+
+---
+
+## Browser Compatibility
+
+| Feature | Chrome | Firefox | Safari | Edge |
+|---------|--------|---------|--------|------|
+| HTML5 Audio | Yes | Yes | Yes | Yes |
+| Range Requests | Yes | Yes | Yes | Yes |
+| AudioContext | Yes | Yes | Yes (webkit) | Yes |
+| MediaElementSource | Yes | Yes | Yes | Yes |
+| AnalyserNode | Yes | Yes | Yes | Yes |
+| requestAnimationFrame | Yes | Yes | Yes | Yes |
+
+**Note:** Safari requires `webkitAudioContext` prefix in older versions.
+
+---
+
+## Testing Strategy
+
+### Unit Tests
+
+1. `useHybridAudio` hook behavior
+2. Audio state transitions
+3. Web Audio graph creation
+4. Frequency data extraction
+
+### Integration Tests
+
+1. Full player component rendering
+2. Play/pause/seek operations
+3. Volume control
+4. Reactive effects updates
+
+### E2E Tests
+
+1. Streaming audio playback
+2. Seek beyond buffered content
+3. Cross-browser playback
+4. Mobile device support
+
+### Manual Testing Scenarios
+
+1. Play audio from streaming URL
+2. Seek to unbuffered position
+3. Verify no crackling during playback
+4. Verify reactive effects respond to music
+5. Test with various audio formats (MP3, AAC, OGG)
+6. Test with different file sizes (1MB - 100MB)
+
+---
+
+## Risks and Mitigations
+
+| Risk | Likelihood | Impact | Mitigation |
+|------|------------|--------|------------|
+| Safari AudioContext issues | Medium | High | Fallback to webkit prefix, test thoroughly |
+| CORS issues with streaming | Medium | Medium | Ensure crossOrigin="anonymous" and server headers |
+| Memory leaks with context | Low | Medium | Proper cleanup in useEffect, singleton pattern |
+| Performance on mobile | Medium | Medium | Reduce fftSize, lower frame rate on mobile |
+| Breaking existing integrations | Medium | High | Provide migration guide, keep old components |
+
+---
+
+## Conclusion
+
+This hybrid architecture provides the best of both worlds:
+
+1. **Native HTML5 Audio for Playback:**
+   - No crackling or audio glitches
+   - Browser-optimized decoding and buffering
+   - Native Range request handling for streaming
+
+2. **Web Audio API for Visualization:**
+   - Real-time frequency analysis
+   - Multiple analyzer nodes for different components
+   - Efficient data extraction for animations
+
+3. **Preserved Reactive Effects:**
+   - All existing effects continue to work
+   - Same audio levels data structure
+   - No changes needed to effect components
+
+4. **Streaming Support:**
+   - Works with chunked audio delivery
+   - Proper buffered region visualization
+   - Seek works even beyond initial buffer
+
+The implementation can be done incrementally without breaking existing functionality, and the migration path is straightforward for consumers of the component library.