@djangocfg/ui-nextjs 2.1.83 → 2.1.85

package/src/tools/AudioPlayer/@refactoring3/02-MEDIA-VIEWER-ANALYSIS.md

@@ -1,560 +0,0 @@
-# Media Viewer Audio Analysis - AudioPlayer Usage in FileWorkspace
-
-**Date:** 2025-12-30
-**Analyzed Directory:** `cmdop/projects/solution/frontend/apps/web/app/_layouts/FileWorkspace/viewers/media`
-
----
-
-## Table of Contents
-
-1. [Overview](#overview)
-2. [Component Architecture](#component-architecture)
-3. [SimpleAudioPlayer vs ProgressiveAudioPlayer](#simpleaudioplayer-vs-progressiveaudioplayer)
-4. [Root Cause of Crackling Issue](#root-cause-of-crackling-issue)
-5. [Chunks/Streaming Handling](#chunksstreaming-handling)
-6. [Audio Element Usage Differences](#audio-element-usage-differences)
-7. [Code References](#code-references)
-8. [Recommendations](#recommendations)
-
----
-
-## Overview
-
-The `AudioViewer.tsx` in FileWorkspace provides two distinct player modes:
-
-1. **WaveSurfer Mode** (default): Uses `SimpleAudioPlayer` with WaveSurfer.js for waveform visualization
-2. **Progressive Mode**: Uses `ProgressiveAudioPlayer` with native HTML5 audio
-
-Users can toggle between modes via a switch in the UI, with preference persisted in localStorage.
-
----
-
-## Component Architecture
-
-### AudioViewer Component Structure
-
-```
-AudioViewer (AudioViewer.tsx)
-    |
-    +-- Mode Toggle (localStorage: 'audio-viewer-progressive-mode')
-    |
-    +-- Progressive Mode?
-    |       |
-    |       YES --> ProgressiveAudioPlayer
-    |       |           |
-    |       |           +-- useAudioElement (native <audio>)
-    |       |           +-- useProgressiveWaveform (chunks + canvas)
-    |       |           +-- WaveformCanvas (custom rendering)
-    |       |
-    |       NO --> Has Metadata?
-    |               |
-    |               YES --> VisualizationProvider
-    |               |           +-- AudioViewerWithMetadata
-    |               |                   +-- AudioProvider (WaveSurfer context)
-    |               |                   +-- AudioReactiveCover
-    |               |                   +-- AudioPlayer (WaveSurfer controls)
-    |               |
-    |               NO --> SimpleAudioPlayer
-    |                           +-- VisualizationProvider
-    |                           +-- AudioProvider (WaveSurfer)
-    |                           +-- AudioPlayer
-    |                           +-- AudioReactiveCover (optional)
-```
-
-### Key File Locations
-
-| File | Path |
-|------|------|
-| AudioViewer | `cmdop/.../viewers/media/AudioViewer.tsx` |
-| SimpleAudioPlayer | `djangocfg/.../AudioPlayer/components/SimpleAudioPlayer.tsx` |
-| AudioPlayer | `djangocfg/.../AudioPlayer/components/AudioPlayer.tsx` |
-| AudioProvider | `djangocfg/.../AudioPlayer/context/AudioProvider.tsx` |
-| ProgressiveAudioPlayer | `djangocfg/.../AudioPlayer/progressive/ProgressiveAudioPlayer.tsx` |
-| useAudioElement | `djangocfg/.../AudioPlayer/progressive/useAudioElement.ts` |
-| useProgressiveWaveform | `djangocfg/.../AudioPlayer/progressive/useProgressiveWaveform.ts` |
-
----
-
-## SimpleAudioPlayer vs ProgressiveAudioPlayer
-
-### SimpleAudioPlayer (WaveSurfer Mode)
-
-**Architecture:**
-```typescript
-// SimpleAudioPlayer.tsx (lines 132-286)
-function SimpleAudioPlayer(props) {
-  return (
-    <VisualizationProvider>
-      <SimpleAudioPlayerContent {...props} />
-    </VisualizationProvider>
-  );
-}
-
-function SimpleAudioPlayerContent({ src, prefetch = true, ... }) {
-  const containerRef = useRef<HTMLDivElement>(null);
-
-  return (
-    <AudioProvider
-      source={{ uri: src, prefetch }}  // <-- Key: prefetch=true by default
-      containerRef={containerRef}
-    >
-      <AudioPlayer ref={containerRef} ... />  // WaveSurfer renders here
-    </AudioProvider>
-  );
-}
-```
-
-**Key Characteristics:**
-1. Uses `@wavesurfer/react` hook internally via `AudioProvider`
-2. **Requires full file load** for waveform generation
-3. Default `prefetch=true` fetches entire file as blob before loading
-4. WaveSurfer creates its own `<audio>` element internally
-5. Supports reactive audio effects (AudioReactiveCover)
-
-**AudioProvider internals (lines 53-371):**
-```typescript
-export function AudioProvider({ source, ... }) {
-  // Uses useAudioSource hook for prefetching
-  const { url: resolvedUrl, isLoading: isPrefetching } = useAudioSource(source);
-
-  // WaveSurfer hook
-  const { wavesurfer, isReady, isPlaying, currentTime } = useWavesurfer(options);
-
-  // Shared Web Audio for reactive effects
-  const audioElement = wavesurfer?.getMediaElement() ?? null;
-  const sharedAudio = useSharedWebAudio(audioElement);
-  const audioLevels = useAudioAnalysis(sharedAudio, isPlaying);
-  // ...
-}
-```
-
-### ProgressiveAudioPlayer (Progressive Mode)
-
-**Architecture:**
-```typescript
-// ProgressiveAudioPlayer.tsx (lines 43-293)
-export function ProgressiveAudioPlayer({ src, ... }) {
-  const audioRef = useRef<HTMLAudioElement>(null);
-
-  // Native HTML5 audio control
-  const audio = useAudioElement({ src, autoPlay, ... });
-
-  // Progressive waveform (separate from playback)
-  const waveform = useProgressiveWaveform({ url: src, duration: audio.duration });
-
-  return (
-    <div>
-      {/* Hidden native audio element */}
-      <audio
-        ref={(el) => { audioRef.current = el; audio.audioRef.current = el; }}
-        preload="metadata"
-        crossOrigin="anonymous"
-      />
-
-      {/* Custom canvas waveform */}
-      <WaveformCanvas
-        peaks={waveform.peaks}
-        currentTime={audio.currentTime}
-        duration={audio.duration}
-        buffered={audio.buffered}
-        onSeek={handleSeek}
-      />
-
-      {/* Controls */}
-    </div>
-  );
-}
-```
-
-**Key Characteristics:**
-1. Uses **native `<audio>` element** for playback
-2. Browser handles Range requests automatically
-3. Waveform loads **progressively** in chunks (512KB default)
-4. Seek works on any position (browser fetches needed data)
-5. **No reactive effects** (no Web Audio context by default)
-
----
-
-## Root Cause of Crackling Issue
-
-### The Problem
-
-The "crackling" issue with WaveSurfer mode (`AudioPlayer`/`SimpleAudioPlayer`) has **multiple potential causes**:
-
-### Cause 1: Web Audio Context Routing (Most Likely)
-
-In `useSharedWebAudio.ts` (lines 39-47):
-```typescript
-if (connectedElementRef.current !== audioElement) {
-  // Disconnect old source
-  if (sourceRef.current) {
-    try { sourceRef.current.disconnect(); } catch { /* ignore */ }
-  }
-
-  // Create new source - this routes audio through Web Audio API
-  sourceRef.current = audioContext.createMediaElementSource(audioElement);
-  sourceRef.current.connect(audioContext.destination);
-}
-```
-
-**Issue:** When `createMediaElementSource()` is called:
-1. Audio is routed through Web Audio API instead of direct playback
-2. This adds latency and can cause timing issues
-3. If AudioContext is not in "running" state, audio may glitch
-4. Multiple connections (analyser nodes) can cause audio artifacts
-
-In `useAudioAnalysis.ts` (lines 31-46):
-```typescript
-useEffect(() => {
-  const analyser = sharedAudio.createAnalyser({ fftSize: 256, smoothing: 0.85 });
-  if (analyser) {
-    analyserRef.current = analyser;
-    // This creates: source -> analyser -> destination chain
-  }
-}, [sharedAudio.sourceNode, ...]);
-```
-
-### Cause 2: WaveSurfer Internal Buffering
-
-WaveSurfer.js uses Web Audio API internally for decoding and playback. When combined with:
-- HTTP Range streaming that doesn't provide complete file
-- Additional Web Audio routing for visualizations
-- Multiple render cycles
-
-This can cause:
-- Buffer underruns (crackling/popping)
-- Timing inconsistencies
-- Decode errors that manifest as audio artifacts
-
-### Cause 3: React Strict Mode Double-Mounting
-
-In development, React Strict Mode can cause:
-1. Double initialization of AudioContext
-2. Race conditions in audio element setup
-3. Multiple event listener attachments
-
-### Why ProgressiveAudioPlayer is Smooth
-
-```typescript
-// useAudioElement.ts - Direct native audio
-<audio
-  ref={audioRef}
-  preload="metadata"
-  crossOrigin="anonymous"
-/>
-// No Web Audio routing, no extra processing
-```
-
-The `ProgressiveAudioPlayer`:
-1. **No Web Audio context** for playback (just for waveform generation)
-2. **No `MediaElementSource`** routing
-3. Browser handles all audio decoding natively
-4. Direct hardware audio output
-
----
-
-## Chunks/Streaming Handling
-
-### WaveSurfer Mode (SimpleAudioPlayer)
-
-**useAudioSource.ts** handles prefetching (lines 55-141):
-```typescript
-const fetchAsBlob = async () => {
-  const response = await fetch(source.uri, {
-    headers: { 'Range': 'bytes=0-' },  // Request full file
-  });
-
-  // Stream chunks for progress tracking
-  const reader = response.body.getReader();
-  const chunks: ArrayBuffer[] = [];
-
-  while (true) {
-    const { done, value } = await reader.read();
-    if (done) break;
-    chunks.push(value.buffer);
-    // Update progress...
-  }
-
-  // Combine all chunks into single blob
-  const blob = new Blob(chunks, { type: 'audio/mpeg' });
-  const blobUrl = URL.createObjectURL(blob);
-  setUrl(blobUrl);  // WaveSurfer uses this blob URL
-};
-```
-
-**Problem:** WaveSurfer needs the **complete file** before it can:
-- Generate accurate waveform
-- Enable seeking to any position
-- Calculate correct duration
-
-If prefetch fails or is disabled, WaveSurfer only has partial data.
-
-### Progressive Mode (ProgressiveAudioPlayer)
-
-**useProgressiveWaveform.ts** (lines 138-227):
-```typescript
-const loadWaveform = useCallback(async () => {
-  // HEAD request for file size
-  const headResponse = await fetch(url, { method: 'HEAD' });
-  const totalSize = parseInt(contentLength, 10);
-
-  // Fetch in chunks (512KB default)
-  let offset = 0;
-  while (offset < totalSize) {
-    const response = await fetch(url, {
-      headers: { Range: `bytes=${offset}-${end}` },
-    });
-
-    const chunk = await response.arrayBuffer();
-    accumulatedDataRef.current.push(chunk);
-
-    // Try to decode and extract peaks periodically
-    await decodeAccumulated();
-
-    offset += chunkSize;
-  }
-});
-```
-
-**For playback**, native `<audio>` handles streaming independently:
-```typescript
-// useAudioElement.ts (lines 303-314)
-useEffect(() => {
-  audio.src = src;  // Browser handles Range requests automatically
-  audio.load();
-}, [src]);
-```
-
----
-
-## Audio Element Usage Differences
-
-### WaveSurfer Mode
-
-| Aspect | Implementation |
-|--------|----------------|
-| Audio Element | Created by WaveSurfer internally |
-| Access | `wavesurfer.getMediaElement()` |
-| Routing | Source -> Web Audio -> Destination |
-| Control | Via WaveSurfer API (`wavesurfer.play()`, etc.) |
-| Buffering | WaveSurfer manages internally |
-
-```typescript
-// AudioProvider.tsx (lines 202-205)
-const audioElement = useMemo(() => {
-  return wavesurfer?.getMediaElement() ?? null;
-}, [wavesurfer]);
-```
-
-### Progressive Mode
-
-| Aspect | Implementation |
-|--------|----------------|
-| Audio Element | Explicit `<audio>` in JSX |
-| Access | Direct ref |
-| Routing | Direct to hardware (no Web Audio for playback) |
-| Control | Native DOM API (`audio.play()`, etc.) |
-| Buffering | Browser handles via `buffered` property |
-
-```typescript
-// ProgressiveAudioPlayer.tsx (lines 111-118)
-<audio
-  ref={(el) => {
-    (audioRef as any).current = el;
-    (audio.audioRef as any).current = el;
-  }}
-  preload="metadata"
-  crossOrigin="anonymous"
-/>
-```
-
-### Event Handling Comparison
-
-**WaveSurfer Mode (via AudioProvider):**
-```typescript
-// Uses WaveSurfer events
-wavesurfer.on('seeking', handleSeeking);
-wavesurfer.on('error', handleError);
-wavesurfer.on('loading', handleLoading);
-wavesurfer.on('finish', handleFinish);
-```
-
-**Progressive Mode (via useAudioElement):**
-```typescript
-// Uses native DOM events (lines 259-272)
-audio.addEventListener('loadedmetadata', handleLoadedMetadata);
-audio.addEventListener('canplay', handleCanPlay);
-audio.addEventListener('play', handlePlay);
-audio.addEventListener('pause', handlePause);
-audio.addEventListener('timeupdate', handleTimeUpdate);
-audio.addEventListener('progress', handleProgress);
-audio.addEventListener('seeking', handleSeeking);
-audio.addEventListener('seeked', handleSeeked);
-audio.addEventListener('waiting', handleWaiting);
-audio.addEventListener('stalled', handleStalled);
-```
-
----
-
-## Code References
-
-### AudioViewer Mode Selection (AudioViewer.tsx)
-
-```typescript
-// Lines 69-86
-const [useProgressivePlayer, setUseProgressivePlayer] = useLocalStorage(
-  'audio-viewer-progressive-mode',
-  false
-);
-
-const handleModeChange = (checked: boolean) => {
-  // Stop all audio before switching
-  document.querySelectorAll('audio').forEach(audio => {
-    audio.pause();
-    audio.currentTime = 0;
-  });
-  setPlayerKey(prev => prev + 1);  // Force remount
-  setUseProgressivePlayer(checked);
-};
-```
-
-### Progressive Player Rendering (AudioViewer.tsx)
-
-```typescript
-// Lines 183-206
-if (useProgressivePlayer) {
-  return (
-    <div key={`progressive-${playerKey}`}>
-      {PlayerModeToggle}
-      <ProgressiveAudioPlayer
-        src={src}
-        title={metadata?.title || file.name}
-        artist={metadata?.artist}
-        showWaveform
-        showControls
-        showTimer
-        showVolume
-        showLoop
-      />
-    </div>
-  );
-}
-```
-
-### WaveSurfer Player Rendering (AudioViewer.tsx)
-
-```typescript
-// Lines 208-237 (simplified)
-if (!metadata) {
-  return (
-    <SimpleAudioPlayer
-      src={src}
-      title={file.name}
-      showWaveform
-      reactiveCover
-    />
-  );
-}
-
-// With metadata
-return (
-  <VisualizationProvider>
-    <AudioViewerWithMetadata src={src} file={file} metadata={metadata} />
-  </VisualizationProvider>
-);
-```
-
-### WaveSurfer Web Audio Connection (useSharedWebAudio.ts)
-
-```typescript
-// Lines 44-47 - The problematic routing
-sourceRef.current = audioContext.createMediaElementSource(audioElement);
-sourceRef.current.connect(audioContext.destination);
-```
-
-### Native Audio Element Setup (useAudioElement.ts)
-
-```typescript
-// Lines 302-314
-useEffect(() => {
-  const audio = audioRef.current;
-  if (!audio) return;
-
-  audioDebug.load(src);
-  setIsReady(false);
-  setError(null);
-  setCurrentTime(0);
-  setDuration(0);
-  audio.src = src;
-  audio.load();
-}, [src]);
-```
-
----
-
-## Recommendations
-
-### To Fix Crackling in WaveSurfer Mode
-
-1. **Lazy Web Audio Initialization**
-   - Only create `MediaElementSource` when reactive effects are actually needed
-   - Add a prop to disable Web Audio routing entirely
-
-2. **AudioContext State Management**
-   ```typescript
-   // Ensure AudioContext is running before creating source
-   if (audioContext.state === 'suspended') {
-     await audioContext.resume();
-   }
-   ```
-
-3. **Single Analyser Node**
-   - Currently multiple components might create separate analysers
-   - Consolidate to a single shared analyser
-
-4. **Disable Reactive Effects by Default**
-   - Make `reactiveCover={false}` the default
-   - Only enable when user explicitly requests
-
-### To Improve Progressive Mode
-
-1. **Add Reactive Effects Support (Optional)**
-   - Create separate Web Audio routing only for visualization
-   - Keep playback through native audio
-
-2. **Pre-generate Waveform Peaks on Backend**
-   - Avoid client-side decoding overhead
-   - Cache peaks alongside audio files
-
-### Configuration Options
-
-Consider adding these options to `AudioViewer`:
-
-```typescript
-interface AudioViewerOptions {
-  // Disable all Web Audio processing
-  disableWebAudio?: boolean;
-
-  // Disable reactive effects specifically
-  disableReactiveEffects?: boolean;
-
-  // Force progressive mode for streaming sources
-  forceProgressiveForStreaming?: boolean;
-}
-```
-
----
-
-## Summary
-
-| Feature | SimpleAudioPlayer (WaveSurfer) | ProgressiveAudioPlayer |
-|---------|------------------------------|----------------------|
-| Audio Routing | Web Audio API | Native Browser |
-| Waveform | Pre-computed (needs full file) | Progressive (chunks) |
-| Seek | Limited by buffered data | Any position |
-| Reactive Effects | Yes | No |
-| Crackling Risk | High (Web Audio routing) | Low (direct playback) |
-| Streaming | Requires prefetch | Native support |
-| Memory Usage | Higher (full file in blob) | Lower (browser cache) |
-
-**Conclusion:** The crackling issue in WaveSurfer mode stems primarily from the Web Audio API routing required for reactive visualizations. The `ProgressiveAudioPlayer` avoids this by using native HTML5 audio playback, resulting in smoother audio output at the cost of reactive visual effects.