@djangocfg/ui-nextjs 2.1.82 → 2.1.83

package/src/tools/AudioPlayer/@refactoring3/04-CRACKLING-ISSUE-DIAGNOSIS.md

@@ -0,0 +1,373 @@
+# Audio Crackling/Distortion Issue Diagnosis
+
+## Executive Summary
+
+The audio crackling and distortion issues in the WaveSurfer-based AudioPlayer stem from **multiple Web Audio API routing conflicts** and **buffer management problems**. The SimpleAudioPlayer (using native HTMLAudioElement) works smoothly because it bypasses all these complexities.
+
+---
+
+## 1. Root Cause Analysis
+
+### 1.1 Double Audio Routing - CRITICAL ISSUE
+
+**Location:** `useSharedWebAudio.ts` lines 44-46 + lines 80-82
+
+```typescript
+// First connection (line 44-46):
+sourceRef.current = audioContext.createMediaElementSource(audioElement);
+sourceRef.current.connect(audioContext.destination);  // Direct connection
+
+// Second connection when creating analyser (line 80-82):
+sourceRef.current.connect(analyser);
+analyser.connect(audioContext.destination);  // DUPLICATE path to destination!
+```
+
+**Problem:** Audio signal is routed to `destination` twice:
+1. Source -> Destination (direct)
+2. Source -> Analyser -> Destination
+
+This causes the audio to be **played twice with slight timing differences**, resulting in:
+- Phase cancellation (certain frequencies cancel out)
+- Crackling from doubled samples
+- Distortion from summed amplitudes exceeding 0dB
+
+### 1.2 WaveSurfer's Dual Decoding Architecture
+
+**Location:** `wavesurfer.ts` lines 502-566, `decoder.ts` lines 2-10
+
+WaveSurfer performs audio decoding **twice**:
+
+1. **For waveform rendering** (lines 556-557):
+```typescript
+const arrayBuffer = await blob.arrayBuffer();
+this.decodedData = await Decoder.decode(arrayBuffer, this.options.sampleRate);
+```
+
+2. **For playback** via MediaElement:
+```typescript
+this.setSrc(url, blob);  // Browser decodes again for playback
+```
+
+The waveform decoder uses a **low sample rate by default** (8000 Hz - see `defaultOptions.sampleRate`), while the browser uses the file's native sample rate for playback. This mismatch can cause visual/audio sync issues but is not the primary crackling cause.
+
+### 1.3 WebAudioPlayer Buffer Recreation on Seek
+
+**Location:** `webaudio.ts` lines 96-128
+
+```typescript
+private _play() {
+  // Clean up old buffer node
+  if (this.bufferNode) {
+    this.bufferNode.onended = null;
+    this.bufferNode.disconnect();
+  }
+
+  // Create NEW buffer source node every time
+  this.bufferNode = this.audioContext.createBufferSource();
+  this.bufferNode.buffer = this.buffer;
+  // ...
+  this.bufferNode.start(this.audioContext.currentTime, currentPos);
+}
+```
+
+**Problem:** `AudioBufferSourceNode` is a **one-shot node** - it can only be started once. WaveSurfer must recreate it on every play/seek, but:
+- Rapid seeks cause rapid node creation/destruction
+- No crossfade between old and new nodes = clicks at transition points
+- `disconnect()` is immediate, not fade-out
+
+### 1.4 Prefetch Streaming Chunking
+
+**Location:** `useAudioSource.ts` lines 97-114
+
+```typescript
+// Stream the response for progress tracking
+const reader = response.body.getReader();
+const chunks: ArrayBuffer[] = [];
+
+while (true) {
+  const { done, value } = await reader.read();
+  if (done) break;
+  chunks.push(value.buffer.slice(value.byteOffset, value.byteOffset + value.byteLength));
+}
+
+// Combine chunks into blob
+const blob = new Blob(chunks, { type: 'audio/mpeg' });
+```
+
+**Potential Issue:** If the stream is interrupted or network latency causes gaps, the resulting blob may have discontinuities. However, since the blob is fully assembled before loading, this is a minor concern compared to the routing issue.
+
+---
+
+## 2. Comparison: SimpleAudioPlayer vs WaveSurfer AudioPlayer
+
+| Aspect | SimpleAudioPlayer | WaveSurfer AudioPlayer |
+|--------|-------------------|------------------------|
+| **Audio Element** | Native `<audio>` | Native `<audio>` (MediaElement backend) |
+| **Web Audio API** | Not used | Used for analyser + SharedWebAudio |
+| **Audio Routing** | Browser-native | Source -> multiple destinations |
+| **Decoding** | Browser-native once | Browser + custom decoder |
+| **Seeks** | Native seeking | Native + buffer recreation |
+| **Analyser** | None | Connected via SharedWebAudio |
+
+### Why SimpleAudioPlayer Works
+
+SimpleAudioPlayer in `SimpleAudioPlayer.tsx` **does use WaveSurfer** via the `AudioProvider`:
+
+```typescript
+// SimpleAudioPlayer.tsx line 200-205
+<AudioProvider
+  source={{ uri: src, prefetch }}
+  containerRef={containerRef}
+  autoPlay={autoPlay}
+  waveformOptions={waveformOptions}
+>
+```
+
+So the crackling issue **also affects SimpleAudioPlayer** when:
+1. `reactiveCover` is enabled (triggers `useAudioAnalysis`)
+2. Audio levels are being tracked
+
+If SimpleAudioPlayer works smoothly for you, check if:
+- `reactiveCover={false}` or `variant="none"`
+- This disables `useAudioAnalysis` which creates the analyser node
+
+---
+
+## 3. Web Audio API Routing Diagram
+
+### Current (Problematic) Architecture
+
+```
+                                   +------------------+
+                                   |                  |
+     +-------------+    connect    |   destination    |  <-- DOUBLE SIGNAL!
+     |   source    | ------------> |   (speakers)     |
+     | (MediaElem) |               |                  |
+     +-------------+               +------------------+
+            |                             ^
+            |                             |
+            |  connect                    |  connect
+            v                             |
+     +-------------+                      |
+     |  analyser   | ---------------------+
+     +-------------+
+```
+
+### Correct Architecture
+
+```
+     +-------------+               +-------------+               +------------------+
+     |   source    |    connect    |  analyser   |    connect    |   destination    |
+     | (MediaElem) | ------------> |             | ------------> |   (speakers)     |
+     +-------------+               +-------------+               +------------------+
+```
+
+---
+
+## 4. Code Fixes
+
+### Fix 1: Remove Direct Connection in useSharedWebAudio
+
+**File:** `hooks/useSharedWebAudio.ts`
+
+```diff
+const initAudio = () => {
+  try {
+    if (!audioContextRef.current) {
+      const AudioContextClass = window.AudioContext ||
+        (window as unknown as { webkitAudioContext: typeof AudioContext }).webkitAudioContext;
+      audioContextRef.current = new AudioContextClass();
+    }
+
+    const audioContext = audioContextRef.current;
+
+    if (connectedElementRef.current !== audioElement) {
+      if (sourceRef.current) {
+        try { sourceRef.current.disconnect(); } catch { /* ignore */ }
+      }
+
+      sourceRef.current = audioContext.createMediaElementSource(audioElement);
+-     // Connect directly to destination (analysers will be inserted in between)
+-     sourceRef.current.connect(audioContext.destination);
++     // DON'T connect to destination here - let the analyser chain handle it
++     // If no analysers are created, we need a passthrough connection
++     if (analyserNodesRef.current.size === 0) {
++       sourceRef.current.connect(audioContext.destination);
++     }
+      connectedElementRef.current = audioElement;
+    }
+  } catch (error) {
+    console.warn('[SharedWebAudio] Could not initialize:', error);
+  }
+};
+```
+
+### Fix 2: Proper Analyser Connection (No Duplicate Path)
+
+**File:** `hooks/useSharedWebAudio.ts`
+
+```diff
+const createAnalyser = useCallback((options?: { fftSize?: number; smoothing?: number }): AnalyserNode | null => {
+  if (!audioContextRef.current || !sourceRef.current) return null;
+
+  try {
+    const analyser = audioContextRef.current.createAnalyser();
+    analyser.fftSize = options?.fftSize ?? 256;
+    analyser.smoothingTimeConstant = options?.smoothing ?? 0.85;
+
+-   // Connect: source -> analyser -> destination
+-   sourceRef.current.connect(analyser);
+-   analyser.connect(audioContextRef.current.destination);
++   // First analyser: disconnect source from destination and insert analyser
++   if (analyserNodesRef.current.size === 0) {
++     try { sourceRef.current.disconnect(audioContextRef.current.destination); } catch { /* not connected */ }
++   }
++
++   // Connect analyser in series (last analyser -> new analyser -> destination)
++   // For simplicity, connect in parallel but only ONE path to destination
++   sourceRef.current.connect(analyser);
++
++   // Only the first analyser connects to destination
++   if (analyserNodesRef.current.size === 0) {
++     analyser.connect(audioContextRef.current.destination);
++   }
+
+    analyserNodesRef.current.add(analyser);
+    return analyser;
+  } catch (error) {
+    console.warn('[SharedWebAudio] Could not create analyser:', error);
+    return null;
+  }
+}, []);
+```
+
+### Fix 3: Alternative - Use AnalyserNode Without Routing to Destination
+
+For analysis-only (no audio modification), analysers don't need to be in the audio path:
+
+```typescript
+const createAnalyser = useCallback((options?: { fftSize?: number; smoothing?: number }): AnalyserNode | null => {
+  if (!audioContextRef.current || !sourceRef.current) return null;
+
+  try {
+    const analyser = audioContextRef.current.createAnalyser();
+    analyser.fftSize = options?.fftSize ?? 256;
+    analyser.smoothingTimeConstant = options?.smoothing ?? 0.85;
+
+    // Connect analyser as a "listener" - doesn't need to output to destination
+    sourceRef.current.connect(analyser);
+    // analyser.connect(destination) - NOT NEEDED for getByteFrequencyData()
+
+    analyserNodesRef.current.add(analyser);
+    return analyser;
+  } catch (error) {
+    console.warn('[SharedWebAudio] Could not create analyser:', error);
+    return null;
+  }
+}, []);
+```
+
+**This is the cleanest fix** - the analyser only needs to be connected to the source to read frequency data. It doesn't need to output anywhere.
+
+---
+
+## 5. Additional Recommendations
+
+### 5.1 Add Crossfade for WebAudioPlayer Seeks
+
+For smoother seeking when using WebAudio backend:
+
+```typescript
+// In webaudio.ts _play() method
+private _play() {
+  if (!this.paused) return;
+  this.paused = false;
+
+  const audioContext = this.audioContext;
+
+  // Crossfade out old node
+  if (this.bufferNode && this.gainNode) {
+    const oldGain = this.gainNode.gain.value;
+    this.gainNode.gain.setValueAtTime(oldGain, audioContext.currentTime);
+    this.gainNode.gain.linearRampToValueAtTime(0, audioContext.currentTime + 0.01);
+    setTimeout(() => {
+      this.bufferNode?.disconnect();
+    }, 15);
+  }
+
+  // Create new node with fade-in
+  this.bufferNode = audioContext.createBufferSource();
+  // ... rest of setup
+
+  this.gainNode.gain.setValueAtTime(0, audioContext.currentTime);
+  this.gainNode.gain.linearRampToValueAtTime(1, audioContext.currentTime + 0.01);
+}
+```
+
+### 5.2 Consider MediaElement Backend Only
+
+If waveform visualization isn't critical, avoid WebAudio backend entirely:
+
+```typescript
+// In AudioProvider options, ensure backend is MediaElement (default)
+const options = useMemo(() => ({
+  // ...
+  backend: 'MediaElement',  // Not 'WebAudio'
+}), []);
+```
+
+### 5.3 Debounce Rapid Seeks
+
+Add debouncing for seek operations to prevent rapid buffer recreation:
+
+```typescript
+const seek = useCallback(
+  debounce((time: number) => {
+    if (wavesurfer) {
+      wavesurfer.setTime(Math.max(0, Math.min(time, duration)));
+    }
+  }, 50),
+  [wavesurfer, duration]
+);
+```
+
+---
+
+## 6. Testing Checklist
+
+After applying fixes, test for:
+
+- [ ] No crackling during normal playback
+- [ ] No crackling when seeking
+- [ ] No crackling when pausing/resuming
+- [ ] Audio analysis (reactive effects) still work
+- [ ] Waveform visualization still works
+- [ ] Volume control doesn't cause distortion
+- [ ] Multiple AudioPlayer instances don't interfere
+
+---
+
+## 7. Quick Diagnostic
+
+To quickly verify the double-routing issue, add this debug code:
+
+```typescript
+// In useSharedWebAudio.ts after creating source
+console.log('[DEBUG] Source node connections:', sourceRef.current?.numberOfOutputs);
+console.log('[DEBUG] Destination inputs:', audioContextRef.current?.destination.numberOfInputs);
+```
+
+If `numberOfOutputs > 1` after creating an analyser, the double-routing is confirmed.
+
+---
+
+## Summary
+
+| Issue | Severity | Fix Complexity |
+|-------|----------|----------------|
+| Double audio routing | **CRITICAL** | Low - Remove duplicate connect() |
+| WebAudioPlayer buffer recreation | Medium | Medium - Add crossfade |
+| Prefetch chunking | Low | N/A - Not primary cause |
+| Dual decoding | Low | N/A - Different purposes |
+
+**Recommended action:** Apply Fix 3 (analyser without destination routing) first - it's the safest change with the biggest impact.