@siteed/audio-studio 3.0.0

package/README.md

@@ -0,0 +1,167 @@
+# @siteed/audio-studio
+
+[![Version](https://img.shields.io/npm/v/@siteed/audio-studio.svg)](https://www.npmjs.com/package/@siteed/audio-studio)
+[![Downloads](https://img.shields.io/npm/dt/@siteed/audio-studio.svg)](https://www.npmjs.com/package/@siteed/audio-studio)
+[![License](https://img.shields.io/npm/l/@siteed/audio-studio.svg)](https://www.npmjs.com/package/@siteed/audio-studio)
+[![GitHub stars](https://img.shields.io/github/stars/deeeed/audiolab.svg?style=social&label=Star)](https://github.com/deeeed/audiolab)
+
+**Give it a GitHub star, if you found this repo useful.**
+
+Cross-platform audio recording, analysis, and processing for React Native and Expo. Mel spectrogram and spectral feature extraction run through a shared C++ layer on iOS, Android, and web (via WASM).
+
+> Formerly `@siteed/expo-audio-studio`. See [MIGRATION.md](../../MIGRATION.md) for upgrade steps.
+
+<div align="center">
+  <div style="display: flex; justify-content: center; gap: 20px; margin: 30px 0;">
+    <div>
+      <h3>iOS</h3>
+      <img src="../../docs/ios.gif" alt="iOS Demo" width="280" />
+    </div>
+    <div>
+      <h3>Android</h3>
+      <img src="../../docs/android.gif" alt="Android Demo" width="280" />
+    </div>
+  </div>
+
+  <p>
+    <a href="https://apps.apple.com/app/audio-playground/id6739774966">
+      <img src="https://developer.apple.com/app-store/marketing/guidelines/images/badge-download-on-the-app-store.svg" alt="App Store" height="40" />
+    </a>
+    <a href="https://play.google.com/store/apps/details?id=net.siteed.audioplayground">
+      <img src="https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png" alt="Google Play" height="40" />
+    </a>
+  </p>
+  <p><a href="https://deeeed.github.io/audiolab/playground">Try the web demo</a></p>
+</div>
+
+## Features
+
+- **Recording** — real-time streaming, dual-stream (raw PCM + compressed), background recording, zero-latency start via `prepareRecording`
+- **Device management** — list/select input devices (Bluetooth, USB, wired), automatic fallback
+- **Interruption handling** — auto pause/resume during phone calls
+- **Audio analysis** — MFCC, spectral features, mel spectrogram, tempo, pitch, waveform preview
+- **Native performance** — mel spectrogram and FFT-based features (MFCC, spectral centroid, etc.) computed in shared C++ on all platforms (JNI on Android, Obj-C++ bridge on iOS, WASM on web)
+- **Trimming** — precision cut with multi-segment support (keep or remove ranges)
+- **Notifications** — Android live waveform in notification, iOS media player integration
+- **Consistent format** — WAV PCM output across all platforms
+
+## Install
+
+```bash
+yarn add @siteed/audio-studio
+```
+
+## Quick Start
+
+```typescript
+import { useAudioRecorder } from '@siteed/audio-studio';
+
+const { startRecording, stopRecording, isRecording } = useAudioRecorder();
+
+// Record
+await startRecording({
+  sampleRate: 44100,
+  channels: 1,
+  encoding: 'pcm_16bit',
+});
+
+// ... later
+const result = await stopRecording();
+console.log('Saved to:', result.fileUri);
+```
+
+### Zero-Latency Recording
+
+Pre-initialize to eliminate startup delay:
+
+```typescript
+const { prepareRecording, startRecording, stopRecording } = useSharedAudioRecorder();
+
+await prepareRecording({ sampleRate: 44100, channels: 1, encoding: 'pcm_16bit' });
+
+// Later — starts instantly
+await startRecording();
+```
+
+### Shared State Across Components
+
+```typescript
+const AudioApp = () => (
+  <AudioRecorderProvider>
+    <RecordButton />
+    <AudioVisualizer />
+  </AudioRecorderProvider>
+);
+```
+
+### Float32 Streaming (for ML / DSP)
+
+Set `streamFormat: 'float32'` to get `Float32Array` on all platforms instead of base64:
+
+```typescript
+await startRecording({
+  sampleRate: 16000,
+  channels: 1,
+  encoding: 'pcm_32bit',
+  streamFormat: 'float32',
+  onAudioStream: async (event) => {
+    const samples = event.data as Float32Array;
+    await myModel.feed(samples);
+  },
+});
+```
+
+## Audio Analysis
+
+```typescript
+import { extractAudioAnalysis, extractPreview, extractMelSpectrogram, trimAudio } from '@siteed/audio-studio';
+
+// Feature extraction
+const analysis = await extractAudioAnalysis({
+  fileUri: 'path/to/recording.wav',
+  features: { rms: true, zcr: true, mfcc: true, spectralCentroid: true }
+});
+
+// Lightweight waveform for visualization
+const preview = await extractPreview({
+  fileUri: 'path/to/recording.wav',
+  pointsPerSecond: 50
+});
+
+// Mel spectrogram for ML
+const mel = await extractMelSpectrogram({
+  fileUri: 'path/to/recording.wav',
+  nMels: 40, hopLengthMs: 10
+});
+
+// Trim audio
+const trimmed = await trimAudio({
+  fileUri: 'path/to/recording.wav',
+  ranges: [{ startTimeMs: 1000, endTimeMs: 5000 }],
+  mode: 'keep'
+});
+```
+
+### Which Method to Use
+
+| Method | Cost | Use case |
+|--------|------|----------|
+| `extractPreview` | Light | Waveform visualization |
+| `extractRawWavAnalysis` | Light | WAV metadata without decoding |
+| `extractAudioData` | Medium | Raw PCM for custom processing |
+| `extractAudioAnalysis` | Medium-Heavy | MFCC, spectral features, pitch, tempo |
+| `extractMelSpectrogram` | Heavy | Frequency-domain for ML |
+
+## Docs
+
+- [Getting Started Guide](https://deeeed.github.io/audiolab/docs/)
+- [Contributing](./CONTRIBUTE.md)
+- [AudioPlayground](https://deeeed.github.io/audiolab/playground) — live demo of all features
+- [@siteed/expo-audio-ui](https://github.com/deeeed/audiolab/tree/main/packages/expo-audio-ui) — waveform, spectrogram, and audio control components
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+---
+<sub>Created by [Arthur Breton](https://siteed.net)</sub>

package/android/build.gradle

@@ -0,0 +1,143 @@
+apply plugin: 'com.android.library'
+apply plugin: 'kotlin-android'
+apply plugin: 'maven-publish'
+
+group = 'net.siteed.audiostudio'
+version = '0.1.0'
+
+// These properties prevent plugin conflicts in all Expo versions
+project.ext.isLibrary = true
+project.ext.preventPluginConflict = true
+
+buildscript {
+  def expoModulesCorePlugin = new File(project(":expo-modules-core").projectDir.absolutePath, "ExpoModulesCorePlugin.gradle")
+  if (expoModulesCorePlugin.exists()) {
+    apply from: expoModulesCorePlugin
+    applyKotlinExpoModulesCorePlugin()
+  }
+
+  // Simple helper that allows the root project to override versions declared by this library.
+  ext.safeExtGet = { prop, fallback ->
+    rootProject.ext.has(prop) ? rootProject.ext.get(prop) : fallback
+  }
+
+  // Ensures backward compatibility
+  ext.getKotlinVersion = {
+    if (ext.has("kotlinVersion")) {
+      ext.kotlinVersion()
+    } else {
+      ext.safeExtGet("kotlinVersion", "1.8.10")
+    }
+  }
+
+  repositories {
+    mavenCentral()
+    google()
+  }
+
+  dependencies {
+    classpath("org.jetbrains.kotlin:kotlin-gradle-plugin:${getKotlinVersion()}")
+  }
+}
+
+afterEvaluate {
+  publishing {
+    publications {
+      release(MavenPublication) {
+        from components.release
+      }
+    }
+    repositories {
+      maven {
+        url = mavenLocal().url
+      }
+    }
+  }
+}
+
+android {
+  compileSdkVersion safeExtGet("compileSdkVersion", 34)
+
+  def agpVersion = com.android.Version.ANDROID_GRADLE_PLUGIN_VERSION
+  if (agpVersion.tokenize('.')[0].toInteger() < 8) {
+    compileOptions {
+      sourceCompatibility JavaVersion.VERSION_17
+      targetCompatibility JavaVersion.VERSION_17
+    }
+
+    tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).configureEach {
+      kotlinOptions {
+        jvmTarget = '17'
+      }
+    }
+  }
+
+  ndkVersion safeExtGet("ndkVersion", "27.1.12297006")
+
+  namespace "net.siteed.audiostudio"
+  defaultConfig {
+    minSdkVersion safeExtGet("minSdkVersion", 21)
+    targetSdkVersion safeExtGet("targetSdkVersion", 34)
+    versionCode 1
+    versionName "0.1.0"
+    testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
+    externalNativeBuild {
+      cmake {
+        cppFlags "-frtti -fexceptions -Wall"
+        arguments "-DANDROID_STL=c++_shared"
+        abiFilters "x86", "x86_64", "arm64-v8a"
+      }
+    }
+  }
+  externalNativeBuild {
+    cmake {
+      path "src/main/CMakeLists.txt"
+    }
+  }
+  lintOptions {
+    abortOnError false
+  }
+  publishing {
+    singleVariant("release") {
+      withSourcesJar()
+    }
+  }
+}
+
+repositories {
+  mavenCentral()
+  google()
+}
+
+dependencies {
+  implementation project(':expo-modules-core')
+  implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:${getKotlinVersion()}"
+
+  // Add AndroidX dependencies
+  implementation 'androidx.core:core-ktx:1.10.1'
+  implementation 'androidx.annotation:annotation:1.6.0'
+  implementation 'androidx.appcompat:appcompat:1.6.1'
+  
+  // Add Kotlinx Coroutines for main code
+  implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3'
+  implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3'
+
+  // Add testing dependencies
+  testImplementation 'junit:junit:4.13.2'
+  testImplementation 'org.jetbrains.kotlin:kotlin-test:1.8.10'
+  testImplementation 'org.jetbrains.kotlin:kotlin-test-junit:1.8.10'
+  
+  // Add Android instrumented test dependencies
+  androidTestImplementation 'androidx.test:runner:1.5.2'
+  androidTestImplementation 'androidx.test.ext:junit:1.1.5'
+  androidTestImplementation 'androidx.test:core:1.5.0'
+  androidTestImplementation 'junit:junit:4.13.2'
+  androidTestImplementation 'androidx.test:rules:1.5.0'
+  androidTestImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.3'
+}
+
+kotlin {
+  jvmToolchain {
+    languageVersion.set(JavaLanguageVersion.of(17))
+  }
+}

package/android/src/androidTest/java/net/siteed/audiostudio/AudioProcessorInstrumentedTest.kt

@@ -0,0 +1,197 @@
+package net.siteed.audiostudio
+
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import androidx.test.platform.app.InstrumentationRegistry
+import org.junit.Test
+import org.junit.Assert.*
+import org.junit.Before
+import org.junit.After
+import org.junit.runner.RunWith
+import java.io.File
+import kotlin.math.abs
+
+/**
+ * Instrumented tests for AudioProcessor that require Android framework components.
+ * These tests run on an Android device/emulator and have access to MediaExtractor/MediaCodec.
+ */
+@RunWith(AndroidJUnit4::class)
+class AudioProcessorInstrumentedTest {
+    private lateinit var context: android.content.Context
+    private lateinit var audioProcessor: AudioProcessor
+    private lateinit var testFilesDir: File
+
+    @Before
+    fun setUp() {
+        context = InstrumentationRegistry.getInstrumentation().targetContext
+        testFilesDir = context.filesDir
+        audioProcessor = AudioProcessor(testFilesDir)
+        
+        // Copy test assets to files directory
+        copyTestAssets()
+    }
+
+    @After
+    fun tearDown() {
+        // Clean up test files
+        testFilesDir.listFiles()?.forEach { file ->
+            if (file.name.endsWith(".wav")) {
+                file.delete()
+            }
+        }
+    }
+
+    private fun copyTestAssets() {
+        val assetManager = context.assets
+        val testFiles = listOf("jfk.wav", "chorus.wav", "recorder_hello_world.wav")
+        
+        testFiles.forEach { fileName ->
+            try {
+                assetManager.open(fileName).use { input ->
+                    File(testFilesDir, fileName).outputStream().use { output ->
+                        input.copyTo(output)
+                    }
+                }
+            } catch (e: Exception) {
+                // If asset doesn't exist, skip it
+                println("Warning: Test asset $fileName not found")
+            }
+        }
+    }
+
+    // ========== Audio Loading Tests ==========
+
+    @Test
+    fun testLoadAudioFromAnyFormat_loadsWavFile() {
+        // Given
+        val wavFile = File(testFilesDir, "jfk.wav")
+        assertTrue("Test file should exist", wavFile.exists())
+
+        // When
+        val audioData = audioProcessor.loadAudioFromAnyFormat(wavFile.absolutePath, null)
+
+        // Then
+        assertNotNull("Audio data should not be null", audioData)
+        assertEquals("Sample rate should be 16000", 16000, audioData!!.sampleRate)
+        assertEquals("Should be mono", 1, audioData.channels)
+        assertEquals("Should be 16-bit", 16, audioData.bitDepth)
+        assertTrue("Should have audio data", audioData.data.isNotEmpty())
+        assertTrue("Duration should be positive", audioData.durationMs > 0)
+    }
+
+    @Test
+    fun testLoadAudioFromAnyFormat_withDecodingConfig() {
+        // Given
+        val wavFile = File(testFilesDir, "jfk.wav")
+        val config = DecodingConfig(
+            targetSampleRate = 44100,
+            targetChannels = 2,  // Test channel conversion - bug fixed
+            targetBitDepth = 16,
+            normalizeAudio = false
+        )
+
+        // When
+        val audioData = audioProcessor.loadAudioFromAnyFormat(wavFile.absolutePath, config)
+
+        // Then
+        assertNotNull("Audio data should not be null", audioData)
+        assertEquals("Sample rate should be converted to 44100", 44100, audioData!!.sampleRate)
+        assertEquals("Should be converted to stereo", 2, audioData.channels)
+    }
+
+    // ========== Audio Trimming Tests ==========
+
+    @Test
+    fun testTrimAudio_basicTrimming() {
+        // Given
+        val wavFile = File(testFilesDir, "jfk.wav")
+        val startTimeMs = 1000L
+        val endTimeMs = 3000L
+
+        // When
+        val trimmedAudio = audioProcessor.trimAudio(
+            fileUri = wavFile.absolutePath,
+            startTimeMs = startTimeMs,
+            endTimeMs = endTimeMs,
+            config = null,
+            outputFileName = "trimmed_jfk.wav"
+        )
+
+        // Then
+        assertNotNull("Trimmed audio should not be null", trimmedAudio)
+        
+        // Verify the trimmed file was created
+        val trimmedFile = File(testFilesDir, "trimmed_jfk.wav")
+        assertTrue("Trimmed file should exist", trimmedFile.exists())
+    }
+
+    // ========== Mel Spectrogram Tests ==========
+
+    @Test
+    fun testExtractMelSpectrogram_basicGeneration() {
+        // Given
+        val wavFile = File(testFilesDir, "jfk.wav")
+        val audioData = audioProcessor.loadAudioFromAnyFormat(wavFile.absolutePath, null)
+        assertNotNull("Audio data should load", audioData)
+
+        // When
+        val melSpectrogram = audioProcessor.extractMelSpectrogram(
+            audioData = audioData!!,
+            windowSizeMs = 25f,
+            hopLengthMs = 10f,
+            nMels = 40,
+            fftLength = 512
+        )
+
+        // Then
+        assertNotNull("Mel spectrogram should not be null", melSpectrogram)
+        assertTrue("Should have time frames", melSpectrogram.spectrogram.isNotEmpty())
+        assertEquals("Should have 40 mel bins", 40, melSpectrogram.spectrogram[0].size)
+        
+        // Verify timestamps
+        assertTrue("Should have timestamps", melSpectrogram.timeStamps.isNotEmpty())
+        assertEquals("Timestamps should match frames", 
+            melSpectrogram.spectrogram.size, melSpectrogram.timeStamps.size)
+        
+        // Verify frequencies
+        assertEquals("Should have 40 frequency bins", 40, melSpectrogram.frequencies.size)
+        assertTrue("Frequencies should be ascending", 
+            melSpectrogram.frequencies.zip(melSpectrogram.frequencies.drop(1))
+                .all { (a, b) -> a < b })
+    }
+
+    // ========== Preview Generation Tests ==========
+
+    @Test
+    fun testGeneratePreview_basicPreview() {
+        // Given
+        val wavFile = File(testFilesDir, "chorus.wav")
+        val audioData = audioProcessor.loadAudioFromAnyFormat(wavFile.absolutePath, null)
+        assertNotNull("Audio data should load", audioData)
+        
+        val config = RecordingConfig(
+            sampleRate = audioData!!.sampleRate,
+            channels = audioData.channels,
+            encoding = "pcm_16bit",
+            segmentDurationMs = 20 // 50 points per second
+        )
+
+        // When
+        val preview = audioProcessor.generatePreview(
+            audioData = audioData,
+            numberOfPoints = 100,
+            startTimeMs = null,
+            endTimeMs = null,
+            config = config
+        )
+
+        // Then
+        assertNotNull("Preview should not be null", preview)
+        assertEquals("Should have 100 data points", 100, preview.dataPoints.size)
+        
+        // Verify amplitude range
+        assertTrue("Min amplitude should be reasonable", preview.amplitudeRange.min >= 0)
+        assertTrue("Max amplitude should be reasonable", preview.amplitudeRange.max <= 1)
+        assertTrue("Max should be greater than min", 
+            preview.amplitudeRange.max > preview.amplitudeRange.min)
+    }
+}