react-native-tts-kit 0.1.0

package/ATTRIBUTIONS.md

@@ -0,0 +1,87 @@
+# Attributions
+
+`react-native-tts-kit` ships and depends on the following components.
+
+---
+
+## Model weights — Supertonic-3 (BigScience OpenRAIL-M)
+
+- **Original source:** https://huggingface.co/Supertone/supertonic-3
+  - Pinned upstream commit: `724fb5abbf5502583fb520898d45929e62f02c0b` (2026-04-29 release)
+- **Mirror used by this package:** https://huggingface.co/ahk-d/supertonic-3
+  - Pinned mirror commit: `5024985bc861c2ae97ef9038dc2fc56f410e95be`
+  - Byte-identical copy of the upstream weights, hosted as a redistribution
+    safeguard (OpenRAIL-M Section III §4 explicitly permits redistribution).
+  - The package downloads from the mirror first; if it's unreachable, falls
+    back to the upstream Supertone repo at the pinned upstream commit.
+- **Copyright:** © 2025 Supertone Inc.
+- **License:** [BigScience OpenRAIL-M](./licenses/OpenRAIL-M.txt) (full text shipped under `licenses/`)
+
+### What you can do
+- Use the weights in commercial apps, no royalty.
+- Redistribute, fine-tune, derive new models.
+- Modify `.onnx` files (mark them as modified per Section III §4(c)).
+
+### What you cannot do (Attachment A — Use Restrictions)
+You may not use the model, or any model derived from it, to:
+1. Violate any law.
+2. Exploit or harm minors.
+3. Generate or spread verifiably false information intended to harm.
+4. Generate or spread personal identifiable information to harm someone.
+5. Generate AI content without **clearly disclosing that it is AI-generated**.
+6. Defame, disparage, or harass.
+7. **Impersonate someone (e.g. deepfakes) without their consent.**
+8. Make fully-automated decisions affecting a person's legal rights.
+9. Discriminate or harm groups based on protected characteristics.
+10. Exploit vulnerable populations.
+11. Provide medical advice / interpret medical results.
+12. Use for law-enforcement, immigration, or asylum prediction.
+
+### What you must do when distributing this package or apps built with it
+Per OpenRAIL-M Section III §4:
+- (a) Bind your end users to the same use restrictions in your ToS or license.
+- (b) Ship a copy of the OpenRAIL-M license with the model (we do — see `licenses/`).
+- (c) Mark any modified model files. (We don't modify the `.onnx` files.)
+- (d) Preserve Supertone's copyright and attribution notices. (We do.)
+
+> **Practical guidance for apps shipping this package:** add a line to your ToS / "About" screen along the lines of:
+> *"This app uses Supertone's Supertonic-3 model under the BigScience OpenRAIL-M License. Your use of this app's voice features is subject to the OpenRAIL-M Use Restrictions, which prohibit impersonation without consent, generation of misleading content, and other harmful uses."*
+
+---
+
+## Source-code reference — Supertone/supertonic GitHub
+
+We **do not vendor** Supertone's code. Our iOS Swift inference and Android
+Kotlin port ([`ios/Supertonic/`](./ios/Supertonic/), [`android/.../supertonic/`](./android/src/main/java/expo/modules/ttskit/supertonic/))
+were written from scratch using the upstream Python and Swift references as a
+specification:
+
+- **Source:** https://github.com/supertone-inc/supertonic
+- **Copyright:** © 2025 Supertone Inc.
+- **License:** [MIT](https://github.com/supertone-inc/supertonic/blob/main/LICENSE)
+
+The MIT license on the upstream code does not impose redistribution
+obligations on our independent port; we credit Supertone here for
+transparency and because the code is closely modeled on theirs.
+
+---
+
+## Runtime dependencies
+
+| Package | Source | License |
+|---|---|---|
+| ONNX Runtime (iOS / Android) | https://github.com/microsoft/onnxruntime | MIT |
+| Expo Modules Core | https://github.com/expo/expo | MIT |
+| `expo-speech` (optional system engine) | https://github.com/expo/expo/tree/main/packages/expo-speech | MIT |
+| `expo-asset`, `expo-constants`, `expo-dev-client` | https://github.com/expo/expo | MIT |
+
+---
+
+## This package
+
+- **Code license:** MIT — see [`LICENSE`](./LICENSE)
+- **Copyright:** © 2026 ahk-d
+
+The MIT license on this repository covers only the code in `src/`, `ios/`,
+`android/`, `example/`, and `benchmarks/`. The Supertonic-3 model weights
+downloaded at runtime remain under the OpenRAIL-M license described above.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ahk-d
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,231 @@
+# react-native-tts-kit
+
+<p align="center">
+  <img src="docs/thumbnail.png" alt="react-native-tts-kit — Neural TTS for React Native · On-device · 31 languages" width="1024">
+</p>
+
+**Neural text-to-speech for React Native and Expo. On-device. Sub-100ms. 31 languages.**
+
+```ts
+import TTSKit from 'react-native-tts-kit';
+
+await TTSKit.speak('Hello, world.');
+```
+
+No API keys. No network. No robotic system voice.
+
+> **Status:** v0.1 alpha. iOS verified on iPhone (iOS 26+). Android scaffolded but not yet validated on a real device. Feedback via [GitHub issues](https://github.com/ahk-d/react-native-tts-kit/issues).
+
+---
+
+## Why
+
+| | Quality | Offline | Cost | Languages |
+|---|---|---|---|---|
+| `expo-speech` (system) | robotic | ✅ | free | OS-bound |
+| ElevenLabs / OpenAI TTS | excellent | ❌ | per-request | 30+ |
+| **`react-native-tts-kit`** | **neural** | **✅** | **free** | **31** |
+
+There was no good answer for on-device neural TTS in React Native. We needed it for [Flowent](https://getflowent.com), so we built it and open-sourced it.
+
+---
+
+## Install
+
+```bash
+npx expo install react-native-tts-kit
+npx expo prebuild --platform ios
+cd ios && pod install && cd ..
+npx expo run:ios --device
+```
+
+Bare RN: same flow, just install `expo-modules-core` as a peer dep.
+
+> Custom native code can't run inside Expo Go — use a dev build. Standard Expo workflow in 2026.
+
+---
+
+## Use
+
+```ts
+import TTSKit from 'react-native-tts-kit';
+
+// Default: F1 voice, English
+await TTSKit.speak('Hello, world.');
+
+// Pair any voice with any of 31 languages
+await TTSKit.speak('Bonjour le monde',  { voice: 'F1', language: 'fr' });
+await TTSKit.speak('こんにちは',          { voice: 'M2', language: 'ja' });
+await TTSKit.speak('안녕하세요',          { voice: 'F3', language: 'ko' });
+
+// Stream long text — first audio arrives before synthesis finishes
+const stream = TTSKit.stream(longArticle, { language: 'en' });
+stream.on('chunk', (pcm) => { /* PCM16LE @ 44.1 kHz */ });
+stream.on('end',   () => {});
+
+// Stop in-flight synthesis
+await TTSKit.stop();
+```
+
+### First-launch UX
+
+The model is ~210 MB (multilingual split — 31 languages, fp16 weights) and downloads on first use. Call `prefetchModel()` from a settings screen so users aren't surprised mid-conversation:
+
+```ts
+await TTSKit.prefetchModel((p) => {
+  setProgress(p.percent); // 0–100
+});
+```
+
+Once downloaded, all calls are instant and offline forever.
+
+### Voices
+
+10 voices, all language-agnostic. Pair any voice with any language.
+
+| Male  | M1, M2, M3, M4, M5 |
+|-------|---------------------|
+| Female | F1, F2, F3, F4, F5 |
+
+```ts
+const voices = await TTSKit.getVoices();
+// [{ id: 'F1', name: 'F1', gender: 'female', engine: 'supertonic' }, ...]
+```
+
+### Languages (31, verified on-device)
+
+All 31 languages produce intelligible neural-quality audio on iPhone.
+The example app ([`example/App.tsx`](example/App.tsx)) ships a tappable
+sample sentence for every one.
+
+| | | | | |
+|---|---|---|---|---|
+| Arabic (`ar`) | Bulgarian (`bg`) | Czech (`cs`) | Danish (`da`) | German (`de`) |
+| Greek (`el`) | English (`en`) | Spanish (`es`) | Estonian (`et`) | Finnish (`fi`) |
+| French (`fr`) | Hindi (`hi`) | Croatian (`hr`) | Hungarian (`hu`) | Indonesian (`id`) |
+| Italian (`it`) | Japanese (`ja`) | Korean (`ko`) | Lithuanian (`lt`) | Latvian (`lv`) |
+| Dutch (`nl`) | Polish (`pl`) | Portuguese (`pt`) | Romanian (`ro`) | Russian (`ru`) |
+| Slovak (`sk`) | Slovenian (`sl`) | Swedish (`sv`) | Turkish (`tr`) | Ukrainian (`uk`) |
+| Vietnamese (`vi`) | | | | |
+
+```ts
+import { SUPERTONIC_LANGUAGES } from 'react-native-tts-kit';
+
+await TTSKit.speak('こんにちは', { voice: 'F1', language: 'ja' });
+await TTSKit.speak('Привет',     { voice: 'M2', language: 'ru' });
+await TTSKit.speak('नमस्ते',      { voice: 'F3', language: 'hi' });
+```
+
+Voices are language-agnostic — any voice can speak any language.
+
+### Engines
+
+```ts
+TTSKit.setEngine('supertonic'); // default — neural, on-device
+TTSKit.setEngine('system');     // expo-speech fallback (robotic but free)
+```
+
+| Engine | Status | Use when |
+|---|---|---|
+| `supertonic` | ✅ shipping | Production neural TTS |
+| `system` | ✅ shipping | Fallback, no model download |
+| `neutts` (voice cloning) | ⏳ v1.1 | 3-second voice clone |
+| `cloud:eleven` / `cloud:openai` | ⏳ v1.2 | When you want premium quality |
+
+---
+
+## Performance
+
+Measured on iPhone (iOS 26.4, Debug build):
+
+| Metric | Value |
+|---|---|
+| Time to first audio (1 sentence) | ~70 ms |
+| Real-time factor | < 0.5× |
+| Cold start (first speak after launch) | ~1–2 s |
+| Memory (peak during synthesis) | ~250 MB |
+
+Reproducible benchmark harness in [`benchmarks/`](benchmarks/). Numbers across more devices (iPhone 14, Pixel 8, mid-tier Android) coming with v1.0.
+
+---
+
+## Privacy
+
+Text passed to `speak()` and `stream()` is processed entirely on-device. Once the model is downloaded (one-time, on first `prefetchModel()` call), **no text or audio crosses the network at runtime** — synthesis runs in the local ONNX session and audio plays through the platform audio engine.
+
+The only network activity this package performs is the initial model download from HuggingFace (see [`ATTRIBUTIONS.md`](ATTRIBUTIONS.md) for endpoints). Downloads are SHA-256-verified against fingerprints baked into the package when present. If you ship your app in a privacy-sensitive context, an offline mode (e.g. bundling the model file with your app's assets) is also supported — see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+---
+
+## Architecture
+
+```
+your app
+   ↓
+TTSKit                        — public API ([src/index.ts](src/index.ts))
+   ↓
+SupertonicEngine                 — JS wrapper, listens for native events
+   ↓
+RNTTSKitModule                — Expo Module bridge ([ios/](ios/), [android/](android/))
+   ↓
+4 ONNX sessions                  — duration_predictor → text_encoder
+                                   → vector_estimator (×8 denoising)
+                                   → vocoder
+   ↓
+Float32 PCM @ 44.1 kHz           — AVAudioEngine / AudioTrack playback
+```
+
+Model weights: [Supertonic-3](https://huggingface.co/Supertone/supertonic-3) (99M params, OpenRAIL-M). We host a [pinned mirror](https://huggingface.co/ahk-d/supertonic-3) for resilience.
+
+---
+
+## License
+
+| Component | License |
+|---|---|
+| This package | MIT |
+| Supertonic-3 model weights | [BigScience OpenRAIL-M](licenses/OpenRAIL-M.txt) |
+| ONNX Runtime, Expo Modules | MIT |
+
+**If you ship this in your app**, OpenRAIL-M requires you to:
+
+1. Bind your end users to the [Use Restrictions](ATTRIBUTIONS.md#what-you-cannot-do-attachment-a--use-restrictions) — most importantly **no impersonation/deepfakes without consent** and **AI-generated audio must be disclosed**.
+2. Ship a copy of [`licenses/OpenRAIL-M.txt`](licenses/OpenRAIL-M.txt) with your app.
+3. Preserve the Supertone copyright notice.
+
+A 2-line ToS clause covers it. See [ATTRIBUTIONS.md](ATTRIBUTIONS.md) for boilerplate.
+
+---
+
+## Roadmap
+
+- **v1.0** — Android validation, benchmarks across 4+ devices, cleaner first-launch UX
+- **v1.1** — Voice cloning via [NeuTTS Air](https://huggingface.co/neuphonic/neutts-air) (3-sec sample → cloned voice)
+- **v1.2** — Cloud engine adapters (ElevenLabs, OpenAI, Cartesia) behind the same API
+- **v2.0** — `@ttskit/web` — same API in the browser, WASM/WebGPU
+
+---
+
+## FAQ
+
+**How big is the model?** ~210 MB at fp16 (the multilingual split that supports all 31 languages — `vector_estimator.onnx` alone is 138 MB because it carries cross-lingual weights). Downloaded once on first launch and stored in Application Support / app-private files. fp32 fallback is available — see [tools/quantize.md](tools/quantize.md).
+
+**Does it work in Expo Go?** No — custom native code requires a dev build. Run `npx expo prebuild` and use `npx expo run:ios --device`.
+
+**Why isn't Chinese in the language list?** Supertonic-3's open-source weights cover 31 languages; Mandarin isn't one of them. Use a cloud engine (v1.2) if you need it.
+
+**Can I use a custom voice?** Voice cloning lands in v1.1. For now, you have 10 preset voices.
+
+**Does this work in production?** It's v0.1 alpha. The pipeline runs but we've validated on one iPhone. Expect rough edges for ~2 weeks post-launch.
+
+**How does this compare to `expo-kokoro-onnx`?** Different model (Supertonic-3 vs Kokoro-82M), more languages (31 vs 8), faster TTFA, no espeak-ng dependency. Both are valid choices; we chose Supertonic for Flowent's needs.
+
+---
+
+## Credits
+
+- [Supertone Inc.](https://www.supertone.ai/) — for [Supertonic-3](https://huggingface.co/Supertone/supertonic-3), the model that does the heavy lifting
+- [Microsoft ONNX Runtime](https://onnxruntime.ai/) — inference engine
+- [Expo Modules](https://docs.expo.dev/modules/overview/) — native bridge
+
+Built by [@ahk-d](https://github.com/ahk-d) for [Flowent](https://getflowent.com).

package/android/build.gradle

@@ -0,0 +1,50 @@
+apply plugin: 'com.android.library'
+apply plugin: 'kotlin-android'
+
+group = 'expo.modules.ttskit'
+version = '0.1.0'
+
+def expoModulesCorePlugin = new File(project(":expo-modules-core").projectDir.absolutePath, "ExpoModulesCorePlugin.gradle")
+apply from: expoModulesCorePlugin
+applyKotlinExpoModulesCorePlugin()
+useCoreDependencies()
+useExpoPublishing()
+
+android {
+  namespace "expo.modules.ttskit"
+  compileSdkVersion safeExtGet('compileSdkVersion', 34)
+
+  defaultConfig {
+    minSdkVersion safeExtGet('minSdkVersion', 24)
+    targetSdkVersion safeExtGet('targetSdkVersion', 34)
+    versionCode 1
+    versionName "0.1.0"
+  }
+
+  buildFeatures {
+    buildConfig true
+  }
+
+  compileOptions {
+    sourceCompatibility JavaVersion.VERSION_17
+    targetCompatibility JavaVersion.VERSION_17
+  }
+
+  kotlinOptions {
+    jvmTarget = '17'
+  }
+
+  packagingOptions {
+    pickFirst '**/libonnxruntime.so'
+  }
+}
+
+dependencies {
+  implementation "com.microsoft.onnxruntime:onnxruntime-android:1.18.0"
+  implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3"
+  implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3"
+}
+
+def safeExtGet(prop, fallback) {
+  rootProject.ext.has(prop) ? rootProject.ext.get(prop) : fallback
+}

package/android/src/main/AndroidManifest.xml

@@ -0,0 +1,3 @@
+<manifest xmlns:android="http://schemas.android.com/apk/res/android">
+  <uses-permission android:name="android.permission.INTERNET" />
+</manifest>

package/android/src/main/java/expo/modules/ttskit/RNTTSKitModule.kt

@@ -0,0 +1,158 @@
+package expo.modules.ttskit
+
+import android.util.Base64
+import expo.modules.kotlin.Promise
+import expo.modules.kotlin.modules.Module
+import expo.modules.kotlin.modules.ModuleDefinition
+import expo.modules.ttskit.supertonic.AudioEngine
+import expo.modules.ttskit.supertonic.ModelLocator
+import expo.modules.ttskit.supertonic.SupertonicSession
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.SupervisorJob
+import kotlinx.coroutines.cancel
+import kotlinx.coroutines.launch
+
+class RNTTSKitModule : Module() {
+  private var session: SupertonicSession? = null
+  private val audio = AudioEngine()
+  private val scope = CoroutineScope(Dispatchers.IO + SupervisorJob())
+  private var prefetchJob: Job? = null
+
+  override fun definition() = ModuleDefinition {
+    Name("RNTTSKit")
+
+    Events(
+      "onPrefetchProgress",
+      "onStreamChunk",
+      "onStreamEnd",
+      "onStreamError",
+      "onSpeakStart",
+      "onSpeakDone"
+    )
+
+    OnCreate {
+      session = SupertonicSession(appContext.reactContext!!)
+    }
+
+    OnDestroy {
+      runCatching { session?.tearDown() }
+      session = null
+      audio.tearDown()
+      scope.cancel()
+    }
+
+    AsyncFunction("isAvailable") {
+      val ctx = appContext.reactContext ?: return@AsyncFunction false
+      ModelLocator.modelExists(ctx)
+    }
+
+    AsyncFunction("prefetch") { promise: Promise ->
+      val ctx = appContext.reactContext
+      if (ctx == null) {
+        promise.reject("CONTEXT_UNAVAILABLE", "React context unavailable", null)
+        return@AsyncFunction
+      }
+      prefetchJob?.cancel()
+      prefetchJob = scope.launch {
+        try {
+          ModelLocator.ensureModel(ctx) { downloaded, total ->
+            sendEvent("onPrefetchProgress", mapOf(
+              "bytesDownloaded" to downloaded,
+              "totalBytes" to total,
+              "percent" to if (total > 0) (downloaded.toDouble() / total * 100.0) else 0.0
+            ))
+          }
+          session?.loadIfNeeded()
+          // Pre-warm default voice so first speak() after prefetch doesn't
+          // pay JSON-decode + tensor-alloc cost.
+          session?.prewarmDefaultVoice()
+          promise.resolve(null)
+        } catch (e: Exception) {
+          promise.reject("PREFETCH_FAILED", e.message ?: "prefetch failed", e)
+        }
+      }
+    }
+
+    AsyncFunction("speak") { id: String, text: String, voiceId: String, lang: String, totalStep: Int, speed: Double, volume: Double, promise: Promise ->
+      scope.launch {
+        try {
+          val s = session ?: throw IllegalStateException("session not initialized")
+          s.loadIfNeeded()
+          // Pipe chunks through AudioTrack as they finish synthesizing rather
+          // than synthesizing the whole utterance first. With multi-sentence
+          // input this drops perceived TTFA from O(total-synthesis) to
+          // O(first-sentence). For a single short input it's the same as
+          // before. onSpeakStart fires when the first chunk hits the speaker.
+          audio.beginStream(s.sampleRate, volume.toFloat())
+          var startEmitted = false
+          s.synthesizeStreaming(text, lang, voiceId, totalStep, speed) { samples ->
+            if (!startEmitted) {
+              sendEvent("onSpeakStart", mapOf("id" to id))
+              startEmitted = true
+            }
+            audio.feedStream(samples)
+          }
+          audio.endStream()
+          sendEvent("onSpeakDone", mapOf("id" to id))
+          promise.resolve(null)
+        } catch (e: Exception) {
+          audio.endStream()
+          promise.reject("SYNTHESIS_FAILED", e.message ?: "synthesis failed", e)
+        }
+      }
+    }
+
+    AsyncFunction("stream") { id: String, text: String, voiceId: String, lang: String, totalStep: Int, speed: Double, volume: Double, promise: Promise ->
+      scope.launch {
+        val s = session ?: run {
+          promise.reject("MODEL_NOT_LOADED", "session not initialized", null)
+          return@launch
+        }
+        try {
+          s.loadIfNeeded()
+          audio.beginStream(s.sampleRate, volume.toFloat())
+          s.synthesizeStreaming(text, lang, voiceId, totalStep, speed) { samples ->
+            val pcm16 = SupertonicSession.toPcm16(samples)
+            val b64 = Base64.encodeToString(pcm16, Base64.NO_WRAP)
+            sendEvent("onStreamChunk", mapOf("id" to id, "pcm" to b64))
+            audio.feedStream(samples)
+          }
+          audio.endStream()
+          sendEvent("onStreamEnd", mapOf("id" to id))
+          promise.resolve(null)
+        } catch (e: Exception) {
+          audio.endStream()
+          sendEvent("onStreamError", mapOf("id" to id, "message" to (e.message ?: "stream failed")))
+          promise.reject("SYNTHESIS_FAILED", e.message ?: "stream failed", e)
+        }
+      }
+    }
+
+    AsyncFunction("stop") { promise: Promise ->
+      audio.stop()
+      session?.cancel()
+      promise.resolve(null)
+    }
+
+    AsyncFunction("clearCache") { promise: Promise ->
+      // Tear down the loaded ONNX sessions before deleting the files they
+      // reference. Otherwise the next loadIfNeeded() would short-circuit
+      // (isReady == true) and skip re-loading from disk. Then build a fresh
+      // session so subsequent prefetch/speak calls have a non-null target.
+      val ctx = appContext.reactContext
+      prefetchJob?.cancel()
+      audio.stop()
+      runCatching { session?.cancel() }
+      runCatching { session?.tearDown() }
+      if (ctx != null) {
+        ModelLocator.clearCache(ctx)
+        session = SupertonicSession(ctx)
+      } else {
+        session = null
+      }
+      promise.resolve(null)
+    }
+  }
+}