fastkokoro 0.2.0__tar.gz

fastkokoro-0.2.0/PKG-INFO

@@ -0,0 +1,256 @@
+Metadata-Version: 2.3
+Name: fastkokoro
+Version: 0.2.0
+Summary: Lightweight OpenAI-compatible Kokoro TTS server powered by ONNX Runtime
+Author: Vilson Rodrigues
+Author-email: Vilson Rodrigues <vilson@msgflux.com>
+Requires-Dist: fastapi>=0.115.0
+Requires-Dist: huggingface-hub>=0.36.0
+Requires-Dist: kokoro-onnx>=0.5.0
+Requires-Dist: numpy>=2.0.0
+Requires-Dist: onnxruntime>=1.20.1
+Requires-Dist: orjson>=3.10.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: soundfile>=0.13.0
+Requires-Dist: uvicorn>=0.32.0
+Requires-Dist: uvloop>=0.21.0 ; sys_platform != 'win32'
+Requires-Dist: onnxruntime-gpu>=1.20.0 ; platform_machine == 'x86_64' and sys_platform != 'darwin' and extra == 'gpu'
+Requires-Python: >=3.12
+Provides-Extra: gpu
+Description-Content-Type: text/markdown
+
+# fastkokoro
+
+Lightweight OpenAI-compatible Kokoro TTS server powered by ONNX Runtime.
+
+`fastkokoro` runs the 82M-parameter Kokoro text-to-speech model with low startup
+overhead, fast local inference, and a small dependency footprint. It supports CPU
+and GPU execution through ONNX Runtime providers, including CUDA, TensorRT, and
+OpenVINO when the matching runtime package is installed. The default model is
+NVIDIA's optimized ONNX export: `nvidia/kokoro-82M-onnx-opt`.
+
+The NVIDIA repo's `voices.bin` uses a raw float32 layout. `fastkokoro` converts it
+once into the `.npz` voice format expected by `kokoro-onnx`, so the default model
+and voices both come from `nvidia/kokoro-82M-onnx-opt`.
+
+## Install
+
+```bash
+uv sync
+```
+
+From PyPI:
+
+```bash
+pip install fastkokoro
+```
+
+For GPU builds on platforms supported by `onnxruntime-gpu`:
+
+```bash
+uv sync --extra gpu
+```
+
+## Run
+
+```bash
+uv run fastkokoro
+```
+
+The server starts on `http://0.0.0.0:8880` by default.
+
+Docker CPU:
+
+```bash
+docker build -f Dockerfile.cpu -t fastkokoro:cpu .
+docker run -p 8880:8880 fastkokoro:cpu
+```
+
+Docker Hub CPU:
+
+```bash
+docker run -p 8880:8880 msgflux/fastkokoro:cpu
+```
+
+Docker GPU:
+
+```bash
+docker build -f Dockerfile.gpu -t fastkokoro:gpu .
+docker run --gpus all -p 8880:8880 fastkokoro:gpu
+```
+
+Docker Hub GPU:
+
+```bash
+docker run --gpus all -p 8880:8880 msgflux/fastkokoro:gpu
+```
+
+Environment variables:
+
+| Variable | Default |
+| --- | --- |
+| `FASTKOKORO_HOST` | `0.0.0.0` |
+| `FASTKOKORO_PORT` | `8880` |
+| `FASTKOKORO_MODEL_REPO` | `nvidia/kokoro-82M-onnx-opt` |
+| `FASTKOKORO_MODEL_FILE` | `kokoro-82m-v1.0.onnx` |
+| `FASTKOKORO_MODEL_PATH` | unset; downloads from Hugging Face |
+| `FASTKOKORO_VOICES_FILE` | `voices.bin` |
+| `FASTKOKORO_VOICES_INDEX_FILE` | `voices.txt` |
+| `FASTKOKORO_VOICES_PATH` | unset; downloads and converts NVIDIA voices |
+| `FASTKOKORO_DEFAULT_VOICE` | `af_heart` |
+| `FASTKOKORO_DEFAULT_LANG` | `en-us` |
+| `FASTKOKORO_WARMUP` | `true` |
+| `FASTKOKORO_WARMUP_TEXT` | `hello` |
+| `FASTKOKORO_ONNX_PROVIDERS` | `CPUExecutionProvider` |
+| `FASTKOKORO_ONNX_AUTO_PROVIDERS` | `false` |
+| `FASTKOKORO_ONNX_INTRA_OP_NUM_THREADS` | unset |
+| `FASTKOKORO_ONNX_INTER_OP_NUM_THREADS` | unset |
+
+`FASTKOKORO_WARMUP=true` runs a short synthesis during startup. This makes the
+server take a little longer to become ready, but avoids paying most of the first
+request latency on the first user request.
+
+## ONNX Runtime Providers
+
+`fastkokoro` creates the ONNX Runtime session directly, so provider selection is
+explicit and predictable.
+
+CPU:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=CPUExecutionProvider uv run fastkokoro
+```
+
+CUDA with CPU fallback:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=CUDAExecutionProvider,CPUExecutionProvider uv run fastkokoro
+```
+
+TensorRT with CUDA and CPU fallback:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider uv run fastkokoro
+```
+
+Intel/OpenVINO builds can use:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=OpenVINOExecutionProvider,CPUExecutionProvider uv run fastkokoro
+```
+
+Set `FASTKOKORO_ONNX_AUTO_PROVIDERS=true` to pass every provider available in the
+installed ONNX Runtime build to the session. Use this mostly for quick local
+experiments; production deployments should pin an explicit provider order.
+
+## API
+
+Health:
+
+```bash
+curl http://localhost:8880/health
+```
+
+Models:
+
+```bash
+curl http://localhost:8880/v1/models
+```
+
+The server exposes the local Kokoro model as `kokoro`. For client compatibility,
+`/v1/audio/speech` also accepts `tts-1` and `gpt-4o-mini-tts` as aliases, but
+they are not listed by `/v1/models` because the server is not running OpenAI TTS
+models.
+
+## Voices and Languages
+
+The official Kokoro voice list maps voices to language codes. `fastkokoro`
+accepts the Kokoro language code and common locale aliases, then validates that
+the requested voice belongs to the resolved language.
+
+| Language | Request `lang` values | Voices |
+| --- | --- | --- |
+| American English | `a`, `en-us`, `american` | `af_heart`, `af_alloy`, `af_aoede`, `af_bella`, `af_jessica`, `af_kore`, `af_nicole`, `af_nova`, `af_river`, `af_sarah`, `af_sky`, `am_adam`, `am_echo`, `am_eric`, `am_fenrir`, `am_liam`, `am_michael`, `am_onyx`, `am_puck`, `am_santa` |
+| British English | `b`, `en-gb`, `british` | `bf_alice`, `bf_emma`, `bf_isabella`, `bf_lily`, `bm_daniel`, `bm_fable`, `bm_george`, `bm_lewis` |
+| Japanese | `j`, `ja`, `ja-jp` | `jf_alpha`, `jf_gongitsune`, `jf_nezumi`, `jf_tebukuro`, `jm_kumo` |
+| Mandarin Chinese | `z`, `zh`, `zh-cn`, `mandarin` | `zf_xiaobei`, `zf_xiaoni`, `zf_xiaoxiao`, `zf_xiaoyi`, `zm_yunjian`, `zm_yunxi`, `zm_yunxia`, `zm_yunyang` |
+| Spanish | `e`, `es`, `es-es` | `ef_dora`, `em_alex`, `em_santa` |
+| French | `f`, `fr`, `fr-fr` | `ff_siwis` |
+| Hindi | `h`, `hi`, `hi-in` | `hf_alpha`, `hf_beta`, `hm_omega`, `hm_psi` |
+| Italian | `i`, `it`, `it-it` | `if_sara`, `im_nicola` |
+| Brazilian Portuguese | `p`, `pt`, `pt-br` | `pf_dora`, `pm_alex`, `pm_santa` |
+
+Speech:
+
+```bash
+curl http://localhost:8880/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "kokoro",
+    "input": "Hello from fastkokoro.",
+    "voice": "af_heart",
+    "response_format": "wav"
+  }' \
+  --output speech.wav
+```
+
+Streaming PCM:
+
+```bash
+curl http://localhost:8880/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "kokoro",
+    "input": "Streaming from fastkokoro.",
+    "voice": "af_heart",
+    "response_format": "pcm",
+    "stream": true
+  }' \
+  --output speech.pcm
+```
+
+## OpenAI SDK Examples
+
+The examples use inline script dependencies, so they can run directly with `uv`
+without adding the OpenAI SDK to the project environment.
+
+Start `fastkokoro` first:
+
+```bash
+uv run fastkokoro
+```
+
+Save synthesized audio to a file:
+
+```bash
+uv run examples/tts_save_file.py
+```
+
+Consume streamed audio chunks:
+
+```bash
+uv run examples/tts_stream_chunks.py
+```
+
+Useful environment variables:
+
+| Variable | Default |
+| --- | --- |
+| `FASTKOKORO_BASE_URL` | `http://localhost:8880/v1` |
+| `FASTKOKORO_API_KEY` | `fastkokoro` |
+| `FASTKOKORO_VOICE` | `pf_dora` |
+| `FASTKOKORO_TEXT` | `Ola, tudo bem?` |
+| `FASTKOKORO_TTS_OUTPUT` | `speech.wav` |
+
+## Python
+
+```python
+from fastkokoro import FastKokoro
+
+engine = FastKokoro()
+audio = engine.create(
+    "Hello from fastkokoro.",
+    voice="af_heart",
+    response_format="wav",
+)
+```

fastkokoro-0.2.0/README.md

@@ -0,0 +1,235 @@
+# fastkokoro
+
+Lightweight OpenAI-compatible Kokoro TTS server powered by ONNX Runtime.
+
+`fastkokoro` runs the 82M-parameter Kokoro text-to-speech model with low startup
+overhead, fast local inference, and a small dependency footprint. It supports CPU
+and GPU execution through ONNX Runtime providers, including CUDA, TensorRT, and
+OpenVINO when the matching runtime package is installed. The default model is
+NVIDIA's optimized ONNX export: `nvidia/kokoro-82M-onnx-opt`.
+
+The NVIDIA repo's `voices.bin` uses a raw float32 layout. `fastkokoro` converts it
+once into the `.npz` voice format expected by `kokoro-onnx`, so the default model
+and voices both come from `nvidia/kokoro-82M-onnx-opt`.
+
+## Install
+
+```bash
+uv sync
+```
+
+From PyPI:
+
+```bash
+pip install fastkokoro
+```
+
+For GPU builds on platforms supported by `onnxruntime-gpu`:
+
+```bash
+uv sync --extra gpu
+```
+
+## Run
+
+```bash
+uv run fastkokoro
+```
+
+The server starts on `http://0.0.0.0:8880` by default.
+
+Docker CPU:
+
+```bash
+docker build -f Dockerfile.cpu -t fastkokoro:cpu .
+docker run -p 8880:8880 fastkokoro:cpu
+```
+
+Docker Hub CPU:
+
+```bash
+docker run -p 8880:8880 msgflux/fastkokoro:cpu
+```
+
+Docker GPU:
+
+```bash
+docker build -f Dockerfile.gpu -t fastkokoro:gpu .
+docker run --gpus all -p 8880:8880 fastkokoro:gpu
+```
+
+Docker Hub GPU:
+
+```bash
+docker run --gpus all -p 8880:8880 msgflux/fastkokoro:gpu
+```
+
+Environment variables:
+
+| Variable | Default |
+| --- | --- |
+| `FASTKOKORO_HOST` | `0.0.0.0` |
+| `FASTKOKORO_PORT` | `8880` |
+| `FASTKOKORO_MODEL_REPO` | `nvidia/kokoro-82M-onnx-opt` |
+| `FASTKOKORO_MODEL_FILE` | `kokoro-82m-v1.0.onnx` |
+| `FASTKOKORO_MODEL_PATH` | unset; downloads from Hugging Face |
+| `FASTKOKORO_VOICES_FILE` | `voices.bin` |
+| `FASTKOKORO_VOICES_INDEX_FILE` | `voices.txt` |
+| `FASTKOKORO_VOICES_PATH` | unset; downloads and converts NVIDIA voices |
+| `FASTKOKORO_DEFAULT_VOICE` | `af_heart` |
+| `FASTKOKORO_DEFAULT_LANG` | `en-us` |
+| `FASTKOKORO_WARMUP` | `true` |
+| `FASTKOKORO_WARMUP_TEXT` | `hello` |
+| `FASTKOKORO_ONNX_PROVIDERS` | `CPUExecutionProvider` |
+| `FASTKOKORO_ONNX_AUTO_PROVIDERS` | `false` |
+| `FASTKOKORO_ONNX_INTRA_OP_NUM_THREADS` | unset |
+| `FASTKOKORO_ONNX_INTER_OP_NUM_THREADS` | unset |
+
+`FASTKOKORO_WARMUP=true` runs a short synthesis during startup. This makes the
+server take a little longer to become ready, but avoids paying most of the first
+request latency on the first user request.
+
+## ONNX Runtime Providers
+
+`fastkokoro` creates the ONNX Runtime session directly, so provider selection is
+explicit and predictable.
+
+CPU:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=CPUExecutionProvider uv run fastkokoro
+```
+
+CUDA with CPU fallback:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=CUDAExecutionProvider,CPUExecutionProvider uv run fastkokoro
+```
+
+TensorRT with CUDA and CPU fallback:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider uv run fastkokoro
+```
+
+Intel/OpenVINO builds can use:
+
+```bash
+FASTKOKORO_ONNX_PROVIDERS=OpenVINOExecutionProvider,CPUExecutionProvider uv run fastkokoro
+```
+
+Set `FASTKOKORO_ONNX_AUTO_PROVIDERS=true` to pass every provider available in the
+installed ONNX Runtime build to the session. Use this mostly for quick local
+experiments; production deployments should pin an explicit provider order.
+
+## API
+
+Health:
+
+```bash
+curl http://localhost:8880/health
+```
+
+Models:
+
+```bash
+curl http://localhost:8880/v1/models
+```
+
+The server exposes the local Kokoro model as `kokoro`. For client compatibility,
+`/v1/audio/speech` also accepts `tts-1` and `gpt-4o-mini-tts` as aliases, but
+they are not listed by `/v1/models` because the server is not running OpenAI TTS
+models.
+
+## Voices and Languages
+
+The official Kokoro voice list maps voices to language codes. `fastkokoro`
+accepts the Kokoro language code and common locale aliases, then validates that
+the requested voice belongs to the resolved language.
+
+| Language | Request `lang` values | Voices |
+| --- | --- | --- |
+| American English | `a`, `en-us`, `american` | `af_heart`, `af_alloy`, `af_aoede`, `af_bella`, `af_jessica`, `af_kore`, `af_nicole`, `af_nova`, `af_river`, `af_sarah`, `af_sky`, `am_adam`, `am_echo`, `am_eric`, `am_fenrir`, `am_liam`, `am_michael`, `am_onyx`, `am_puck`, `am_santa` |
+| British English | `b`, `en-gb`, `british` | `bf_alice`, `bf_emma`, `bf_isabella`, `bf_lily`, `bm_daniel`, `bm_fable`, `bm_george`, `bm_lewis` |
+| Japanese | `j`, `ja`, `ja-jp` | `jf_alpha`, `jf_gongitsune`, `jf_nezumi`, `jf_tebukuro`, `jm_kumo` |
+| Mandarin Chinese | `z`, `zh`, `zh-cn`, `mandarin` | `zf_xiaobei`, `zf_xiaoni`, `zf_xiaoxiao`, `zf_xiaoyi`, `zm_yunjian`, `zm_yunxi`, `zm_yunxia`, `zm_yunyang` |
+| Spanish | `e`, `es`, `es-es` | `ef_dora`, `em_alex`, `em_santa` |
+| French | `f`, `fr`, `fr-fr` | `ff_siwis` |
+| Hindi | `h`, `hi`, `hi-in` | `hf_alpha`, `hf_beta`, `hm_omega`, `hm_psi` |
+| Italian | `i`, `it`, `it-it` | `if_sara`, `im_nicola` |
+| Brazilian Portuguese | `p`, `pt`, `pt-br` | `pf_dora`, `pm_alex`, `pm_santa` |
+
+Speech:
+
+```bash
+curl http://localhost:8880/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "kokoro",
+    "input": "Hello from fastkokoro.",
+    "voice": "af_heart",
+    "response_format": "wav"
+  }' \
+  --output speech.wav
+```
+
+Streaming PCM:
+
+```bash
+curl http://localhost:8880/v1/audio/speech \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "kokoro",
+    "input": "Streaming from fastkokoro.",
+    "voice": "af_heart",
+    "response_format": "pcm",
+    "stream": true
+  }' \
+  --output speech.pcm
+```
+
+## OpenAI SDK Examples
+
+The examples use inline script dependencies, so they can run directly with `uv`
+without adding the OpenAI SDK to the project environment.
+
+Start `fastkokoro` first:
+
+```bash
+uv run fastkokoro
+```
+
+Save synthesized audio to a file:
+
+```bash
+uv run examples/tts_save_file.py
+```
+
+Consume streamed audio chunks:
+
+```bash
+uv run examples/tts_stream_chunks.py
+```
+
+Useful environment variables:
+
+| Variable | Default |
+| --- | --- |
+| `FASTKOKORO_BASE_URL` | `http://localhost:8880/v1` |
+| `FASTKOKORO_API_KEY` | `fastkokoro` |
+| `FASTKOKORO_VOICE` | `pf_dora` |
+| `FASTKOKORO_TEXT` | `Ola, tudo bem?` |
+| `FASTKOKORO_TTS_OUTPUT` | `speech.wav` |
+
+## Python
+
+```python
+from fastkokoro import FastKokoro
+
+engine = FastKokoro()
+audio = engine.create(
+    "Hello from fastkokoro.",
+    voice="af_heart",
+    response_format="wav",
+)
+```

fastkokoro-0.2.0/pyproject.toml

@@ -0,0 +1,50 @@
+[project]
+name = "fastkokoro"
+version = "0.2.0"
+description = "Lightweight OpenAI-compatible Kokoro TTS server powered by ONNX Runtime"
+readme = "README.md"
+authors = [
+    { name = "Vilson Rodrigues", email = "vilson@msgflux.com" }
+]
+requires-python = ">=3.12"
+dependencies = [
+    "fastapi>=0.115.0",
+    "huggingface-hub>=0.36.0",
+    "kokoro-onnx>=0.5.0",
+    "numpy>=2.0.0",
+    "onnxruntime>=1.20.1",
+    "orjson>=3.10.0",
+    "pydantic>=2.0.0",
+    "soundfile>=0.13.0",
+    "uvicorn>=0.32.0",
+    "uvloop>=0.21.0 ; sys_platform != 'win32'",
+]
+
+[project.optional-dependencies]
+gpu = [
+    "onnxruntime-gpu>=1.20.0; platform_machine == 'x86_64' and sys_platform != 'darwin'",
+]
+
+[project.scripts]
+fastkokoro = "fastkokoro.cli:main"
+
+[dependency-groups]
+dev = [
+    "httpx>=0.28.0",
+    "packaging>=24.0",
+    "pytest>=8.0.0",
+    "pytest-asyncio>=1.0.0",
+    "ruff>=0.12.0",
+    "twine>=6.0.0",
+]
+
+[tool.ruff]
+line-length = 88
+target-version = "py312"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]
+
+[build-system]
+requires = ["uv_build>=0.11.14,<0.12.0"]
+build-backend = "uv_build"

fastkokoro-0.2.0/src/fastkokoro/__init__.py

@@ -0,0 +1,3 @@
+from fastkokoro.engine import FastKokoro
+
+__all__ = ["FastKokoro"]

fastkokoro-0.2.0/src/fastkokoro/assets.py

@@ -0,0 +1,78 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import numpy as np
+from huggingface_hub import hf_hub_download
+
+from fastkokoro.config import Settings
+
+VOICE_STYLE_SHAPE = (510, 1, 256)
+
+
+def resolve_model_path(settings: Settings) -> Path:
+    if settings.model_path is not None:
+        return settings.model_path
+
+    path = hf_hub_download(
+        repo_id=settings.model_repo,
+        filename=settings.model_file,
+        cache_dir=settings.cache_dir,
+    )
+    return Path(path)
+
+
+def resolve_voices_path(settings: Settings) -> Path:
+    if settings.voices_path is not None:
+        return settings.voices_path
+
+    voices_bin = Path(
+        hf_hub_download(
+            repo_id=settings.model_repo,
+            filename=settings.voices_file,
+            cache_dir=settings.cache_dir,
+        )
+    )
+    voices_index = Path(
+        hf_hub_download(
+            repo_id=settings.model_repo,
+            filename=settings.voices_index_file,
+            cache_dir=settings.cache_dir,
+        )
+    )
+    return convert_raw_voices_to_npz(voices_bin, voices_index, settings.cache_dir)
+
+
+def convert_raw_voices_to_npz(
+    voices_bin: Path, voices_index: Path, cache_dir: Path
+) -> Path:
+    destination = cache_dir / "voices-kokoro-onnx.npz"
+    if destination.exists():
+        return destination
+
+    names = parse_voice_names(voices_index)
+    raw = np.fromfile(voices_bin, dtype=np.float32)
+    expected_values = len(names) * np.prod(VOICE_STYLE_SHAPE)
+    if raw.size != expected_values:
+        raise ValueError(
+            "Unexpected voices.bin shape: "
+            f"got {raw.size} float32 values for {len(names)} voices, "
+            f"expected {expected_values}."
+        )
+
+    styles = raw.reshape((len(names), *VOICE_STYLE_SHAPE))
+    voices = {name: styles[index] for index, name in enumerate(names)}
+    destination.parent.mkdir(parents=True, exist_ok=True)
+    np.savez(destination, **voices)
+    return destination
+
+
+def parse_voice_names(path: Path) -> list[str]:
+    names: list[str] = []
+    for line in path.read_text(encoding="utf-8").splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        _, name = line.split("=", 1)
+        names.append(name)
+    return names

fastkokoro-0.2.0/src/fastkokoro/audio.py

@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+from io import BytesIO
+from typing import Literal
+
+import numpy as np
+import soundfile as sf
+
+AudioFormat = Literal["pcm", "wav", "mp3", "opus", "flac"]
+
+
+def encode_audio(
+    samples: np.ndarray, sample_rate: int, audio_format: AudioFormat
+) -> bytes:
+    if audio_format == "pcm":
+        clipped = np.clip(samples, -1.0, 1.0)
+        return (clipped * 32767).astype("<i2").tobytes()
+
+    subtype = None
+    container = audio_format.upper()
+    if audio_format == "wav":
+        container = "WAV"
+        subtype = "PCM_16"
+    elif audio_format == "mp3":
+        container = "MP3"
+    elif audio_format == "opus":
+        container = "OGG"
+        subtype = "OPUS"
+    elif audio_format == "flac":
+        container = "FLAC"
+
+    with BytesIO() as buffer:
+        sf.write(buffer, samples, sample_rate, format=container, subtype=subtype)
+        return buffer.getvalue()
+
+
+def media_type(audio_format: AudioFormat) -> str:
+    return {
+        "pcm": "audio/pcm",
+        "wav": "audio/wav",
+        "mp3": "audio/mpeg",
+        "opus": "audio/ogg",
+        "flac": "audio/flac",
+    }[audio_format]

fastkokoro-0.2.0/src/fastkokoro/cli.py

@@ -0,0 +1,16 @@
+from __future__ import annotations
+
+import uvicorn
+
+from fastkokoro.config import Settings
+
+
+def main() -> None:
+    settings = Settings.from_env()
+    uvicorn.run(
+        "fastkokoro.server:app",
+        host=settings.host,
+        port=settings.port,
+        loop="auto",
+        reload=False,
+    )

fastkokoro-0.2.0/src/fastkokoro/config.py

@@ -0,0 +1,89 @@
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+
+DEFAULT_MODEL_REPO = "nvidia/kokoro-82M-onnx-opt"
+DEFAULT_MODEL_FILE = "kokoro-82m-v1.0.onnx"
+DEFAULT_VOICES_FILE = "voices.bin"
+DEFAULT_VOICES_INDEX_FILE = "voices.txt"
+DEFAULT_VOICE = "af_heart"
+DEFAULT_LANG = "en-us"
+DEFAULT_HOST = "0.0.0.0"
+DEFAULT_PORT = 8880
+DEFAULT_ONNX_PROVIDERS = ("CPUExecutionProvider",)
+DEFAULT_WARMUP_TEXT = "hello"
+SAMPLE_RATE = 24000
+
+
+@dataclass(frozen=True)
+class Settings:
+    model_repo: str
+    model_file: str
+    voices_file: str
+    voices_index_file: str
+    model_path: Path | None
+    voices_path: Path | None
+    cache_dir: Path
+    default_voice: str
+    default_lang: str
+    host: str
+    port: int
+    onnx_providers: tuple[str, ...]
+    onnx_auto_providers: bool
+    onnx_intra_op_num_threads: int | None
+    onnx_inter_op_num_threads: int | None
+    warmup: bool
+    warmup_text: str
+
+    @classmethod
+    def from_env(cls) -> Settings:
+        model_path = os.getenv("FASTKOKORO_MODEL_PATH")
+        voices_path = os.getenv("FASTKOKORO_VOICES_PATH")
+        cache_dir = os.getenv("FASTKOKORO_CACHE_DIR")
+        providers = os.getenv("FASTKOKORO_ONNX_PROVIDERS")
+
+        return cls(
+            model_repo=os.getenv("FASTKOKORO_MODEL_REPO", DEFAULT_MODEL_REPO),
+            model_file=os.getenv("FASTKOKORO_MODEL_FILE", DEFAULT_MODEL_FILE),
+            voices_file=os.getenv("FASTKOKORO_VOICES_FILE", DEFAULT_VOICES_FILE),
+            voices_index_file=os.getenv(
+                "FASTKOKORO_VOICES_INDEX_FILE", DEFAULT_VOICES_INDEX_FILE
+            ),
+            model_path=Path(model_path).expanduser() if model_path else None,
+            voices_path=Path(voices_path).expanduser() if voices_path else None,
+            cache_dir=Path(cache_dir or "~/.cache/fastkokoro").expanduser(),
+            default_voice=os.getenv("FASTKOKORO_DEFAULT_VOICE", DEFAULT_VOICE),
+            default_lang=os.getenv("FASTKOKORO_DEFAULT_LANG", DEFAULT_LANG),
+            host=os.getenv("FASTKOKORO_HOST", DEFAULT_HOST),
+            port=int(os.getenv("FASTKOKORO_PORT", str(DEFAULT_PORT))),
+            onnx_providers=parse_csv(providers) or DEFAULT_ONNX_PROVIDERS,
+            onnx_auto_providers=parse_bool(os.getenv("FASTKOKORO_ONNX_AUTO_PROVIDERS")),
+            onnx_intra_op_num_threads=parse_optional_int(
+                os.getenv("FASTKOKORO_ONNX_INTRA_OP_NUM_THREADS")
+            ),
+            onnx_inter_op_num_threads=parse_optional_int(
+                os.getenv("FASTKOKORO_ONNX_INTER_OP_NUM_THREADS")
+            ),
+            warmup=parse_bool(os.getenv("FASTKOKORO_WARMUP"), default=True),
+            warmup_text=os.getenv("FASTKOKORO_WARMUP_TEXT", DEFAULT_WARMUP_TEXT),
+        )
+
+
+def parse_csv(value: str | None) -> tuple[str, ...]:
+    if not value:
+        return ()
+    return tuple(item.strip() for item in value.split(",") if item.strip())
+
+
+def parse_bool(value: str | None, *, default: bool = False) -> bool:
+    if value is None:
+        return default
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
+def parse_optional_int(value: str | None) -> int | None:
+    if value is None or value.strip() == "":
+        return None
+    return int(value)

fastkokoro-0.2.0/src/fastkokoro/engine.py

@@ -0,0 +1,99 @@
+from __future__ import annotations
+
+import logging
+from collections.abc import AsyncGenerator
+
+import numpy as np
+from kokoro_onnx import Kokoro
+
+from fastkokoro.assets import resolve_model_path, resolve_voices_path
+from fastkokoro.audio import AudioFormat, encode_audio
+from fastkokoro.config import Settings
+from fastkokoro.onnx import create_session
+from fastkokoro.voices import normalize_language, validate_voice_language
+
+logger = logging.getLogger("uvicorn.error")
+
+
+class FastKokoro:
+    def __init__(self, settings: Settings | None = None):
+        self.settings = settings or Settings.from_env()
+        self.model_path = resolve_model_path(self.settings)
+        self.voices_path = resolve_voices_path(self.settings)
+        self.session = create_session(self.model_path, self.settings)
+        self.kokoro = Kokoro.from_session(self.session, str(self.voices_path))
+        logger.info(
+            "fastkokoro engine initialized: model_repo=%s model_file=%s "
+            "model_path=%s voices_path=%s active_providers=%s "
+            "default_voice=%s default_lang=%s warmup=%s",
+            self.settings.model_repo,
+            self.settings.model_file,
+            self.model_path,
+            self.voices_path,
+            self.session.get_providers(),
+            self.settings.default_voice,
+            self.settings.default_lang,
+            self.settings.warmup,
+        )
+
+    def voices(self) -> list[str]:
+        return self.kokoro.get_voices()
+
+    def warmup(self) -> None:
+        self.create(
+            self.settings.warmup_text,
+            voice=self.settings.default_voice,
+            response_format="pcm",
+            lang=self.settings.default_lang,
+        )
+
+    def resolve_request(self, voice: str | None, lang: str | None) -> tuple[str, str]:
+        resolved_voice = voice or self.settings.default_voice
+        resolved_lang = normalize_language(
+            lang, resolved_voice, self.settings.default_lang
+        )
+        validate_voice_language(resolved_voice, resolved_lang, set(self.voices()))
+        return resolved_voice, resolved_lang
+
+    def create(
+        self,
+        text: str,
+        *,
+        voice: str | None = None,
+        speed: float = 1.0,
+        response_format: AudioFormat = "mp3",
+        lang: str | None = None,
+    ) -> bytes:
+        resolved_voice, resolved_lang = self.resolve_request(voice, lang)
+
+        samples, sample_rate = self.kokoro.create(
+            text,
+            voice=resolved_voice,
+            speed=speed,
+            lang=resolved_lang,
+        )
+        return encode_audio(samples, sample_rate, response_format)
+
+    async def create_stream(
+        self,
+        text: str,
+        *,
+        voice: str | None = None,
+        speed: float = 1.0,
+        response_format: AudioFormat = "pcm",
+        lang: str | None = None,
+    ) -> AsyncGenerator[bytes, None]:
+        resolved_voice, resolved_lang = self.resolve_request(voice, lang)
+
+        stream = self.kokoro.create_stream(
+            text,
+            voice=resolved_voice,
+            speed=speed,
+            lang=resolved_lang,
+        )
+        async for samples, sample_rate in stream:
+            yield encode_audio(
+                samples.astype(np.float32),
+                sample_rate,
+                response_format,
+            )

fastkokoro-0.2.0/src/fastkokoro/json.py

@@ -0,0 +1,13 @@
+from __future__ import annotations
+
+from typing import Any
+
+import orjson
+from starlette.responses import Response
+
+
+class FastJSONResponse(Response):
+    media_type = "application/json"
+
+    def render(self, content: Any) -> bytes:
+        return orjson.dumps(content)

fastkokoro-0.2.0/src/fastkokoro/onnx.py

@@ -0,0 +1,44 @@
+from __future__ import annotations
+
+import logging
+from pathlib import Path
+
+import onnxruntime as ort
+
+from fastkokoro.config import Settings
+
+logger = logging.getLogger("uvicorn.error")
+
+
+def create_session(model_path: Path, settings: Settings) -> ort.InferenceSession:
+    available = ort.get_available_providers()
+    providers = (
+        available if settings.onnx_auto_providers else list(settings.onnx_providers)
+    )
+    missing = [provider for provider in providers if provider not in available]
+    if missing:
+        raise ValueError(
+            "Requested ONNX Runtime provider(s) are not available: "
+            f"{', '.join(missing)}. Available providers: {', '.join(available)}"
+        )
+
+    session_options = ort.SessionOptions()
+    if settings.onnx_intra_op_num_threads is not None:
+        session_options.intra_op_num_threads = settings.onnx_intra_op_num_threads
+    if settings.onnx_inter_op_num_threads is not None:
+        session_options.inter_op_num_threads = settings.onnx_inter_op_num_threads
+
+    session = ort.InferenceSession(
+        str(model_path),
+        providers=providers,
+        sess_options=session_options,
+    )
+    logger.info(
+        "ONNX Runtime session initialized: model=%s requested_providers=%s "
+        "active_providers=%s available_providers=%s",
+        model_path,
+        providers,
+        session.get_providers(),
+        available,
+    )
+    return session

fastkokoro-0.2.0/src/fastkokoro/openai.py

@@ -0,0 +1,30 @@
+from __future__ import annotations
+
+import time
+from typing import Literal
+
+from pydantic import BaseModel, Field
+
+from fastkokoro.audio import AudioFormat
+
+
+class SpeechRequest(BaseModel):
+    model: str = "kokoro"
+    input: str = Field(min_length=1)
+    voice: str = "af_heart"
+    response_format: AudioFormat = "mp3"
+    speed: float = Field(default=1.0, ge=0.5, le=2.0)
+    stream: bool | None = None
+    lang: str | None = None
+
+
+class ModelObject(BaseModel):
+    id: str
+    object: Literal["model"] = "model"
+    created: int = Field(default_factory=lambda: int(time.time()))
+    owned_by: str = "fastkokoro"
+
+
+class ModelList(BaseModel):
+    object: Literal["list"] = "list"
+    data: list[ModelObject]

fastkokoro-0.2.0/src/fastkokoro/server.py

@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+from collections.abc import AsyncGenerator
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI, HTTPException
+from fastapi.responses import Response, StreamingResponse
+
+from fastkokoro.audio import media_type
+from fastkokoro.engine import FastKokoro
+from fastkokoro.json import FastJSONResponse
+from fastkokoro.openai import ModelList, ModelObject, SpeechRequest
+from fastkokoro.voices import KOKORO_MODEL_ID, SUPPORTED_MODEL_IDS
+
+
+def create_app(engine: FastKokoro | None = None) -> FastAPI:
+    @asynccontextmanager
+    async def lifespan(app: FastAPI):
+        app_engine = get_engine()
+        if app_engine.settings.warmup:
+            app_engine.warmup()
+        yield
+
+    app = FastAPI(
+        title="fastkokoro",
+        version="0.1.0",
+        default_response_class=FastJSONResponse,
+        lifespan=lifespan,
+    )
+    app.state.engine = engine
+
+    def get_engine() -> FastKokoro:
+        if app.state.engine is None:
+            app.state.engine = FastKokoro()
+        return app.state.engine
+
+    @app.get("/health")
+    def health() -> dict[str, str]:
+        return {"status": "healthy"}
+
+    @app.get("/v1/models")
+    def models() -> ModelList:
+        return ModelList(
+            data=[
+                ModelObject(id=KOKORO_MODEL_ID),
+            ]
+        )
+
+    @app.get("/v1/audio/voices")
+    def voices() -> dict[str, list[str]]:
+        return {"voices": get_engine().voices()}
+
+    @app.post("/v1/audio/speech")
+    async def speech(request: SpeechRequest) -> Response:
+        if request.model not in SUPPORTED_MODEL_IDS:
+            raise HTTPException(
+                status_code=400,
+                detail=f"Unsupported model {request.model!r}. Use {KOKORO_MODEL_ID!r}.",
+            )
+
+        engine = get_engine()
+        content_type = media_type(request.response_format)
+        should_stream = request.stream is True
+
+        try:
+            resolved_voice, resolved_lang = engine.resolve_request(
+                request.voice, request.lang
+            )
+        except ValueError as exc:
+            raise HTTPException(status_code=400, detail=str(exc)) from exc
+
+        if should_stream:
+
+            async def chunks() -> AsyncGenerator[bytes, None]:
+                async for chunk in engine.create_stream(
+                    request.input,
+                    voice=resolved_voice,
+                    speed=request.speed,
+                    response_format=request.response_format,
+                    lang=resolved_lang,
+                ):
+                    yield chunk
+
+            return StreamingResponse(chunks(), media_type=content_type)
+
+        try:
+            audio = engine.create(
+                request.input,
+                voice=resolved_voice,
+                speed=request.speed,
+                response_format=request.response_format,
+                lang=resolved_lang,
+            )
+        except (AssertionError, RuntimeError, ValueError) as exc:
+            raise HTTPException(status_code=400, detail=str(exc)) from exc
+        return Response(content=audio, media_type=content_type)
+
+    return app
+
+
+app = create_app()

fastkokoro-0.2.0/src/fastkokoro/voices.py

@@ -0,0 +1,154 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+KOKORO_MODEL_ID = "kokoro"
+OPENAI_MODEL_ALIASES = frozenset({"tts-1", "gpt-4o-mini-tts"})
+SUPPORTED_MODEL_IDS = frozenset({KOKORO_MODEL_ID, *OPENAI_MODEL_ALIASES})
+
+
+@dataclass(frozen=True)
+class LanguageSpec:
+    kokoro_code: str
+    runtime_lang: str
+    aliases: tuple[str, ...]
+    voices: tuple[str, ...]
+
+
+LANGUAGES: tuple[LanguageSpec, ...] = (
+    LanguageSpec(
+        kokoro_code="a",
+        runtime_lang="en-us",
+        aliases=("a", "en-us", "en_us", "american", "american-english"),
+        voices=(
+            "af_heart",
+            "af_alloy",
+            "af_aoede",
+            "af_bella",
+            "af_jessica",
+            "af_kore",
+            "af_nicole",
+            "af_nova",
+            "af_river",
+            "af_sarah",
+            "af_sky",
+            "am_adam",
+            "am_echo",
+            "am_eric",
+            "am_fenrir",
+            "am_liam",
+            "am_michael",
+            "am_onyx",
+            "am_puck",
+            "am_santa",
+        ),
+    ),
+    LanguageSpec(
+        kokoro_code="b",
+        runtime_lang="en-gb",
+        aliases=("b", "en-gb", "en_gb", "british", "british-english"),
+        voices=(
+            "bf_alice",
+            "bf_emma",
+            "bf_isabella",
+            "bf_lily",
+            "bm_daniel",
+            "bm_fable",
+            "bm_george",
+            "bm_lewis",
+        ),
+    ),
+    LanguageSpec(
+        kokoro_code="j",
+        runtime_lang="ja",
+        aliases=("j", "ja", "ja-jp", "ja_jp", "japanese"),
+        voices=("jf_alpha", "jf_gongitsune", "jf_nezumi", "jf_tebukuro", "jm_kumo"),
+    ),
+    LanguageSpec(
+        kokoro_code="z",
+        runtime_lang="zh",
+        aliases=("z", "zh", "zh-cn", "zh_cn", "mandarin", "mandarin-chinese"),
+        voices=(
+            "zf_xiaobei",
+            "zf_xiaoni",
+            "zf_xiaoxiao",
+            "zf_xiaoyi",
+            "zm_yunjian",
+            "zm_yunxi",
+            "zm_yunxia",
+            "zm_yunyang",
+        ),
+    ),
+    LanguageSpec(
+        kokoro_code="e",
+        runtime_lang="es",
+        aliases=("e", "es", "es-es", "es_es", "spanish"),
+        voices=("ef_dora", "em_alex", "em_santa"),
+    ),
+    LanguageSpec(
+        kokoro_code="f",
+        runtime_lang="fr-fr",
+        aliases=("f", "fr", "fr-fr", "fr_fr", "french"),
+        voices=("ff_siwis",),
+    ),
+    LanguageSpec(
+        kokoro_code="h",
+        runtime_lang="hi",
+        aliases=("h", "hi", "hi-in", "hi_in", "hindi"),
+        voices=("hf_alpha", "hf_beta", "hm_omega", "hm_psi"),
+    ),
+    LanguageSpec(
+        kokoro_code="i",
+        runtime_lang="it",
+        aliases=("i", "it", "it-it", "it_it", "italian"),
+        voices=("if_sara", "im_nicola"),
+    ),
+    LanguageSpec(
+        kokoro_code="p",
+        runtime_lang="pt-br",
+        aliases=("p", "pt", "pt-br", "pt_br", "brazilian-portuguese", "portuguese"),
+        voices=("pf_dora", "pm_alex", "pm_santa"),
+    ),
+)
+
+LANGUAGE_BY_ALIAS = {
+    alias: language for language in LANGUAGES for alias in language.aliases
+}
+VOICE_TO_LANGUAGE = {
+    voice: language for language in LANGUAGES for voice in language.voices
+}
+SUPPORTED_LANGUAGE_ALIASES = frozenset(LANGUAGE_BY_ALIAS)
+
+
+def normalize_language(lang: str | None, voice: str | None, default_lang: str) -> str:
+    raw_lang = lang or ""
+    key = raw_lang.strip().lower().replace("_", "-")
+    if key:
+        language = LANGUAGE_BY_ALIAS.get(key)
+        if language is None:
+            raise ValueError(
+                "Unsupported language. Supported values: "
+                + ", ".join(sorted(SUPPORTED_LANGUAGE_ALIASES))
+            )
+        return language.runtime_lang
+
+    if voice and voice in VOICE_TO_LANGUAGE:
+        return VOICE_TO_LANGUAGE[voice].runtime_lang
+
+    default_key = default_lang.strip().lower().replace("_", "-")
+    language = LANGUAGE_BY_ALIAS.get(default_key)
+    if language is None:
+        raise ValueError(f"Unsupported default language: {default_lang}")
+    return language.runtime_lang
+
+
+def validate_voice_language(voice: str, lang: str, available_voices: set[str]) -> None:
+    if voice not in available_voices:
+        raise ValueError(f"Voice {voice!r} is not available")
+
+    language = VOICE_TO_LANGUAGE.get(voice)
+    if language is not None and language.runtime_lang != lang:
+        raise ValueError(
+            f"Voice {voice!r} belongs to language {language.runtime_lang!r}, "
+            f"but request language resolved to {lang!r}"
+        )