huri-module-tts 0.1.0__tar.gz

huri_module_tts-0.1.0/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.4
+Name: huri-module-tts
+Version: 0.1.0
+Summary: HuRI modules: text to speech with Piper (CPU, any machine) or CosyVoice3 (voice cloning, NVIDIA GPU)
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: huri-core[server]<0.5,>=0.4
+Requires-Dist: huri-events<0.2,>=0.1
+Requires-Dist: numpy<2,>=1.26
+Provides-Extra: piper
+Requires-Dist: piper-tts<1.9,>=1.8; extra == "piper"
+Provides-Extra: cosyvoice
+Requires-Dist: torch; extra == "cosyvoice"
+Requires-Dist: torchaudio; extra == "cosyvoice"
+
+# huri-module-tts
+
+HuRI's text-to-speech modules: they speak the assistant's reply. One package,
+two engines, each a module of its own with its own dependencies (an extra), so
+a host installs only the engine it runs:
+
+| Module | Engine | Extra | Topics | Deployment |
+|---|---|---|---|---|
+| `tts` | [Piper](https://github.com/OHF-Voice/piper1-gpl): ONNX only, no torch, no GPU, ~30x faster than realtime on a laptop CPU. The default: it runs on any machine. | `piper` | `huri.token` → `huri.audio.out` | `TTSHandle` |
+| `tts_cosyvoice` | CosyVoice3: clones the voice of a reference sample. Needs an NVIDIA GPU for real time (~7x slower than realtime on CPU; its vocoder faults on consumer RDNA3 GPUs). | `cosyvoice` | `huri.token` → `huri.audio.out` | `CosyVoiceTTSHandle` |
+
+Both are built on `huri_tts.streaming.StreamingTTS`, the engine-agnostic half:
+it opens one synthesis session per utterance, pushes each `huri.token` into it
+in order and streams `huri.audio.out` back with presentation timestamps.
+
+The base install has no engine library; `[tool.huri-modules.extras]` in
+`pyproject.toml` names the extra each module needs (`huri hub install` and
+`scripts/install_local.sh` add it). CosyVoice's hardware pins are
+`requirements-<variant>.tts_cosyvoice.txt`: only a host that plans
+`tts_cosyvoice` gets them, so they never meet Piper's `onnxruntime`.
+
+Upgrading from the three packages this one replaced (huri-tts,
+huri-module-tts-piper, huri-module-tts-cosyvoice): `pip uninstall huri-tts
+huri-module-tts-piper huri-module-tts-cosyvoice` first (`./install.sh` does
+it). Left installed, they claim the module names `tts` and `tts_cosyvoice` too,
+and huri-core then loads no module at all.
+
+Piper (`tts`):
+
+```sh
+pip install -e 'packages/huri-module-tts[piper]'
+python -m piper.download_voices --download-dir assets/models/piper en_US-lessac-medium
+HURI_PIPER_VOICE=assets/models/piper/en_US-lessac-medium.onnx HURI_MODULES=...,tts ...
+```
+
+CosyVoice3 (`tts_cosyvoice`):
+
+```sh
+pip install -c constraints.txt --extra-index-url https://download.pytorch.org/whl/cu121 \
+  -r packages/huri-module-tts/requirements-nvidia.tts_cosyvoice.txt
+pip install -e 'packages/huri-module-tts[cosyvoice]'
+packages/huri-module-tts/install_cosyvoice.sh /opt/cosyvoice   # not pip-installable
+export HURI_COSY_DIR=/opt/cosyvoice   # the handle puts it and its Matcha-TTS on sys.path
+HURI_MODULES=...,tts_cosyvoice ...   # instead of tts; see the env vars in huri_tts/cosyvoice/handle.py
+```
+
+`huri hub install tts_cosyvoice` (or `pip install
+'huri-module-tts[cosyvoice]'`) does only the second line: torch and torchaudio,
+unpinned. The pins, the CosyVoice checkout and `HURI_COSY_DIR` stay the steps
+above (`./install.sh --tts-engine cosyvoice` does all of them).
+
+Use `tts_cosyvoice` in place of `tts` in the server's `HURI_MODULES` and the
+client's `modules:`, and name its Serve deployment `CosyVoiceTTSHandle`.
+
+A distribution of
+[huri-modules](https://github.com/Lucy-Robotics/huri-modules), versioned in its
+`pyproject.toml` (see RELEASING.md there). It depends on `huri-core` and
+`huri-events`, which are not on PyPI: without a checkout, install them from git
+in the same pip command, so pip never looks them up on an index (`[cosyvoice]`
+for the other engine):
+
+```sh
+ref=0.1.0   # a huri-modules release tag
+core=0.4.0  # HURI_CORE_REF in that release's versions.env
+pip install \
+  "huri-core[server] @ git+https://github.com/Lucy-Robotics/huri-core.git@$core" \
+  "huri-events @ git+https://github.com/Lucy-Robotics/huri-modules.git@$ref#subdirectory=packages/huri-events" \
+  "huri-module-tts[piper] @ git+https://github.com/Lucy-Robotics/huri-modules.git@$ref#subdirectory=packages/huri-module-tts"
+```

huri_module_tts-0.1.0/README.md

@@ -0,0 +1,69 @@
+# huri-module-tts
+
+HuRI's text-to-speech modules: they speak the assistant's reply. One package,
+two engines, each a module of its own with its own dependencies (an extra), so
+a host installs only the engine it runs:
+
+| Module | Engine | Extra | Topics | Deployment |
+|---|---|---|---|---|
+| `tts` | [Piper](https://github.com/OHF-Voice/piper1-gpl): ONNX only, no torch, no GPU, ~30x faster than realtime on a laptop CPU. The default: it runs on any machine. | `piper` | `huri.token` → `huri.audio.out` | `TTSHandle` |
+| `tts_cosyvoice` | CosyVoice3: clones the voice of a reference sample. Needs an NVIDIA GPU for real time (~7x slower than realtime on CPU; its vocoder faults on consumer RDNA3 GPUs). | `cosyvoice` | `huri.token` → `huri.audio.out` | `CosyVoiceTTSHandle` |
+
+Both are built on `huri_tts.streaming.StreamingTTS`, the engine-agnostic half:
+it opens one synthesis session per utterance, pushes each `huri.token` into it
+in order and streams `huri.audio.out` back with presentation timestamps.
+
+The base install has no engine library; `[tool.huri-modules.extras]` in
+`pyproject.toml` names the extra each module needs (`huri hub install` and
+`scripts/install_local.sh` add it). CosyVoice's hardware pins are
+`requirements-<variant>.tts_cosyvoice.txt`: only a host that plans
+`tts_cosyvoice` gets them, so they never meet Piper's `onnxruntime`.
+
+Upgrading from the three packages this one replaced (huri-tts,
+huri-module-tts-piper, huri-module-tts-cosyvoice): `pip uninstall huri-tts
+huri-module-tts-piper huri-module-tts-cosyvoice` first (`./install.sh` does
+it). Left installed, they claim the module names `tts` and `tts_cosyvoice` too,
+and huri-core then loads no module at all.
+
+Piper (`tts`):
+
+```sh
+pip install -e 'packages/huri-module-tts[piper]'
+python -m piper.download_voices --download-dir assets/models/piper en_US-lessac-medium
+HURI_PIPER_VOICE=assets/models/piper/en_US-lessac-medium.onnx HURI_MODULES=...,tts ...
+```
+
+CosyVoice3 (`tts_cosyvoice`):
+
+```sh
+pip install -c constraints.txt --extra-index-url https://download.pytorch.org/whl/cu121 \
+  -r packages/huri-module-tts/requirements-nvidia.tts_cosyvoice.txt
+pip install -e 'packages/huri-module-tts[cosyvoice]'
+packages/huri-module-tts/install_cosyvoice.sh /opt/cosyvoice   # not pip-installable
+export HURI_COSY_DIR=/opt/cosyvoice   # the handle puts it and its Matcha-TTS on sys.path
+HURI_MODULES=...,tts_cosyvoice ...   # instead of tts; see the env vars in huri_tts/cosyvoice/handle.py
+```
+
+`huri hub install tts_cosyvoice` (or `pip install
+'huri-module-tts[cosyvoice]'`) does only the second line: torch and torchaudio,
+unpinned. The pins, the CosyVoice checkout and `HURI_COSY_DIR` stay the steps
+above (`./install.sh --tts-engine cosyvoice` does all of them).
+
+Use `tts_cosyvoice` in place of `tts` in the server's `HURI_MODULES` and the
+client's `modules:`, and name its Serve deployment `CosyVoiceTTSHandle`.
+
+A distribution of
+[huri-modules](https://github.com/Lucy-Robotics/huri-modules), versioned in its
+`pyproject.toml` (see RELEASING.md there). It depends on `huri-core` and
+`huri-events`, which are not on PyPI: without a checkout, install them from git
+in the same pip command, so pip never looks them up on an index (`[cosyvoice]`
+for the other engine):
+
+```sh
+ref=0.1.0   # a huri-modules release tag
+core=0.4.0  # HURI_CORE_REF in that release's versions.env
+pip install \
+  "huri-core[server] @ git+https://github.com/Lucy-Robotics/huri-core.git@$core" \
+  "huri-events @ git+https://github.com/Lucy-Robotics/huri-modules.git@$ref#subdirectory=packages/huri-events" \
+  "huri-module-tts[piper] @ git+https://github.com/Lucy-Robotics/huri-modules.git@$ref#subdirectory=packages/huri-module-tts"
+```

huri_module_tts-0.1.0/huri_module_tts.egg-info/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.4
+Name: huri-module-tts
+Version: 0.1.0
+Summary: HuRI modules: text to speech with Piper (CPU, any machine) or CosyVoice3 (voice cloning, NVIDIA GPU)
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: huri-core[server]<0.5,>=0.4
+Requires-Dist: huri-events<0.2,>=0.1
+Requires-Dist: numpy<2,>=1.26
+Provides-Extra: piper
+Requires-Dist: piper-tts<1.9,>=1.8; extra == "piper"
+Provides-Extra: cosyvoice
+Requires-Dist: torch; extra == "cosyvoice"
+Requires-Dist: torchaudio; extra == "cosyvoice"
+
+# huri-module-tts
+
+HuRI's text-to-speech modules: they speak the assistant's reply. One package,
+two engines, each a module of its own with its own dependencies (an extra), so
+a host installs only the engine it runs:
+
+| Module | Engine | Extra | Topics | Deployment |
+|---|---|---|---|---|
+| `tts` | [Piper](https://github.com/OHF-Voice/piper1-gpl): ONNX only, no torch, no GPU, ~30x faster than realtime on a laptop CPU. The default: it runs on any machine. | `piper` | `huri.token` → `huri.audio.out` | `TTSHandle` |
+| `tts_cosyvoice` | CosyVoice3: clones the voice of a reference sample. Needs an NVIDIA GPU for real time (~7x slower than realtime on CPU; its vocoder faults on consumer RDNA3 GPUs). | `cosyvoice` | `huri.token` → `huri.audio.out` | `CosyVoiceTTSHandle` |
+
+Both are built on `huri_tts.streaming.StreamingTTS`, the engine-agnostic half:
+it opens one synthesis session per utterance, pushes each `huri.token` into it
+in order and streams `huri.audio.out` back with presentation timestamps.
+
+The base install has no engine library; `[tool.huri-modules.extras]` in
+`pyproject.toml` names the extra each module needs (`huri hub install` and
+`scripts/install_local.sh` add it). CosyVoice's hardware pins are
+`requirements-<variant>.tts_cosyvoice.txt`: only a host that plans
+`tts_cosyvoice` gets them, so they never meet Piper's `onnxruntime`.
+
+Upgrading from the three packages this one replaced (huri-tts,
+huri-module-tts-piper, huri-module-tts-cosyvoice): `pip uninstall huri-tts
+huri-module-tts-piper huri-module-tts-cosyvoice` first (`./install.sh` does
+it). Left installed, they claim the module names `tts` and `tts_cosyvoice` too,
+and huri-core then loads no module at all.
+
+Piper (`tts`):
+
+```sh
+pip install -e 'packages/huri-module-tts[piper]'
+python -m piper.download_voices --download-dir assets/models/piper en_US-lessac-medium
+HURI_PIPER_VOICE=assets/models/piper/en_US-lessac-medium.onnx HURI_MODULES=...,tts ...
+```
+
+CosyVoice3 (`tts_cosyvoice`):
+
+```sh
+pip install -c constraints.txt --extra-index-url https://download.pytorch.org/whl/cu121 \
+  -r packages/huri-module-tts/requirements-nvidia.tts_cosyvoice.txt
+pip install -e 'packages/huri-module-tts[cosyvoice]'
+packages/huri-module-tts/install_cosyvoice.sh /opt/cosyvoice   # not pip-installable
+export HURI_COSY_DIR=/opt/cosyvoice   # the handle puts it and its Matcha-TTS on sys.path
+HURI_MODULES=...,tts_cosyvoice ...   # instead of tts; see the env vars in huri_tts/cosyvoice/handle.py
+```
+
+`huri hub install tts_cosyvoice` (or `pip install
+'huri-module-tts[cosyvoice]'`) does only the second line: torch and torchaudio,
+unpinned. The pins, the CosyVoice checkout and `HURI_COSY_DIR` stay the steps
+above (`./install.sh --tts-engine cosyvoice` does all of them).
+
+Use `tts_cosyvoice` in place of `tts` in the server's `HURI_MODULES` and the
+client's `modules:`, and name its Serve deployment `CosyVoiceTTSHandle`.
+
+A distribution of
+[huri-modules](https://github.com/Lucy-Robotics/huri-modules), versioned in its
+`pyproject.toml` (see RELEASING.md there). It depends on `huri-core` and
+`huri-events`, which are not on PyPI: without a checkout, install them from git
+in the same pip command, so pip never looks them up on an index (`[cosyvoice]`
+for the other engine):
+
+```sh
+ref=0.1.0   # a huri-modules release tag
+core=0.4.0  # HURI_CORE_REF in that release's versions.env
+pip install \
+  "huri-core[server] @ git+https://github.com/Lucy-Robotics/huri-core.git@$core" \
+  "huri-events @ git+https://github.com/Lucy-Robotics/huri-modules.git@$ref#subdirectory=packages/huri-events" \
+  "huri-module-tts[piper] @ git+https://github.com/Lucy-Robotics/huri-modules.git@$ref#subdirectory=packages/huri-module-tts"
+```

huri_module_tts-0.1.0/huri_module_tts.egg-info/SOURCES.txt

@@ -0,0 +1,21 @@
+README.md
+pyproject.toml
+huri_module_tts.egg-info/PKG-INFO
+huri_module_tts.egg-info/SOURCES.txt
+huri_module_tts.egg-info/dependency_links.txt
+huri_module_tts.egg-info/entry_points.txt
+huri_module_tts.egg-info/requires.txt
+huri_module_tts.egg-info/top_level.txt
+huri_tts/__init__.py
+huri_tts/streaming.py
+huri_tts/cosyvoice/__init__.py
+huri_tts/cosyvoice/handle.py
+huri_tts/cosyvoice/module.py
+huri_tts/manifests/tts/huri_module.toml
+huri_tts/manifests/tts_cosyvoice/huri_module.toml
+huri_tts/piper/__init__.py
+huri_tts/piper/handle.py
+huri_tts/piper/module.py
+tests/test_cosyvoice.py
+tests/test_piper.py
+tests/test_streaming_tts.py

huri_module_tts-0.1.0/huri_module_tts.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

huri_module_tts-0.1.0/huri_module_tts.egg-info/entry_points.txt

@@ -0,0 +1,3 @@
+[huri.modules]
+tts = huri_tts.piper.module:PiperTTS
+tts_cosyvoice = huri_tts.cosyvoice.module:CosyVoiceTTS

huri_module_tts-0.1.0/huri_module_tts.egg-info/requires.txt

@@ -0,0 +1,10 @@
+huri-core[server]<0.5,>=0.4
+huri-events<0.2,>=0.1
+numpy<2,>=1.26
+
+[cosyvoice]
+torch
+torchaudio
+
+[piper]
+piper-tts<1.9,>=1.8

huri_module_tts-0.1.0/huri_module_tts.egg-info/top_level.txt

@@ -0,0 +1 @@
+huri_tts

huri_module_tts-0.1.0/huri_tts/__init__.py

@@ -0,0 +1,4 @@
+"""HuRI's text-to-speech modules: `tts` (Piper, huri_tts.piper) and
+`tts_cosyvoice` (CosyVoice3, huri_tts.cosyvoice), both built on
+huri_tts.streaming.StreamingTTS. Each engine's dependencies are an extra of
+huri-module-tts (`piper`, `cosyvoice`)."""

huri_module_tts-0.1.0/huri_tts/cosyvoice/handle.py

@@ -0,0 +1,222 @@
+import asyncio
+import os
+import queue
+import sys
+import traceback
+from typing import AsyncGenerator, Optional
+
+import numpy as np
+from huri_events.audio import Audio
+from ray import serve
+
+# Defaults — overridden by env vars in production (see README.md)
+_MODEL_PATH = os.environ.get(
+    "HURI_MODEL_PATH", "/models/cosytts/FunAudioLLM/Fun-CosyVoice3-0.5B-2512"
+)
+_VOICE_SAMPLE_PATH = os.environ.get("HURI_VOICE_SAMPLE_PATH", "/assets/voice.wav")
+_DEFAULT_INSTRUCTION = "You are a helpful assistant."
+
+
+def _normalize_transcript(raw: str) -> str:
+    """Make a reference transcript safe for CosyVoice3.
+
+    CosyVoice3 expects "<instruction><|endofprompt|><transcript-of-voice.wav>".
+    If the configured transcript supplies a bare transcript (no marker), prepend
+    the default instruction so the transcript lands AFTER <|endofprompt|> —
+    otherwise the LM treats it as an instruction and intermittently renders it as
+    speech (prompt leakage).
+    """
+    return (
+        raw
+        if "<|endofprompt|>" in raw
+        else f"{_DEFAULT_INSTRUCTION}<|endofprompt|>{raw}"
+    )
+
+
+def _add_cosyvoice_to_path(cosy_dir: str) -> None:
+    """Make a CosyVoice checkout importable: its root and Matcha-TTS submodule.
+
+    CosyVoice is not pip-installable, so `HURI_COSY_DIR` stands in for a
+    PYTHONPATH entry and the Serve config needs no absolute, per-host path.
+    """
+    for path in (cosy_dir, os.path.join(cosy_dir, "third_party", "Matcha-TTS")):
+        path = os.path.abspath(path)
+        if os.path.isdir(path) and path not in sys.path:
+            sys.path.insert(0, path)
+
+
+_END_TEXT = object()  # sentinel pushed into the text queue to close synth
+_END_AUDIO = object()  # sentinel pushed into the audio queue when synth completes
+_DONE = object()  # sentinel for exhausted sync generator
+
+
+@serve.deployment(name="CosyVoiceTTSHandle", max_ongoing_requests=200)
+class CosyVoiceTTSHandle:
+    """CosyVoice3 wrapper with per-session bistream synthesis.
+
+    The model's `inference_zero_shot` accepts a Python generator as `tts_text`
+    and yields audio chunks as text arrives — that's the "bistream" mode.
+    Because the model call is fully synchronous, each session runs in a thread
+    via `run_in_executor` and is fed by a thread-safe `queue.Queue` that the
+    asyncio side pushes text into.
+    """
+
+    def __init__(
+        self,
+        model_path: str = _MODEL_PATH,
+        voice_sample_path: str = _VOICE_SAMPLE_PATH,
+        voice_sample_transcript: Optional[str] = None,
+    ):
+        cosy_dir = os.environ.get("HURI_COSY_DIR")
+        if cosy_dir:
+            _add_cosyvoice_to_path(cosy_dir)
+
+        from cosyvoice.cli.cosyvoice import CosyVoice3
+
+        # Resolve the reference transcript here (deploy time on the GPU worker)
+        # rather than at module import: importing this module must not require
+        # HURI_VOICE_TRANSCRIPT, since modules.py imports it inside a broad
+        # try/except that would otherwise make TTS silently vanish from the
+        # pipeline when the var is unset. Fail loudly and locally instead.
+        if voice_sample_transcript is None:
+            raw = os.environ.get("HURI_VOICE_TRANSCRIPT")
+            if not raw:
+                raise RuntimeError(
+                    "HURI_VOICE_TRANSCRIPT is not set. The TTS deployment needs the "
+                    "transcript of the reference voice sample (voice.wav). Set it in "
+                    "the Serve app runtime_env.env_vars (see deploy values.yaml)."
+                )
+            voice_sample_transcript = raw
+        voice_sample_transcript = _normalize_transcript(voice_sample_transcript)
+
+        # fp16 is the whole point of running on a bandwidth-rich GPU (e.g. V100):
+        # it routes CosyVoice's LM/flow/vocoder matmuls through the fp16 tensor
+        # cores and halves memory traffic. CosyVoice3 defaults fp16=False (fp32),
+        # which leaves that advantage entirely unused. Default ON when a CUDA
+        # device is present; override with HURI_TTS_FP16 (fp16 needs CUDA, so it
+        # is forced off on CPU regardless).
+        import torch
+
+        _fp16_env = os.environ.get("HURI_TTS_FP16")
+        if _fp16_env is not None:
+            fp16 = _fp16_env.strip().lower() in ("1", "true", "yes", "on")
+        else:
+            fp16 = torch.cuda.is_available()
+        if fp16 and not torch.cuda.is_available():
+            print("[TTS] fp16 requested but no CUDA device; falling back to fp32")
+            fp16 = False
+
+        # TensorRT accelerates the flow-matching estimator (often the single
+        # biggest TTS speedup) but builds a device-specific engine at startup
+        # (~minutes) and needs validation per GPU arch. OFF by default; flip
+        # HURI_TTS_TRT=1 to experiment once fp16 is confirmed working.
+        load_trt = os.environ.get("HURI_TTS_TRT", "").strip().lower() in (
+            "1",
+            "true",
+            "yes",
+            "on",
+        )
+
+        print(
+            f"[TTS] loading CosyVoice3 (fp16={fp16}, load_trt={load_trt}) "
+            f"from {model_path!r}"
+        )
+        self.model = CosyVoice3(model_dir=model_path, load_trt=load_trt, fp16=fp16)
+        self.sample_rate: int = self.model.sample_rate
+
+        self.prompt_speech = voice_sample_path
+        self.prompt_text: str = voice_sample_transcript
+
+        self._text_queues: dict[str, queue.Queue] = {}
+
+        # Pay CosyVoice's first-synth costs now, at deploy time, rather than on
+        # the first user utterance (see _warmup).
+        self._warmup()
+
+    def _warmup(self) -> None:
+        """Run one throwaway synthesis so the first real utterance is hot.
+
+        The first CosyVoice call pays one-time costs — CUDA context init,
+        cuBLAS/cuDNN algorithm selection, the LM/flow/vocoder first-call
+        compiles, and the caching allocator's first growth — that otherwise land
+        on the first user utterance and stall it for seconds. Draining a dummy
+        synth through the *real* fp16 path (same prompt, stream=True) pays them
+        upfront, mirroring the MOV module's warmup.
+
+        Best-effort: never fatal. The reference sample (voice.wav) is uploaded to
+        its PVC *after* the worker starts, so on a brand-new volume it may be
+        absent on first boot — warmup is skipped then and runs on the next
+        restart once the PVC holds the file.
+        """
+        import time
+
+        if not os.path.isfile(self.prompt_speech):
+            print(
+                f"[TTS] warmup skipped: reference sample {self.prompt_speech!r} "
+                "not present yet (upload voice.wav, then restart to warm)"
+            )
+            return
+        try:
+            t0 = time.time()
+
+            def _dummy_text():
+                yield "Hello, this is a warm up."
+
+            audio_iter = self.model.inference_zero_shot(
+                _dummy_text(),
+                self.prompt_text,
+                self.prompt_speech,
+                stream=True,
+            )
+            chunks = sum(1 for _ in audio_iter)
+            print(f"[TTS] warmup done ({chunks} chunks) in {time.time() - t0:.2f}s")
+        except Exception as e:  # noqa: BLE001 — warmup is an optimisation, never fatal
+            print(f"[TTS] WARNING warmup failed: {e!r}")
+
+    async def get_sample_rate(self) -> int:
+        return self.sample_rate
+
+    async def start_session(self, session_id: str) -> None:
+        self._text_queues[session_id] = queue.Queue()
+
+    async def push_text(self, session_id: str, text: str, end: bool) -> None:
+        q = self._text_queues.get(session_id)
+        if q is None:
+            return
+        if text:
+            q.put(text)
+        if end:
+            q.put(_END_TEXT)
+
+    async def stream_audio(self, session_id: str) -> AsyncGenerator[Audio, None]:
+        text_q = self._text_queues[session_id]
+        loop = asyncio.get_running_loop()
+        chunk_count = 0
+
+        def text_gen():
+            while True:
+                item = text_q.get()
+                if item is _END_TEXT:
+                    return
+                yield item
+
+        try:
+            audio_iter = self.model.inference_zero_shot(
+                text_gen(),
+                self.prompt_text,
+                self.prompt_speech,
+                stream=True,
+            )
+            while True:
+                result = await loop.run_in_executor(None, next, audio_iter, _DONE)
+                if result is _DONE:
+                    break
+                assert isinstance(result, dict)
+                chunk_count += 1
+                speech = result["tts_speech"].squeeze(0).numpy().astype(np.float32)
+                yield Audio(data=speech, sample_rate=self.sample_rate)
+        except Exception:
+            traceback.print_exc()
+            raise
+        finally:
+            self._text_queues.pop(session_id, None)

huri_module_tts-0.1.0/huri_tts/cosyvoice/module.py

@@ -0,0 +1,11 @@
+from huri_tts.cosyvoice.handle import CosyVoiceTTSHandle
+from huri_tts.streaming import StreamingTTS
+
+
+class CosyVoiceTTS(StreamingTTS):
+    """Text to speech with CosyVoice3: zero-shot voice cloning from a reference
+    sample (HURI_VOICE_SAMPLE_PATH + HURI_VOICE_TRANSCRIPT). Needs an NVIDIA GPU
+    to run in real time, and the CosyVoice sources on PYTHONPATH
+    (install_cosyvoice.sh)."""
+
+    _handle_cls = CosyVoiceTTSHandle

huri_module_tts-0.1.0/huri_tts/manifests/tts/huri_module.toml

@@ -0,0 +1,7 @@
+name = "tts"
+summary = "Speaks the reply with Piper (ONNX, CPU, ~30x realtime)"
+input = { topic = "huri.token", event = "huri_events.text:Token" }
+output = { topic = "huri.audio.out", event = "huri_events.audio:Audio" }
+events = ["huri-events>=0.1,<0.2"]
+requires = { gpu = false, actuation = false }
+huri_core = ">=0.4,<0.5"

huri_module_tts-0.1.0/huri_tts/manifests/tts_cosyvoice/huri_module.toml

@@ -0,0 +1,7 @@
+name = "tts_cosyvoice"
+summary = "Speaks the reply in a cloned voice with CosyVoice3"
+input = { topic = "huri.token", event = "huri_events.text:Token" }
+output = { topic = "huri.audio.out", event = "huri_events.audio:Audio" }
+events = ["huri-events>=0.1,<0.2"]
+requires = { gpu = true, actuation = false }
+huri_core = ">=0.4,<0.5"

huri_module_tts-0.1.0/huri_tts/piper/handle.py

@@ -0,0 +1,182 @@
+"""Piper text-to-speech deployment — the portable TTS engine.
+
+Why this exists alongside CosyVoice3 (huri_tts.cosyvoice, module `tts_cosyvoice`):
+
+CosyVoice gives zero-shot voice cloning but needs torch plus a working
+CUDA/ROCm stack, and on CPU it synthesises at roughly 7x *slower* than
+realtime — unusable in a conversation. It also cannot run on consumer RDNA3
+GPUs at all (the HiFi-GAN vocoder faults inside a MIOpen convolution), so on
+an AMD machine there is no fast path.
+
+Piper is ONNX-only: no torch, no GPU, a 61 MB voice model, and measured at
+~0.03x realtime on a Ryzen 7840HS — about 30x faster than it speaks, with the
+first chunk in ~30 ms. That makes it the engine that works on *any* machine,
+which is why it is the default. CosyVoice (huri_tts.cosyvoice) is the
+other engine, for when an NVIDIA GPU is present and the cloned voice matters
+more than latency: a separate module, `tts_cosyvoice`, wired in its place.
+
+The public surface is the one huri_tts.streaming.StreamingTTS drives —
+``get_sample_rate`` / ``start_session`` / ``push_text`` / ``stream_audio``. The
+deployment keeps the name ``TTSHandle``, so existing ``deployments:`` blocks
+still apply to the default engine.
+"""
+
+import asyncio
+import os
+import re
+import traceback
+from typing import AsyncGenerator, Dict, List
+
+import numpy as np
+from huri_events.audio import Audio
+from ray import serve
+
+_VOICE_PATH = os.environ.get(
+    "HURI_PIPER_VOICE", "assets/models/piper/en_US-lessac-medium.onnx"
+)
+# Sentence-ish boundaries. Piper synthesises a whole utterance at a time, so we
+# flush on these to start speaking before the LLM has finished writing.
+_BOUNDARY = re.compile(r"[.!?…]['\"\)\]]?\s")
+# Don't flush on a boundary this early — "Dr." or "1." would cut the phrase into
+# unnatural fragments, and Piper's prosody is better with more context.
+_MIN_FLUSH_CHARS = 24
+
+
+@serve.deployment(name="TTSHandle")
+class PiperTTSHandle:
+    """Piper wrapper with per-session incremental synthesis.
+
+    Stateless across sessions apart from the per-session text buffer: the ONNX
+    session is shared and thread-safe for inference, so one replica serves every
+    client.
+    """
+
+    def __init__(
+        self,
+        voice_path: str = _VOICE_PATH,
+        length_scale: float | None = None,
+        volume: float | None = None,
+    ):
+        from piper import PiperVoice
+
+        if not os.path.isfile(voice_path):
+            raise RuntimeError(
+                f"[PiperTTS] voice model not found: {voice_path!r}. "
+                "Download one with: python -m piper.download_voices "
+                "--download-dir assets/models/piper en_US-lessac-medium"
+            )
+
+        self._voice = PiperVoice.load(voice_path)
+        self._syn_config = self._build_syn_config(length_scale, volume)
+        # Probe the real output rate rather than trusting a constant: it differs
+        # per voice (22.05 kHz for -medium, 16 kHz for some -low voices) and the
+        # client resamples against whatever we report.
+        self._sample_rate = int(self._voice.config.sample_rate)
+        self._buffers: Dict[str, List[str]] = {}
+        self._queues: Dict[str, asyncio.Queue] = {}
+        print(
+            f"[PiperTTS] loaded {os.path.basename(voice_path)} "
+            f"@ {self._sample_rate}Hz (onnxruntime, CPU)"
+        )
+
+    @staticmethod
+    def _build_syn_config(length_scale: float | None, volume: float | None):
+        """None when nothing is overridden, so Piper uses the voice's own defaults."""
+
+        from piper import SynthesisConfig
+
+        if length_scale is None and volume is None:
+            return None
+        # Fields set individually rather than via **kwargs: SynthesisConfig mixes
+        # int/bool/float fields, so a kwargs dict cannot be type-checked.
+        syn_config = SynthesisConfig()
+        if length_scale is not None:
+            syn_config.length_scale = length_scale
+        if volume is not None:
+            syn_config.volume = volume
+        return syn_config
+
+    async def get_sample_rate(self) -> int:
+        return self._sample_rate
+
+    async def start_session(self, session_id: str) -> None:
+        self._buffers[session_id] = []
+        self._queues[session_id] = asyncio.Queue()
+
+    async def push_text(self, session_id: str, text: str, end: bool) -> None:
+        """Accumulate LLM tokens, flushing complete phrases for synthesis.
+
+        Unlike CosyVoice there is no bistream mode to push tokens into, so the
+        phrase boundary is ours to choose. Flushing per sentence keeps the
+        time-to-first-audio low while still giving Piper enough context for
+        sensible prosody.
+        """
+        buf = self._buffers.setdefault(session_id, [])
+        queue = self._queues.setdefault(session_id, asyncio.Queue())
+        if text:
+            buf.append(text)
+
+        joined = "".join(buf)
+        if not end:
+            # Only flush on a boundary that is followed by whitespace, so a
+            # decimal point or an abbreviation does not split the phrase.
+            match = None
+            for m in _BOUNDARY.finditer(joined):
+                match = m
+            if match is None or match.end() < _MIN_FLUSH_CHARS:
+                return
+            phrase, remainder = joined[: match.end()], joined[match.end() :]
+            buf.clear()
+            if remainder:
+                buf.append(remainder)
+        else:
+            phrase, _ = joined, ""
+            buf.clear()
+
+        phrase = phrase.strip()
+        if phrase:
+            await queue.put(phrase)
+        if end:
+            await queue.put(None)  # end-of-utterance sentinel
+
+    async def stream_audio(self, session_id: str) -> AsyncGenerator[Audio, None]:
+        queue = self._queues.setdefault(session_id, asyncio.Queue())
+        loop = asyncio.get_running_loop()
+        pts = 0.0
+        try:
+            while True:
+                phrase = await queue.get()
+                if phrase is None:
+                    # Zero-length end marker, matching TTSHandle's contract.
+                    yield Audio(
+                        data=np.zeros(0, dtype=np.float32),
+                        sample_rate=self._sample_rate,
+                        end=True,
+                        pts=pts,
+                    )
+                    return
+                # Synthesis is CPU-bound C++ inside onnxruntime; run it off the
+                # event loop so the replica keeps serving other sessions.
+                chunks = await loop.run_in_executor(None, self._synthesize, phrase)
+                for samples in chunks:
+                    if samples.size == 0:
+                        continue
+                    yield Audio(
+                        data=samples,
+                        sample_rate=self._sample_rate,
+                        end=False,
+                        pts=pts,
+                    )
+                    pts += samples.size / self._sample_rate
+        except Exception:
+            traceback.print_exc()
+            raise
+        finally:
+            self._buffers.pop(session_id, None)
+            self._queues.pop(session_id, None)
+
+    def _synthesize(self, phrase: str) -> List[np.ndarray]:
+        out: List[np.ndarray] = []
+        for chunk in self._voice.synthesize(phrase, syn_config=self._syn_config):
+            out.append(np.asarray(chunk.audio_float_array, dtype=np.float32))
+        return out

huri_module_tts-0.1.0/huri_tts/piper/module.py

@@ -0,0 +1,11 @@
+from huri_tts.piper.handle import PiperTTSHandle
+from huri_tts.streaming import StreamingTTS
+
+
+class PiperTTS(StreamingTTS):
+    """Text to speech with Piper (ONNX): CPU-only, ~30x faster than realtime.
+
+    The default engine: it runs on any machine. See handle.py for why.
+    """
+
+    _handle_cls = PiperTTSHandle

huri_module_tts-0.1.0/huri_tts/streaming.py

@@ -0,0 +1,130 @@
+"""The engine-agnostic half of HuRI's text-to-speech modules."""
+
+import asyncio
+import uuid
+from typing import TYPE_CHECKING, Any, AsyncGenerator
+
+import numpy as np
+from huri_events.audio import Audio
+from huri_events.text import Token
+
+from huri_core.core.module import ModuleWithHandle
+
+if TYPE_CHECKING:
+    from ray.serve import handle
+
+_END_AUDIO = object()  # sentinel pushed into the audio queue when synthesis ends
+
+
+class StreamingTTS(ModuleWithHandle):
+    """Streaming TTS module — bistream tokens-in / audio-out, engine-agnostic.
+
+    Opens one synthesis session per utterance (delimited by `token.end`). Each
+    incoming token is pushed straight into the model's text generator so audio
+    starts coming back before the LLM has finished producing the response.
+    No clause buffering on our side — CosyVoice's frontend handles segmentation
+    and stitches LM calls together across the whole utterance.
+
+    It drives any deployment exposing ``get_sample_rate()``,
+    ``start_session(sid)``, ``push_text(sid, text, end)`` and
+    ``stream_audio(sid)`` (an async generator of Audio); an engine package
+    subclasses it and sets ``_handle_cls`` to its deployment.
+
+    input: huri.token (Token)
+    output: huri.audio.out (Audio)
+    """
+
+    input_type = "huri.token"
+    output_type = "huri.audio.out"
+    input_event = Token
+    output_event = Audio
+
+    def __init__(self, _handle: "handle.DeploymentHandle"):
+        super().__init__(_handle)
+        self._session_id: str | None = None
+        self._audio_q: asyncio.Queue | None = None
+        self._stream_task: asyncio.Task | None = None
+        # The EventGraph fans each token out as its own concurrent process()
+        # task on this shared instance. This lock serialises session setup and
+        # text pushes so tokens reach CosyVoice's text queue in arrival order,
+        # exactly once. asyncio.Lock wakes waiters FIFO and tokens are created
+        # in order, so order is preserved — crucially the end-of-utterance token
+        # can no longer overtake a content token (which would truncate synthesis
+        # and silently drop trailing words).
+        self._push_lock = asyncio.Lock()
+
+    async def process(self, token: Token) -> AsyncGenerator[Audio, None]:
+        # Acquire BEFORE any await so lock-acquisition order matches token order.
+        # Setup + push happen under the lock; only the first token of an
+        # utterance goes on to drain/yield audio (outside the lock, so pushes of
+        # later tokens are never blocked by the long-running drain).
+        async with self._push_lock:
+            is_first = self._session_id is None
+            if is_first:
+                self._session_id = str(uuid.uuid4())
+                self._audio_q = asyncio.Queue()
+                print(
+                    f"[TTS-client] [{self._session_id}] opening new utterance session"
+                )
+                await self._handle.start_session.remote(self._session_id)
+                self._stream_task = asyncio.create_task(
+                    self._drain_audio(self._session_id, self._audio_q)
+                )
+
+            sid = self._session_id
+            audio_q = self._audio_q
+            stream_task = self._stream_task
+            print(f"[TTS-client] [{sid}] push token: {token.text!r} (end={token.end})")
+            await self._handle.push_text.remote(sid, token.text, token.end)
+
+        if not is_first:
+            return
+
+        assert audio_q is not None and stream_task is not None
+        try:
+            count = 0
+            while True:
+                item = await audio_q.get()
+                if item is _END_AUDIO:
+                    break
+                count += 1
+                print(f"[TTS-client] [{sid}] yield chunk #{count}")
+                yield item
+            await stream_task
+            print(f"[TTS-client] [{sid}] utterance complete ({count} chunks)")
+
+            sample_rate = await self._handle.get_sample_rate.remote()
+            yield Audio(
+                data=np.array([], dtype=np.float32), sample_rate=sample_rate, end=True
+            )
+        finally:
+            async with self._push_lock:
+                self._session_id = None
+                self._audio_q = None
+                self._stream_task = None
+
+    async def _drain_audio(self, session_id: str, audio_q: asyncio.Queue) -> None:
+        try:
+            # Any: the Ray response type differs with/without ray installed
+            # (CI lints without it), so a `type: ignore` would flip between
+            # needed and unused.
+            response: Any = self._handle.options(stream=True).stream_audio.remote(
+                session_id
+            )
+            count = 0
+            pts = 0.0
+            async for audio in response:
+                count += 1
+                audio.pts = pts
+                pts += audio.data.shape[0] / audio.sample_rate
+                print(
+                    f"[TTS-client] [{session_id}] drain received chunk #{count} "
+                    f"pts={audio.pts:.3f}s next={pts:.3f}s",
+                )
+                await audio_q.put(audio)
+        except Exception as e:
+            print(f"[TTS-client] [{session_id}] drain task FAILED: {e!r}")
+            raise
+        finally:
+            await audio_q.put(_END_AUDIO)
+            print(f"[TTS-client] [{session_id}] drain task finished")

huri_module_tts-0.1.0/pyproject.toml

@@ -0,0 +1,41 @@
+[build-system]
+requires = ["setuptools>=64,<82"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "huri-module-tts"
+version = "0.1.0"
+description = "HuRI modules: text to speech with Piper (CPU, any machine) or CosyVoice3 (voice cloning, NVIDIA GPU)"
+readme = "README.md"
+requires-python = ">=3.10"
+# The streaming logic both engines share. No engine library here: each comes
+# with its extra, so a host that runs one engine never installs the other's
+# stack (piper's onnxruntime and CosyVoice's onnxruntime-gpu do not mix).
+dependencies = [
+  "huri-core[server]>=0.4,<0.5",
+  "huri-events>=0.1,<0.2",
+  "numpy>=1.26,<2",
+]
+
+[project.optional-dependencies]
+piper = ["piper-tts>=1.8,<1.9"]
+# CosyVoice itself is not pip-installable (install_cosyvoice.sh); its hardware
+# pins are requirements-<variant>.tts_cosyvoice.txt.
+cosyvoice = ["torch", "torchaudio"]
+
+[project.entry-points."huri.modules"]
+tts = "huri_tts.piper.module:PiperTTS"
+tts_cosyvoice = "huri_tts.cosyvoice.module:CosyVoiceTTS"
+
+# The extra each module needs: read by huri-modules' tooling
+# (scripts/module_requirements.py, check_registry.py against registry.toml's
+# `extra`), not by huri-core.
+[tool.huri-modules.extras]
+tts = "piper"
+tts_cosyvoice = "cosyvoice"
+
+[tool.setuptools.packages.find]
+include = ["huri_tts*"]
+
+[tool.setuptools.package-data]
+huri_tts = ["manifests/*/huri_module.toml"]

huri_module_tts-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

huri_module_tts-0.1.0/tests/test_cosyvoice.py

@@ -0,0 +1,44 @@
+"""What can be checked of CosyVoice without its sources and model."""
+
+import sys
+
+from huri_tts.cosyvoice.handle import (
+    CosyVoiceTTSHandle,
+    _add_cosyvoice_to_path,
+    _normalize_transcript,
+)
+from huri_tts.cosyvoice.module import CosyVoiceTTS
+
+
+def test_a_bare_transcript_is_placed_after_the_prompt_marker():
+    # Before the marker the LM reads it as an instruction and may speak it.
+    bare = "Instinct creates its own oppressors."
+    assert _normalize_transcript(bare) == (
+        f"You are a helpful assistant.<|endofprompt|>{bare}"
+    )
+
+
+def test_an_explicit_instruction_is_kept():
+    raw = "Speak softly.<|endofprompt|>Hello."
+    assert _normalize_transcript(raw) == raw
+
+
+def test_the_module_binds_its_own_deployment():
+    assert CosyVoiceTTS._handle_cls is CosyVoiceTTSHandle
+    # Not "TTSHandle": both engines can then run in one Serve app.
+    assert CosyVoiceTTSHandle.name == "CosyVoiceTTSHandle"
+    assert (CosyVoiceTTS.input_type, CosyVoiceTTS.output_type) == (
+        "huri.token",
+        "huri.audio.out",
+    )
+
+
+def test_cosy_dir_puts_the_checkout_and_matcha_on_the_path(tmp_path, monkeypatch):
+    # The Serve configs set HURI_COSY_DIR only, no per-host PYTHONPATH.
+    matcha = tmp_path / "third_party" / "Matcha-TTS"
+    matcha.mkdir(parents=True)
+    monkeypatch.setattr(sys, "path", list(sys.path))
+    _add_cosyvoice_to_path(str(tmp_path))
+    _add_cosyvoice_to_path(str(tmp_path))
+    assert sys.path.count(str(tmp_path)) == 1
+    assert sys.path.count(str(matcha)) == 1

huri_module_tts-0.1.0/tests/test_piper.py

@@ -0,0 +1,78 @@
+"""Piper's deployment logic, with the ONNX voice replaced by a fake."""
+
+import asyncio
+from types import SimpleNamespace
+
+import numpy as np
+from huri_events.text import Token
+from huri_tts.piper.handle import PiperTTSHandle
+from huri_tts.piper.module import PiperTTS
+
+from huri_core.testing import LocalHandle, ModuleHarness
+
+
+class FakeVoice:
+    """10 samples per character, in two chunks per phrase."""
+
+    def synthesize(self, phrase, syn_config=None):
+        half = len(phrase) * 5
+        for _ in range(2):
+            yield SimpleNamespace(audio_float_array=np.ones(half, dtype=np.float32))
+
+
+def handle():
+    deployment = PiperTTSHandle.func_or_class.__new__(PiperTTSHandle.func_or_class)
+    deployment._voice = FakeVoice()
+    deployment._syn_config = None
+    deployment._sample_rate = 22050
+    deployment._buffers, deployment._queues = {}, {}
+    return deployment
+
+
+async def phrases(h, tokens):
+    await h.start_session("s")
+    for text, end in tokens:
+        await h.push_text("s", text, end)
+    queue, out = h._queues["s"], []
+    while not queue.empty():
+        out.append(await queue.get())
+    return out
+
+
+def test_tokens_are_flushed_by_sentence_once_long_enough():
+    tokens = [
+        ("Dr. Smith is here. ", False),
+        ("He says hello to everyone! ", False),
+        ("Bye", False),
+        ("", True),
+    ]
+    assert asyncio.run(phrases(handle(), tokens)) == [
+        "Dr. Smith is here. He says hello to everyone!",
+        "Bye",
+        None,  # end of utterance
+    ]
+
+
+def test_streamed_audio_is_timed_and_ends_with_a_marker():
+    async def run():
+        h = handle()
+        await h.start_session("s")
+        await h.push_text("s", "Hello there.", True)
+        return [a async for a in h.stream_audio("s")]
+
+    audio = asyncio.run(run())
+    assert [a.data.size for a in audio] == [60, 60, 0]
+    assert [a.pts for a in audio] == [0.0, 60 / 22050, 120 / 22050]
+    assert audio[-1].end and not audio[0].end
+
+
+def test_the_module_drives_the_deployment_end_to_end():
+    harness = ModuleHarness(PiperTTS, handle=LocalHandle(handle()))
+
+    async def speak():
+        tokens = [Token("Hello there, this is HuRI. ", False), Token("", True)]
+        return await asyncio.gather(*(harness.feed(t) for t in tokens))
+
+    first, rest = asyncio.run(speak())
+    assert rest == [] and first[-1].end
+    assert sum(a.data.size for a in first) == len("Hello there, this is HuRI.") * 10

huri_module_tts-0.1.0/tests/test_streaming_tts.py

@@ -0,0 +1,75 @@
+"""StreamingTTS on a fake engine, fed the way the bus feeds it: one concurrent
+process() call per token."""
+
+import asyncio
+
+import numpy as np
+from huri_events.audio import Audio
+from huri_events.text import Token
+from huri_tts.streaming import StreamingTTS
+
+from huri_core.testing import LocalHandle, ModuleHarness
+
+
+class FakeEngine:
+    """The four methods StreamingTTS drives; 10 samples per pushed character."""
+
+    def __init__(self):
+        self.sessions = {}
+
+    async def get_sample_rate(self):
+        return 16000
+
+    async def start_session(self, sid):
+        self.sessions[sid] = {"texts": [], "done": asyncio.Event()}
+
+    async def push_text(self, sid, text, end):
+        await asyncio.sleep(0)  # let concurrent pushes interleave if they could
+        session = self.sessions[sid]
+        session["texts"].append(text)
+        if end:
+            session["done"].set()
+
+    async def stream_audio(self, sid):
+        session = self.sessions[sid]
+        await session["done"].wait()
+        for text in session["texts"]:
+            if text:
+                yield Audio(np.ones(len(text) * 10, dtype=np.float32), 16000)
+
+
+class Engine(StreamingTTS):
+    _handle_cls = FakeEngine
+
+
+async def speak(harness, tokens):
+    return await asyncio.gather(*(harness.feed(t) for t in tokens))
+
+
+def test_an_utterance_streams_in_token_order_and_ends_with_a_marker():
+    engine = FakeEngine()
+    harness = ModuleHarness(Engine, handle=LocalHandle(engine))
+    tokens = [Token("Hello", False), Token(" there", False), Token("", True)]
+    first, *rest = asyncio.run(speak(harness, tokens))
+
+    assert rest == [[], []]  # only the utterance's first call yields audio
+    (session,) = engine.sessions.values()
+    assert session["texts"] == ["Hello", " there", ""]
+    assert [chunk.data.size for chunk in first] == [50, 60, 0]
+    assert [chunk.pts for chunk in first[:2]] == [0.0, 50 / 16000]
+    assert first[-1].end and first[-1].sample_rate == 16000
+
+
+def test_each_utterance_gets_its_own_session():
+    engine = FakeEngine()
+    harness = ModuleHarness(Engine, handle=LocalHandle(engine))
+
+    async def two_utterances():  # one event loop, like the server
+        await speak(harness, [Token("One.", False), Token("", True)])
+        await speak(harness, [Token("Two.", False), Token("", True)])
+
+    asyncio.run(two_utterances())
+    assert [s["texts"] for s in engine.sessions.values()] == [
+        ["One.", ""],
+        ["Two.", ""],
+    ]