pipecat-mirai 0.1.0__tar.gz

pipecat_mirai-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,31 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python: ["3.11", "3.12"]
+        pipecat: ["1.8.1", "latest"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+      - name: Install
+        run: |
+          uv venv --python ${{ matrix.python }}
+          if [ "${{ matrix.pipecat }}" = "latest" ]; then PIPECAT="pipecat-ai[websocket]"; else PIPECAT="pipecat-ai[websocket]==${{ matrix.pipecat }}"; fi
+          uv pip install -e . "$PIPECAT" pytest pytest-asyncio ruff
+      - name: Lint
+        run: |
+          .venv/bin/ruff check .
+          .venv/bin/ruff format --check src tests examples benchmarks
+      - name: Test
+        run: .venv/bin/python -m pytest -q
+      - name: Examples compile
+        run: .venv/bin/python -m py_compile examples/foundational/01-say-hello.py examples/phone/twilio_bot.py benchmarks/phone-breaks/phone_breaks.py

pipecat_mirai-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,49 @@
+name: Release
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+      - name: Check tag matches version
+        run: |
+          VERSION=$(grep -m1 '^version = ' pyproject.toml | cut -d'"' -f2)
+          test "v$VERSION" = "${GITHUB_REF_NAME}" || { echo "tag ${GITHUB_REF_NAME} != v$VERSION"; exit 1; }
+      - run: uv build
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write   # PyPI trusted publishing; no API token stored in GitHub
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1
+
+  github-release:
+    needs: publish
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - env:
+          GH_TOKEN: ${{ github.token }}
+        run: gh release create "$GITHUB_REF_NAME" dist/* --title "$GITHUB_REF_NAME" --notes "See CHANGELOG.md"

pipecat_mirai-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+.venv/
+__pycache__/
+*.pyc
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+*.wav
+results/

pipecat_mirai-0.1.0/CHANGELOG.md

@@ -0,0 +1,23 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
+[Semantic Versioning](https://semver.org/).
+
+## [0.1.0] - 2026-10-02
+
+### Added
+
+- `MiraiTTSService`: streaming Mirai text-to-speech for Pipecat. Audio is
+  resampled from Mirai's 48 kHz to the pipeline's output rate per utterance, and
+  network reads that end mid-sample are handled. It reports TTFB and usage
+  metrics, supports tracing and `TTSUpdateSettingsFrame`, and closes the HTTP
+  stream on interruption.
+- `apply_output_lead()`: lets Pipecat's websocket output transports (FastAPI,
+  websocket server and websocket client) send up to 0.4 s ahead of real time, so
+  event-loop stalls no longer break audio on phone calls.
+- Examples: a foundational speech check and a Twilio phone bot.
+- `benchmarks/phone-breaks`: a reproducible harness that measures breaks on phone
+  calls under event-loop stalls.
+
+Tested with Pipecat 1.8.1 and 1.12.0.

pipecat_mirai-0.1.0/LICENSE

@@ -0,0 +1,24 @@
+BSD 2-Clause License
+
+Copyright (c) 2026, Sona Labs Pvt Ltd
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

pipecat_mirai-0.1.0/PKG-INFO

@@ -0,0 +1,140 @@
+Metadata-Version: 2.5
+Name: pipecat-mirai
+Version: 0.1.0
+Summary: Mirai text-to-speech (Hindi, Hinglish, Gujarati) for Pipecat voice agents
+Project-URL: Homepage, https://github.com/MiraiMinds/pipecat-mirai
+Project-URL: Documentation, https://docs.miraiminds.co
+Project-URL: Issues, https://github.com/MiraiMinds/pipecat-mirai/issues
+Author-email: Sona Labs Pvt Ltd <sneh.mehta@miraiminds.co>
+License-Expression: BSD-2-Clause
+License-File: LICENSE
+Keywords: hindi,mirai,pipecat,telephony,text-to-speech,tts,voice-agent
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Requires-Python: >=3.11
+Requires-Dist: httpx<1,>=0.27
+Requires-Dist: numpy>=1.26
+Requires-Dist: pipecat-ai<2,>=1.8.1
+Requires-Dist: soxr>=0.3
+Description-Content-Type: text/markdown
+
+# pipecat-mirai
+
+[Mirai](https://miraiminds.co) text-to-speech for [Pipecat](https://github.com/pipecat-ai/pipecat)
+voice agents: natural Hindi, Hinglish and Gujarati voices, streamed with about
+100 ms to first audio.
+
+- **`MiraiTTSService`**: a Pipecat TTS service for Mirai's streaming API. Audio is
+  resampled to your pipeline's rate (8 kHz for phone calls), with no voice
+  registration or sample-rate workarounds.
+- **`apply_output_lead()`**: stops audio breaking up on phone calls when your
+  server is busy (see [Phone calls](#phone-calls)). This works with any TTS service.
+
+## Installation
+
+```bash
+uv add pipecat-mirai     # or: pip install pipecat-mirai
+```
+
+## Prerequisites
+
+- A Mirai API key (`sk_live_…`) from the [Mirai console](https://sandbox.voice.miraiminds.co).
+  Set it as `MIRAI_API_KEY`, or pass `api_key=`.
+- Pipecat 1.8.1 or newer and Python 3.11+.
+
+## Usage
+
+```python
+from pipecat_mirai import MiraiTTSService
+
+tts = MiraiTTSService(settings=MiraiTTSService.Settings(voice="shruti"))
+
+pipeline = Pipeline([transport.input(), stt, user_aggregator, llm, tts, transport.output(), assistant_aggregator])
+```
+
+| Voice | |
+|---|---|
+| `ashu` | male |
+| `neha` | female (default) |
+| `shruti` | female |
+| `sameer` | male |
+
+Write Hindi in Devanagari and English words in Latin script, as people actually
+type Hinglish: `आपका order कल deliver होगा।`
+
+Change the voice mid-call with Pipecat's standard settings frame:
+
+```python
+await task.queue_frame(TTSUpdateSettingsFrame(delta=MiraiTTSService.Settings(voice="sameer")))
+```
+
+### Parameters
+
+| Argument | Default | |
+|---|---|---|
+| `api_key` | `$MIRAI_API_KEY` | Your Mirai API key |
+| `settings` | `voice="neha"`, `model="mira-tts"` | `MiraiTTSService.Settings(...)` |
+| `voice`, `model` | | Shortcuts for the same settings |
+| `sample_rate` | pipeline `audio_out_sample_rate` | Output rate; Mirai's 48 kHz audio is resampled to it |
+| `base_url` | `https://sandbox.voice.miraiminds.co/v1` | API base URL |
+| `http_client` | own client | An `httpx.AsyncClient` you manage |
+
+The service reports time-to-first-byte and character usage metrics and supports
+Pipecat tracing. Interrupting the bot closes the HTTP stream at once.
+
+## Phone calls
+
+Pipecat's websocket transports (`FastAPIWebsocketTransport` with Twilio, Plivo,
+Exotel or Telnyx serializers, `WebsocketServerTransport`, `WebsocketClientTransport`)
+send audio at exactly real time. The phone provider then never holds more than
+about 40 ms of audio. If your server's event loop stalls for longer (a blocking
+call, a heavy parse, many calls on one CPU), the provider runs out and the caller
+hears the voice break up. This happens with every TTS vendor, and it gets worse
+as you add concurrent calls.
+
+`apply_output_lead()` lets the transport send up to 0.4 s ahead, so short stalls
+go unnoticed:
+
+```python
+from pipecat_mirai import MiraiTTSService, apply_output_lead
+
+transport = FastAPIWebsocketTransport(websocket, FastAPIWebsocketParams(
+    audio_out_enabled=True, add_wav_header=False, serializer=serializer, ...))
+apply_output_lead(transport)              # default 0.4 s; apply_output_lead(transport, 0.6) for more
+```
+
+We measured this on 8 kHz Twilio-protocol calls, with Pipecat 1.8.1, real Mirai audio
+and 50–250 ms event-loop stalls every ~3 s per call
+([benchmarks/phone-breaks](benchmarks/phone-breaks)):
+
+| Concurrent calls | Stock Pipecat: speech stretched by breaks | With `apply_output_lead` |
+|---|---|---|
+| 3 | 6.6% (3.5 breaks per sentence) | **0.08%** |
+| 10 | 29.1% (14.5 breaks per sentence) | **0.01%** |
+
+When the caller interrupts, Pipecat still tells the provider to clear its queue, so
+at most the lead (0.4 s) of already-sent audio is discarded. One side effect:
+Pipecat's "bot stopped speaking" event fires up to the lead earlier than the caller
+actually stops hearing the bot.
+
+## Examples
+
+- [`examples/foundational/01-say-hello.py`](examples/foundational/01-say-hello.py):
+  a minimal Pipecat pipeline that speaks one line and saves `hello.wav`.
+- [`examples/phone/twilio_bot.py`](examples/phone/twilio_bot.py): a Twilio Media
+  Streams bot with `apply_output_lead`.
+
+## Compatibility
+
+Tested with Pipecat 1.8.1 and 1.12.0 on Python 3.11–3.12. `apply_output_lead` changes how
+Pipecat's websocket output transports time their writes. A test in this repo
+fails if a new Pipecat release changes that, and the function logs a warning and
+does nothing for transports it doesn't support.
+
+## License
+
+BSD-2-Clause. Maintained by [Mirai](https://miraiminds.co) (Sona Labs Pvt Ltd).
+This is a community integration, not maintained by the Pipecat team.

pipecat_mirai-0.1.0/README.md

@@ -0,0 +1,117 @@
+# pipecat-mirai
+
+[Mirai](https://miraiminds.co) text-to-speech for [Pipecat](https://github.com/pipecat-ai/pipecat)
+voice agents: natural Hindi, Hinglish and Gujarati voices, streamed with about
+100 ms to first audio.
+
+- **`MiraiTTSService`**: a Pipecat TTS service for Mirai's streaming API. Audio is
+  resampled to your pipeline's rate (8 kHz for phone calls), with no voice
+  registration or sample-rate workarounds.
+- **`apply_output_lead()`**: stops audio breaking up on phone calls when your
+  server is busy (see [Phone calls](#phone-calls)). This works with any TTS service.
+
+## Installation
+
+```bash
+uv add pipecat-mirai     # or: pip install pipecat-mirai
+```
+
+## Prerequisites
+
+- A Mirai API key (`sk_live_…`) from the [Mirai console](https://sandbox.voice.miraiminds.co).
+  Set it as `MIRAI_API_KEY`, or pass `api_key=`.
+- Pipecat 1.8.1 or newer and Python 3.11+.
+
+## Usage
+
+```python
+from pipecat_mirai import MiraiTTSService
+
+tts = MiraiTTSService(settings=MiraiTTSService.Settings(voice="shruti"))
+
+pipeline = Pipeline([transport.input(), stt, user_aggregator, llm, tts, transport.output(), assistant_aggregator])
+```
+
+| Voice | |
+|---|---|
+| `ashu` | male |
+| `neha` | female (default) |
+| `shruti` | female |
+| `sameer` | male |
+
+Write Hindi in Devanagari and English words in Latin script, as people actually
+type Hinglish: `आपका order कल deliver होगा।`
+
+Change the voice mid-call with Pipecat's standard settings frame:
+
+```python
+await task.queue_frame(TTSUpdateSettingsFrame(delta=MiraiTTSService.Settings(voice="sameer")))
+```
+
+### Parameters
+
+| Argument | Default | |
+|---|---|---|
+| `api_key` | `$MIRAI_API_KEY` | Your Mirai API key |
+| `settings` | `voice="neha"`, `model="mira-tts"` | `MiraiTTSService.Settings(...)` |
+| `voice`, `model` | | Shortcuts for the same settings |
+| `sample_rate` | pipeline `audio_out_sample_rate` | Output rate; Mirai's 48 kHz audio is resampled to it |
+| `base_url` | `https://sandbox.voice.miraiminds.co/v1` | API base URL |
+| `http_client` | own client | An `httpx.AsyncClient` you manage |
+
+The service reports time-to-first-byte and character usage metrics and supports
+Pipecat tracing. Interrupting the bot closes the HTTP stream at once.
+
+## Phone calls
+
+Pipecat's websocket transports (`FastAPIWebsocketTransport` with Twilio, Plivo,
+Exotel or Telnyx serializers, `WebsocketServerTransport`, `WebsocketClientTransport`)
+send audio at exactly real time. The phone provider then never holds more than
+about 40 ms of audio. If your server's event loop stalls for longer (a blocking
+call, a heavy parse, many calls on one CPU), the provider runs out and the caller
+hears the voice break up. This happens with every TTS vendor, and it gets worse
+as you add concurrent calls.
+
+`apply_output_lead()` lets the transport send up to 0.4 s ahead, so short stalls
+go unnoticed:
+
+```python
+from pipecat_mirai import MiraiTTSService, apply_output_lead
+
+transport = FastAPIWebsocketTransport(websocket, FastAPIWebsocketParams(
+    audio_out_enabled=True, add_wav_header=False, serializer=serializer, ...))
+apply_output_lead(transport)              # default 0.4 s; apply_output_lead(transport, 0.6) for more
+```
+
+We measured this on 8 kHz Twilio-protocol calls, with Pipecat 1.8.1, real Mirai audio
+and 50–250 ms event-loop stalls every ~3 s per call
+([benchmarks/phone-breaks](benchmarks/phone-breaks)):
+
+| Concurrent calls | Stock Pipecat: speech stretched by breaks | With `apply_output_lead` |
+|---|---|---|
+| 3 | 6.6% (3.5 breaks per sentence) | **0.08%** |
+| 10 | 29.1% (14.5 breaks per sentence) | **0.01%** |
+
+When the caller interrupts, Pipecat still tells the provider to clear its queue, so
+at most the lead (0.4 s) of already-sent audio is discarded. One side effect:
+Pipecat's "bot stopped speaking" event fires up to the lead earlier than the caller
+actually stops hearing the bot.
+
+## Examples
+
+- [`examples/foundational/01-say-hello.py`](examples/foundational/01-say-hello.py):
+  a minimal Pipecat pipeline that speaks one line and saves `hello.wav`.
+- [`examples/phone/twilio_bot.py`](examples/phone/twilio_bot.py): a Twilio Media
+  Streams bot with `apply_output_lead`.
+
+## Compatibility
+
+Tested with Pipecat 1.8.1 and 1.12.0 on Python 3.11–3.12. `apply_output_lead` changes how
+Pipecat's websocket output transports time their writes. A test in this repo
+fails if a new Pipecat release changes that, and the function logs a warning and
+does nothing for transports it doesn't support.
+
+## License
+
+BSD-2-Clause. Maintained by [Mirai](https://miraiminds.co) (Sona Labs Pvt Ltd).
+This is a community integration, not maintained by the Pipecat team.

pipecat_mirai-0.1.0/benchmarks/phone-breaks/README.md

@@ -0,0 +1,53 @@
+# Phone-call breaks benchmark
+
+Do callers hear the bot's voice break up when the bot's server is busy? This
+harness measures it end to end. A real Pipecat bot (`FastAPIWebsocketTransport` +
+`TwilioFrameSerializer`, 8 kHz μ-law, `MiraiTTSService`) talks to a simulated phone
+provider in a separate process. The provider plays the received audio in real
+time from a 60 ms jitter buffer and records every moment it has nothing to play.
+
+The bot's event loop gets two kinds of load, per call:
+
+- steady work: 1 ms of CPU every 20 ms, like VAD or STT processing
+- stalls: on average every 3 s, the loop is blocked for 50–250 ms, the way a
+  synchronous HTTP call, a large JSON parse or a CPU-heavy step blocks a real bot
+
+Each call speaks seven lines (about 45 s of speech) at fixed offsets over 80 s.
+
+## Results
+
+Pipecat 1.8.1, real Mirai audio (recorded engine streams replayed on their
+original timing), 4-core arm64 Linux host:
+
+| Concurrent calls | Output pacing | Breaks per sentence | Silence inserted per sentence | Speech stretched |
+|---|---|---|---|---|
+| 3 | stock Pipecat | 3.46 | 365 ms | 6.6% |
+| 3 | `apply_output_lead(0.4)` | 0.04 | 4 ms | **0.08%** |
+| 10 | stock Pipecat | 14.54 | 1,608 ms | 29.1% |
+| 10 | `apply_output_lead(0.4)` | 0.01 | 1 ms | **0.01%** |
+| 3 | stock, with the stand-in server below | 3.88 | 383 ms | 6.3% |
+| 3 | `apply_output_lead(0.4)`, with the stand-in | 0 | 0 ms | **0%** |
+
+Stock Pipecat sends audio at exactly real time, so the provider never holds more
+than one 40 ms chunk, and every stall longer than that is heard as a gap. The
+damage grows with the number of calls, because each call adds stalls to the same
+event loop. This doesn't depend on the TTS vendor.
+
+## Running it
+
+```bash
+uv venv && uv pip install -e ../.. "pipecat-ai[websocket]" uvicorn aiohttp websockets
+
+# Mirai stand-in: streams a 48 kHz mono WAV (10 s or more) with Mirai's delivery timing.
+python phone_breaks.py standin --wav speech48k.wav &
+# Or skip the stand-in: export MIRAI_API_KEY=... and pass --tts-url https://sandbox.voice.miraiminds.co/v1
+
+python phone_breaks.py bot --port 8765 --lead 0 &      # stock Pipecat pacing
+python phone_breaks.py bot --port 8766 --lead 0.4 &    # with apply_output_lead
+
+python phone_breaks.py phone --ws-url ws://127.0.0.1:8765/ws --calls 3 --out results/stock_c3
+python phone_breaks.py phone --ws-url ws://127.0.0.1:8766/ws --calls 3 --out results/lead_c3
+```
+
+Each run prints a JSON summary, writes `result.json` with per-sentence numbers,
+and saves `caller_hears_call0.wav`, which is what the first caller heard.