echospeaks 0.0.1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,113 @@
1
+ Metadata-Version: 2.5
2
+ Name: echospeaks
3
+ Version: 0.0.1
4
+ Summary: Sentence-level TTS audio cache for pipecat voice pipelines — in-memory store, cache keying, and EnableTTSCache/TTSCacheCapture pipeline integration
5
+ Author-email: Futwork <om@futwork.com>
6
+ License: Proprietary
7
+ Classifier: License :: Other/Proprietary License
8
+ Classifier: Programming Language :: Python :: 3
9
+ Requires-Python: >=3.11
10
+ Requires-Dist: aiohttp>=3.9.0
11
+ Provides-Extra: pipecat
12
+ Requires-Dist: pipecat-ai>=1.0.0; extra == 'pipecat'
13
+ Description-Content-Type: text/markdown
14
+
15
+ # echovoice
16
+
17
+ Sentence-level TTS audio cache for [pipecat](https://github.com/pipecat-ai/pipecat)
18
+ voice pipelines. **In-memory only for now** — audio captured on one call is
19
+ served on every later call in the same process; no external service, no
20
+ persistence.
21
+
22
+ ## Layout
23
+
24
+ ```
25
+ echovoice/
26
+ defaults.py every SDK-wide tunable default, in one place
27
+ common/ shared dataclasses (CachedAudio, AudioMeta) + logger shim
28
+ cache/ keying (normalization + cache keys) and the in-memory store
29
+ pipecat_tts/ pipeline integration — mixin.py (read), capture.py (write),
30
+ pending.py (read→write handoff), bracket_filter.py (merges
31
+ per-sentence Started/Stopped into turn-level brackets so the
32
+ transport behaves like stock); needs pipecat-ai
33
+ remote/ dormant echo-service HTTP path (client + config)
34
+ ```
35
+
36
+ ## How it works
37
+
38
+ Two pieces, wired around the host's existing TTS service:
39
+
40
+ - **Read — `EnableTTSCache`** (`echovoice.pipecat_tts.mixin`): drop-in
41
+ replacement for instantiating the provider class. Every sentence's
42
+ `run_tts` first checks the shared in-memory store: a **hit** replays the
43
+ cached PCM as `TTSAudioRawFrame`s (zero provider traffic, barge-in aware),
44
+ a **miss** synthesizes live and marks the sentence for capture.
45
+ - **Write — `TTSCacheCapture`** (`echovoice.pipecat_tts.capture`): a
46
+ pass-through processor placed right after the TTS service in the pipeline.
47
+ It buffers each miss-marked sentence's audio and stores the finished
48
+ sentence in the shared store (`echovoice.cache.store.AUDIO_STORE`, a bounded
49
+ LRU map — 500 entries / 200 MB by default). Interruptions discard partial
50
+ buffers, so truncated audio is never stored.
51
+
52
+ Cache keys (`echovoice.cache.keying`) hash normalized text + provider + voice
53
+ + model + sample rate, so whitespace/casing never causes a false miss and two
54
+ voices in one process can never serve each other's audio. Normalization is
55
+ for key matching only — synthesis always uses the real text.
56
+
57
+ ElevenLabs specifics handled by the wrapper (validated against the
58
+ multi-context websocket API):
59
+
60
+ - One provider context **per sentence**, closed right after its text is sent
61
+ (close is graceful — the provider flushes the audio, then sends the final).
62
+ - A client-side gate caps simultaneous contexts (default 4 — ElevenLabs kills
63
+ the socket at >5; tune via `EnableTTSCache(cache_config={"max_concurrent_context": ...,
64
+ "sentence_wait_timeout_s": ...})`, all defaults live in `echovoice/defaults.py`).
65
+ Slots free on the context's final, interruption, or websocket reconnect.
66
+ - Cache hits take no context and no gate slot; the replayed context is
67
+ completed explicitly so the sentence serializer never stalls.
68
+
69
+ ## Install
70
+
71
+ ```bash
72
+ pip install -e /path/to/echovoice # core (no pipecat needed)
73
+ pip install -e "/path/to/echovoice[pipecat]" # with the pipecat wrapper
74
+ ```
75
+
76
+ With uv, in the host app's `pyproject.toml`:
77
+
78
+ ```toml
79
+ dependencies = ["echovoice"]
80
+
81
+ [tool.uv.sources]
82
+ echovoice = { path = "../echovoice", editable = true }
83
+ ```
84
+
85
+ ## Usage (fw-aisha style)
86
+
87
+ ```python
88
+ from echovoice.pipecat_tts import EnableTTSCache, TTSCacheCapture
89
+
90
+ # read side — instead of CustomElevenLabsTTSService(...):
91
+ tts = EnableTTSCache(CustomElevenLabsTTSService,
92
+ url="wss://api.in.residency.elevenlabs.io",
93
+ api_key=ELEVENLABS_API_KEY,
94
+ settings=ElevenLabsTTSService.Settings(model=..., voice=...),
95
+ )
96
+
97
+ # write side + bracket filter — right after the TTS service in the pipeline:
98
+ pipeline = Pipeline([..., tts, TTSCacheCapture(), TTSTurnBracketFilter(),
99
+ transport.output(), ...])
100
+ ```
101
+
102
+ No configuration needed. The wrapped service exposes
103
+ `get_and_reset_turn_stats()` and `log_cache_summary(call_sid=...)` for call
104
+ logs. Skip both pieces and the provider runs fully stock.
105
+
106
+ `echovoice.remote` holds the HTTP client for a future server-backed cache
107
+ (echo-service); it is dormant — nothing on the active path uses it.
108
+
109
+ ## Tests
110
+
111
+ ```bash
112
+ uv sync && uv run pytest
113
+ ```
@@ -0,0 +1,19 @@
1
+ echovoice/__init__.py,sha256=b1deXn2Pgmx1OH8l_N07AeXdw030UDdDW42Jo9vzwPE,1026
2
+ echovoice/defaults.py,sha256=EBgsFDE3jkbgFKeq6SsWc9SqgNtY86Nrt8rmlPUMvRM,1630
3
+ echovoice/cache/__init__.py,sha256=14GeUAAs0MqtndFCd7K9BCF_BlQ68bFpWtAJylmwEjQ,270
4
+ echovoice/cache/keying.py,sha256=D2tqOnS1PwuP65Bqo8F8QHbeivA5g99hCGDpl34h6MU,3669
5
+ echovoice/cache/store.py,sha256=7xkBTkhlPXXz0YXUK2EDnVlFrKpk6QVmZS2wbQGINwY,3465
6
+ echovoice/common/__init__.py,sha256=oMSIcb0q-bjgp9516PT3CpA99GnOn2Rx5ZF4J-asARA,117
7
+ echovoice/common/log.py,sha256=ZT48yRFZdP_W1nyLMYFZEpHo08hn3mqTXgdkZrB8OI8,232
8
+ echovoice/common/models.py,sha256=xfdrn8seoeBNI9KMqWeGbFdCKCan4CP-HY58bRNEy0o,1448
9
+ echovoice/pipecat_tts/__init__.py,sha256=5YkSr1NzGTL7H0QQCb6jj06SuMd2U9jNFbnziFnaysM,1214
10
+ echovoice/pipecat_tts/bracket_filter.py,sha256=2wMF6yHOI_pz2thyJj7PIWHOJMizlU5pB6wDwChDMAU,6917
11
+ echovoice/pipecat_tts/capture.py,sha256=GWnmvSIMHK0NviD80nu11SerH9TqflwyaUD1NaB_xaY,4367
12
+ echovoice/pipecat_tts/mixin.py,sha256=1AI2GWHOztlxBkFk2cM8VFbw8_QgWz_mFybjOQ2QqL0,16789
13
+ echovoice/pipecat_tts/pending.py,sha256=ngSsSg4cjnwiU4HMQMkkNqIerH1C2NzxNuY1yU2-a7A,1515
14
+ echovoice/remote/__init__.py,sha256=IiKn9KCZzkrr6h0X_JxwH3Z-p4e0goPp5i4MOXZy-4k,420
15
+ echovoice/remote/client.py,sha256=5cPP6gPhGZWVnN77sU8lRsGSgpSgYMBDYpMaEGo0r70,5161
16
+ echovoice/remote/config.py,sha256=LUCUgtThK3rlHwtWCLn2wKkLOfvtbkZDzjoYVijaVPs,1651
17
+ echospeaks-0.0.1.dist-info/METADATA,sha256=3UDfYxOV7xafc_JjcZgZhYvkpSe-lQTHRNMa5CWm6cs,4561
18
+ echospeaks-0.0.1.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
19
+ echospeaks-0.0.1.dist-info/RECORD,,
@@ -0,0 +1,4 @@
1
+ Wheel-Version: 1.0
2
+ Generator: hatchling 1.32.0
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
echovoice/__init__.py ADDED
@@ -0,0 +1,27 @@
1
+ """
2
+ echovoice — sentence-level TTS audio cache for pipecat voice pipelines.
3
+
4
+ Package layout:
5
+ defaults.py every SDK-wide tunable default, in one place
6
+ common/ shared dataclasses (CachedAudio, AudioMeta) + logger shim
7
+ cache/ keying (normalization + cache keys) and the in-memory store
8
+ pipecat_tts/ the pipeline integration: EnableTTSCache (read) and
9
+ TTSCacheCapture (write) — import explicitly, needs pipecat-ai
10
+ remote/ dormant echo-service HTTP path for a future server-backed cache
11
+
12
+ echovoice.pipecat_tts is not re-exported here so the core SDK stays
13
+ import-safe without pipecat installed.
14
+ """
15
+ from .cache.keying import generate_cache_key, normalize_for_cache, settings_signature
16
+ from .cache.store import AUDIO_STORE, InMemoryAudioStore
17
+ from .common.models import AudioMeta, CachedAudio
18
+
19
+ __all__ = [
20
+ "AUDIO_STORE",
21
+ "AudioMeta",
22
+ "CachedAudio",
23
+ "InMemoryAudioStore",
24
+ "generate_cache_key",
25
+ "normalize_for_cache",
26
+ "settings_signature",
27
+ ]
@@ -0,0 +1,10 @@
1
+ from .keying import generate_cache_key, normalize_for_cache, settings_signature
2
+ from .store import AUDIO_STORE, InMemoryAudioStore
3
+
4
+ __all__ = [
5
+ "AUDIO_STORE",
6
+ "InMemoryAudioStore",
7
+ "generate_cache_key",
8
+ "normalize_for_cache",
9
+ "settings_signature",
10
+ ]
@@ -0,0 +1,99 @@
1
+ """
2
+ Cache-key text normalization + key generation (shared by all providers).
3
+
4
+ IMPORTANT: normalization is for KEY MATCHING ONLY. Audio is always synthesized
5
+ from the real, punctuated text + real settings — never from the normalized key.
6
+ """
7
+ import hashlib
8
+ import json
9
+ import re
10
+ import unicodedata
11
+
12
+ # ── text normalization ────────────────────────────────────────────────────────
13
+
14
+ _ZERO_WIDTH = {"​", "‌", "‍", ""}
15
+ _KEEP_PUNCT = {"?", "!"} # prosodic — keep
16
+ _APOSTROPHES = set("'’‘\"“”`") # drop, no space (contractions)
17
+
18
+ _REPEAT_MARK = re.compile(r"([?!])\1+") # "??" -> "?"
19
+ _SPACE_BEFORE_MARK = re.compile(r"\s+([?!])") # "ok ?" -> "ok?"
20
+
21
+
22
+ def normalize_for_cache(text: str) -> str:
23
+ if not text:
24
+ return ""
25
+ text = unicodedata.normalize("NFC", text).lower()
26
+ chars = list(text)
27
+ n = len(chars)
28
+ out = []
29
+ for i, ch in enumerate(chars):
30
+ if ch in _ZERO_WIDTH or ch in _APOSTROPHES:
31
+ continue
32
+ if ch in _KEEP_PUNCT:
33
+ out.append(ch)
34
+ continue
35
+ prev = chars[i - 1] if i > 0 else ""
36
+ nxt = chars[i + 1] if i + 1 < n else ""
37
+ if ch in (",", "."):
38
+ out.append(ch if (prev.isdigit() and nxt.isdigit()) else " ")
39
+ continue
40
+ if ch == "%":
41
+ out.append("%" if prev.isdigit() else " ")
42
+ continue
43
+ cat = unicodedata.category(ch)
44
+ if cat == "Sc": # currency symbol next to a number
45
+ out.append(ch if (prev.isdigit() or nxt.isdigit()) else " ")
46
+ continue
47
+ out.append(" " if cat[0] in ("P", "S") else ch) # keep L/M/N + space
48
+ text = "".join(out)
49
+ text = _REPEAT_MARK.sub(r"\1", text)
50
+ text = _SPACE_BEFORE_MARK.sub(r"\1", text)
51
+ return " ".join(text.split())
52
+
53
+
54
+ # ── settings signature ────────────────────────────────────────────────────────
55
+
56
+ _AUDIO_SETTING_KEYS = frozenset({
57
+ "speed", "stability", "similarity_boost", "style", "use_speaker_boost",
58
+ "language", "controls", "emotion", "pitch", "pronunciation_dict_id",
59
+ })
60
+
61
+
62
+ def _canon(v):
63
+ if isinstance(v, float):
64
+ return round(v, 6)
65
+ if isinstance(v, dict):
66
+ return {k: _canon(v[k]) for k in sorted(v)}
67
+ if isinstance(v, (list, tuple)):
68
+ return [_canon(x) for x in v]
69
+ return v
70
+
71
+
72
+ def settings_signature(settings: dict | None) -> str:
73
+ """Canonical 12-char hash of the audio-affecting settings only."""
74
+ if not settings:
75
+ return ""
76
+ sig = {k: _canon(settings[k]) for k in sorted(settings)
77
+ if k in _AUDIO_SETTING_KEYS and settings.get(k) is not None}
78
+ if not sig:
79
+ return ""
80
+ return hashlib.sha256(json.dumps(sig, sort_keys=True).encode("utf-8")).hexdigest()[:12]
81
+
82
+
83
+ # ── key generation ────────────────────────────────────────────────────────────
84
+
85
+ def generate_cache_key(
86
+ text: str,
87
+ *,
88
+ provider: str,
89
+ voice_id: str,
90
+ model_id: str,
91
+ sample_rate: int = 16000,
92
+ num_channels: int = 1,
93
+ encoding: str = "pcm_s16le",
94
+ settings: dict | None = None,
95
+ ) -> str:
96
+ normalized = normalize_for_cache(text)
97
+ sig = settings_signature(settings)
98
+ raw = f"{provider}:{model_id}:{voice_id}:{sample_rate}:{num_channels}:{encoding}:{sig}:{normalized}"
99
+ return hashlib.sha256(raw.encode("utf-8")).hexdigest()
@@ -0,0 +1,94 @@
1
+ """
2
+ In-memory TTS audio store shared by the read side (EnableTTSCache) and the
3
+ write side (TTSCacheCapture).
4
+
5
+ Process-wide by design: a sentence captured on one call is served on every
6
+ later call in the same process. In-memory only for now — no persistence, no
7
+ external service; a Redis/echo-service backend can slot in behind the same
8
+ get/put surface later.
9
+ """
10
+ import time
11
+ from collections import OrderedDict
12
+
13
+ from ..common.log import logger
14
+ from ..common.models import CachedAudio
15
+ from ..defaults import STORE_MAX_BYTES, STORE_MAX_ENTRIES
16
+
17
+
18
+ class InMemoryAudioStore:
19
+ """Bounded LRU map: cache key -> CachedAudio. Newest capture wins."""
20
+
21
+ def __init__(self, max_entries: int = STORE_MAX_ENTRIES,
22
+ max_bytes: int = STORE_MAX_BYTES):
23
+ self._max_entries = max_entries
24
+ self._max_bytes = max_bytes
25
+ self._entries: "OrderedDict[str, CachedAudio]" = OrderedDict()
26
+ self._total_bytes = 0
27
+ # Process-wide lookup counters, recorded by the read side (note_hit /
28
+ # note_miss) so a rate-mismatched lookup counts as a miss, not a hit.
29
+ self.total_hits = 0
30
+ self.total_misses = 0
31
+
32
+ def get(self, key: str) -> CachedAudio | None:
33
+ entry = self._entries.get(key)
34
+ if entry is not None:
35
+ self._entries.move_to_end(key) # LRU touch
36
+ return entry
37
+
38
+ def put(self, entry: CachedAudio) -> None:
39
+ if not entry.key or not entry.audio:
40
+ return
41
+ old = self._entries.pop(entry.key, None)
42
+ if old is not None:
43
+ self._total_bytes -= len(old.audio)
44
+ entry.hits += old.hits # re-capture keeps the sentence's serve count
45
+ if not entry.captured_at:
46
+ entry.captured_at = time.time()
47
+ self._entries[entry.key] = entry
48
+ self._total_bytes += len(entry.audio)
49
+ while self._entries and (
50
+ len(self._entries) > self._max_entries or self._total_bytes > self._max_bytes
51
+ ):
52
+ evicted_key, evicted = self._entries.popitem(last=False)
53
+ self._total_bytes -= len(evicted.audio)
54
+ logger.debug(f"TTS_CACHE evicted key={evicted_key[:12]} bytes={len(evicted.audio)}")
55
+
56
+ def delete(self, key: str) -> bool:
57
+ """Remove one entry (e.g. via the host's inspection endpoint, to
58
+ re-create miss scenarios). Returns True if the entry existed."""
59
+ entry = self._entries.pop(key, None)
60
+ if entry is None:
61
+ return False
62
+ self._total_bytes -= len(entry.audio)
63
+ return True
64
+
65
+ def note_hit(self, entry: CachedAudio) -> None:
66
+ """Record that this entry was served from the cache."""
67
+ entry.hits += 1
68
+ self.total_hits += 1
69
+
70
+ def note_miss(self) -> None:
71
+ """Record a lookup that ended in live synthesis."""
72
+ self.total_misses += 1
73
+
74
+ def snapshot(self) -> list[CachedAudio]:
75
+ """Current entries, most recently used/stored first. Read-only view for
76
+ inspection endpoints/debugging — does not touch LRU order."""
77
+ return list(reversed(self._entries.values()))
78
+
79
+ def clear(self) -> None:
80
+ self._entries.clear()
81
+ self._total_bytes = 0
82
+ self.total_hits = 0
83
+ self.total_misses = 0
84
+
85
+ def __len__(self) -> int:
86
+ return len(self._entries)
87
+
88
+ @property
89
+ def total_bytes(self) -> int:
90
+ return self._total_bytes
91
+
92
+
93
+ # The process-wide store both pipeline pieces use.
94
+ AUDIO_STORE = InMemoryAudioStore()
@@ -0,0 +1,4 @@
1
+ from .log import logger
2
+ from .models import AudioMeta, CachedAudio
3
+
4
+ __all__ = ["AudioMeta", "CachedAudio", "logger"]
@@ -0,0 +1,7 @@
1
+ """Logger shim: ride the host app's loguru if present, else stdlib logging."""
2
+ try:
3
+ from loguru import logger # noqa: F401
4
+ except ImportError: # pragma: no cover
5
+ import logging
6
+
7
+ logger = logging.getLogger("echovoice")
@@ -0,0 +1,39 @@
1
+ """Shared dataclasses used across the SDK."""
2
+ from dataclasses import dataclass, field
3
+
4
+
5
+ @dataclass
6
+ class CachedAudio:
7
+ """One cached sentence: the PCM plus enough metadata to replay it."""
8
+
9
+ key: str # full cache key (provider/voice/model/rate/text)
10
+ text: str # raw sentence text, kept for logs/debugging only
11
+ audio: bytes # PCM s16le
12
+ sample_rate: int
13
+ num_channels: int = 1
14
+ captured_at: float = 0.0
15
+ hits: int = 0 # times this entry was served (see store.note_hit)
16
+ # Word timings captured off the live synthesis, as (word, seconds from
17
+ # sentence start). Replayed on a hit so the transcript/context is built
18
+ # word-by-word exactly like a live sentence; None when never captured.
19
+ word_times: list | None = None
20
+ words_include_spaces: bool = False # TTSTextFrame.includes_inter_frame_spaces
21
+
22
+ @property
23
+ def duration_s(self) -> float:
24
+ return len(self.audio) / (max(self.sample_rate, 1) * 2 * max(self.num_channels, 1))
25
+
26
+
27
+ @dataclass
28
+ class AudioMeta:
29
+ """Synthesis parameters shipped with a generation job so echo-service can
30
+ re-create the audio server-side (the cache key is a hash — not reversible).
31
+ Used only by the dormant remote (echo-service) path."""
32
+
33
+ provider: str
34
+ voice_id: str
35
+ model_id: str
36
+ sample_rate: int
37
+ num_channels: int = 1
38
+ encoding: str = "pcm_s16le"
39
+ settings: dict = field(default_factory=dict)
echovoice/defaults.py ADDED
@@ -0,0 +1,31 @@
1
+ """
2
+ Every SDK-wide tunable default in one place.
3
+
4
+ Hosts override the gate/registry values per instance via
5
+ EnableTTSCache(cache_config={...}) — see echovoice.pipecat_tts.mixin. Any key
6
+ missing from cache_config falls back to the value here.
7
+ """
8
+
9
+ # ── context gate (echovoice.pipecat_tts.mixin) ────────────────────────────────
10
+ # ElevenLabs allows at most 5 simultaneous contexts per websocket; exceeding it
11
+ # is a 1008 policy violation that kills the whole socket. Stay one below.
12
+ MAX_CONCURRENT_CONTEXT = 4
13
+
14
+ # Longest a sentence waits at a full gate. A context's final normally lands
15
+ # 150-300ms after close; if no slot has freed in this long, assume the finals
16
+ # were lost (e.g. socket died) and reclaim every slot.
17
+ SENTENCE_WAIT_TIMEOUT_S = 3.0
18
+
19
+ # ── hit replay (echovoice.pipecat_tts.mixin) ──────────────────────────────────
20
+ # Cached PCM is replayed in slices of this many bytes (~30ms @ 16kHz mono
21
+ # s16le) so a barge-in can stop the replay between slices.
22
+ REPLAY_CHUNK_BYTES = 960
23
+
24
+ # ── pending-capture registry (echovoice.pipecat_tts.pending) ──────────────────
25
+ # Bound on the read→write handoff registry; marks whose contexts never finish
26
+ # (dead socket, interruption races) age out past this many entries.
27
+ PENDING_MAX_ENTRIES = 512
28
+
29
+ # ── in-memory audio store (echovoice.cache.store) ─────────────────────────────
30
+ STORE_MAX_ENTRIES = 500
31
+ STORE_MAX_BYTES = 200 * 1024 * 1024
@@ -0,0 +1,33 @@
1
+ """
2
+ Sentence-level TTS cache for pipecat pipelines. In-memory, two pieces:
3
+
4
+ READ — EnableTTSCache(ProviderClass, ...) (mixin.py): drop-in replacement
5
+ for ProviderClass(...). Hits replay cached PCM with zero provider
6
+ traffic; misses synthesize live and are marked for capture.
7
+
8
+ WRITE — TTSCacheCapture() (capture.py): a pass-through processor placed right
9
+ after the TTS service in the pipeline; stores each miss-marked
10
+ sentence's finished audio in the shared store.
11
+
12
+ Host integration is two lines:
13
+
14
+ tts = EnableTTSCache(CustomElevenLabsTTSService, url=..., api_key=..., settings=...)
15
+ pipeline = Pipeline([..., tts, TTSCacheCapture(), transport.output(), ...])
16
+
17
+ Requires pipecat-ai (install extra: echovoice[pipecat]); the rest of the SDK
18
+ imports without it.
19
+ """
20
+ from ..defaults import MAX_CONCURRENT_CONTEXT, SENTENCE_WAIT_TIMEOUT_S
21
+ from .bracket_filter import TTSTurnBracketFilter
22
+ from .capture import TTSCacheCapture
23
+ from .mixin import EnableTTSCache, TTSCacheMixin, make_cached
24
+
25
+ __all__ = [
26
+ "EnableTTSCache",
27
+ "MAX_CONCURRENT_CONTEXT",
28
+ "SENTENCE_WAIT_TIMEOUT_S",
29
+ "TTSCacheCapture",
30
+ "TTSCacheMixin",
31
+ "TTSTurnBracketFilter",
32
+ "make_cached",
33
+ ]
@@ -0,0 +1,155 @@
1
+ """
2
+ TTSTurnBracketFilter — coalesces per-sentence Started/Stopped brackets into
3
+ one turn-level bracket, restoring stock pipecat semantics downstream.
4
+
5
+ Per-sentence contexts (the cache's read side) make the TTS service emit a
6
+ TTSStartedFrame/TTSStoppedFrame pair PER SENTENCE. The output transport
7
+ treats every Stopped as "bot stopped speaking", so bot-speaking state flaps
8
+ at each sentence boundary: noisy logs, per-sentence bot started/stopped
9
+ events, VAD-only barge-in windows between sentences, and confused turn
10
+ metrics. Stock pipecat (one context per turn) emits ONE bracket per turn —
11
+ which is what every downstream consumer was designed for.
12
+
13
+ This processor sits AFTER TTSCacheCapture (which needs the per-sentence
14
+ Stops) and BEFORE transport.output(). It forwards the turn's first Started,
15
+ swallows intermediate Stopped/Started pairs, and releases only the final
16
+ Stopped:
17
+
18
+ in: S1▶ audio S1■ S2▶ audio S2■ S3▶ audio S3■
19
+ out: S1▶ audio audio audio S3■
20
+
21
+ "Is this the final Stopped?" is answered by asking the TTS SERVICE, not by
22
+ watching frame order. (Frame markers lie here: pipecat re-pushes the
23
+ LLMFullResponseEndFrame at the end of the FIRST context — see
24
+ tts_service._handle_llm_response_end — so in per-sentence mode the End
25
+ marker passes downstream after sentence 1, not after the last sentence.)
26
+ The service holds the truth:
27
+ - _turn_context_id not None → the LLM turn is still producing text
28
+ - _serialization_queue non-empty → more sentence contexts queued to play
29
+ - _playing_context_id → the context the serializer is draining;
30
+ if it moved past the held Stop's
31
+ context, another sentence is under way
32
+ The held Stopped is released only when none of those indicate pending
33
+ speech. Release is re-checked when a Stopped arrives and when an
34
+ LLMFullResponseEndFrame passes (covers the last sentence finishing playout
35
+ before the LLM turn was fully flushed).
36
+
37
+ Interruption: the InterruptionFrame bypasses this filter's bookkeeping
38
+ entirely (the transport declares bot-stopped from the interruption itself),
39
+ so on interruption we just reset — drop any held Stopped, ignore stray
40
+ Stops until the next Started — and pass the frame through untouched.
41
+ """
42
+ from pipecat.frames.frames import (
43
+ Frame,
44
+ InterruptionFrame,
45
+ LLMFullResponseEndFrame,
46
+ LLMFullResponseStartFrame,
47
+ TTSStartedFrame,
48
+ TTSStoppedFrame,
49
+ )
50
+ from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
51
+
52
+ from ..common.log import logger
53
+
54
+
55
+ class TTSTurnBracketFilter(FrameProcessor):
56
+ """Merge per-sentence TTS Started/Stopped brackets into per-turn ones.
57
+
58
+ Args:
59
+ tts: the (cache-wrapped) TTS service instance from the same pipeline.
60
+ Consulted read-only to decide whether more sentences are pending
61
+ in the current turn. Without it the filter cannot know, and
62
+ passes every bracket through unmerged (stock flapping behavior).
63
+ """
64
+
65
+ def __init__(self, tts=None):
66
+ super().__init__()
67
+ self._tts = tts
68
+ self._warned_no_tts = False
69
+ self._run_open = False # forwarded Started, Stopped pending
70
+ self._held_stop: TTSStoppedFrame | None = None
71
+
72
+ async def process_frame(self, frame: Frame, direction: FrameDirection):
73
+ await super().process_frame(frame, direction)
74
+ if direction != FrameDirection.DOWNSTREAM:
75
+ await self.push_frame(frame, direction)
76
+ return
77
+ for out in self._transform(frame):
78
+ await self.push_frame(out, direction)
79
+
80
+ # ── pure frame logic (returns the frames to forward, in order) ──────────
81
+
82
+ def _transform(self, frame: Frame) -> list[Frame]:
83
+ if isinstance(frame, InterruptionFrame):
84
+ # The transport handles bot-stopped itself on interruption; any
85
+ # held Stopped belongs to a turn that no longer exists.
86
+ self._reset()
87
+ return [frame]
88
+
89
+ if isinstance(frame, LLMFullResponseStartFrame):
90
+ out = []
91
+ if self._held_stop is not None:
92
+ # Previous run never released (e.g. turn died mid-generation)
93
+ # — close its bracket before the new turn begins.
94
+ logger.debug("TTS_BRACKET closing stale bracket at new turn start")
95
+ out.append(self._held_stop)
96
+ self._reset()
97
+ out.append(frame)
98
+ return out
99
+
100
+ if isinstance(frame, LLMFullResponseEndFrame):
101
+ # Not a reliable end-of-speech marker (see module docstring), but
102
+ # a good moment to re-check the service state.
103
+ return [frame] + self._maybe_release()
104
+
105
+ if isinstance(frame, TTSStartedFrame):
106
+ self._held_stop = None # another sentence follows: held stop was intermediate
107
+ if not self._run_open:
108
+ self._run_open = True
109
+ return [frame] # first Started of the turn passes
110
+ return [] # intermediate Started swallowed
111
+
112
+ if isinstance(frame, TTSStoppedFrame):
113
+ if not self._run_open:
114
+ return [] # stray stop after interruption/reset
115
+ self._held_stop = frame
116
+ return self._maybe_release()
117
+
118
+ return [frame]
119
+
120
+ # ── turn-pending check against the TTS service ──────────────────────────
121
+
122
+ def _more_speech_pending(self) -> bool:
123
+ svc = self._tts
124
+ if svc is None:
125
+ if not self._warned_no_tts:
126
+ self._warned_no_tts = True
127
+ logger.warning(
128
+ "TTS_BRACKET no tts service reference — cannot detect "
129
+ "turn end, passing brackets through unmerged"
130
+ )
131
+ return False
132
+ # The LLM turn is still aggregating/flushing sentences.
133
+ if getattr(svc, "_turn_context_id", None) is not None:
134
+ return True
135
+ # More sentence contexts queued behind the one that just drained.
136
+ queue = getattr(svc, "_serialization_queue", None)
137
+ if queue is not None and not queue.empty():
138
+ return True
139
+ # The serializer already moved on to draining another context.
140
+ playing = getattr(svc, "_playing_context_id", None)
141
+ held_ctx = self._held_stop.context_id if self._held_stop else None
142
+ if playing is not None and playing != held_ctx:
143
+ return True
144
+ return False
145
+
146
+ def _maybe_release(self) -> list[Frame]:
147
+ if self._held_stop is not None and not self._more_speech_pending():
148
+ stop = self._held_stop
149
+ self._reset()
150
+ return [stop]
151
+ return []
152
+
153
+ def _reset(self) -> None:
154
+ self._run_open = False
155
+ self._held_stop = None