pipecat-backchannel 0.2.0__tar.gz

pipecat_backchannel-0.2.0/.gitignore

@@ -0,0 +1,10 @@
+__pycache__/
+*.py[cod]
+.venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+.env
+.clip_cache/

pipecat_backchannel-0.2.0/CLAUDE.md

@@ -0,0 +1,86 @@
+# pipecat-backchannel
+
+## The idea
+
+Human conversation runs on two channels at once. One person holds the floor and talks. The other, without ever taking the floor, keeps feeding back short signals — "mhm", "yeah", "right". These are **backchannels**, and they mean *keep going, I'm with you*.
+
+They are not turns. The speaker doesn't stop for them, doesn't answer them, and doesn't remember them as things the listener said. They're a parallel stream of evidence that someone is still on the other end.
+
+Voice agents have no such channel. They are silent for as long as you speak, then reply once you stop. From the user's side that silence is indistinguishable from a hung connection, so they compensate: trailing off, repeating themselves, cutting a thought short to check the agent is still there. The conversation stops feeling live.
+
+This library adds that missing channel to a voice agent, and nothing else. It is not a feature of the agent's intelligence — the agent never knows it happened. It's a thin behavioral layer that makes the agent *sound* like it's listening while it listens.
+
+## Why it's shaped the way it is
+
+The problem decomposes into decisions that are genuinely independent, and the architecture's whole job is to keep them that way:
+
+- **When to signal.** A timing decision, made from audio alone. This is the hard part and the only one that can ruin the experience.
+- **What to signal.** Which flavor of acknowledgment fits the moment.
+- **Where the sound comes from.** Synthesis, recording, a vendor — an infrastructure concern.
+- **How it reaches the user.** Turning a decision into audio in the pipeline.
+
+Collapsing any two of these together is the mistake this design exists to prevent. They change for different reasons, at different rates, under different constraints.
+
+Three properties fall out of the domain and are not negotiable:
+
+**Timing is a hard deadline, not a goal.** A backchannel is only correct inside a narrow window. Slightly late is worse than never — an acknowledgment that lands after the moment has passed sounds like an interruption. Anything slow must therefore happen ahead of time, never in the moment. Work is precomputed at startup so the live decision is pure and instant.
+
+**The decisions differ wildly in stakes.** Getting the timing wrong makes the agent talk over people. Getting the flavor wrong makes it sound very slightly off. These must not be coupled, and the cheap, sloppy decision must never be allowed to influence the expensive, careful one.
+
+**Listening is not speaking.** A backchannel must leave no trace in the agent's memory of the conversation. The moment it's recorded as something the agent said, it has become a turn, and the whole premise is gone.
+
+Read those three as the acceptance criteria for any change here. A change that's elegant but violates one of them is wrong.
+
+---
+
+# Engineering principles
+
+## Dependency inversion — the one that matters most
+
+**Depend on abstractions, never on concretions.** High-level policy must not know which low-level implementation serves it. If a module names a vendor, a library, or a driver, it is welded to it.
+
+**Inject dependencies; never construct them inside.** A component that builds its own collaborators has silently made them mandatory and untestable. Pass them in, with a sensible default where there's an obvious one.
+
+This is what buys everything else:
+
+- **Swappability.** A new provider is a new implementation, not an edit.
+- **Testability.** Inject a fake and the test is fast, offline, deterministic. If something is hard to test, that's the design telling you it's too coupled — fix the coupling, don't mock harder.
+- **Blast radius.** Concrete details stay at the edges. A change out there doesn't reach the core.
+
+The abstraction must be defined by the **consumer's** need, not the implementation's shape. An interface that mirrors one vendor's API is that vendor wearing a hat — the second implementation will not fit it.
+
+## Simple first
+
+Write the obvious thing. Complexity should arrive because something demanded it, not because you predicted it would. Speculative generality is paid for now, for a benefit that usually never comes.
+
+You cannot guess the right abstraction from one use case. Wait for the second. Duplication is cheaper than the wrong abstraction — it's easy to merge later, while a bad abstraction is hard to remove once everything depends on it.
+
+## Architecturally correct first
+
+Simple is not sloppy. Get the **seams** right early — where things plug together is cheap to place now and expensive to move later. Everything else can be crude and improved in place.
+
+A wrong abstraction is worse than none.
+
+## SOLID
+
+- **S** — one reason to change. If describing a component needs "and", split it.
+- **O** — extend by adding, not by editing. New behavior shouldn't mean touching working code.
+- **L** — a subtype must be usable anywhere its base is, without surprises.
+- **I** — many small interfaces beat one fat one. Don't force a caller to depend on methods it ignores.
+- **D** — see above.
+
+## DRY
+
+Deduplicate **knowledge, not text.** Two pieces of code that look alike but change for different reasons are not duplicates — merging them couples them forever. One fact should have one home; that's the point.
+
+## Abstraction
+
+An abstraction earns its place by hiding something the caller genuinely shouldn't know. If it only forwards calls, delete it — it's a layer, not an abstraction.
+
+Name things for what they mean, not how they work. The name is the abstraction; if you can't name it clearly, the boundary is wrong.
+
+## KISS
+
+The reader is the constraint. Code is read far more than written, and the next reader has none of your context. Clever costs more than it saves.
+
+Prefer boring, obvious, explicit. Fewer moving parts, fewer states, fewer ways to hold it wrong.

pipecat_backchannel-0.2.0/LICENSE

@@ -0,0 +1,24 @@
+BSD 2-Clause License
+
+Copyright (c) 2026 Taras Maister
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

pipecat_backchannel-0.2.0/PKG-INFO

@@ -0,0 +1,215 @@
+Metadata-Version: 2.4
+Name: pipecat-backchannel
+Version: 0.2.0
+Summary: Make voice AI conversations feel live — Pipecat agents that listen back with natural "mhm"s and "yeah"s while the user is still talking.
+Project-URL: Homepage, https://github.com/maisterr/pipecat-backchannel
+Project-URL: Repository, https://github.com/maisterr/pipecat-backchannel
+Project-URL: Issues, https://github.com/maisterr/pipecat-backchannel/issues
+Author: Taras Maister
+License-Expression: BSD-2-Clause
+License-File: LICENSE
+Keywords: backchannel,conversational-ai,pipecat,turn-taking,voice-ai
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Requires-Python: >=3.11
+Requires-Dist: pipecat-ai<2,>=1.6.0
+Description-Content-Type: text/markdown
+
+<div align="center">
+  <a href="https://github.com/pipecat-ai/pipecat">
+    <img alt="Pipecat" width="220px" height="auto" src="https://raw.githubusercontent.com/pipecat-ai/pipecat/main/pipecat.png">
+  </a>
+  <p><em>A community integration for <a href="https://github.com/pipecat-ai/pipecat">Pipecat</a></em></p>
+</div>
+
+# pipecat-backchannel
+
+[![PyPI](https://img.shields.io/pypi/v/pipecat-backchannel)](https://pypi.org/project/pipecat-backchannel)
+[![Python](https://img.shields.io/pypi/pyversions/pipecat-backchannel)](https://pypi.org/project/pipecat-backchannel)
+[![License](https://img.shields.io/badge/license-BSD--2--Clause-blue)](LICENSE)
+[![Pipecat](https://img.shields.io/badge/Pipecat-1.6.0-brightgreen)](https://github.com/pipecat-ai/pipecat)
+
+Your Pipecat agent listens back: quiet "mhm"s and "yeah"s while the user is
+still talking.
+
+## Why
+
+Listen to two people on a phone call. While one tells a story, the other keeps
+feeding back small signals: "mhm", "yeah". Linguists call these
+**backchannels**. They mean _keep going, I'm with you_, and the speaker talks
+straight through them.
+
+Talk to a voice agent and you get silence until you finish. Silence on a call
+reads as a dropped connection, so you hesitate, trail off, or ask "hello?".
+
+This library adds the listening sounds. The agent keeps the floor with you the
+whole time: a backchannel takes no turn and leaves no trace in your LLM
+context.
+
+## Install
+
+```bash
+uv add pipecat-backchannel
+```
+
+## Usage
+
+```python
+from pipecat_backchannel import Backchannel
+
+backchannel = Backchannel()
+
+pipeline = Pipeline(backchannel([
+    transport.input(),
+    stt,
+    context_aggregator.user(),
+    llm,
+    tts,
+    transport.output(),
+    context_aggregator.assistant(),
+]))
+```
+
+One wrapper call is the whole setup. The default configuration works without
+an API key, a voice ID, or a sample rate, and places every processor for you.
+
+The first run records the clips through **your own TTS**, so they come out in
+your bot's voice, and caches them in `.clip_cache/`. Every later run starts
+from the cache. Delete that directory after changing voice.
+
+## Record before the first call
+
+The first session waits a few seconds while the recorder fills the clip cache.
+Move that wait to app startup if you want to avoid delay for first caller:
+
+```python
+backchannel = Backchannel()          # one per process
+
+async def run_bot(...):
+    pipeline = Pipeline(backchannel([... tts ...]))
+
+if __name__ == "__main__":
+    asyncio.run(backchannel.prewarm(tts=make_tts()))   # before the server opens
+    main()
+```
+
+## Tuning
+
+```python
+Backchannel(
+    volume=0.6,                              # how loud, relative to the recording
+    params=BackchannelParams(
+        fire_probability=0.8,                # chance of speaking up at a good moment
+        cooldown_s=2.5,                      # minimum gap between backchannels
+        min_speech_before_eligible_s=0.7,    # ignore pauses right after a turn starts
+    ),
+)
+```
+
+`fire_probability` and `cooldown_s` set how present the listener feels. Raise
+the first and lower the second for a more active one. Keep some slack: a bot
+that reacts to each eligible pause sounds mechanical, because human listeners
+skip some of them.
+
+`volume` sits below 1.0 by default so clips play under the user's voice.
+
+Set `loguru` to `DEBUG` to see each fire/skip decision and its reason.
+
+### Going quiet mid-session
+
+Some stretches of a call want a silent listener: the user dictating a card
+number, or the bot walking through a form where each pause belongs to a
+field, and an "mhm" would read as confirmation.
+
+The component that decides when to fire is the `BackchannelProcessor`, one
+per pipeline. `backchannel([...])` returns your processor list with it
+inserted, so keep that list, build the `Pipeline` from it, and pull the gate
+out of it:
+
+```python
+from pipecat_backchannel.processor import BackchannelProcessor
+
+processors = backchannel([transport.input(), stt, llm, tts, transport.output()])
+pipeline = Pipeline(processors)
+
+gate = next(p for p in processors if isinstance(p, BackchannelProcessor))
+```
+
+Then flip it from any event handler, as many times as the call needs:
+
+```python
+gate.enabled = False   # stop firing; every frame still passes through
+...
+gate.enabled = True    # resume
+```
+
+`enabled` is a plain attribute, safe to set at any moment. While it is
+`False` the processor keeps passing audio through and keeps tracking speech,
+so nothing downstream notices and re-enabling picks up mid-turn. It fires no
+clips and runs no end-of-turn classification in that state. The pipeline, the
+LLM context, and the other sessions' gates stay untouched.
+
+## Custom clips
+
+The library groups clips by conversational function: continuer, agreement,
+hesitation, surprise.
+
+```python
+Backchannel(clip_groups={"continue": ["Mhm.", "Right."], "affirm": ["Yeah.", "Yep."]})
+```
+
+Two rules for a custom inventory:
+
+- **Give each group at least two clips.** A one-clip group repeats itself.
+- **Get variety from punctuation, and keep the vocabulary small.** Real
+  backchannels draw on a handful of sounds: _uh-huh, yeah, mm-hmm, right,
+  okay, oh, huh, hm_. A written phrase like "absolutely" reads fine on the
+  page, then sounds like an answer when spoken over someone mid-sentence.
+  Punctuation is the handle your TTS gives you: `"Mhm."` falls, `"Mhm,"` stays
+  up, `"Hm..."` draws out.
+
+Pass `synthesizer=` to use a different voice, or `cache=` to supply your own
+recorded `.wav` files. See `pipecat_backchannel.synth` and `.cache`.
+
+## Run the example
+
+```bash
+cp examples/.env.template examples/.env   # DEEPGRAM_API_KEY, CARTESIA_API_KEY, CEREBRAS_API_KEY
+uv run examples/bot.py
+```
+
+Open `http://localhost:7860/client`, talk in mid-length sentences, and pause
+mid-thought. A quiet "mhm" lands on clause-complete pauses. Word-search pauses
+("we need the, uh...") and finished questions get silence, and the questions
+get a real reply.
+
+## Limitations
+
+- **Pauses only.** Humans backchannel during speech too, cued by pitch. This
+  library reacts to pauses. The roadmap covers the rest.
+- **About 2× inference while the user speaks.** It runs a second VAD and
+  turn-detector on purpose: the pipeline's copies are tuned for turn-taking,
+  and pause timing needs the opposite settings. A handful of concurrent
+  sessions runs fine; measure before a large deployment.
+- **English only.** The clips and the regexes are English. The underlying
+  models handle other languages.
+
+## Roadmap
+
+- [ ] **Fire during speech.** Real listeners backchannel mid-utterance, cued
+      by pitch and rhythm rather than silence. That takes Voice Activity
+      Projection; [MaAI](https://github.com/MaAI-Kyoto/MaAI) ships a usable
+      model. It reads a different signal, so it slots in beside the pause
+      gate rather than replacing it, and it is the largest quality jump left.
+- [ ] **Match the speaker's prosody.** The selector picks a clip at random
+      within its group. Picking the one whose pitch and energy fit the last
+      utterance adds no runtime cost, because the clips already sit on disk.
+
+## License
+
+BSD 2-Clause © Taras Maister, matching Pipecat's own license. Not affiliated
+with Daily or the Pipecat core team.

pipecat_backchannel-0.2.0/README.md

@@ -0,0 +1,194 @@
+<div align="center">
+  <a href="https://github.com/pipecat-ai/pipecat">
+    <img alt="Pipecat" width="220px" height="auto" src="https://raw.githubusercontent.com/pipecat-ai/pipecat/main/pipecat.png">
+  </a>
+  <p><em>A community integration for <a href="https://github.com/pipecat-ai/pipecat">Pipecat</a></em></p>
+</div>
+
+# pipecat-backchannel
+
+[![PyPI](https://img.shields.io/pypi/v/pipecat-backchannel)](https://pypi.org/project/pipecat-backchannel)
+[![Python](https://img.shields.io/pypi/pyversions/pipecat-backchannel)](https://pypi.org/project/pipecat-backchannel)
+[![License](https://img.shields.io/badge/license-BSD--2--Clause-blue)](LICENSE)
+[![Pipecat](https://img.shields.io/badge/Pipecat-1.6.0-brightgreen)](https://github.com/pipecat-ai/pipecat)
+
+Your Pipecat agent listens back: quiet "mhm"s and "yeah"s while the user is
+still talking.
+
+## Why
+
+Listen to two people on a phone call. While one tells a story, the other keeps
+feeding back small signals: "mhm", "yeah". Linguists call these
+**backchannels**. They mean _keep going, I'm with you_, and the speaker talks
+straight through them.
+
+Talk to a voice agent and you get silence until you finish. Silence on a call
+reads as a dropped connection, so you hesitate, trail off, or ask "hello?".
+
+This library adds the listening sounds. The agent keeps the floor with you the
+whole time: a backchannel takes no turn and leaves no trace in your LLM
+context.
+
+## Install
+
+```bash
+uv add pipecat-backchannel
+```
+
+## Usage
+
+```python
+from pipecat_backchannel import Backchannel
+
+backchannel = Backchannel()
+
+pipeline = Pipeline(backchannel([
+    transport.input(),
+    stt,
+    context_aggregator.user(),
+    llm,
+    tts,
+    transport.output(),
+    context_aggregator.assistant(),
+]))
+```
+
+One wrapper call is the whole setup. The default configuration works without
+an API key, a voice ID, or a sample rate, and places every processor for you.
+
+The first run records the clips through **your own TTS**, so they come out in
+your bot's voice, and caches them in `.clip_cache/`. Every later run starts
+from the cache. Delete that directory after changing voice.
+
+## Record before the first call
+
+The first session waits a few seconds while the recorder fills the clip cache.
+Move that wait to app startup if you want to avoid delay for first caller:
+
+```python
+backchannel = Backchannel()          # one per process
+
+async def run_bot(...):
+    pipeline = Pipeline(backchannel([... tts ...]))
+
+if __name__ == "__main__":
+    asyncio.run(backchannel.prewarm(tts=make_tts()))   # before the server opens
+    main()
+```
+
+## Tuning
+
+```python
+Backchannel(
+    volume=0.6,                              # how loud, relative to the recording
+    params=BackchannelParams(
+        fire_probability=0.8,                # chance of speaking up at a good moment
+        cooldown_s=2.5,                      # minimum gap between backchannels
+        min_speech_before_eligible_s=0.7,    # ignore pauses right after a turn starts
+    ),
+)
+```
+
+`fire_probability` and `cooldown_s` set how present the listener feels. Raise
+the first and lower the second for a more active one. Keep some slack: a bot
+that reacts to each eligible pause sounds mechanical, because human listeners
+skip some of them.
+
+`volume` sits below 1.0 by default so clips play under the user's voice.
+
+Set `loguru` to `DEBUG` to see each fire/skip decision and its reason.
+
+### Going quiet mid-session
+
+Some stretches of a call want a silent listener: the user dictating a card
+number, or the bot walking through a form where each pause belongs to a
+field, and an "mhm" would read as confirmation.
+
+The component that decides when to fire is the `BackchannelProcessor`, one
+per pipeline. `backchannel([...])` returns your processor list with it
+inserted, so keep that list, build the `Pipeline` from it, and pull the gate
+out of it:
+
+```python
+from pipecat_backchannel.processor import BackchannelProcessor
+
+processors = backchannel([transport.input(), stt, llm, tts, transport.output()])
+pipeline = Pipeline(processors)
+
+gate = next(p for p in processors if isinstance(p, BackchannelProcessor))
+```
+
+Then flip it from any event handler, as many times as the call needs:
+
+```python
+gate.enabled = False   # stop firing; every frame still passes through
+...
+gate.enabled = True    # resume
+```
+
+`enabled` is a plain attribute, safe to set at any moment. While it is
+`False` the processor keeps passing audio through and keeps tracking speech,
+so nothing downstream notices and re-enabling picks up mid-turn. It fires no
+clips and runs no end-of-turn classification in that state. The pipeline, the
+LLM context, and the other sessions' gates stay untouched.
+
+## Custom clips
+
+The library groups clips by conversational function: continuer, agreement,
+hesitation, surprise.
+
+```python
+Backchannel(clip_groups={"continue": ["Mhm.", "Right."], "affirm": ["Yeah.", "Yep."]})
+```
+
+Two rules for a custom inventory:
+
+- **Give each group at least two clips.** A one-clip group repeats itself.
+- **Get variety from punctuation, and keep the vocabulary small.** Real
+  backchannels draw on a handful of sounds: _uh-huh, yeah, mm-hmm, right,
+  okay, oh, huh, hm_. A written phrase like "absolutely" reads fine on the
+  page, then sounds like an answer when spoken over someone mid-sentence.
+  Punctuation is the handle your TTS gives you: `"Mhm."` falls, `"Mhm,"` stays
+  up, `"Hm..."` draws out.
+
+Pass `synthesizer=` to use a different voice, or `cache=` to supply your own
+recorded `.wav` files. See `pipecat_backchannel.synth` and `.cache`.
+
+## Run the example
+
+```bash
+cp examples/.env.template examples/.env   # DEEPGRAM_API_KEY, CARTESIA_API_KEY, CEREBRAS_API_KEY
+uv run examples/bot.py
+```
+
+Open `http://localhost:7860/client`, talk in mid-length sentences, and pause
+mid-thought. A quiet "mhm" lands on clause-complete pauses. Word-search pauses
+("we need the, uh...") and finished questions get silence, and the questions
+get a real reply.
+
+## Limitations
+
+- **Pauses only.** Humans backchannel during speech too, cued by pitch. This
+  library reacts to pauses. The roadmap covers the rest.
+- **About 2× inference while the user speaks.** It runs a second VAD and
+  turn-detector on purpose: the pipeline's copies are tuned for turn-taking,
+  and pause timing needs the opposite settings. A handful of concurrent
+  sessions runs fine; measure before a large deployment.
+- **English only.** The clips and the regexes are English. The underlying
+  models handle other languages.
+
+## Roadmap
+
+- [ ] **Fire during speech.** Real listeners backchannel mid-utterance, cued
+      by pitch and rhythm rather than silence. That takes Voice Activity
+      Projection; [MaAI](https://github.com/MaAI-Kyoto/MaAI) ships a usable
+      model. It reads a different signal, so it slots in beside the pause
+      gate rather than replacing it, and it is the largest quality jump left.
+- [ ] **Match the speaker's prosody.** The selector picks a clip at random
+      within its group. Picking the one whose pitch and energy fit the last
+      utterance adds no runtime cost, because the clips already sit on disk.
+
+## License
+
+BSD 2-Clause © Taras Maister, matching Pipecat's own license. Not affiliated
+with Daily or the Pipecat core team.

pipecat_backchannel-0.2.0/examples/.env.template

@@ -0,0 +1,14 @@
+DEEPGRAM_API_KEY=your_deepgram_api_key_here
+CARTESIA_API_KEY=your_cartesia_api_key_here
+CARTESIA_VOICE_ID=71a7ad14-091c-4e8e-a314-022ece01c121
+CEREBRAS_API_KEY=your_cerebras_api_key_here
+CEREBRAS_MODEL=gemma-4-31b
+
+# Daily WebRTC room (leave unset to run in local "webrtc" test mode instead)
+DAILY_API_KEY=your_daily_api_key_here
+
+# Main-pipeline VAD start_secs (barge-in/interruption sensitivity). Higher =
+# fewer spurious interruptions from brief sounds, slower to react to real
+# barge-in. Independent of backchannel pause timing, which BackchannelProcessor
+# runs its own VAD for (BackchannelParams.vad_start_secs, default 0.2).
+MAIN_VAD_START_SECS=0.4