nardial 0.4.1__tar.gz → 0.4.2__tar.gz

{nardial-0.4.1 → nardial-0.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nardial
-Version: 0.4.1
+Version: 0.4.2
 Summary: Nardial Dialog System
 Author-email: Mike Ligthart <m.e.u.ligthart@vu.nl>
 Maintainer: Clara Fayyad, Landon Brown, Evelyne Prodescu, Eva Biesot
@@ -9,6 +9,7 @@ Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: social-interaction-cloud
+Requires-Dist: pynput>=1.7
 Provides-Extra: google-tts
 Requires-Dist: social-interaction-cloud[google-tts]; extra == "google-tts"
 Provides-Extra: elevenlabs
@@ -269,6 +270,37 @@ Then pass `device` and `tts` to `ConversationAgent` as above.
 
 ---
 
+### Pause and skip controls
+
+Bind keys to pause the interaction or to skip a listen that takes too long by passing `controls` in an `InteractionConfig`. Keys work globally (the terminal does not need focus), with or without `SessionManager`:
+
+```python
+from nardial.interaction_orchestrator import InteractionConfig
+
+agent = ConversationAgent(
+    ...,  # device and providers as in the setups above
+    int_config=InteractionConfig(
+        controls={
+            "start_paused": True,          # wait for the pause key before starting
+            "toggle_pause": ["space"],     # pause / resume
+            "skip_listen": ["s"],          # stop a listen that takes too long
+        },
+    ),
+)
+```
+
+| Setting | Effect |
+|---|---|
+| `toggle_pause` | Pauses the interaction. Speech in progress is stopped and spoken again from the beginning after resume. Listening in progress is stopped, what was heard so far is kept, and the dialog continues after the listen once resumed. Works in every dialog type, including LLM dialogs. |
+| `skip_listen` | Stops the current listen early. What was said so far is still recognised and used as the answer: Dialogflow gets a `StopListeningMessage`, and local Whisper gets an end-of-audio signal and then transcribes what it recorded (this can take a few seconds; Whisper waits up to 30 s for it, which a faster model can lower with `LocalWhisperNLUProvider(..., stop_grace_seconds=5)`). The written keyword provider cannot stop early and gives up after 3 seconds with an empty answer. |
+| `start_paused` | `true` makes the interaction wait for `toggle_pause` before it starts. |
+
+A question whose listen was stopped by a pause or skip is not asked again, even when the answer is empty: the dialog continues with whatever was heard, and an empty answer follows the move's `default_outcome`.
+
+Key names are single characters (`"p"`) or [pynput key names](https://pynput.readthedocs.io/en/latest/keyboard.html#pynput.keyboard.Key) (`"space"`, `"enter"`, `"f5"`). Moves that are not speech or listening (e.g. `timed_wait`, `wait_for_button`) are not paused; the pause takes effect at the next speech or listen.
+
+---
+
 ### Using SessionManager
 
 `SessionManager` wraps `ConversationAgent` and adds dialog loading, eligibility checking, and session state:
@@ -670,6 +702,7 @@ Asks a free-text question and listens for any spoken reply. The answer can be st
 | `add_interest_from_answer` | boolean | | If `true`, adds the answer to the user's topics of interest |
 | `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence after the user replies |
 | `llm_extract_entity` | boolean | | If `true`, the LLM extracts the key entity from the answer (e.g. `"dog"` from `"my favorite animal is a dog"`) before it is stored; unrelated answers store nothing. Default `false` |
+| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |
 
 ```json
 {
@@ -695,6 +728,7 @@ Asks a yes/no question. The detected intent (`"yes"`, `"no"`, `"dontknow"`) driv
 | `default_outcome` | string | | Outcome label used as fallback |
 | `add_interest` | string | | Topic added to interest list when the user answers `"yes"` |
 | `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence |
+| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |
 
 ```json
 {
@@ -723,6 +757,7 @@ Presents a multiple-choice question. The selected option drives branching and ca
 | `default_outcome` | string | | Outcome label when nothing matches |
 | `add_interest_from_variable` | string | | After storing, adds the named variable's value as a topic of interest |
 | `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence |
+| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |
 
 ```json
 {

{nardial-0.4.1 → nardial-0.4.2}/README.md

@@ -241,6 +241,37 @@ Then pass `device` and `tts` to `ConversationAgent` as above.
 
 ---
 
+### Pause and skip controls
+
+Bind keys to pause the interaction or to skip a listen that takes too long by passing `controls` in an `InteractionConfig`. Keys work globally (the terminal does not need focus), with or without `SessionManager`:
+
+```python
+from nardial.interaction_orchestrator import InteractionConfig
+
+agent = ConversationAgent(
+    ...,  # device and providers as in the setups above
+    int_config=InteractionConfig(
+        controls={
+            "start_paused": True,          # wait for the pause key before starting
+            "toggle_pause": ["space"],     # pause / resume
+            "skip_listen": ["s"],          # stop a listen that takes too long
+        },
+    ),
+)
+```
+
+| Setting | Effect |
+|---|---|
+| `toggle_pause` | Pauses the interaction. Speech in progress is stopped and spoken again from the beginning after resume. Listening in progress is stopped, what was heard so far is kept, and the dialog continues after the listen once resumed. Works in every dialog type, including LLM dialogs. |
+| `skip_listen` | Stops the current listen early. What was said so far is still recognised and used as the answer: Dialogflow gets a `StopListeningMessage`, and local Whisper gets an end-of-audio signal and then transcribes what it recorded (this can take a few seconds; Whisper waits up to 30 s for it, which a faster model can lower with `LocalWhisperNLUProvider(..., stop_grace_seconds=5)`). The written keyword provider cannot stop early and gives up after 3 seconds with an empty answer. |
+| `start_paused` | `true` makes the interaction wait for `toggle_pause` before it starts. |
+
+A question whose listen was stopped by a pause or skip is not asked again, even when the answer is empty: the dialog continues with whatever was heard, and an empty answer follows the move's `default_outcome`.
+
+Key names are single characters (`"p"`) or [pynput key names](https://pynput.readthedocs.io/en/latest/keyboard.html#pynput.keyboard.Key) (`"space"`, `"enter"`, `"f5"`). Moves that are not speech or listening (e.g. `timed_wait`, `wait_for_button`) are not paused; the pause takes effect at the next speech or listen.
+
+---
+
 ### Using SessionManager
 
 `SessionManager` wraps `ConversationAgent` and adds dialog loading, eligibility checking, and session state:
@@ -642,6 +673,7 @@ Asks a free-text question and listens for any spoken reply. The answer can be st
 | `add_interest_from_answer` | boolean | | If `true`, adds the answer to the user's topics of interest |
 | `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence after the user replies |
 | `llm_extract_entity` | boolean | | If `true`, the LLM extracts the key entity from the answer (e.g. `"dog"` from `"my favorite animal is a dog"`) before it is stored; unrelated answers store nothing. Default `false` |
+| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |
 
 ```json
 {
@@ -667,6 +699,7 @@ Asks a yes/no question. The detected intent (`"yes"`, `"no"`, `"dontknow"`) driv
 | `default_outcome` | string | | Outcome label used as fallback |
 | `add_interest` | string | | Topic added to interest list when the user answers `"yes"` |
 | `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence |
+| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |
 
 ```json
 {
@@ -695,6 +728,7 @@ Presents a multiple-choice question. The selected option drives branching and ca
 | `default_outcome` | string | | Outcome label when nothing matches |
 | `add_interest_from_variable` | string | | After storing, adds the named variable's value as a topic of interest |
 | `llm_followup` | string | | System prompt for an LLM-generated follow-up sentence |
+| `timeout` | number | | Seconds to listen for an answer. Defaults to `InteractionConfig(listen_timeout=...)` (10 s) |
 
 ```json
 {

{nardial-0.4.1 → nardial-0.4.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "nardial"
-version = "0.4.1"
+version = "0.4.2"
 description = "Nardial Dialog System"
 readme = "README.md"
 requires-python = ">=3.12"
@@ -23,6 +23,7 @@ maintainers = [
 
 dependencies = [
   "social-interaction-cloud",
+  "pynput>=1.7",
 ]
 
 [project.optional-dependencies]

nardial-0.4.2/src/nardial/controls.py

@@ -0,0 +1,184 @@
+"""Session-wide controls: pause/resume and skipping a listen, bound to input triggers.
+
+Example config (passed as ``InteractionConfig(controls=...)``)::
+
+    {
+        "start_paused": True,          # wait for toggle_pause before the first speech or listen
+        "toggle_pause": ["space"],     # keyboard keys...
+        "skip_listen": ["s"],
+    }
+
+A trigger is a keyboard key name, or ``{"<event type>": "<value>"}`` for other
+input sources (e.g. ``{"robot_button": "head_front"}`` once such a source exists).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+
+from nardial.events.bus import EventBus
+from nardial.events.sources import KeyboardSource
+
+logger = logging.getLogger(__name__)
+
+ACTIONS = ("toggle_pause", "skip_listen")
+
+
+class SessionControl:
+    """Maps input events to actions and lets the orchestrator honour pause/skip."""
+
+    def __init__(self, config: dict):
+        config = dict(config or {})
+        unknown = set(config) - set(ACTIONS) - {"start_paused"}
+        if unknown:
+            raise ValueError(f"Unknown controls: {sorted(unknown)}. Allowed: {list(ACTIONS)} and 'start_paused'.")
+
+        # (event type, value) -> action
+        self.bindings: dict[tuple[str, str], str] = {}
+        for action in ACTIONS:
+            for trigger in config.get(action) or []:
+                if isinstance(trigger, str):
+                    trigger = {KeyboardSource.event_type: trigger}
+                for event_type, value in trigger.items():
+                    self.bindings[(event_type, str(value).lower())] = action
+
+        event_types = {event_type for event_type, _ in self.bindings}
+        self.sources = [KeyboardSource()] if KeyboardSource.event_type in event_types else []
+
+        self._running = asyncio.Event()
+        self._paused = asyncio.Event()
+        self._skip = asyncio.Event()
+        if config.get("start_paused"):
+            self._paused.set()
+        else:
+            self._running.set()
+
+        self._bus: EventBus | None = None
+        self._sub = None
+        self._task: asyncio.Task | None = None
+
+    # ------------------------------------------------------------------
+    # Actions
+    # ------------------------------------------------------------------
+
+    @property
+    def started(self) -> bool:
+        return self._task is not None
+
+    @property
+    def paused(self) -> bool:
+        return self._paused.is_set()
+
+    def pause(self) -> None:
+        self._running.clear()
+        self._paused.set()
+        print(f"[INFO] Session paused. Press {self._triggers_for('toggle_pause')} to continue.")
+
+    def resume(self) -> None:
+        self._paused.clear()
+        self._running.set()
+        print("[INFO] Session resumed.")
+
+    def toggle_pause(self) -> None:
+        self.resume() if self.paused else self.pause()
+
+    def skip_listen(self) -> None:
+        """Stop the current listen early, keeping what was heard so far. No-op when not listening."""
+        self._skip.set()
+
+    def action_for(self, event) -> str | None:
+        """The action bound to ``event``, or None."""
+        if not isinstance(event.data, dict):
+            return None
+        return self.bindings.get((event.type, str(event.data.get("value")).lower()))
+
+    def handle(self, event) -> None:
+        action = self.action_for(event)
+        if action is not None:
+            getattr(self, action)()
+
+    def _triggers_for(self, action: str) -> str:
+        return " / ".join(value for (_, value), a in self.bindings.items() if a == action) or "(no trigger)"
+
+    # ------------------------------------------------------------------
+    # Lifecycle
+    # ------------------------------------------------------------------
+
+    async def start(self, bus: EventBus) -> None:
+        """Listen for bound input events on ``bus`` and start the input sources."""
+        self._bus = bus
+        self._sub = bus.subscribe(lambda ev: self.action_for(ev) is not None)
+        self._task = asyncio.create_task(self._consume())
+        for source in self.sources:
+            source.start(bus)
+        if self.paused:
+            print(f"[INFO] Session starts paused. Press {self._triggers_for('toggle_pause')} to start.")
+
+    async def stop(self) -> None:
+        for source in self.sources:
+            source.stop()
+        if self._task is not None:
+            self._task.cancel()
+            await asyncio.gather(self._task, return_exceptions=True)
+            self._task = None
+        if self._bus is not None and self._sub is not None:
+            self._bus.unsubscribe(self._sub)
+            self._sub = None
+
+    async def _consume(self) -> None:
+        while True:
+            event = await self._sub.get()
+            try:
+                self.handle(event)
+            except Exception:
+                logger.exception("SessionControl: handling %r failed", event)
+
+    # ------------------------------------------------------------------
+    # Used by the orchestrator
+    # ------------------------------------------------------------------
+
+    async def wait_until_running(self) -> None:
+        await self._running.wait()
+
+    async def run(self, make_coro, on_stop=None, stop_grace_seconds: float = 3.0):
+        """Run ``make_coro()`` and return its result, honouring pause and skip.
+
+        Without ``on_stop`` (speech), a pause cancels the coroutine and starts it
+        again from scratch after resume, so the robot repeats the sentence
+        instead of finishing it mid-word.
+
+        With ``on_stop`` (listening), both a pause and ``skip_listen`` call it to
+        make the coroutine finish early (e.g. stop listening, keeping the audio
+        so far) and return the coroutine's result. After a pause, that result is
+        held until resume, so the dialog continues after the listen. If the
+        coroutine has not finished within ``stop_grace_seconds`` it is cancelled
+        and ``None`` is returned.
+        """
+        while True:
+            await self._running.wait()
+            self._skip.clear()
+            task = asyncio.ensure_future(make_coro())
+            waiters = {asyncio.ensure_future(self._paused.wait())}
+            if on_stop is not None:
+                waiters.add(asyncio.ensure_future(self._skip.wait()))
+            stopped = False
+            try:
+                await asyncio.wait({task, *waiters}, return_when=asyncio.FIRST_COMPLETED)
+                if task.done():
+                    return task.result()
+                if on_stop is not None:
+                    # Paused or skipped: ask the work to finish early and keep its result.
+                    on_stop()
+                    await asyncio.wait({task}, timeout=stop_grace_seconds)
+                    stopped = True
+            finally:
+                for waiter in waiters:
+                    waiter.cancel()
+                if not task.done():
+                    task.cancel()
+                    await asyncio.gather(task, return_exceptions=True)
+            if stopped:
+                await self._running.wait()  # after a pause, continue only once resumed
+                return None if task.cancelled() else task.result()
+            # Paused speech: loop around, wait for resume and start again.

{nardial-0.4.1 → nardial-0.4.2}/src/nardial/conversation_agent.py

@@ -112,7 +112,7 @@ class ConversationAgent:
         """
         self.orchestrator.play_animation(animation_name, run_async=run_async)
 
-    async def ask_yesno(self, question, max_attempts=1, **kwargs):
+    async def ask_yesno(self, question, max_attempts=1, timeout=None, **kwargs):
         """
         Ask a yes/no question and interpret the response via the NLU provider.
 
@@ -122,6 +122,8 @@ class ConversationAgent:
             The question to ask the user.
         max_attempts : int, optional
             Number of retries if no valid answer is detected.
+        timeout : float, optional
+            Seconds to listen for an answer. Defaults to ``InteractionConfig.listen_timeout``.
 
         Returns
         -------
@@ -131,7 +133,7 @@ class ConversationAgent:
         attempts = 0
         while attempts < max_attempts:
             await self.say(question, **kwargs)
-            result = await self.orchestrator.listen(context={'answer_yesno': 1})
+            result = await self.orchestrator.listen(timeout=timeout, context={'answer_yesno': 1})
             if result.intent:
                 print(f'context: answer_yesno, recognized_intent: {result.intent}')
                 if result.intent == INTENT_YESNO_YES:
@@ -140,10 +142,12 @@ class ConversationAgent:
                     return "no"
                 elif result.intent == INTENT_YESNO_DONTKNOW:
                     return "dontknow"
+            if result.interrupted:
+                break  # stopped by pause/skip: continue the dialog instead of asking again
             attempts += 1
         return None
 
-    async def ask_open(self, question, max_attempts=2, **kwargs):
+    async def ask_open(self, question, max_attempts=2, timeout=None, **kwargs):
         """
         Ask an open-ended question and return the user's spoken response.
 
@@ -153,6 +157,8 @@ class ConversationAgent:
             The question to ask.
         max_attempts : int, optional
             Number of retries if no response is captured.
+        timeout : float, optional
+            Seconds to listen for an answer. Defaults to ``InteractionConfig.listen_timeout``.
 
         Returns
         -------
@@ -162,13 +168,15 @@ class ConversationAgent:
         attempts = 0
         while attempts < max_attempts:
             await self.say(question, **kwargs)
-            result = await self.orchestrator.listen()
+            result = await self.orchestrator.listen(timeout=timeout)
             if result.transcript:
                 return result.transcript
+            if result.interrupted:
+                break  # stopped by pause/skip: continue the dialog instead of asking again
             attempts += 1
         return None
 
-    async def ask_options(self, question, options, max_attempts=2, **kwargs):
+    async def ask_options(self, question, options, max_attempts=2, timeout=None, **kwargs):
         """
         Ask a question and match the response against a set of predefined options.
 
@@ -180,6 +188,8 @@ class ConversationAgent:
             List of expected keywords/options to match against the response.
         max_attempts : int, optional
             Number of retries.
+        timeout : float, optional
+            Seconds to listen for an answer. Defaults to ``InteractionConfig.listen_timeout``.
 
         Returns
         -------
@@ -190,7 +200,7 @@ class ConversationAgent:
         -----
         Matching is case-insensitive and based on substring presence.
         """
-        answer = await self.ask_open(question, max_attempts=max_attempts, **kwargs)
+        answer = await self.ask_open(question, max_attempts=max_attempts, timeout=timeout, **kwargs)
         if answer:
             answer_lower = answer.lower()
             for opt in options:

nardial-0.4.2/src/nardial/events/sources.py

@@ -0,0 +1,68 @@
+"""Input sources: things outside the dialog (keys, robot buttons, ...) that emit bus events.
+
+A source only reports *what* happened, e.g. ``Event(type="keyboard", data={"value": "space"})``.
+What that input *does* (pausing, skipping a listen) is decided by
+:class:`~nardial.controls.SessionControl`.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Protocol
+
+from nardial.events.bus import EventBus
+from nardial.events.types import Event
+
+logger = logging.getLogger(__name__)
+
+
+class InputSource(Protocol):
+    """Emits events onto the session's bus between ``start()`` and ``stop()``."""
+
+    event_type: str
+
+    def start(self, bus: EventBus) -> None: ...
+
+    def stop(self) -> None: ...
+
+
+def key_name(key) -> str | None:
+    """Readable name for a pynput key: ``"a"``, ``"space"``, ``"f5"``."""
+    char = getattr(key, "char", None)
+    if char:
+        return char.lower()
+    return getattr(key, "name", None)
+
+
+class KeyboardSource:
+    """Emits ``Event(type="keyboard", data={"value": <key name>})`` for every key press.
+
+    Uses pynput, which listens globally (the terminal does not need focus) on its
+    own thread. If pynput cannot start (e.g. no display on a headless Linux
+    machine), keyboard controls are disabled with a warning instead of crashing.
+    """
+
+    event_type = "keyboard"
+
+    def __init__(self):
+        self._listener = None
+
+    def start(self, bus: EventBus) -> None:
+        try:
+            from pynput import keyboard
+        except Exception as e:
+            logger.warning("Keyboard controls disabled, pynput is unavailable: %s", e)
+            return
+
+        def on_press(key):
+            name = key_name(key)
+            if name:
+                bus.emit_sync(Event(priority=0, type=self.event_type, source="keyboard", data={"value": name}))
+
+        self._listener = keyboard.Listener(on_press=on_press)
+        self._listener.start()
+
+    def stop(self) -> None:
+        if self._listener is not None:
+            self._listener.stop()
+            self._listener = None

{nardial-0.4.1 → nardial-0.4.2}/src/nardial/interaction_orchestrator.py

@@ -11,6 +11,8 @@ from time import strftime
 from sic_framework.core import sic_logging
 from sic_framework.core.sic_application import SICApplication
 
+from nardial.controls import SessionControl
+from nardial.events import EventBus
 from nardial.providers.device import DeviceAdapter, AnimationStyle
 from nardial.providers.tts import TTSProvider, _amplify_audio
 from nardial.providers.nlu import NLUProvider, NLUResult
@@ -26,10 +28,18 @@ class InteractionConfig:
     """
 
     def __init__(self, language="en", post_speech_delay=None,
-                 signal_listening_behavior=True):
+                 signal_listening_behavior=True, listen_timeout=10, controls=None):    
+        """
+        :param controls: Optional pause/skip controls, e.g.
+            `{"start_paused": True, "toggle_pause": ["space"], "skip_listen": ["s"]}`.
+            See `nardial.controls.SessionControl`.
+        """
         self.language = language
         self.post_speech_delay = post_speech_delay
         self.signal_listening_behavior = signal_listening_behavior
+         # Default seconds to listen for a user answer; moves can override it with their own "timeout".
+        self.listen_timeout = listen_timeout
+        self.controls = controls
         self.animated = True
         self.animation_style = AnimationStyle.EXPLANATORY
         self.always_regenerate = False
@@ -54,6 +64,9 @@ class InteractionConfig:
 
 
 class InteractionOrchestrator:
+    # Seconds a stopped listen may take to return its result, used when the NLU provider sets no stop_grace_seconds.
+    DEFAULT_STOP_GRACE_SECONDS = 3.0
+
     def __init__(self, device: DeviceAdapter, tts_provider: TTSProvider,
                  nlu_provider: NLUProvider, llm_provider: LLMProvider | None = None,
                  vector_store: VectorStoreProvider | None = None,
@@ -103,6 +116,9 @@ class InteractionOrchestrator:
         self.nlu_provider = nlu_provider
         self.logger.info("Complete and ready for interaction!")
 
+        # Pause/skip controls from InteractionConfig.controls; started by start_controls().
+        self.control = SessionControl(int_config.controls) if int_config.controls else None
+
     def start_logging(self, log_id, init_data: dict):
         folder = Path("logs")
         folder.mkdir(parents=True, exist_ok=True)
@@ -162,6 +178,17 @@ class InteractionOrchestrator:
     @InteractionConfig.apply_config_defaults('interaction_conf', ['post_speech_delay', 'animated', 'always_regenerate', 'chunk_audio'])
     async def say(self, text, post_speech_delay=None, animated=False, amplified=False, always_regenerate=False, chunk_audio=False,
             voice_settings=None):
+        speak = functools.partial(self._say, text, post_speech_delay=post_speech_delay, animated=animated,
+                                  amplified=amplified, always_regenerate=always_regenerate,
+                                  chunk_audio=chunk_audio, voice_settings=voice_settings)
+        control = await self._active_control()
+        if control is None:
+            return await speak()
+        # A pause stops the sentence; it is spoken again from the start after resume.
+        return await control.run(speak)
+
+    async def _say(self, text, post_speech_delay, animated, amplified, always_regenerate, chunk_audio,
+                   voice_settings):
         if animated:
             self.device.play_speaking_animation(self.interaction_conf.animation_style)
         try:
@@ -180,7 +207,33 @@ class InteractionOrchestrator:
         if post_speech_delay and post_speech_delay > 0:
             await asyncio.sleep(post_speech_delay)
 
-    async def listen(self, context=None, timeout=10) -> NLUResult:
+    async def listen(self, context=None, timeout=None) -> NLUResult:
+        if timeout is None:
+            timeout = self.interaction_conf.listen_timeout
+        
+        hear = functools.partial(self._listen, context=context, timeout=timeout)
+        control = await self._active_control()
+        if control is None:
+            return await hear()
+        # A pause or skip_listen stops listening (Dialogflow sends StopListeningMessage), so the
+        # audio heard so far is still recognised. After a pause the dialog continues after the listen.
+        stopped = False
+
+        def stop():
+            nonlocal stopped
+            stopped = True
+            self.nlu_provider.cancel()
+
+        # How long to wait for the result after stopping; providers that transcribe after
+        # recording (Whisper) need longer than the default.
+        stop_grace_seconds = getattr(self.nlu_provider, "stop_grace_seconds", self.DEFAULT_STOP_GRACE_SECONDS)
+        result = await control.run(hear, on_stop=stop, stop_grace_seconds=stop_grace_seconds)
+        if result is None:
+            result = NLUResult(transcript="")
+        result.interrupted = stopped
+        return result
+
+    async def _listen(self, context, timeout) -> NLUResult:
         if self.interaction_conf.signal_listening_behavior:
             self.device.signal_listening(start=True)
         try:
@@ -190,8 +243,9 @@ class InteractionOrchestrator:
         except asyncio.CancelledError:
             self.nlu_provider.cancel()
             raise
-        if self.interaction_conf.signal_listening_behavior:
-            self.device.signal_listening(start=False)
+        finally:
+            if self.interaction_conf.signal_listening_behavior:
+                self.device.signal_listening(start=False)
         if result.transcript:
             self.log_utterance(speaker='user', text=result.transcript)
             await self._push_user_transcript(result.transcript)
@@ -259,7 +313,35 @@ class InteractionOrchestrator:
     def play_motion(self, motion_name):
         self.device.play_motion_sequence(motion_name)
 
+    async def start_controls(self, bus: EventBus | None = None) -> None:
+        """Start the pause/skip controls on ``bus`` (the session's), or on a bus of their own.
+
+        SessionManager calls this with its session bus. Without a SessionManager the
+        controls are started on their own bus at the first ``say``/``listen``.
+        """
+        if self.control is None:
+            return
+        await self.control.stop()  # restart cleanly if they were already running on another bus
+        if bus is None:
+            bus = EventBus()
+            bus.set_loop(asyncio.get_running_loop())
+        await self.control.start(bus)
+
+    async def stop_controls(self) -> None:
+        if self.control is not None:
+            await self.control.stop()
+
+    async def _active_control(self):
+        """The started controls, or None when no controls are configured."""
+        control = getattr(self, "control", None)
+        if control is not None and not control.started:
+            await self.start_controls()
+        return control
+
     def disconnect(self):
+        if getattr(self, "control", None) is not None:
+            for source in self.control.sources:
+                source.stop()
         self.tts_provider.close()
         self.device.disconnect()
         if self.vector_store is not None:
@@ -273,3 +355,4 @@ class InteractionOrchestrator:
 
     def set_interaction_conf(self, interaction_conf: InteractionConfig):
         self.interaction_conf = interaction_conf
+        self.control = SessionControl(interaction_conf.controls) if interaction_conf.controls else None