vision-agents-plugins-qwen 0.2.4__tar.gz

vision_agents_plugins_qwen-0.2.4/.gitignore

@@ -0,0 +1,90 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.cursor/*
+# Distribution / packaging
+.Python
+build/
+dist/
+downloads/
+develop-eggs/
+eggs/
+.eggs/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+pip-wheel-metadata/
+MANIFEST
+*.egg-info/
+*.egg
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+coverage.xml
+nosetests.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Type checker / lint caches
+.mypy_cache/
+.dmypy.json
+dmypy.json
+.pytype/
+.pyre/
+.ruff_cache/
+
+# Environments
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+.env
+.env.local
+.env.*.local
+.env.bak
+pyvenv.cfg
+.python-version
+
+# Editors / IDEs
+.vscode/
+.idea/
+
+# Jupyter Notebook
+.ipynb_checkpoints/
+
+# OS / Misc
+.DS_Store
+*.log
+
+# Tooling & repo-specific
+pyrightconfig.json
+shell.nix
+bin/*
+lib/*
+stream-py/
+
+# Artifacts / assets
+*.pt
+*.kef
+*.onnx
+profile.html
+
+/opencode.json

vision_agents_plugins_qwen-0.2.4/PKG-INFO

@@ -0,0 +1,83 @@
+Metadata-Version: 2.4
+Name: vision-agents-plugins-qwen
+Version: 0.2.4
+Summary: Qwen Omni plugin for vision agents
+Project-URL: Documentation, https://visionagents.ai/
+Project-URL: Website, https://visionagents.ai/
+Project-URL: Source, https://github.com/GetStream/Vision-Agents
+License-Expression: MIT
+Requires-Python: >=3.10
+Requires-Dist: numpy
+Requires-Dist: vision-agents
+Requires-Dist: websockets>=15.0.1
+Description-Content-Type: text/markdown
+
+# Qwen Realtime Plugin for Vision Agents
+
+Qwen3 Realtime LLM integration for Vision Agents framework with native audio output and built-in speech recognition using WebSocket-based realtime communication.
+
+## Features
+
+- **Native audio output**: No TTS service needed - audio comes directly from the model
+- **Built-in STT**: Integrated speech-to-text using `gummy-realtime-v1` - no external STT service required
+- **Server-side VAD**: Automatic turn detection with configurable silence thresholds
+- **Video understanding**: Optional video frame support for multimodal interactions
+- **Real-time streaming**: WebSocket-based bidirectional communication for low-latency responses
+- **Interruption handling**: Automatic cancellation when user starts speaking
+
+## Installation
+
+```bash
+uv add vision-agents[qwen]
+```
+
+## Usage
+
+```python
+from vision_agents.core import User, Agent
+from vision_agents.plugins import getstream, qwen
+
+agent = Agent(
+    edge=getstream.Edge(),
+    agent_user=User(name="Qwen Assistant"),
+    instructions="Be helpful and friendly",
+    llm=qwen.Realtime(
+        model="qwen3-omni-flash-realtime",
+        voice="Cherry",
+        fps=1,
+    ),
+    # No STT or TTS needed - Qwen Realtime provides both
+)
+```
+
+## Configuration
+
+| Parameter | Description | Default | Accepted Values |
+|-----------|-------------|---------|----------------|
+| `model` | Qwen Realtime model identifier | `"qwen3-omni-flash-realtime"` | Model name string |
+| `api_key` | DashScope API key | `None` (from env) | String or `None` |
+| `base_url` | WebSocket API base URL | `"wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime"` | URL string |
+| `voice` | Voice for audio output | `"Cherry"` | Voice name string |
+| `fps` | Video frames per second | `1` | Integer |
+| `include_video` | Include video frames in requests | `False` | Boolean |
+| `video_width` | Video frame width | `1280` | Integer |
+| `video_height` | Video frame height | `720` | Integer |
+
+## Environment Variables
+
+Set `DASHSCOPE_API_KEY` in your environment or `.env` file:
+
+```bash
+DASHSCOPE_API_KEY=your_dashscope_api_key_here
+```
+
+## Example
+
+See `plugins/qwen/example/qwen_realtime_example.py` for a complete working example.
+
+## Dependencies
+
+- vision-agents
+- websockets
+- aiortc
+- av

vision_agents_plugins_qwen-0.2.4/README.md

@@ -0,0 +1,69 @@
+# Qwen Realtime Plugin for Vision Agents
+
+Qwen3 Realtime LLM integration for Vision Agents framework with native audio output and built-in speech recognition using WebSocket-based realtime communication.
+
+## Features
+
+- **Native audio output**: No TTS service needed - audio comes directly from the model
+- **Built-in STT**: Integrated speech-to-text using `gummy-realtime-v1` - no external STT service required
+- **Server-side VAD**: Automatic turn detection with configurable silence thresholds
+- **Video understanding**: Optional video frame support for multimodal interactions
+- **Real-time streaming**: WebSocket-based bidirectional communication for low-latency responses
+- **Interruption handling**: Automatic cancellation when user starts speaking
+
+## Installation
+
+```bash
+uv add vision-agents[qwen]
+```
+
+## Usage
+
+```python
+from vision_agents.core import User, Agent
+from vision_agents.plugins import getstream, qwen
+
+agent = Agent(
+    edge=getstream.Edge(),
+    agent_user=User(name="Qwen Assistant"),
+    instructions="Be helpful and friendly",
+    llm=qwen.Realtime(
+        model="qwen3-omni-flash-realtime",
+        voice="Cherry",
+        fps=1,
+    ),
+    # No STT or TTS needed - Qwen Realtime provides both
+)
+```
+
+## Configuration
+
+| Parameter | Description | Default | Accepted Values |
+|-----------|-------------|---------|----------------|
+| `model` | Qwen Realtime model identifier | `"qwen3-omni-flash-realtime"` | Model name string |
+| `api_key` | DashScope API key | `None` (from env) | String or `None` |
+| `base_url` | WebSocket API base URL | `"wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime"` | URL string |
+| `voice` | Voice for audio output | `"Cherry"` | Voice name string |
+| `fps` | Video frames per second | `1` | Integer |
+| `include_video` | Include video frames in requests | `False` | Boolean |
+| `video_width` | Video frame width | `1280` | Integer |
+| `video_height` | Video frame height | `720` | Integer |
+
+## Environment Variables
+
+Set `DASHSCOPE_API_KEY` in your environment or `.env` file:
+
+```bash
+DASHSCOPE_API_KEY=your_dashscope_api_key_here
+```
+
+## Example
+
+See `plugins/qwen/example/qwen_realtime_example.py` for a complete working example.
+
+## Dependencies
+
+- vision-agents
+- websockets
+- aiortc
+- av

vision_agents_plugins_qwen-0.2.4/example/README.md

@@ -0,0 +1,94 @@
+# Qwen Realtime Example
+
+This example demonstrates how to use Qwen Realtime with Vision Agents for real-time conversations.
+
+## Features
+
+- **Real-time streaming**: Direct audio streaming from Qwen Realtime API
+- **No text input**: The model does not support text input, so start speaking once you join the call
+- **Video support**: Configure frames per second for video processing
+
+## Installation
+
+```bash
+uv add vision-agents[qwen]
+```
+
+## Quick Start
+
+1. Set your API key in your environment:
+
+```bash
+export DASHSCOPE_API_KEY=your_dashscope_api_key_here
+```
+
+Or create a `.env` file:
+
+```
+DASHSCOPE_API_KEY=your_dashscope_api_key_here
+```
+
+2. Run the example:
+
+```bash
+uv run python qwen_realtime_example.py
+```
+
+## Code Example
+
+```python
+from dotenv import load_dotenv
+from vision_agents.core import Agent, User, cli
+from vision_agents.core.agents import AgentLauncher
+from vision_agents.plugins import getstream, qwen
+
+load_dotenv()
+
+async def create_agent(**kwargs) -> Agent:
+    llm = qwen.Realtime(fps=1)
+
+    agent = Agent(
+        edge=getstream.Edge(),
+        agent_user=User(name="Qwen Assistant", id="agent"),
+        instructions="You are a helpful AI assistant. Be friendly and conversational.",
+        llm=llm,
+    )
+    return agent
+
+async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
+    await agent.create_user()
+    call = await agent.create_call(call_type, call_id)
+
+    with await agent.join(call):
+        await agent.edge.open_demo(call)
+        await agent.finish()
+
+if __name__ == "__main__":
+    cli(AgentLauncher(create_agent=create_agent, join_call=join_call))
+```
+
+## Configuration
+
+### Environment Variables
+
+- **`DASHSCOPE_API_KEY`**: Your DashScope/Alibaba API key (required)
+
+### Realtime Parameters
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `fps` | Video frames per second | `1` |
+| `api_key` | DashScope API key | `None` (from env) |
+
+## Requirements
+
+- Python 3.10+
+- DashScope API key
+- Stream API credentials (configured via `getstream.Edge()`)
+- `vision-agents` framework
+
+## Notes
+
+- The model is hosted in Singapore, so latency may vary depending on your location
+- The model does not support text input - once you join the call, simply start speaking to the agent
+- This example uses the CLI interface for easy interaction

vision_agents_plugins_qwen-0.2.4/example/env.example

@@ -0,0 +1,6 @@
+# DashScope API key for Qwen Omni
+DASHSCOPE_API_KEY=your_dashscope_api_key_here
+
+# Stream API credentials
+STREAM_API_KEY=your_stream_api_key
+STREAM_API_SECRET=your_stream_api_secret

vision_agents_plugins_qwen-0.2.4/example/pyproject.toml

@@ -0,0 +1,16 @@
+[project]
+name = "qwen-omni-example"
+version = "0.1.0"
+description = "Example using Qwen Omni with Vision Agents"
+requires-python = ">=3.10"
+dependencies = [
+    "vision-agents",
+    "vision-agents-plugins-qwen",
+    "vision-agents-plugins-getstream",
+    "python-dotenv",
+]
+
+[tool.uv.sources]
+vision-agents = { workspace = true }
+vision-agents-plugins-qwen = { workspace = true }
+vision-agents-plugins-getstream = { workspace = true }

vision_agents_plugins_qwen-0.2.4/example/qwen_realtime_example.py

@@ -0,0 +1,36 @@
+# This is a basic example using Qwen Realtime with Vision Agents
+# To run this example, you must have DASHSCOPE_API_KEY set in your env.
+# Do note that the model is hosted in Singapore so depending on your location, the latency may vary.
+# This model also does not support text input so once you join the call, simply start speaking to the agent.
+
+from dotenv import load_dotenv
+from vision_agents.core import Agent, User, cli
+from vision_agents.core.agents import AgentLauncher
+from vision_agents.plugins import getstream, qwen
+
+load_dotenv()
+
+
+async def create_agent(**kwargs) -> Agent:
+    llm = qwen.Realtime(fps=1)
+
+    agent = Agent(
+        edge=getstream.Edge(),
+        agent_user=User(name="Qwen Assistant", id="agent"),
+        instructions="You are a helpful AI assistant. Be friendly and conversational.",
+        llm=llm,
+    )
+    return agent
+
+
+async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
+    await agent.create_user()
+    call = await agent.create_call(call_type, call_id)
+
+    with await agent.join(call):
+        await agent.edge.open_demo(call)
+        await agent.finish()
+
+
+if __name__ == "__main__":
+    cli(AgentLauncher(create_agent=create_agent, join_call=join_call))

vision_agents_plugins_qwen-0.2.4/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["hatchling", "hatch-vcs"]
+build-backend = "hatchling.build"
+
+[project]
+name = "vision-agents-plugins-qwen"
+dynamic = ["version"]
+description = "Qwen Omni plugin for vision agents"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+dependencies = [
+    "vision-agents",
+    "numpy",
+    "websockets>=15.0.1",
+]
+
+[project.urls]
+Documentation = "https://visionagents.ai/"
+Website = "https://visionagents.ai/"
+Source = "https://github.com/GetStream/Vision-Agents"
+
+[tool.hatch.version]
+source = "vcs"
+raw-options = { root = "..", search_parent_directories = true, fallback_version = "0.0.0" }
+
+[tool.hatch.build.targets.wheel]
+packages = ["."]
+
+[tool.uv.sources]
+vision-agents = { workspace = true }
+
+[dependency-groups]
+dev = [
+    "pytest>=8.4.1",
+    "pytest-asyncio>=1.0.0",
+]

vision_agents_plugins_qwen-0.2.4/tests/test_qwen_realtime.py

@@ -0,0 +1,86 @@
+import asyncio
+
+import dotenv
+import pytest
+from vision_agents.core.llm.events import (
+    RealtimeAudioOutputEvent,
+)
+from vision_agents.plugins.qwen import Realtime
+
+dotenv.load_dotenv()
+
+
+@pytest.fixture()
+async def llm():
+    """Create and manage Realtime connection lifecycle"""
+    realtime = Realtime(
+        fps=1, vad_silence_duration_ms=0, vad_prefix_padding_ms=0, vad_threshold=0.1
+    )
+    yield realtime
+    await realtime.close()
+
+
+class TestQwen3Realtime:
+    """Integration tests for Qwen3Realtime connect flow"""
+
+    @pytest.mark.integration
+    async def test_audio_sending_flow(self, llm, mia_audio_16khz, silence_1s_16khz):
+        """Test sending real audio data and verify connection remains stable"""
+        events = []
+
+        @llm.events.subscribe
+        async def on_audio(event: RealtimeAudioOutputEvent):
+            events.append(event)
+
+        # Connect the llm
+        await llm.connect()
+        # Let it handle the connection events
+        await asyncio.sleep(5.0)
+
+        # Send 1s of silence first
+        await llm.simple_audio_response(silence_1s_16khz)
+        # Send audio
+        await llm.simple_audio_response(mia_audio_16khz)
+        # Send silence again
+        await llm.simple_audio_response(silence_1s_16khz)
+
+        # Let it run for a few sec
+        await asyncio.sleep(10.0)
+
+        # Verify that the model replied with audio
+        assert len(events) > 0
+
+    @pytest.mark.integration
+    async def test_video_sending_flow(
+        self,
+        llm,
+        bunny_video_track,
+        describe_what_you_see_audio_16khz,
+        silence_1s_16khz,
+    ):
+        """Test sending real video data and verify connection remains stable"""
+        events = []
+
+        @llm.events.subscribe
+        async def on_audio(event: RealtimeAudioOutputEvent):
+            events.append(event)
+
+        await llm.connect()
+        # Let the model to handle all connection events
+        await asyncio.sleep(5.0)
+
+        # Send 1s of silence first
+        await llm.simple_audio_response(silence_1s_16khz)
+        # Start video sender with low FPS to avoid overwhelming the connection
+        await llm.watch_video_track(bunny_video_track)
+        # Send audio to the model (it does not support text inputs)
+        await llm.simple_audio_response(describe_what_you_see_audio_16khz)
+        # Send silence again
+        await llm.simple_audio_response(silence_1s_16khz)
+        # Let it run for a few seconds
+        await asyncio.sleep(10.0)
+
+        # Stop video sender
+        await llm._stop_watching_video_track()
+        # Verify that the model replied
+        assert len(events) > 0

vision_agents_plugins_qwen-0.2.4/vision_agents/plugins/qwen/__init__.py

@@ -0,0 +1,3 @@
+from .qwen_realtime import Qwen3Realtime as Realtime
+
+__all__ = ["Realtime"]

vision_agents_plugins_qwen-0.2.4/vision_agents/plugins/qwen/client.py

@@ -0,0 +1,151 @@
+import asyncio
+import base64
+import contextlib
+import json
+import logging
+import time
+from typing import Any, AsyncIterator, Optional
+
+import websockets
+from getstream.video.rtc import PcmData
+
+logger = logging.getLogger(__name__)
+
+
+class Qwen3RealtimeClient:
+    """
+    A wrapper around WebSocket connection for Qwen3Realtime API.
+    It automatically reconnects in case of connection failures.
+    """
+
+    def __init__(
+        self,
+        model: str,
+        api_key: str,
+        base_url: str,
+        config: dict[str, Any],
+        reconnect_backoff: float = 1.0,
+    ) -> None:
+        self._base_url = f"{base_url}?model={model}"
+        self._api_key = api_key
+        self._real_ws: Optional[websockets.ClientConnection] = None
+        self._exit_stack = contextlib.AsyncExitStack()
+        self._config = config
+        self._conn_lock = asyncio.Lock()
+        self._closed = False
+        self._reconnect_backoff = reconnect_backoff
+
+    async def connect(self) -> None:
+        if self._conn_lock.locked():
+            return None
+
+        async with self._conn_lock:
+            logger.debug(f"Connecting to Qwen3Realtime at {self._base_url}")
+            self._real_ws = await self._exit_stack.enter_async_context(
+                websockets.connect(
+                    uri=self._base_url,
+                    additional_headers={"Authorization": f"Bearer {self._api_key}"},
+                )
+            )
+            # Initialize session with config params
+            await self.update_session(self._config)
+        return None
+
+    async def close(self) -> None:
+        self._closed = True
+        try:
+            await self._exit_stack.aclose()
+        except Exception as e:
+            logger.warning(f"Error closing session: {e}")
+
+    async def read(self) -> AsyncIterator[dict[str, Any]]:
+        while not self._closed:
+            try:
+                async for msg in self._ws:
+                    event = json.loads(msg)
+                    yield event
+            except websockets.ConnectionClosedError as e:
+                if not _should_reconnect(e):
+                    raise
+                await asyncio.sleep(self._reconnect_backoff)
+                await self.connect()
+
+    async def send_event(self, event: dict[str, Any]) -> None:
+        event["event_id"] = f"event_{int(time.time() * 1000)}"
+
+        try:
+            await self._ws.send(json.dumps(event))
+        except websockets.ConnectionClosedError as e:
+            if not _should_reconnect(e):
+                raise
+            logger.warning(
+                f"Re-establishing Qwen3Realtime connection due to error: {e}"
+            )
+            await asyncio.sleep(self._reconnect_backoff)
+            await self.connect()
+
+    async def update_session(self, config: dict[str, Any]) -> None:
+        """Update the session configuration."""
+        await self.send_event(event={"type": "session.update", "session": config})
+
+    async def send_audio(self, pcm: PcmData) -> None:
+        """Stream raw audio data to the API."""
+        # Only 16-bit, 16 kHz, mono PCM is supported.
+        audio_bytes = pcm.resample(
+            target_sample_rate=16000, target_channels=1
+        ).samples.tobytes()
+        audio_b64 = base64.b64encode(audio_bytes).decode()
+        append_event = {"type": "input_audio_buffer.append", "audio": audio_b64}
+        await self.send_event(append_event)
+
+    async def commit_audio(self) -> None:
+        """Commit the audio buffer to trigger processing."""
+        event = {"type": "input_audio_buffer.commit"}
+        await self.send_event(event)
+
+    async def send_frame(self, frame_bytes: bytes) -> None:
+        """
+        Append image data to the image buffer.
+
+        Note:
+            - The image format must be JPG or JPEG. A resolution of 480p or 720p is recommended.
+                The maximum supported resolution is 1080p.
+            - A single image should not exceed 500 KB in size.
+            - Encode the image data to Base64 before sending.
+            - We recommend sending images to the server at a rate of no more than 2 frames per second.
+            - You must send audio data at least once before sending image data.
+        """
+        image_b64 = base64.b64encode(frame_bytes).decode()
+        event = {"type": "input_image_buffer.append", "image": image_b64}
+        await self.send_event(event)
+
+    async def cancel_response(self) -> None:
+        """Cancel the current response."""
+        event = {"type": "response.cancel"}
+        await self.send_event(event)
+
+    @property
+    def _ws(self) -> websockets.ClientConnection:
+        if self._real_ws is None:
+            raise ValueError("The websocket connection is not established yet")
+        return self._real_ws
+
+
+def _should_reconnect(exc: Exception) -> bool:
+    """
+    Temporary errors should typically trigger a reconnect.
+    So if the websocket breaks this should return True and trigger a reconnect
+    """
+    reconnect_close_codes = [
+        1011,  # Server-side exception or session timeout
+        1012,  # Service restart
+        1013,  # Try again later
+        1014,  # Bad gateway
+    ]
+    if (
+        isinstance(exc, websockets.ConnectionClosedError)
+        and exc.rcvd
+        and exc.rcvd.code in reconnect_close_codes
+    ):
+        return True
+    return False

vision_agents_plugins_qwen-0.2.4/vision_agents/plugins/qwen/events.py

@@ -0,0 +1,13 @@
+from dataclasses import dataclass, field
+from vision_agents.core.events import PluginBaseEvent
+from typing import Optional, Any
+
+
+@dataclass
+class LLMErrorEvent(PluginBaseEvent):
+    """Event emitted when an LLM encounters an error."""
+
+    type: str = field(default="plugin.llm.error", init=False)
+    plugin_name: str = ""
+    error_message: Optional[str] = None
+    event_data: Optional[Any] = None

vision_agents_plugins_qwen-0.2.4/vision_agents/plugins/qwen/qwen_realtime.py

@@ -0,0 +1,283 @@
+import asyncio
+import base64
+import logging
+import os
+import uuid
+from concurrent.futures import ThreadPoolExecutor
+from typing import Any, Optional, cast
+
+import aiortc
+import av
+from aiortc import VideoStreamTrack
+from getstream.video.rtc import PcmData
+from vision_agents.core.edge.types import Participant
+from vision_agents.core.llm import Realtime
+from vision_agents.core.llm.events import LLMResponseChunkEvent
+from vision_agents.core.llm.llm import LLMResponseEvent
+from vision_agents.core.processors import Processor
+from vision_agents.core.utils.video_forwarder import VideoForwarder
+from vision_agents.core.utils.video_utils import frame_to_jpeg_bytes
+
+from . import events
+from .client import Qwen3RealtimeClient
+
+DEFAULT_BASE_URL = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime"
+PLUGIN_NAME = "Qwen3Realtime"
+
+logger = logging.getLogger(__name__)
+
+
+class Qwen3Realtime(Realtime):
+    def __init__(
+        self,
+        model: str = "qwen3-omni-flash-realtime",
+        api_key: Optional[str] = None,
+        base_url: Optional[str] = None,
+        voice: str = "Cherry",
+        fps: int = 1,
+        include_video: bool = False,
+        video_width: int = 1280,
+        video_height: int = 720,
+        audio_transcription_model: str = "gummy-realtime-v1",
+        vad_threshold: float = 0.1,
+        vad_prefix_padding_ms: int = 500,
+        vad_silence_duration_ms: int = 900,
+    ):
+        super().__init__(fps=fps)
+        self.model = model
+        self.voice = voice
+        self.session_id = str(uuid.uuid4())
+        self.events.register_events_from_module(events)
+
+        self._base_url = base_url or DEFAULT_BASE_URL
+
+        api_key = api_key or os.getenv("DASHSCOPE_API_KEY")
+        if not api_key:
+            raise ValueError("api_key is required")
+        self._api_key = cast(str, api_key)
+
+        self._video_forwarder: Optional[VideoForwarder] = None
+        self._include_video = include_video
+        self._real_client: Optional[Qwen3RealtimeClient] = None
+        self._processing_task: Optional[asyncio.Task] = None
+        self._video_width = video_width
+        self._video_height = video_height
+        self._executor = ThreadPoolExecutor(max_workers=1)
+
+        self._is_responding = False
+        self._current_response_id = None
+        self._current_item_id = None
+        self._current_participant: Optional[Participant] = None
+        # The model requires us not to send any video frames until the audio is sent
+        self._audio_emitted_once = False
+        self._audio_transcription_model = audio_transcription_model
+        self._vad_threshold = vad_threshold
+        self._vad_prefix_padding_ms = vad_prefix_padding_ms
+        self._vad_silence_duration_ms = vad_silence_duration_ms
+
+    async def connect(self):
+        # Stop the processing task first in case we're reconnecting
+        await self._stop_processing_task()
+
+        # Session configuration
+        session_config = {
+            "modalities": ["text", "audio"],
+            "voice": self.voice,
+            "instructions": self._instructions,
+            "input_audio_format": "pcm16",
+            "output_audio_format": "pcm24",
+            "input_audio_transcription": {"model": self._audio_transcription_model},
+            "turn_detection": {
+                "type": "server_vad",
+                "threshold": self._vad_threshold,
+                "prefix_padding_ms": self._vad_prefix_padding_ms,
+                "silence_duration_ms": self._vad_silence_duration_ms,
+            },
+        }
+        self._real_client = Qwen3RealtimeClient(
+            api_key=self._api_key,
+            base_url=self._base_url,
+            model=self.model,
+            config=session_config,
+        )
+        await self._real_client.connect()
+        self.connected = True
+        logger.debug(f"Started Qwen3Realtime session at {self._base_url}")
+
+        # Start the loop task
+        self._start_processing_task()
+
+    async def simple_audio_response(
+        self, pcm: PcmData, participant: Optional[Participant] = None
+    ):
+        if not self.connected:
+            return
+        self._current_participant = participant
+        await self._client.send_audio(pcm=pcm)
+        self._audio_emitted_once = True
+
+    async def simple_response(
+        self,
+        text: str,
+        processors: Optional[list[Processor]] = None,
+        participant: Optional[Participant] = None,
+    ) -> LLMResponseEvent[Any]:
+        logger.warning(
+            f'Cannot reply to "{text}"; reason - Qwen3Realtime does not support text inputs'
+        )
+        return LLMResponseEvent(text="", original=None)
+
+    async def close(self):
+        self.connected = False
+        await self._stop_watching_video_track()
+        if self._processing_task is not None:
+            self._processing_task.cancel()
+            await self._processing_task
+
+        self._executor.shutdown(wait=False)
+
+        if self._real_client is not None:
+            await self._real_client.close()
+            self._real_client = None
+
+    async def watch_video_track(
+        self,
+        track: aiortc.mediastreams.MediaStreamTrack,
+        shared_forwarder: Optional[VideoForwarder] = None,
+    ) -> None:
+        """
+        Start sending video frames using VideoForwarder.
+
+        Args:
+            track: Video track to watch
+            shared_forwarder: Optional shared VideoForwarder to use instead of creating a new one
+        """
+
+        # This method can be called multiple times with different forwarders
+        # Remove handler from old forwarder if it exists
+        await self._stop_watching_video_track()
+        if self._video_forwarder is not None:
+            await self._video_forwarder.remove_frame_handler(self._send_video_frame)
+
+        self._video_forwarder = shared_forwarder or VideoForwarder(
+            input_track=cast(VideoStreamTrack, track),
+            max_buffer=5,
+            fps=float(self.fps),
+            name="qwen3realtime_forwarder",
+        )
+
+        # Add frame handler (starts automatically)
+        self._video_forwarder.add_frame_handler(self._send_video_frame, fps=self.fps)
+        logger.info(f"Started video forwarding with {self.fps} FPS")
+
+    async def _send_video_frame(self, frame: av.VideoFrame) -> None:
+        """
+        Send a video frame to Qwen3 Realtime API using send_realtime_input
+
+        Parameters:
+            frame: Video frame to send.
+        """
+        if not self._audio_emitted_once:
+            # Wait until the audio is sent at least once before forwarding frames
+            # per the model spec.
+            return
+
+        loop = asyncio.get_running_loop()
+
+        # Run frame conversion in a separate thread to avoid blocking the loop.
+        jpg_bytes = await loop.run_in_executor(
+            self._executor,
+            frame_to_jpeg_bytes,
+            frame,
+            self._video_width,
+            self._video_height,
+        )
+
+        try:
+            await self._client.send_frame(jpg_bytes)
+        except Exception:
+            logger.exception("Failed to send a video frame to Qwen3 Realtime API")
+
+    async def _stop_watching_video_track(self) -> None:
+        if self._video_forwarder is not None:
+            await self._video_forwarder.remove_frame_handler(self._send_video_frame)
+
+    @property
+    def _client(self) -> Qwen3RealtimeClient:
+        if self._real_client is None:
+            raise ValueError("The Qwen3Realtime session is not established yet")
+        return self._real_client
+
+    async def _processing_loop(self) -> None:
+        logger.debug("Start processing events by Qwen3Realtime")
+        try:
+            await self._process_events()
+        except asyncio.CancelledError:
+            logger.debug("Stop processing events by Qwen3Realtime")
+
+    def _start_processing_task(self) -> None:
+        self._processing_task = asyncio.create_task(self._processing_loop())
+
+    async def _stop_processing_task(self) -> None:
+        if self._processing_task is not None:
+            self._processing_task.cancel()
+            await self._processing_task
+
+    async def _process_events(self):
+        async for event in self._client.read():
+            event_type = event.get("type")
+            if event_type == "error":
+                error = event["error"]
+                logger.error(
+                    f"Error received from Qwen3Realtime API: {error}",
+                )
+                self.events.send(
+                    events.LLMErrorEvent(plugin_name=PLUGIN_NAME, error_message=error)
+                )
+                continue
+
+            elif event_type == "session.created":
+                logger.debug("Qwen3Realtime session initialized successfully")
+
+            elif event_type == "response.created":
+                self._current_response_id = event.get("response", {}).get("id")
+                self._is_responding = True
+            elif event_type == "response.output_item.added":
+                self._current_item_id = event.get("item", {}).get("id")
+            elif event_type == "response.done":
+                self._is_responding = False
+                self._current_response_id = None
+                self._current_item_id = None
+            elif event_type == "input_audio_buffer.speech_started":
+                if self._is_responding:
+                    await self._on_interruption()
+            elif event_type == "response.text.delta":
+                self.events.send(
+                    LLMResponseChunkEvent(
+                        plugin_name=PLUGIN_NAME, delta=str(event["delta"])
+                    )
+                )
+            elif event_type == "response.audio.delta":
+                audio_bytes = base64.b64decode(event["delta"])
+                pcm = PcmData.from_bytes(audio_bytes, 24000)
+                self._emit_audio_output_event(audio_data=pcm)
+            elif event_type == "conversation.item.input_audio_transcription.completed":
+                transcript = event.get("transcript", "")
+                if transcript:
+                    self._emit_user_speech_transcription(text=transcript)
+            elif event_type == "response.audio_transcript.delta":
+                delta = event.get("delta", "")
+                if delta:
+                    self._emit_agent_speech_transcription(text=delta)
+
+    async def _on_interruption(self):
+        """Handle user interruption of the current response."""
+        if not self._is_responding:
+            return
+
+        if self._current_response_id:
+            await self._client.cancel_response()
+
+        self._is_responding = False
+        self._current_response_id = None
+        self._current_item_id = None