stackchan-mcp 0.3.0__tar.gz → 0.5.0__tar.gz

{stackchan_mcp-0.3.0 → stackchan_mcp-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: stackchan-mcp
-Version: 0.3.0
+Version: 0.5.0
 Summary: Two-faced MCP gateway for StackChan (xiaozhi-esp32): bridges stdio MCP clients to the ESP32 over WebSocket + HTTP.
 Project-URL: Homepage, https://github.com/kisaragi-mochi/stackchan-mcp
 Project-URL: Repository, https://github.com/kisaragi-mochi/stackchan-mcp
@@ -27,6 +27,12 @@ Requires-Dist: mcp>=1.0
 Requires-Dist: pydantic>=2
 Requires-Dist: python-dotenv
 Requires-Dist: websockets>=12
+Provides-Extra: tts
+Requires-Dist: httpx>=0.27; extra == 'tts'
+Requires-Dist: opuslib>=3; extra == 'tts'
+Provides-Extra: tts-voicevox
+Requires-Dist: httpx>=0.27; extra == 'tts-voicevox'
+Requires-Dist: opuslib>=3; extra == 'tts-voicevox'
 Description-Content-Type: text/markdown
 
 # gateway
@@ -114,6 +120,29 @@ will notice the dropped WebSocket and retry while idle. The retry delay starts
 at 5 seconds and backs off up to 60 seconds. After the gateway is listening
 again, check `get_status` from the stdio MCP side to confirm the device is back.
 
+## Configuration changes
+
+The gateway reads `.env` once at process start. Because the gateway runs as a
+**stdio MCP server** (it has no standalone CLI mode beyond `--help` /
+`--version` / `--check`), editing `.env` while it is connected to an MCP
+client does not take effect on the running process — and killing the gateway
+process directly will not auto-restart it; the MCP client owns the lifecycle.
+
+After editing `.env` (for example to update `STACKCHAN_TOKEN`, `VISION_URL`,
+or `VISION_TOKEN`):
+
+1. Reconnect the MCP client. In Claude Code this is `/mcp` to reconnect, or a
+   full Claude Code restart.
+2. Confirm `mcp__stackchan-mcp__get_status` returns `connected: true` with the
+   expected `tools_count`.
+3. If the ESP32 was already connected with a stale auth credential, hard-reset
+   the device (`esptool.py --before default_reset --after hard_reset chip_id`,
+   or DTR/RTS toggle via pyserial) so it reconnects with the fresh
+   configuration.
+
+`STACKCHAN_TOKEN` takes precedence over the legacy `BEARER_TOKEN`; setting
+either is enough, but if you have both, keep them aligned.
+
 ## Tests
 
 ```bash
@@ -165,8 +194,19 @@ Same shape, under `mcpServers`.
 | `get_touch_state` | Touch sensor state (press/release/stroke) |
 | `set_avatar(face)` | Switch avatar expression (`idle` / `happy` / `thinking` / `sad` / `surprised` / `embarrassed`), or `off` to hide the avatar and disable blink so the underlying xiaozhi-esp32 screens (WiFi config UI, OTA, settings) are visible. A subsequent `set_avatar(<other face>)` brings it back and restores blink. |
 | `set_blink(state)` | Blink animation on/off |
-| `set_mouth(state)` | Mouth shape (`closed` / `half` / `open` / `e` / `u`) |
+| `set_mouth(state)` | Mouth shape (`closed` / `half` / `open` / `e` / `u`), one-shot, held until next call |
+| `set_mouth_sequence(steps)` | Queue and play a list of `{shape, duration_ms}` steps locally for TTS lip-sync. The firmware walks the queue without per-step network RTT. Calling `set_mouth`, `set_avatar`, or this tool again interrupts the in-flight sequence; autonomous blink is paused while a sequence is playing. |
 | `check_vm_en` | Read PY32 VM EN GPIO state (servo power supply diagnostic) |
+| `set_led(index, r, g, b)` | Set one of the 12 base RGB LEDs by index (`0..11`); channels `0..255`. Updates immediately. |
+| `set_all_leds(r, g, b)` | Set all 12 base RGB LEDs to the same color. Updates immediately. |
+| `set_leds(colors)` | Batch-set the first N LEDs from a `[[r,g,b], ...]` array (1..12 entries). Single I2C burst + one latch — use this for animations / multi-color patterns instead of N individual `set_led` calls. Trailing LEDs (beyond `len(colors)`) keep their previous color. Validation is atomic: a malformed entry rejects the whole call without mutating any LED. |
+| `clear_leds` | Turn all 12 base RGB LEDs off. |
+
+The 12 base LEDs are 12× WS2812C wired to the PY32L020 IO expander
+(expander pin 13, not an ESP32 GPIO), so all four LED tools share the
+PY32 I2C bus with the servo-power and Si12T touch paths. If the PY32
+init fails at boot, the LED tools degrade with `available=false`
+instead of cascading errors.
 
 The mapping from these names to ESP32-side `self.*` MCP tools is in
 `stackchan_mcp/stdio_server.py`.

{stackchan_mcp-0.3.0 → stackchan_mcp-0.5.0}/README.md

@@ -83,6 +83,29 @@ will notice the dropped WebSocket and retry while idle. The retry delay starts
 at 5 seconds and backs off up to 60 seconds. After the gateway is listening
 again, check `get_status` from the stdio MCP side to confirm the device is back.
 
+## Configuration changes
+
+The gateway reads `.env` once at process start. Because the gateway runs as a
+**stdio MCP server** (it has no standalone CLI mode beyond `--help` /
+`--version` / `--check`), editing `.env` while it is connected to an MCP
+client does not take effect on the running process — and killing the gateway
+process directly will not auto-restart it; the MCP client owns the lifecycle.
+
+After editing `.env` (for example to update `STACKCHAN_TOKEN`, `VISION_URL`,
+or `VISION_TOKEN`):
+
+1. Reconnect the MCP client. In Claude Code this is `/mcp` to reconnect, or a
+   full Claude Code restart.
+2. Confirm `mcp__stackchan-mcp__get_status` returns `connected: true` with the
+   expected `tools_count`.
+3. If the ESP32 was already connected with a stale auth credential, hard-reset
+   the device (`esptool.py --before default_reset --after hard_reset chip_id`,
+   or DTR/RTS toggle via pyserial) so it reconnects with the fresh
+   configuration.
+
+`STACKCHAN_TOKEN` takes precedence over the legacy `BEARER_TOKEN`; setting
+either is enough, but if you have both, keep them aligned.
+
 ## Tests
 
 ```bash
@@ -134,8 +157,19 @@ Same shape, under `mcpServers`.
 | `get_touch_state` | Touch sensor state (press/release/stroke) |
 | `set_avatar(face)` | Switch avatar expression (`idle` / `happy` / `thinking` / `sad` / `surprised` / `embarrassed`), or `off` to hide the avatar and disable blink so the underlying xiaozhi-esp32 screens (WiFi config UI, OTA, settings) are visible. A subsequent `set_avatar(<other face>)` brings it back and restores blink. |
 | `set_blink(state)` | Blink animation on/off |
-| `set_mouth(state)` | Mouth shape (`closed` / `half` / `open` / `e` / `u`) |
+| `set_mouth(state)` | Mouth shape (`closed` / `half` / `open` / `e` / `u`), one-shot, held until next call |
+| `set_mouth_sequence(steps)` | Queue and play a list of `{shape, duration_ms}` steps locally for TTS lip-sync. The firmware walks the queue without per-step network RTT. Calling `set_mouth`, `set_avatar`, or this tool again interrupts the in-flight sequence; autonomous blink is paused while a sequence is playing. |
 | `check_vm_en` | Read PY32 VM EN GPIO state (servo power supply diagnostic) |
+| `set_led(index, r, g, b)` | Set one of the 12 base RGB LEDs by index (`0..11`); channels `0..255`. Updates immediately. |
+| `set_all_leds(r, g, b)` | Set all 12 base RGB LEDs to the same color. Updates immediately. |
+| `set_leds(colors)` | Batch-set the first N LEDs from a `[[r,g,b], ...]` array (1..12 entries). Single I2C burst + one latch — use this for animations / multi-color patterns instead of N individual `set_led` calls. Trailing LEDs (beyond `len(colors)`) keep their previous color. Validation is atomic: a malformed entry rejects the whole call without mutating any LED. |
+| `clear_leds` | Turn all 12 base RGB LEDs off. |
+
+The 12 base LEDs are 12× WS2812C wired to the PY32L020 IO expander
+(expander pin 13, not an ESP32 GPIO), so all four LED tools share the
+PY32 I2C bus with the servo-power and Si12T touch paths. If the PY32
+init fails at boot, the LED tools degrade with `available=false`
+instead of cascading errors.
 
 The mapping from these names to ESP32-side `self.*` MCP tools is in
 `stackchan_mcp/stdio_server.py`.

{stackchan_mcp-0.3.0 → stackchan_mcp-0.5.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "stackchan-mcp"
-version = "0.3.0"
+version = "0.5.0"
 description = "Two-faced MCP gateway for StackChan (xiaozhi-esp32): bridges stdio MCP clients to the ESP32 over WebSocket + HTTP."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -32,6 +32,22 @@ dependencies = [
     "aiohttp>=3",
 ]
 
+[project.optional-dependencies]
+# Phase 4 TTS — see Issue #70.
+# Concrete engines (VOICEVOX, Irodori) consume these libraries:
+#   * httpx     — VOICEVOX HTTP engine client
+#   * opuslib   — Opus encoding for the device's audio decoder
+# `tts-voicevox` is a no-op alias provided so users can declare intent
+# explicitly; the VOICEVOX engine itself is an external HTTP process and
+# adds no Python dependencies of its own.
+tts = [
+    "httpx>=0.27",
+    "opuslib>=3",
+]
+tts-voicevox = [
+    "stackchan-mcp[tts]",
+]
+
 [project.urls]
 Homepage = "https://github.com/kisaragi-mochi/stackchan-mcp"
 Repository = "https://github.com/kisaragi-mochi/stackchan-mcp"

stackchan_mcp-0.5.0/stackchan_mcp/audio_stream.py

@@ -0,0 +1,52 @@
+"""Opus audio frame handling for the gateway <-> device link.
+
+Outbound (TTS) frames are produced by
+:mod:`stackchan_mcp.tts.audio_utils` and pushed here to the connected
+ESP32 via :meth:`stackchan_mcp.esp32_client.ESP32Manager.send_audio_frame`.
+
+The inbound side (STT pipeline, Phase 4 / Issue #8) is still a stub —
+binary frames coming up from the device are logged and discarded for
+now. Wiring that up belongs to the STT half of Phase 4.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, Iterable
+
+if TYPE_CHECKING:
+    from .esp32_client import ESP32Manager
+
+logger = logging.getLogger(__name__)
+
+
+async def handle_audio_frame(data: bytes, session_id: str) -> None:
+    """Process an incoming binary Opus frame from the device (stub).
+
+    The STT half of Phase 4 will pipe this into a recogniser; until
+    then we just log the size at debug level.
+    """
+    logger.debug(
+        "audio_frame session=%s bytes=%d (discarded — STT not wired up)",
+        session_id,
+        len(data),
+    )
+
+
+async def push_opus_frames(
+    esp32: ESP32Manager,
+    frames: Iterable[bytes],
+) -> int:
+    """Push Opus frames to the connected ESP32.
+
+    Returns the number of frames sent so the caller can report this to
+    the MCP client. Raises :class:`ConnectionError` (via
+    :meth:`ESP32Manager.send_audio_frame`) if the device disconnects
+    mid-stream — the orchestrator turns that into a clean MCP error
+    rather than letting it bubble up as a stack trace.
+    """
+    sent = 0
+    for frame in frames:
+        await esp32.send_audio_frame(frame)
+        sent += 1
+    return sent

{stackchan_mcp-0.3.0 → stackchan_mcp-0.5.0}/stackchan_mcp/cli.py

@@ -16,6 +16,7 @@ import asyncio
 import errno
 import logging
 import os
+import platform
 import shutil
 import socket
 import subprocess
@@ -377,6 +378,56 @@ def _load_dotenv() -> None:
     load_dotenv()
 
 
+# Default Homebrew prefixes that ship libopus.dylib on macOS. Apple
+# Silicon installs default to ``/opt/homebrew``; Intel Macs use
+# ``/usr/local``. Keeping both keeps the helper portable across
+# contributor machines.
+_HOMEBREW_LIB_DIRS = ("/opt/homebrew/lib", "/usr/local/lib")
+
+
+def _ensure_libopus_findable() -> None:
+    """Make libopus reachable to opuslib's ``ctypes.find_library`` on macOS.
+
+    ``opuslib.api`` calls ``ctypes.util.find_library("opus")`` at
+    import time. On macOS that walks ``DYLD_LIBRARY_PATH`` plus a
+    couple of system-default directories — but not Homebrew's
+    ``/opt/homebrew/lib`` (Apple Silicon) or ``/usr/local/lib`` (Intel),
+    so a vanilla ``brew install opus`` lands a working libopus that
+    opuslib still cannot find. Users then see ``Could not find Opus
+    library`` even though the dylib is on disk.
+
+    Prepend any Homebrew-style lib directories that exist so the next
+    ``find_library`` call (triggered by the lazy ``import opuslib``
+    inside :func:`audio_utils.encode_opus_frames`) succeeds. We
+    deliberately *prepend* and skip duplicates so an explicit
+    ``DYLD_LIBRARY_PATH`` set by the operator (e.g. for a custom build
+    of libopus) keeps priority. No-op on non-macOS hosts.
+    """
+    if platform.system() != "Darwin":
+        return
+
+    existing = os.environ.get("DYLD_LIBRARY_PATH", "")
+    paths: list[str] = [p for p in existing.split(":") if p]
+
+    prepended: list[str] = []
+    for candidate in _HOMEBREW_LIB_DIRS:
+        if candidate in paths:
+            continue
+        if not os.path.isdir(candidate):
+            continue
+        prepended.append(candidate)
+
+    if not prepended:
+        return
+
+    os.environ["DYLD_LIBRARY_PATH"] = ":".join(prepended + paths)
+    logger.debug(
+        "Prepended Homebrew lib dirs to DYLD_LIBRARY_PATH so opuslib "
+        "can find libopus: %s",
+        prepended,
+    )
+
+
 def _run_preflight() -> int:
     """Run preflight diagnostics. Returns the desired process exit code.
 
@@ -387,6 +438,7 @@ def _run_preflight() -> int:
     warns about a missing ``STACKCHAN_TOKEN``.
     """
     _load_dotenv()
+    _ensure_libopus_findable()
 
     issues = 0
     print(f"stackchan-mcp {__version__} preflight")
@@ -527,6 +579,7 @@ def main(argv: list[str] | None = None) -> None:
         sys.exit(_run_preflight())
 
     _load_dotenv()
+    _ensure_libopus_findable()
 
     logging.basicConfig(
         level=logging.INFO,

{stackchan_mcp-0.3.0 → stackchan_mcp-0.5.0}/stackchan_mcp/esp32_client.py

@@ -14,6 +14,7 @@ import uuid
 from typing import Any
 
 import websockets
+import websockets.exceptions
 from websockets.asyncio.server import ServerConnection
 
 from .protocol import HelloResponse, make_mcp_message, parse_jsonrpc_response
@@ -36,6 +37,13 @@ class ESP32Connection:
         self._pending: dict[int, asyncio.Future[dict[str, Any]]] = {}
         self._connected = True
         self._initialized = False
+        # Device-declared WebSocket protocol version (from the hello
+        # message). Defaults to 1, which matches the firmware's default
+        # (firmware/main/protocols/websocket_protocol.h: ``version_ = 1``)
+        # and the audio framing this gateway emits today (raw Opus
+        # payload). v2/v3 add a BinaryProtocol header that this gateway
+        # does not yet wrap — see Issue follow-up to #70.
+        self.protocol_version: int = 1
 
     @property
     def connected(self) -> bool:
@@ -142,6 +150,62 @@ class ESP32Connection:
             method = payload.get("method", "")
             logger.info("ESP32 notification: %s", method)
 
+    async def _ws_send(self, payload: bytes | str) -> None:
+        """Send a payload, translating websockets errors to ConnectionError.
+
+        The ``websockets`` library raises its own exception hierarchy
+        (``ConnectionClosed`` and friends), which is *not* a subclass
+        of the built-in :class:`ConnectionError`. Without translation
+        the orchestrator's ``except ConnectionError`` filter — and the
+        MCP handler's ``except RuntimeError`` filter — would let those
+        errors leak as raw tracebacks into the MCP transport, breaking
+        the say() tool's clean error JSON contract on mid-stream
+        disconnect.
+        """
+        try:
+            await self._ws.send(payload)
+        except (
+            websockets.exceptions.ConnectionClosed,
+            OSError,
+        ) as exc:
+            # Mark the connection dead so subsequent calls fail fast
+            # rather than each one re-discovering the broken socket.
+            self.disconnect()
+            raise ConnectionError(f"WebSocket send failed: {exc}") from exc
+
+    async def send_audio_frame(self, opus_frame: bytes) -> None:
+        """Send a single Opus frame to the ESP32 as a WebSocket binary frame.
+
+        The device's ``OnData`` handler (firmware/main/protocols/
+        websocket_protocol.cc) treats every binary frame as an Opus
+        audio payload to feed into its decoder, so this method is the
+        TTS pipeline's egress point.
+        """
+        if not self._connected:
+            raise ConnectionError("ESP32 not connected")
+        await self._ws_send(opus_frame)
+
+    async def send_tts_state(self, state: str) -> None:
+        """Send a TTS state notification (``start`` / ``stop`` / ...).
+
+        The device's :func:`Application::OnIncomingJson` translates
+        ``{"type":"tts","state":"start"}`` into
+        :data:`kDeviceStateSpeaking`, which is the gate for
+        :func:`OnIncomingAudio` pushing packets into the decode queue
+        (see ``firmware/main/application.cc``). Without bracketing the
+        audio frames in start/stop, the device drops them on the floor
+        and the speaker stays silent — the TTS tool returns success
+        without anything actually playing.
+        """
+        if not self._connected:
+            raise ConnectionError("ESP32 not connected")
+        message = {
+            "session_id": self.session_id,
+            "type": "tts",
+            "state": state,
+        }
+        await self._ws_send(json.dumps(message))
+
     def disconnect(self) -> None:
         """Mark connection as disconnected."""
         self._connected = False
@@ -167,6 +231,17 @@ class ESP32Manager:
         self._init_tasks: list[asyncio.Task] = []
         self._vision_url: str = ""
         self._vision_token: str = ""
+        # Per-device serialisation for TTS send sequences. Acquired by
+        # the orchestrator around the entire start → frames → stop
+        # block so concurrent ``say()`` invocations cannot interleave
+        # their Opus frames on the same WebSocket or overlap their
+        # ``tts.start``/``tts.stop`` notifications (which would yank
+        # the firmware out of ``kDeviceStateSpeaking`` mid-utterance
+        # and silently drop the remaining audio). The lock is scoped
+        # to the manager because the manager owns the device today —
+        # if multi-device support lands later, the lock should move
+        # onto :class:`ESP32Connection` instead.
+        self._tts_lock = asyncio.Lock()
 
     @property
     def device_connected(self) -> bool:
@@ -176,6 +251,15 @@ class ESP32Manager:
     def connection(self) -> ESP32Connection | None:
         return self._connection
 
+    @property
+    def tts_lock(self) -> asyncio.Lock:
+        """Per-device lock guarding the TTS send sequence.
+
+        See :attr:`_tts_lock` for the rationale; the orchestrator wraps
+        the start → frames → stop block in ``async with`` on this lock.
+        """
+        return self._tts_lock
+
     async def start(
         self,
         host: str = "0.0.0.0",
@@ -265,6 +349,27 @@ class ESP32Manager:
                         await ws.close()
                         return
 
+                    # Capture the device's WebSocket protocol version
+                    # so callers (e.g. the TTS pipeline) can decide
+                    # whether their wire format is compatible. The
+                    # firmware accepts raw Opus only on v1; v2/v3 wrap
+                    # the payload in a BinaryProtocol header.
+                    raw_version = data.get("version", 1)
+                    try:
+                        connection.protocol_version = int(raw_version)
+                    except (TypeError, ValueError):
+                        connection.protocol_version = 1
+                    if connection.protocol_version != 1:
+                        logger.warning(
+                            "ESP32 negotiated WebSocket protocol "
+                            "version=%s; the gateway emits raw Opus "
+                            "binary frames matching v1 only. TTS "
+                            "calls (say) will be blocked at the "
+                            "orchestrator until v2/v3 BinaryProtocol "
+                            "header wrapping is implemented",
+                            connection.protocol_version,
+                        )
+
                     # Send hello response
                     resp = HelloResponse(session_id=session_id)
                     await ws.send(resp.model_dump_json())
@@ -323,6 +428,29 @@ class ESP32Manager:
             return None, {"code": -32000, "message": "ESP32 not initialized"}
         return await self._connection.call_tool(name, arguments)
 
+    async def send_audio_frame(self, opus_frame: bytes) -> None:
+        """Push a single Opus frame to the connected device.
+
+        Used by the TTS pipeline to deliver synthesised audio. Raises
+        :class:`ConnectionError` if no device is currently attached so
+        the orchestrator can surface a clean error to the MCP client
+        instead of silently dropping audio.
+        """
+        if not self._connection or not self._connection.connected:
+            raise ConnectionError("No ESP32 device connected")
+        await self._connection.send_audio_frame(opus_frame)
+
+    async def send_tts_state(self, state: str) -> None:
+        """Send a TTS state notification (``start`` / ``stop`` / ...).
+
+        Required around audio frame egress so the device transitions
+        into ``kDeviceStateSpeaking`` and back; see
+        :meth:`ESP32Connection.send_tts_state` for the full rationale.
+        """
+        if not self._connection or not self._connection.connected:
+            raise ConnectionError("No ESP32 device connected")
+        await self._connection.send_tts_state(state)
+
     def get_status(self) -> dict[str, Any]:
         """Get current connection status."""
         if not self._connection or not self._connection.connected: