@rainbow-robotics/muscat-plugin-kit 1.3.6-dev.8 → 1.3.7

package/cli/templates/common/README.md

@@ -398,6 +398,103 @@ if __name__ == "__main__":
         time.sleep(1)
 ```
 
+#### External endpoints (`plugin.router`)
+
+`plugin.serve` / `plugin.subscribe` / `plugin.publish` are **internal only** (plugin
+UI ↔ backend, plugin ↔ plugin). To expose an endpoint to the outside world —
+REST, Socket.IO, TCP, MQTT, WebRTC — use `plugin.router`. Declaring one is a
+single decorator; the running Muscat (`common`) picks it up without a restart,
+and it disappears when the plugin stops or is removed.
+
+> Requires a robot running `rainbow-rb-sdk` **0.0.9.dev5 or newer** (the SDK is
+> provided by the plugin container at runtime). On older robots `plugin.router`
+> does not exist and raises `AttributeError`.
+
+```python
+from rainbow.rb_sdk import RBPluginCall
+from rainbow.rb_sdk.plugin import PluginRouteError
+
+plugin = RBPluginCall()  # plugin_id comes from env RB_PLUGIN_ID
+
+
+@plugin.router.on("/hello")                                  # all 5 transports
+def hello(payload: dict, meta: dict):
+    return {"msg": f"hi {payload.get('name')}"}              # return value = response body
+
+
+@plugin.router.on("/admin", explicit_routes=["api"])         # REST only
+def admin(payload, meta):
+    if not (meta.get("principal") or {}).get("is_admin"):
+        raise PluginRouteError(403, "admin only")            # becomes the external status
+    ...
+
+
+@plugin.router.subscribe("/cmd", explicit_events=["tcp"])    # receive external TCP publish
+def on_cmd(payload: dict):
+    ...
+
+
+plugin.router.emit("/state", {"busy": True})                 # push to external subscribers
+```
+
+| API | Meaning | External address |
+| --- | --- | --- |
+| `router.on(path, explicit_routes=None)` | request/response handler | `common/plugin/{plugin_id}/{path}` (REST is always `POST`) |
+| `router.subscribe(topic, explicit_events=None)` | receive external publishes | external side publishes to `common/plugin/{plugin_id}/{topic}` |
+| `router.emit(topic, payload)` | plugin → external push | external side subscribes to `common/plugin/{plugin_id}/{topic}` |
+
+- The `plugin_id` prefix is added by the SDK — write only the path in plugin code.
+- Paths may have several segments (`/tool/echo`). Declaring the same path twice raises `ValueError`.
+- Transport names: `api | socket | tcp | mqtt | webrtc`. `subscribe` accepts only `tcp | mqtt | webrtc`. Unknown names raise `ValueError`.
+- `explicit_routes` / `explicit_events` of `None` means every transport. A request on a transport not listed gets 404.
+
+Handler contract:
+
+- Signature `(payload, meta)`, `(payload)` or `()` — only the arguments you declare are injected. Sync and async both work.
+- `payload`: what the caller sent (for REST: JSON body merged with query params). The shape is up to the plugin.
+- `meta`: `{"transport": "api"|"socket"|"tcp"|"mqtt"|"webrtc", "principal": dict | None}`. When the robot's login policy is on, `principal` holds the token subject. **Role checks are the plugin's job** — inspect `meta["principal"]` yourself.
+- The return value is the response body (must be JSON-serialisable). `None` → empty response.
+- Status codes: `raise PluginRouteError(code, message)`. Any other exception → 500 with `str(exc)` as the message.
+- `subscribe` handlers take `(payload)` only and return nothing.
+
+#### Calling plugin endpoints from outside
+
+| Transport | How |
+| --- | --- |
+| REST | `POST /common/plugin/{plugin_id}/{path}` (through the api-gateway), JSON body |
+| Socket.IO | emit event `common/plugin/{plugin_id}/{path}`, response arrives in the ack |
+| TCP | topic `common/plugin/{plugin_id}/{path}` request / publish |
+| MQTT | request: `common/plugin/{plugin_id}/{path}/__req__`, publish: `common/plugin/{plugin_id}/{topic}` |
+| WebRTC | data channel topic `common/plugin/{plugin_id}/{path}` |
+
+Auth: when the robot's login policy is on, a token is **required** (any role — GUEST passes). When it is off, calls go straight through. REST uses the same Bearer token as the other Muscat APIs.
+
+```bash
+curl -s -X POST http://<robot>/common/plugin/my-plugin/hello \
+  -H 'Content-Type: application/json' -d '{"name":"kim"}'
+```
+
+| Situation | Status |
+| --- | --- |
+| No plugin declares that path / plugin is down / transport not allowed | 404 |
+| Plugin did not answer within 10 s | 504 |
+| `common` ↔ gateway not connected | 503 |
+| Handler raised | 500 (`PluginRouteError` → its `code`) |
+
+Known limits:
+
+- A plugin path named `install` cannot be called over REST (`POST /common/plugin/{id}/install` is taken by the registry install route first). Socket/TCP are unaffected.
+- Swagger shows one catch-all route, not per-plugin paths.
+- The MQTT broker echoes to the sender: a plugin that `emit`s and `subscribe`s the same topic receives its own push over `mqtt`. Drop `mqtt` from `explicit_events` or use separate topics. (TCP is filtered by `sender_id`.)
+- `on` and `subscribe` may share a name — they live in separate tables.
+
+During `muscat-pk dev` / `dev:back` the local bridge (port `10000`) accepts the
+`router.*` registrations, but **external calls need a running `common`** — it is
+`common` that routes `common/plugin/...` to the gateway. Use a robot (or a local
+`common`) to exercise the external addresses. After restarting the plugin process,
+external calls may return 504 for up to about a minute (until the gateway notices
+the old socket is gone), then 404 — that is expected.
+
 #### Program command backend (`program.panel`)
 
 A `program.panel` contribution defines a **custom command** your plugin adds to
@@ -637,9 +734,8 @@ below).
 
 ### C++
 
-C++ plugins use `cpp/` and are built with CMake inside Docker. The current C++
-template is a build/run skeleton; wire it to the plugin gateway when the C++ SDK
-is available.
+C++ plugins use `cpp/` and are built with CMake inside Docker against the
+Rainbow Robotics C++ SDK (see _C++ SDK_ below).
 
 ```cpp
 #include <chrono>
@@ -661,10 +757,31 @@ int main() {
 }
 ```
 
+#### C++ SDK (`rainbow-rb-sdk-dev`)
+
+Both C++ runtimes (`cpp`, `ros2` + `rclcpp`) link the Rainbow Robotics C++ SDK
+from the apt package `rainbow-rb-sdk-dev`. The `muscat-pk` dev containers already
+have it, so `muscat-pk dev` / `build` need nothing extra. To build outside the
+container (your own Ubuntu 24.04 machine, or the robot PC):
+
+```bash
+curl -fsSL https://rainbowrobotics.github.io/rb-sdk/rb-sdk-apt-repo.gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/rb-sdk-apt.gpg
+echo 'deb [signed-by=/usr/share/keyrings/rb-sdk-apt.gpg] https://rainbowrobotics.github.io/rb-sdk stable main' | sudo tee /etc/apt/sources.list.d/rb-sdk.list
+sudo apt update && sudo apt install rainbow-rb-sdk-dev
+```
+
+- CMake: `find_package(rb_sdk REQUIRED)` + `target_link_libraries(<target> rb_sdk::rb_sdk)` (C++20). pkg-config: `rb_sdk`.
+- Headers: `<rainbow/rb_sdk/manipulate/rb_manipulate_sdk.hpp>` (`RBManipulateSDK`),
+  `<rainbow/rb_sdk/program/plugin_program_context.hpp>` (`rb_plugin::PluginProgramContext`, for `program.panel` commands — the header's top comment has a full `main()` example).
+- The binary is linked **dynamically**; `muscat-pk build` copies the matching `librb_sdk.so` into `dist/<arch>/lib/` (the robot has no SDK installed), so each plugin keeps the SDK version it was built with. Change the version with `MUSCAT_PK_RB_SDK_VERSION=<version> MUSCAT_PK_REBUILD_IMAGE=1 muscat-pk dev`.
+- UI ↔ backend messaging works from C++ too: `rb_plugin::RBPluginCall` in `<rainbow/rb_sdk/plugin/rb_plugin.hpp>` (see the scaffolded `cpp/main.cpp`).
+
 ### ROS2 (rclpy)
 
-ROS2 Python plugins use `ros2_ws/` with `runtime.rosClient: "rclpy"`. The node
-is launched with `--ros-args -r __ns:=/{slug}` automatically by `dev:back`.
+ROS2 Python plugins use `ros2_ws/` with `runtime.rosClient: "rclpy"`. relative topic names are placed under the plugin's ROS namespace — the plugin id
+with "." → "/" and "-" → "_" (`com.example.my-plugin` → `/com/example/my_plugin`), in dev
+and on the robot. Use an absolute name (`/cmd_vel`) for topics shared with other nodes.
+`RBPluginCall` works in the node module too (UI messaging and `plugin.router`).
 
 ```python
 # ros2_ws/src/__PLUGIN_SLUG__/__PLUGIN_SLUG__/node.py
@@ -682,9 +799,29 @@ class __PLUGIN_CLASS__Node(Node):
         # use sdk alongside ROS2 publishers/subscribers/services as needed
 ```
 
+
+#### Talking to an external PC over ROS 2
+
+ROS 2 discovery stays on the robot (`ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST`; `muscat-pk dev`
+runs the same), so robots on one LAN don't mix topics. Allow specific PCs from the plugin UI —
+Muscat keeps the list per plugin, starts your backend with `ROS_STATIC_PEERS` set to it, and
+restarts the plugin when it changes. Your ROS code needs no changes; the PC needs no extra settings.
+
+```tsx
+const rb = useRb();
+const peers = await rb.rosPeers.get();               // ["192.168.0.50"]
+await rb.rosPeers.set(["192.168.0.50", "192.168.0.51"]); // saves + restarts the plugin; rejects on a bad IP
+```
+
+- Per plugin: plugin B keeps its allowed PCs when plugin A (allowing the same PC) is stopped or removed.
+- It opens communication; it is not access control — a PC that knows the robot's IP can still
+  add the robot to its own `ROS_STATIC_PEERS`.
+- In `muscat-pk dev` the list is kept in `.muscat-dev/ros-peers.json` (ROS 2 backends).
+
 ### ROS2 (rclcpp)
 
-ROS2 C++ plugins use `ros2_ws/` with `runtime.rosClient: "rclcpp"`.
+ROS2 C++ plugins use `ros2_ws/` with `runtime.rosClient: "rclcpp"`. The node links
+the Rainbow C++ SDK the same way as the `cpp` runtime (see _C++ SDK_ above).
 
 ```cpp
 #include "rclcpp/rclcpp.hpp"
@@ -699,6 +836,65 @@ public:
 
 ---
 
+## Video streaming (MediaMTX)
+
+Muscat runs a MediaMTX media server. The backend publishes a stream over RTSP and
+the UI plays it over WebRTC. `muscat-pk dev` runs the same server (or uses a local
+Muscat's), so the addresses are identical in dev and on the robot.
+
+| | Address |
+| --- | --- |
+| Publish (backend) | `rtsp://127.0.0.1:8554/<plugin_id>/<name>` |
+| Play (UI) | `http://<host>:8889/<plugin_id>/<name>/` |
+
+- Prefix the path with your plugin id — paths are shared with robot cameras and other plugins.
+- The robot runtime has no ffmpeg, GStreamer or OpenCV; bring your own encoder (below).
+
+**Backend (Python)** — use PyAV: add `av` and `numpy` to `requirements.txt`. Its wheels
+include FFmpeg, so they ship in the ZIP like any other dependency.
+
+```python
+import fractions, time
+import av, numpy as np
+
+out = av.open("rtsp://127.0.0.1:8554/__PLUGIN_ID__/cam", mode="w", format="rtsp",
+              options={"rtsp_transport": "tcp"})
+stream = out.add_stream("libx264", rate=25, options={"preset": "ultrafast", "tune": "zerolatency"})
+stream.width, stream.height, stream.pix_fmt = 640, 360, "yuv420p"
+
+i = 0
+while True:
+    img = np.zeros((360, 640, 3), dtype=np.uint8)          # your RGB frame here
+    frame = av.VideoFrame.from_ndarray(img, format="rgb24")
+    frame.pts, frame.time_base = i, fractions.Fraction(1, 25)
+    for packet in stream.encode(frame):
+        out.mux(packet)
+    i += 1
+    time.sleep(1 / 25)
+```
+
+Run the encoder loop in a thread (or `asyncio.to_thread`) so it doesn't block the
+`RBPluginCall` handlers.
+
+**UI** — embed MediaMTX's WebRTC player. `location.hostname` is `localhost` in the
+preview and the robot's address on the robot:
+
+```tsx
+const STREAM = "__PLUGIN_ID__/cam"; // the path the backend publishes
+
+export default function Camera() {
+  return (
+    <iframe
+      src={`${location.protocol}//${location.hostname}:8889/${STREAM}/`}
+      allow="autoplay"
+      className="aspect-video w-full border-0"
+    />
+  );
+}
+```
+
+---
+
 ## manifest.json
 
 ```jsonc
@@ -927,6 +1123,8 @@ muscat-pk type change --to python     # → Python
 
 > Existing backend dirs (`ros2_ws/`, `cpp/`) are not deleted when switching away. Remove them manually if no longer needed.
 
+> Switching between a Python and a C++ backend asks whether to replace this README with the matching one (the old file is kept as `README.md.bak`). Add `--readme` to answer yes without the prompt.
+
 ---
 
 ## AI coding assistants (MCP)
@@ -943,7 +1141,9 @@ letting it guess. `muscat-pk mcp` is an MCP server exposing:
 - `search_rb_sdk` — how to use `useRb()`, `rb.*`, `useRbFormData`/`useRbSave`, and
   the backend runtimes.
 - `list_rb_sdk` / `get_rb_sdk_api` — the installed `rainbow-rb-sdk` (Python
-  backend) classes and signatures.
+  backend) classes and signatures; with `lang: "cpp"`, the C++ SDK headers
+  (`rainbow-rb-sdk-dev`, read out of the `muscat-pk` cpp dev image — build it
+  once with `muscat-pk dev:back`).
 
 **Browse the components live** with `muscat-pk storybook`.
 
@@ -991,6 +1191,10 @@ muscat-pk build            # build + ZIP
 muscat-pk build --no-zip   # build only, no ZIP
 ```
 
+> Building the other arch (e.g. arm64 on an amd64 PC) runs in an emulated container for
+> C++ backends. Docker Desktop has it built in; on Linux enable it once with
+> `docker run --privileged --rm tonistiigi/binfmt --install all`, or pass `--arch` for one arch.
+
 - Builds `ui/content.tsx` → `dist/content.js` (minified)
 - Pins `requirements.txt` to latest versions (stable-only on `main` branch)
 - Copies `assets/`, `manifest.json`, `README*.md`, `py/`, and `requirements.txt` into `dist/`

package/cli/templates/common/py/main.py

@@ -10,6 +10,13 @@ Remote dev (from your laptop):
     MUSCAT_ROBOT_HOST=192.168.1.100
     MUSCAT_ROBOT_TOKEN=<service token>
   Then: muscat-pk dev:back
+
+Handlers:
+  plugin.serve / plugin.subscribe / plugin.publish  -> plugin UI <-> backend only
+  plugin.router.on / subscribe / emit               -> external endpoints
+    (REST / Socket.IO / TCP / MQTT / WebRTC at common/plugin/<plugin_id>/<path>).
+    Needs a robot running rainbow-rb-sdk >= 0.0.9.dev5; older robots have no
+    `plugin.router` (AttributeError). See README "External endpoints".
 """
 
 import asyncio
@@ -17,6 +24,7 @@ import os
 from typing import Any
 
 from rainbow.rb_sdk import RBManipulateSDK, RBPluginCall
+from rainbow.rb_sdk.rb_sdk_cpp import ConnectOptions
 
 
 PLUGIN_NAME = "__PLUGIN_NAME__"
@@ -29,6 +37,8 @@ def register_handlers():
     # Register handlers here — called after connect() so subscriptions survive
     # session reconnection in remote-dev mode.
 
+    # --- Plugin UI <-> backend (not reachable from outside Muscat) ---
+
     @plugin.subscribe("event")
     def on_ui_event(payload: dict[str, Any] | None = None):
         """Handle messages published from the plugin UI with rb.pub("event", ...)."""
@@ -47,6 +57,19 @@ def register_handlers():
         """Small query example callable from the UI with rb.action("tool/echo", ...)."""
         return {"echo": payload, "plugin": PLUGIN_NAME}
 
+    # --- External endpoints (REST / Socket.IO / TCP / MQTT / WebRTC) ---
+    # Exposed as common/plugin/<plugin_id>/hello, e.g.
+    #   curl -X POST http://<robot>/common/plugin/<plugin_id>/hello \
+    #        -H 'Content-Type: application/json' -d '{"name": "kim"}'
+    # Restrict transports with explicit_routes=["api"] (REST only), etc.
+    # Raise PluginRouteError(403, "msg") to set the response status.
+
+    @plugin.router.on("/hello")
+    def hello(payload: dict[str, Any] | None = None, meta: dict[str, Any] | None = None):
+        """meta = {"transport": "api"|"socket"|"tcp"|"mqtt"|"webrtc", "principal": dict | None}."""
+        name = (payload or {}).get("name", "there")
+        return {"msg": f"hi {name}", "via": (meta or {}).get("transport")}
+
 
 def move_j_example():
     """
@@ -73,11 +96,15 @@ async def main():
         # Remote dev: connect to the robot before registering any handlers.
         # connect() closes and reopens the Zenoh session, so handlers registered
         # before this call would be lost.
-        await manipulate.connect(token=robot_token, host=robot_host)
+        opts = ConnectOptions()
+        opts.token = robot_token
+        opts.host = robot_host
+        manipulate.connect(opts)
 
     register_handlers()
 
-    plugin.publish("state", {"ready": True, "plugin": PLUGIN_NAME})
+    plugin.publish("state", {"ready": True, "plugin": PLUGIN_NAME})       # -> plugin UI
+    plugin.router.emit("/state", {"ready": True, "plugin": PLUGIN_NAME})  # -> external subscribers of common/plugin/<plugin_id>/state
     print(f"[{PLUGIN_NAME}] backend ready")
 
     try:

package/cli/templates/cpp/cpp/CMakeLists.txt

@@ -1,14 +1,13 @@
 cmake_minimum_required(VERSION 3.16)
 project(__PLUGIN_SLUG__ CXX)
 
-set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD 20)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 
+# Rainbow Robotics C++ SDK — provided by the apt package rainbow-rb-sdk-dev
+# (pre-installed in the muscat-pk dev container).
+find_package(rb_sdk REQUIRED)
+
 # The executable name must match runtime.entry in manifest.json.
 add_executable(__PLUGIN_SLUG__ main.cpp)
-
-# TODO(cpp-sdk): link the rainbow cpp SDK dynamically (the robot provides the
-# .so at runtime — do NOT bundle/static-link it). Once the SDK ships an apt
-# package with CMake config, enable:
-#   find_package(rainbow_cpp_sdk REQUIRED)
-#   target_link_libraries(__PLUGIN_SLUG__ PRIVATE rainbow::cpp_sdk)
+target_link_libraries(__PLUGIN_SLUG__ PRIVATE rb_sdk::rb_sdk)

package/cli/templates/cpp/cpp/main.cpp

@@ -1,9 +1,19 @@
 // __PLUGIN_NAME__ — C++ plugin back end.
 //
-// The C++ plugin SDK is not available yet, so this is a build/run skeleton.
-// Once the SDK lands, connect to the plugin gateway here (the host exposes it
-// at the RB_PLUGIN_GATEWAY / RB_PLUGIN_ID environment variables) and wire up
-// pub / sub / serve, mirroring the Python rainbow.rb_sdk.RBPluginCall API.
+// The Rainbow Robotics C++ SDK (apt: rainbow-rb-sdk-dev) is installed in the
+// dev container and linked via CMake (rb_sdk::rb_sdk). Headers live under
+// <rainbow/rb_sdk/...>:
+//   - plugin/rb_plugin.hpp               rb_plugin::RBPluginCall — plugin UI <-> backend
+//                                        (serve/subscribe/publish) and external endpoints (router)
+//   - manipulate/rb_manipulate_sdk.hpp   RBManipulateSDK — control the robot
+//   - program/plugin_program_context.hpp rb_plugin::PluginProgramContext —
+//                                        program.panel commands (done/pause/stop,
+//                                        program variables)
+
+#include <rainbow/rb_sdk/manipulate/rb_manipulate_sdk.hpp>
+#include <rainbow/rb_sdk/plugin/rb_plugin.hpp>
+
+#include <nlohmann/json.hpp>
 
 #include <chrono>
 #include <cstdlib>
@@ -11,14 +21,51 @@
 #include <string>
 #include <thread>
 
+using nlohmann::json;
+using rb_plugin::RBPluginCall;
+
 int main() {
-  const char* plugin_id = std::getenv("RB_PLUGIN_ID");
-  std::cout << "[__PLUGIN_NAME__] backend ready"
-            << (plugin_id ? std::string(" (id=") + plugin_id + ")" : "")
-            << std::endl;
+  const std::string name = "__PLUGIN_NAME__";
+
+  // Remote dev: connect before registering handlers (connect() reopens the session).
+  RBManipulateSDK manipulate;
+  const char* host = std::getenv("MUSCAT_ROBOT_HOST");
+  const char* token = std::getenv("MUSCAT_ROBOT_TOKEN");
+  if (host && token) manipulate.connect({.token = token, .host = host});
+
+  // plugin id comes from RB_PLUGIN_ID.
+  auto plugin = RBPluginCall::get_instance(RBPluginCall::resolve_plugin_id(std::nullopt));
+
+  // --- Plugin UI <-> backend (not reachable from outside Muscat) ---
+
+  // UI: await rb.action("grip", { width })
+  plugin->serve("grip", [&](const json& payload) {
+    const int width = payload.is_object() ? payload.value("width", 20) : 20;
+    std::cout << "[" << name << "] grip requested: width=" << width << std::endl;
+    plugin->publish("state", {{"ready", true}, {"last_grip_width", width}});  // -> rb.sub("state")
+    return json{{"ok", true}, {"width", width}};
+  });
+
+  // UI: rb.pub("event", ...)
+  auto event_sub = plugin->subscribe("event", [&](const std::string&, const json& payload) {
+    std::cout << "[" << name << "] event: " << payload.dump() << std::endl;
+  });
+
+  // --- External endpoints (REST / Socket.IO / TCP / MQTT / WebRTC) ---
+  //   curl -X POST http://<robot>/common/plugin/<plugin_id>/hello -d '{"name":"kim"}'
+  plugin->router().on("/hello", [](const json& payload, const json& meta) {
+    const std::string who = payload.is_object() ? payload.value("name", std::string("there")) : "there";
+    return json{{"msg", "hi " + who}, {"via", meta.value("transport", std::string())}};
+  });
+
+  // Example: robot control (not called automatically — add safety checks first).
+  //   IPC::MoveInput_TargetT t; t.tar_values = {0, 0, 0, 0, 0, 0};
+  //   manipulate.move.call_move_j("C500920", t);
+
+  plugin->publish("state", {{"ready", true}, {"plugin", name}});
+  std::cout << "[" << name << "] backend ready" << std::endl;
 
-  // TODO: connect to the plugin gateway and register handlers once the
-  //       C++ SDK is available. For now, just keep the process alive.
+  // Keep the process alive.
   while (true) {
     std::this_thread::sleep_for(std::chrono::seconds(1));
   }