hivemind-mqtt-protocol 0.1.0a2__tar.gz

hivemind_mqtt_protocol-0.1.0a2/PKG-INFO

@@ -0,0 +1,18 @@
+Metadata-Version: 2.4
+Name: hivemind-mqtt-protocol
+Version: 0.1.0a2
+Summary: MQTT broker-mediated network protocol plugin for hivemind-core
+Author-email: JarbasAi <jarbasai@mailfence.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/JarbasHiveMind/hivemind-mqtt-protocol
+Requires-Python: >=3.10
+Requires-Dist: paho-mqtt>=1.6
+Requires-Dist: hivemind-plugin-manager
+Requires-Dist: poorman_handshake>=0.1.0
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: hivemind-core; extra == "test"
+Requires-Dist: hivemind-bus-client; extra == "test"
+Requires-Dist: ovos-bus-client; extra == "test"
+Requires-Dist: ovos-utils; extra == "test"
+Requires-Dist: hivescope; extra == "test"

hivemind_mqtt_protocol-0.1.0a2/README.md

@@ -0,0 +1,126 @@
+# hivemind-mqtt-protocol
+
+An MQTT broker-mediated network protocol plugin for [hivemind-core](https://github.com/JarbasHiveMind/hivemind-core).
+
+Satellites connect to the same MQTT broker they already use for sensors (Home
+Assistant, ESPHome, Tasmota, ESP32) and exchange encrypted `HiveMessage` frames
+over that broker.  No bespoke inbound port or WebSocket stack is required on the
+hub; both hub and satellites are broker clients.
+
+## The broker-mediated model
+
+```
+satellite ──pub──▶  broker  ◀──sub── hub
+hub       ──pub──▶  broker  ◀──sub── satellite
+```
+
+The hub runs ONE paho-mqtt client.  Logical per-satellite connections are
+derived from the topic hierarchy.
+
+### Topic scheme
+
+```
+<prefix>/<hub_id>/c2s/<satellite_id>     # satellite → hub  (hub subscribes …/c2s/+)
+<prefix>/<hub_id>/s2c/<satellite_id>     # hub → satellite
+<prefix>/<hub_id>/status/<satellite_id>  # retained LWT presence (online / offline)
+```
+
+Defaults: `prefix = hivemind`, `hub_id` = node identity name.
+
+### Privacy option
+
+Set `hash_topics: true` to SHA-256-hash the `satellite_id` segment.  The broker
+then sees only an opaque 16-char hex token — useful when the broker is shared or
+untrusted.
+
+## Crypto
+
+The MQTT payload carries the **same encrypted HiveMessage frame** the WebSocket
+transport sends.  The broker only ever sees ciphertext.  HiveMind's full
+AES-GCM / RSA / PAKE handshake runs unchanged inside the payload.
+
+## Authentication
+
+Two independent layers:
+
+1. **Broker-level** — MQTT `username` / `password`, or TLS client-cert.
+   Configure the broker's ACL so each satellite may only publish to its own
+   `c2s/<key>` topic and subscribe to its own `s2c/<key>` topic.
+
+2. **HiveMind-level** — the HELLO / HANDSHAKE exchange embedded in the
+   encrypted payload, identical to the WebSocket path.  The satellite's MQTT
+   username **must equal** its HiveMind access key so the hub can look up the
+   DB record on first contact.
+
+## QoS
+
+| Traffic type | QoS |
+|---|---|
+| Control frames (default) | 1 (at-least-once) |
+| Binary / audio frames | 0 (fire-and-forget, low latency) |
+
+## Configuration keys
+
+| Key | Default | Description |
+|---|---|---|
+| `broker_host` | `localhost` | MQTT broker hostname or IP |
+| `broker_port` | `1883` | Broker port (8883 for TLS) |
+| `broker_username` | — | MQTT username for the hub |
+| `broker_password` | — | MQTT password for the hub |
+| `tls` | `false` | Enable TLS |
+| `tls_ca_certs` | — | Path to CA bundle |
+| `tls_certfile` | — | Path to client cert (mTLS) |
+| `tls_keyfile` | — | Path to client key (mTLS) |
+| `hub_id` | node identity name | Hub identifier in topics |
+| `topic_prefix` | `hivemind` | Topic namespace prefix |
+| `qos` | `1` | Default MQTT QoS for control frames |
+| `hash_topics` | `false` | Hash `satellite_id` in topics |
+| `idle_timeout` | `300` | Seconds of silence before evicting a peer (0 = off) |
+
+## Usage
+
+```python
+from hivemind_plugin_manager import NetworkProtocolFactory
+
+server = NetworkProtocolFactory.create(
+    "hivemind-mqtt-plugin",
+    config={
+        "broker_host": "192.168.1.100",
+        "broker_port": 1883,
+        "hub_id": "living-room-hub",
+    },
+)
+server.run()   # blocks
+```
+
+## Satellite side
+
+The matching satellite client (publish to `c2s`, subscribe to `s2c`, set the
+LWT on `status/<satellite_id>`) is a planned follow-up as a transport option in
+`hivemind-bus-client` or a dedicated `hivemind-mqtt-client`.  An
+ESPHome / Tasmota external-component example for ESP32 satellites is also
+planned.
+
+## Where it fits
+
+```
+hivemind-core
+  └── hivemind-plugin-manager  (NetworkProtocolFactory loads plugins by entry-point)
+        └── hivemind-mqtt-protocol  ← this repo
+              └── paho-mqtt client connected to an external MQTT broker
+```
+
+The plugin registers under the `hivemind.network.protocol` entry-point group as
+`hivemind-mqtt-plugin`.
+
+## Docs
+
+- [docs/architecture.md](docs/architecture.md) — topic scheme, crypto, QoS, idle eviction
+- [docs/configuration.md](docs/configuration.md) — full configuration reference
+- [docs/operations.md](docs/operations.md) — broker setup, TLS/mTLS, authoring a transport plugin
+
+## Install
+
+```bash
+pip install hivemind-mqtt-protocol
+```

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol/__init__.py

@@ -0,0 +1,389 @@
+"""
+HiveMind MQTT Network Protocol Plugin
+
+Transports encrypted HiveMessage frames over an MQTT broker so that any
+satellite — including embedded ESP32 devices — can ride an existing IoT/HA
+MQTT bus rather than opening a dedicated WebSocket connection.
+
+Topology
+--------
+The hub runs ONE paho-mqtt client that connects to an external broker.  There
+is no accept() loop; logical per-satellite "connections" are derived from the
+topic hierarchy:
+
+    <prefix>/<hub_id>/c2s/<satellite_id>     satellite → hub (hub subscribes …/c2s/+)
+    <prefix>/<hub_id>/s2c/<satellite_id>     hub → satellite
+    <prefix>/<hub_id>/status/<satellite_id>  retained LWT presence (online/offline)
+
+Crypto
+------
+The MQTT payload IS the same encrypted HiveMessage frame that the WebSocket
+transport sends.  The broker only ever sees ciphertext.  No extra encryption
+layer is added here; hivemind-core's AES-GCM / RSA / PAKE handshake runs
+unchanged inside the payload bytes.
+
+Auth
+----
+Two layers, consistent with the design doc:
+  1. Broker-level: MQTT username/password or TLS client-cert (config keys
+     ``broker_username`` / ``broker_password`` / ``tls`` / ``cert``).
+  2. HiveMind-level: the HELLO/HANDSHAKE in-payload exchange, identical to the
+     WebSocket path.  The access key is passed as the MQTT username (the
+     satellite MUST set username=<access_key>).
+"""
+
+import dataclasses
+import hashlib
+import threading
+import time
+from dataclasses import dataclass, field
+from typing import Any, Dict, Optional
+
+import paho.mqtt.client as mqtt
+from ovos_bus_client.session import Session
+from ovos_utils.log import LOG
+from poorman_handshake import PasswordHandShake
+
+from hivemind_core.protocol import (
+    HiveMindClientConnection,
+    HiveMindListenerProtocol,
+    HiveMindNodeType,
+)
+from hivemind_plugin_manager.protocols import ClientCallbacks, NetworkProtocol
+
+_ONLINE = "online"
+_OFFLINE = "offline"
+
+# Seconds of silence from a peer before it is considered idle-disconnected.
+# Set to 0 to disable the sweep.
+_DEFAULT_IDLE_TIMEOUT = 300
+
+
+@dataclass
+class HiveMindMqttProtocol(NetworkProtocol):
+    """MQTT broker-mediated network protocol for hivemind-core.
+
+    Config keys (all optional, with defaults shown):
+        broker_host        (str)  "localhost"
+        broker_port        (int)  1883
+        broker_username    (str)  None  — MQTT username (not the HiveMind key)
+        broker_password    (str)  None
+        tls                (bool) False — enable TLS
+        tls_ca_certs       (str)  None  — path to CA bundle
+        tls_certfile       (str)  None  — path to client cert (mTLS)
+        tls_keyfile        (str)  None  — path to client key  (mTLS)
+        hub_id             (str)  NodeIdentity.name or "hivemind-hub"
+        topic_prefix       (str)  "hivemind"
+        qos                (int)  1
+        hash_topics        (bool) False — hash satellite_id in topics for privacy
+        idle_timeout       (int)  300   — seconds; 0 disables
+    """
+
+    config: Dict[str, Any] = field(default_factory=dict)
+    hm_protocol: Optional[HiveMindListenerProtocol] = None
+    callbacks: ClientCallbacks = field(default_factory=ClientCallbacks)
+
+    # Internal state — not part of the dataclass constructor signature.
+    _peers: Dict[str, HiveMindClientConnection] = field(default_factory=dict, init=False, repr=False)
+    _last_seen: Dict[str, float] = field(default_factory=dict, init=False, repr=False)
+    _mqtt: Optional[mqtt.Client] = field(default=None, init=False, repr=False)
+    _lock: threading.Lock = field(default_factory=threading.Lock, init=False, repr=False)
+
+    # ------------------------------------------------------------------
+    # Helpers
+    # ------------------------------------------------------------------
+
+    def _cfg(self, key: str, default: Any = None) -> Any:
+        return self.config.get(key, default)
+
+    def _hub_id(self) -> str:
+        return str(self._cfg("hub_id") or self.identity.name or "hivemind-hub")
+
+    def _prefix(self) -> str:
+        return str(self._cfg("topic_prefix") or "hivemind")
+
+    def _qos(self, is_bin: bool = False) -> int:
+        """QoS 0 for binary/audio (latency-sensitive); QoS 1 for control."""
+        if is_bin:
+            return 0
+        v = self._cfg("qos")
+        return int(v) if v is not None else 1
+
+    def _sat_topic_seg(self, satellite_id: str) -> str:
+        """Return the topic segment for satellite_id, optionally hashed."""
+        if self._cfg("hash_topics", False):
+            return hashlib.sha256(satellite_id.encode()).hexdigest()[:16]
+        return satellite_id
+
+    # topic builders ---------------------------------------------------
+
+    def c2s_topic(self, satellite_id: str) -> str:
+        """Inbound topic: satellite → hub."""
+        return f"{self._prefix()}/{self._hub_id()}/c2s/{self._sat_topic_seg(satellite_id)}"
+
+    def s2c_topic(self, satellite_id: str) -> str:
+        """Outbound topic: hub → satellite."""
+        return f"{self._prefix()}/{self._hub_id()}/s2c/{self._sat_topic_seg(satellite_id)}"
+
+    def status_topic(self, satellite_id: str) -> str:
+        """Retained LWT presence topic."""
+        return f"{self._prefix()}/{self._hub_id()}/status/{self._sat_topic_seg(satellite_id)}"
+
+    def c2s_wildcard(self) -> str:
+        """Wildcard subscription for all satellites' c2s traffic."""
+        return f"{self._prefix()}/{self._hub_id()}/c2s/+"
+
+    def status_wildcard(self) -> str:
+        """Wildcard subscription for all satellites' status topics."""
+        return f"{self._prefix()}/{self._hub_id()}/status/+"
+
+    # satellite_id extraction ------------------------------------------
+
+    @staticmethod
+    def _satellite_id_from_topic(topic: str) -> Optional[str]:
+        """Parse the satellite_id segment from a c2s or status topic."""
+        parts = topic.split("/")
+        # expected: <prefix>/<hub_id>/<segment>/<satellite_id>
+        if len(parts) >= 4:
+            return parts[-1]
+        return None
+
+    # ------------------------------------------------------------------
+    # Connection lifecycle
+    # ------------------------------------------------------------------
+
+    def _build_client_connection(
+        self, satellite_id: str, key: str, useragent: str
+    ) -> Optional[HiveMindClientConnection]:
+        """Create and register a HiveMindClientConnection for a new peer.
+
+        Mirrors HiveMindTornadoWebSocket.open() exactly: look up DB client by
+        key, populate ACL fields, then call handle_new_client.
+
+        Returns the new connection or None if auth fails.
+        """
+        prefix = self._prefix()
+        hub_id = self._hub_id()
+        mqttclient = self._mqtt
+        qos_fn = self._qos
+        s2c = self.s2c_topic(satellite_id)
+        status = self.status_topic(satellite_id)
+
+        def do_send(payload: Any, is_bin: bool = False) -> None:
+            if isinstance(payload, str):
+                payload = payload.encode()
+            mqttclient.publish(s2c, payload, qos=qos_fn(is_bin))
+
+        def do_disconnect() -> None:
+            # Publish an offline tombstone then clean up.
+            mqttclient.publish(status, _OFFLINE, qos=1, retain=True)
+            with self._lock:
+                self._peers.pop(satellite_id, None)
+                self._last_seen.pop(satellite_id, None)
+
+        conn = HiveMindClientConnection(
+            key=key,
+            disconnect=do_disconnect,
+            send_msg=do_send,
+            sess=Session(session_id="default"),
+            name=useragent,
+            hm_protocol=self.hm_protocol,
+        )
+
+        self.hm_protocol.db.sync()
+        user = self.hm_protocol.db.get_client_by_api_key(key)
+
+        if not user:
+            LOG.error(f"[MQTT] Client {satellite_id!r} provided invalid API key")
+            self.hm_protocol.handle_invalid_key_connected(conn)
+            return None
+
+        conn.name = f"{useragent}::{user.client_id}::{user.name}"
+        conn.crypto_key = user.crypto_key
+        conn.msg_blacklist = user.message_blacklist or []
+        conn.skill_blacklist = user.skill_blacklist or []
+        conn.intent_blacklist = user.intent_blacklist or []
+        conn.allowed_types = user.allowed_types
+        conn.can_broadcast = user.can_broadcast
+        conn.can_propagate = user.can_propagate
+        conn.can_escalate = user.can_escalate
+        conn.is_admin = user.is_admin
+        if user.password:
+            conn.pswd_handshake = PasswordHandShake(user.password)
+
+        conn.node_type = HiveMindNodeType.NODE
+
+        if (
+            not conn.crypto_key
+            and not self.hm_protocol.handshake_enabled
+            and self.hm_protocol.require_crypto
+        ):
+            LOG.error(
+                "[MQTT] No pre-shared crypto key and handshake disabled, "
+                "but require_crypto=True"
+            )
+            self.hm_protocol.handle_invalid_protocol_version(conn)
+            return None
+
+        with self._lock:
+            self._peers[satellite_id] = conn
+            self._last_seen[satellite_id] = time.monotonic()
+
+        self.hm_protocol.handle_new_client(conn)
+        LOG.info(f"[MQTT] New logical connection: {satellite_id!r} → {conn.name!r}")
+        return conn
+
+    def _disconnect_peer(self, satellite_id: str) -> None:
+        """Tear down the logical connection for satellite_id (LWT or timeout)."""
+        with self._lock:
+            conn = self._peers.pop(satellite_id, None)
+            self._last_seen.pop(satellite_id, None)
+        if conn is not None:
+            LOG.info(f"[MQTT] Disconnecting peer {satellite_id!r}")
+            self.hm_protocol.handle_client_disconnected(conn)
+
+    # ------------------------------------------------------------------
+    # paho callbacks
+    # ------------------------------------------------------------------
+
+    def _on_connect(self, client: mqtt.Client, userdata: Any, flags: Any, rc: int) -> None:
+        if rc != 0:
+            LOG.error(f"[MQTT] Broker connection failed, rc={rc}")
+            return
+        LOG.info("[MQTT] Connected to broker")
+        client.subscribe(self.c2s_wildcard(), qos=self._qos())
+        client.subscribe(self.status_wildcard(), qos=1)
+        LOG.debug(f"[MQTT] Subscribed to {self.c2s_wildcard()} and {self.status_wildcard()}")
+
+    def _on_message(self, client: mqtt.Client, userdata: Any, msg: mqtt.MQTTMessage) -> None:
+        topic: str = msg.topic
+        payload: bytes = msg.payload
+
+        # Determine segment type (c2s or status).
+        parts = topic.split("/")
+        if len(parts) < 4:
+            LOG.warning(f"[MQTT] Unexpected topic shape: {topic!r}")
+            return
+
+        segment = parts[-2]  # "c2s" or "status"
+        satellite_id = parts[-1]
+
+        if segment == "status":
+            status_val = payload.decode(errors="replace").strip()
+            if status_val == _OFFLINE:
+                LOG.info(f"[MQTT] LWT offline for {satellite_id!r}")
+                self._disconnect_peer(satellite_id)
+            return
+
+        if segment != "c2s":
+            return
+
+        # Inbound data frame.
+        with self._lock:
+            conn = self._peers.get(satellite_id)
+            if conn is not None:
+                self._last_seen[satellite_id] = time.monotonic()
+
+        if conn is None:
+            # Unknown satellite — must authenticate via the MQTT username.
+            # The hub's MQTT client receives the *message*; MQTT does not expose
+            # the sender's credentials directly here.  We use the satellite_id
+            # itself as a key placeholder so the payload HELLO frame can carry
+            # the real access key through the normal HiveMind handshake.
+            # For brokers that enforce ACLs, the MQTT username IS the HiveMind
+            # access key; we record it from the first-frame satellite_id.
+            # Treat satellite_id as the key for the initial lookup so that the
+            # broker-ACL username == HiveMind access key pattern works.
+            useragent = satellite_id
+            key = satellite_id  # satellite_id == MQTT username == HiveMind key
+            conn = self._build_client_connection(satellite_id, key, useragent)
+            if conn is None:
+                # Auth failed; drop frame.
+                return
+
+        try:
+            message = conn.decode(payload)
+        except Exception as e:
+            LOG.warning(f"[MQTT] Failed to decode frame from {satellite_id!r}: {e}")
+            return
+
+        self.hm_protocol.handle_message(message, conn)
+
+    def _on_disconnect(self, client: mqtt.Client, userdata: Any, rc: int) -> None:
+        if rc != 0:
+            LOG.warning(f"[MQTT] Unexpected broker disconnect, rc={rc}")
+
+    # ------------------------------------------------------------------
+    # Idle-timeout sweep
+    # ------------------------------------------------------------------
+
+    def _idle_sweep(self, idle_timeout: float) -> None:
+        """Background thread: evict peers that have been silent too long."""
+        while True:
+            time.sleep(max(idle_timeout / 4, 30))
+            now = time.monotonic()
+            with self._lock:
+                stale = [
+                    sid
+                    for sid, ts in list(self._last_seen.items())
+                    if (now - ts) > idle_timeout
+                ]
+            for sid in stale:
+                LOG.info(f"[MQTT] Idle timeout for peer {sid!r}")
+                self._disconnect_peer(sid)
+
+    # ------------------------------------------------------------------
+    # run() — the blocking entry point called by hivemind-core
+    # ------------------------------------------------------------------
+
+    def run(self) -> None:
+        LOG.debug(f"[MQTT] protocol config: {self.config}")
+
+        broker_host: str = str(self._cfg("broker_host") or "localhost")
+        broker_port: int = int(self._cfg("broker_port") or 1883)
+        hub_id = self._hub_id()
+
+        client_id = f"hivemind-hub-{hub_id}"
+        self._mqtt = mqtt.Client(client_id=client_id)
+
+        # Broker-level auth.
+        username: Optional[str] = self._cfg("broker_username")
+        password: Optional[str] = self._cfg("broker_password")
+        if username:
+            self._mqtt.username_pw_set(username, password)
+
+        # TLS.
+        if self._cfg("tls", False):
+            self._mqtt.tls_set(
+                ca_certs=self._cfg("tls_ca_certs"),
+                certfile=self._cfg("tls_certfile"),
+                keyfile=self._cfg("tls_keyfile"),
+            )
+
+        # Hub's own LWT — signals the hub going offline to any listener.
+        hub_status_topic = f"{self._prefix()}/{hub_id}/status/hub"
+        self._mqtt.will_set(hub_status_topic, _OFFLINE, qos=1, retain=True)
+
+        self._mqtt.on_connect = self._on_connect
+        self._mqtt.on_message = self._on_message
+        self._mqtt.on_disconnect = self._on_disconnect
+
+        self._mqtt.connect(broker_host, broker_port, keepalive=60)
+
+        # Publish hub online status once connected (done in on_connect would
+        # need the client reference; simpler to publish after connect()).
+        self._mqtt.publish(hub_status_topic, _ONLINE, qos=1, retain=True)
+
+        # Start idle-timeout sweep if configured.
+        idle_timeout = float(self._cfg("idle_timeout") or _DEFAULT_IDLE_TIMEOUT)
+        if idle_timeout > 0:
+            t = threading.Thread(
+                target=self._idle_sweep,
+                args=(idle_timeout,),
+                daemon=True,
+                name="mqtt-idle-sweep",
+            )
+            t.start()
+
+        LOG.info(f"[MQTT] listener started — broker={broker_host}:{broker_port}, hub_id={hub_id!r}")
+        self._mqtt.loop_forever()  # blocking — mirrors tornado ioloop.start()

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol/version.py

@@ -0,0 +1,8 @@
+# START_VERSION_BLOCK
+VERSION_MAJOR = 0
+VERSION_MINOR = 1
+VERSION_BUILD = 0
+VERSION_ALPHA = 2
+# END_VERSION_BLOCK
+
+__version__ = f"{VERSION_MAJOR}.{VERSION_MINOR}.{VERSION_BUILD}" + (f"a{VERSION_ALPHA}" if VERSION_ALPHA else "")

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol.egg-info/PKG-INFO

@@ -0,0 +1,18 @@
+Metadata-Version: 2.4
+Name: hivemind-mqtt-protocol
+Version: 0.1.0a2
+Summary: MQTT broker-mediated network protocol plugin for hivemind-core
+Author-email: JarbasAi <jarbasai@mailfence.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/JarbasHiveMind/hivemind-mqtt-protocol
+Requires-Python: >=3.10
+Requires-Dist: paho-mqtt>=1.6
+Requires-Dist: hivemind-plugin-manager
+Requires-Dist: poorman_handshake>=0.1.0
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: hivemind-core; extra == "test"
+Requires-Dist: hivemind-bus-client; extra == "test"
+Requires-Dist: ovos-bus-client; extra == "test"
+Requires-Dist: ovos-utils; extra == "test"
+Requires-Dist: hivescope; extra == "test"

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+README.md
+pyproject.toml
+hivemind_mqtt_protocol/__init__.py
+hivemind_mqtt_protocol/version.py
+hivemind_mqtt_protocol.egg-info/PKG-INFO
+hivemind_mqtt_protocol.egg-info/SOURCES.txt
+hivemind_mqtt_protocol.egg-info/dependency_links.txt
+hivemind_mqtt_protocol.egg-info/entry_points.txt
+hivemind_mqtt_protocol.egg-info/requires.txt
+hivemind_mqtt_protocol.egg-info/top_level.txt
+tests/test_mqtt_protocol.py

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[hivemind.network.protocol]
+hivemind-mqtt-plugin = hivemind_mqtt_protocol:HiveMindMqttProtocol

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol.egg-info/requires.txt

@@ -0,0 +1,11 @@
+paho-mqtt>=1.6
+hivemind-plugin-manager
+poorman_handshake>=0.1.0
+
+[test]
+pytest
+hivemind-core
+hivemind-bus-client
+ovos-bus-client
+ovos-utils
+hivescope

hivemind_mqtt_protocol-0.1.0a2/hivemind_mqtt_protocol.egg-info/top_level.txt

@@ -0,0 +1 @@
+hivemind_mqtt_protocol

hivemind_mqtt_protocol-0.1.0a2/pyproject.toml

@@ -0,0 +1,41 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "hivemind-mqtt-protocol"
+dynamic = ["version"]
+description = "MQTT broker-mediated network protocol plugin for hivemind-core"
+license = "Apache-2.0"
+authors = [
+    { name = "JarbasAi", email = "jarbasai@mailfence.com" }
+]
+requires-python = ">=3.10"
+dependencies = [
+    "paho-mqtt>=1.6",
+    "hivemind-plugin-manager",
+    "poorman_handshake>=0.1.0",
+]
+
+[project.optional-dependencies]
+test = [
+    "pytest",
+    "hivemind-core",
+    "hivemind-bus-client",
+    "ovos-bus-client",
+    "ovos-utils",
+    "hivescope",
+]
+
+[project.urls]
+Homepage = "https://github.com/JarbasHiveMind/hivemind-mqtt-protocol"
+
+[project.entry-points."hivemind.network.protocol"]
+"hivemind-mqtt-plugin" = "hivemind_mqtt_protocol:HiveMindMqttProtocol"
+
+[tool.setuptools.dynamic]
+version = { attr = "hivemind_mqtt_protocol.version.__version__" }
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["hivemind_mqtt_protocol*"]

hivemind_mqtt_protocol-0.1.0a2/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

hivemind_mqtt_protocol-0.1.0a2/tests/test_mqtt_protocol.py

@@ -0,0 +1,451 @@
+"""
+Unit tests for hivemind_mqtt_protocol.HiveMindMqttProtocol.
+
+All tests use mocked paho-mqtt and mocked hivemind-core objects — no live
+broker is required.
+"""
+
+import hashlib
+import threading
+import time
+import types
+from dataclasses import dataclass, field
+from typing import Any, Dict, Optional
+from unittest.mock import MagicMock, call, patch
+
+import pytest
+
+# ---------------------------------------------------------------------------
+# Minimal stubs so we can import the protocol without a full HiveMind install.
+# ---------------------------------------------------------------------------
+
+import sys
+
+
+def _make_stub_module(name: str, **attrs) -> types.ModuleType:
+    mod = types.ModuleType(name)
+    for k, v in attrs.items():
+        setattr(mod, k, v)
+    return mod
+
+
+# ovos_utils.log
+log_stub = _make_stub_module("ovos_utils")
+log_stub.log = _make_stub_module("ovos_utils.log", LOG=MagicMock())
+sys.modules.setdefault("ovos_utils", log_stub)
+sys.modules.setdefault("ovos_utils.log", log_stub.log)
+
+# ovos_bus_client.session
+session_cls = MagicMock(name="Session")
+session_stub = _make_stub_module("ovos_bus_client")
+session_stub.session = _make_stub_module("ovos_bus_client.session", Session=session_cls)
+sys.modules.setdefault("ovos_bus_client", session_stub)
+sys.modules.setdefault("ovos_bus_client.session", session_stub.session)
+
+# poorman_handshake
+psh_stub = _make_stub_module("poorman_handshake", PasswordHandShake=MagicMock())
+sys.modules.setdefault("poorman_handshake", psh_stub)
+
+# hivemind_plugin_manager.protocols
+@dataclass
+class _FakeNetworkProtocol:
+    config: Dict[str, Any] = field(default_factory=dict)
+    hm_protocol: Optional[Any] = None
+    callbacks: Any = None
+
+    @property
+    def identity(self):
+        return MagicMock(name="identity")
+
+    @property
+    def database(self):
+        return None
+
+    @property
+    def clients(self):
+        return {}
+
+    @property
+    def agent_protocol(self):
+        return None
+
+    def run(self):  # abstract placeholder
+        pass
+
+
+class _FakeCallbacks:
+    pass
+
+
+proto_mod = _make_stub_module(
+    "hivemind_plugin_manager.protocols",
+    NetworkProtocol=_FakeNetworkProtocol,
+    ClientCallbacks=_FakeCallbacks,
+)
+pm_mod = _make_stub_module("hivemind_plugin_manager", protocols=proto_mod)
+sys.modules.setdefault("hivemind_plugin_manager", pm_mod)
+sys.modules.setdefault("hivemind_plugin_manager.protocols", proto_mod)
+
+# hivemind_core.protocol
+class _FakeClientConnection:
+    def __init__(self, **kwargs):
+        for k, v in kwargs.items():
+            setattr(self, k, v)
+        self.peer = kwargs.get("name", "peer")
+        self.crypto_key = None
+        self.msg_blacklist = []
+        self.skill_blacklist = []
+        self.intent_blacklist = []
+        self.allowed_types = []
+        self.can_broadcast = True
+        self.can_propagate = True
+        self.can_escalate = True
+        self.is_admin = False
+        self.pswd_handshake = None
+        self.node_type = None
+        self._decode_result = MagicMock(name="HiveMessage")
+
+    def decode(self, payload):
+        return self._decode_result
+
+
+class _FakeNodeType:
+    NODE = "NODE"
+
+
+hmc_mod = _make_stub_module(
+    "hivemind_core.protocol",
+    HiveMindClientConnection=_FakeClientConnection,
+    HiveMindListenerProtocol=MagicMock(),
+    HiveMindNodeType=_FakeNodeType,
+)
+sys.modules.setdefault("hivemind_core", _make_stub_module("hivemind_core"))
+sys.modules.setdefault("hivemind_core.protocol", hmc_mod)
+
+# Now patch NetworkProtocol base in the module under test before importing.
+import hivemind_mqtt_protocol  # noqa: E402  (import after stubs)
+
+# Patch the base class reference inside the module so HiveMindMqttProtocol
+# inherits from our stub instead of the real NetworkProtocol.
+hivemind_mqtt_protocol.NetworkProtocol = _FakeNetworkProtocol
+hivemind_mqtt_protocol.HiveMindClientConnection = _FakeClientConnection
+hivemind_mqtt_protocol.HiveMindNodeType = _FakeNodeType
+hivemind_mqtt_protocol.ClientCallbacks = _FakeCallbacks
+
+from hivemind_mqtt_protocol import HiveMindMqttProtocol  # noqa: E402
+
+
+# ---------------------------------------------------------------------------
+# Fixtures
+# ---------------------------------------------------------------------------
+
+
+def _make_protocol(config=None):
+    """Return a protocol instance with a mocked hm_protocol and mqtt client."""
+    hm = MagicMock(name="hm_protocol")
+    hm.identity = MagicMock(name="identity")
+    hm.identity.name = "testhub"
+    hm.handshake_enabled = True
+    hm.require_crypto = False
+    # Always inject hub_id via config so _hub_id() doesn't go through the
+    # identity property chain (MagicMock.name is a reserved attribute).
+    base_config = {"hub_id": "testhub"}
+    if config:
+        base_config.update(config)
+    config = base_config
+
+    # DB returns a valid user for any key.
+    user = MagicMock(name="user")
+    user.client_id = 42
+    user.name = "testclient"
+    user.crypto_key = None
+    user.message_blacklist = []
+    user.skill_blacklist = []
+    user.intent_blacklist = []
+    user.allowed_types = []
+    user.can_broadcast = True
+    user.can_propagate = True
+    user.can_escalate = True
+    user.is_admin = False
+    user.password = None
+    hm.db.get_client_by_api_key.return_value = user
+
+    p = HiveMindMqttProtocol(config=config or {}, hm_protocol=hm)
+    p._peers = {}
+    p._last_seen = {}
+    p._lock = threading.Lock()
+    mock_mqtt = MagicMock(name="mqtt_client")
+    p._mqtt = mock_mqtt
+    return p
+
+
+# ---------------------------------------------------------------------------
+# Topic build / parse tests
+# ---------------------------------------------------------------------------
+
+
+class TestTopics:
+    def test_c2s_topic_default_prefix(self):
+        p = _make_protocol()
+        assert p.c2s_topic("sat1") == "hivemind/testhub/c2s/sat1"
+
+    def test_s2c_topic_default_prefix(self):
+        p = _make_protocol()
+        assert p.s2c_topic("sat1") == "hivemind/testhub/s2c/sat1"
+
+    def test_status_topic_default_prefix(self):
+        p = _make_protocol()
+        assert p.status_topic("sat1") == "hivemind/testhub/status/sat1"
+
+    def test_custom_prefix_and_hub_id(self):
+        p = _make_protocol({"topic_prefix": "hm", "hub_id": "myhub"})
+        assert p.c2s_topic("x") == "hm/myhub/c2s/x"
+
+    def test_c2s_wildcard(self):
+        p = _make_protocol()
+        assert p.c2s_wildcard() == "hivemind/testhub/c2s/+"
+
+    def test_status_wildcard(self):
+        p = _make_protocol()
+        assert p.status_wildcard() == "hivemind/testhub/status/+"
+
+    def test_hash_topics(self):
+        p = _make_protocol({"hash_topics": True})
+        sat_id = "mysat"
+        expected = hashlib.sha256(sat_id.encode()).hexdigest()[:16]
+        assert p.c2s_topic(sat_id) == f"hivemind/testhub/c2s/{expected}"
+
+    def test_satellite_id_from_c2s_topic(self):
+        topic = "hivemind/testhub/c2s/sat42"
+        assert HiveMindMqttProtocol._satellite_id_from_topic(topic) == "sat42"
+
+    def test_satellite_id_from_status_topic(self):
+        topic = "hivemind/myhub/status/sat99"
+        assert HiveMindMqttProtocol._satellite_id_from_topic(topic) == "sat99"
+
+    def test_satellite_id_short_topic_returns_none(self):
+        assert HiveMindMqttProtocol._satellite_id_from_topic("bad") is None
+
+
+# ---------------------------------------------------------------------------
+# Peer-map: new peer / known peer logic
+# ---------------------------------------------------------------------------
+
+
+class TestPeerMap:
+    def test_new_peer_added_to_map(self):
+        p = _make_protocol()
+        conn = p._build_client_connection("sat1", "sat1", "sat1")
+        assert "sat1" in p._peers
+        assert conn is not None
+
+    def test_known_peer_not_duplicated(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        first_conn = p._peers["sat1"]
+        # Simulating message from known peer — we do NOT call _build again.
+        assert p._peers["sat1"] is first_conn
+
+    def test_invalid_key_not_added(self):
+        p = _make_protocol()
+        p.hm_protocol.db.get_client_by_api_key.return_value = None
+        conn = p._build_client_connection("bad", "bad", "bad")
+        assert conn is None
+        assert "bad" not in p._peers
+
+    def test_handle_new_client_called(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        p.hm_protocol.handle_new_client.assert_called_once()
+
+    def test_invalid_key_triggers_invalid_key_handler(self):
+        p = _make_protocol()
+        p.hm_protocol.db.get_client_by_api_key.return_value = None
+        p._build_client_connection("bad", "bad", "bad")
+        p.hm_protocol.handle_invalid_key_connected.assert_called_once()
+
+
+# ---------------------------------------------------------------------------
+# send_msg publishes to the correct topic
+# ---------------------------------------------------------------------------
+
+
+class TestSendMsg:
+    def test_send_msg_publishes_to_s2c(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        conn = p._peers["sat1"]
+
+        payload = b"encrypted-frame"
+        conn.send_msg(payload, False)
+
+        expected_topic = p.s2c_topic("sat1")
+        p._mqtt.publish.assert_called_with(expected_topic, payload, qos=1)
+
+    def test_send_msg_bin_uses_qos0(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        conn = p._peers["sat1"]
+
+        conn.send_msg(b"audio", True)
+
+        # QoS 0 for binary.
+        call_args = p._mqtt.publish.call_args
+        assert call_args[1]["qos"] == 0 or (len(call_args[0]) >= 3 and call_args[0][2] == 0)
+
+    def test_send_msg_str_payload_encoded(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        conn = p._peers["sat1"]
+
+        conn.send_msg("string-payload", False)
+
+        topic = p.s2c_topic("sat1")
+        p._mqtt.publish.assert_called_with(topic, b"string-payload", qos=1)
+
+
+# ---------------------------------------------------------------------------
+# Disconnect: do_disconnect and _disconnect_peer
+# ---------------------------------------------------------------------------
+
+
+class TestDisconnect:
+    def test_disconnect_removes_from_map(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        assert "sat1" in p._peers
+
+        p._disconnect_peer("sat1")
+
+        assert "sat1" not in p._peers
+
+    def test_disconnect_calls_handle_client_disconnected(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        p.hm_protocol.handle_client_disconnected.reset_mock()
+
+        p._disconnect_peer("sat1")
+
+        p.hm_protocol.handle_client_disconnected.assert_called_once()
+
+    def test_do_disconnect_publishes_tombstone(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        conn = p._peers["sat1"]
+        p._mqtt.publish.reset_mock()
+
+        conn.disconnect()
+
+        status_topic = p.status_topic("sat1")
+        p._mqtt.publish.assert_called_with(status_topic, "offline", qos=1, retain=True)
+        assert "sat1" not in p._peers
+
+    def test_disconnect_unknown_peer_is_noop(self):
+        p = _make_protocol()
+        # Should not raise.
+        p._disconnect_peer("ghost")
+        p.hm_protocol.handle_client_disconnected.assert_not_called()
+
+
+# ---------------------------------------------------------------------------
+# LWT / status message handling
+# ---------------------------------------------------------------------------
+
+
+class TestLWT:
+    def _make_msg(self, topic: str, payload: bytes) -> MagicMock:
+        m = MagicMock()
+        m.topic = topic
+        m.payload = payload
+        return m
+
+    def test_lwt_offline_triggers_disconnect(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        p.hm_protocol.handle_client_disconnected.reset_mock()
+
+        msg = self._make_msg("hivemind/testhub/status/sat1", b"offline")
+        p._on_message(p._mqtt, None, msg)
+
+        p.hm_protocol.handle_client_disconnected.assert_called_once()
+        assert "sat1" not in p._peers
+
+    def test_lwt_online_ignored(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        p.hm_protocol.handle_client_disconnected.reset_mock()
+
+        msg = self._make_msg("hivemind/testhub/status/sat1", b"online")
+        p._on_message(p._mqtt, None, msg)
+
+        p.hm_protocol.handle_client_disconnected.assert_not_called()
+
+    def test_c2s_message_routed_to_handle_message(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        p.hm_protocol.handle_message.reset_mock()
+
+        msg = self._make_msg("hivemind/testhub/c2s/sat1", b"payload")
+        p._on_message(p._mqtt, None, msg)
+
+        p.hm_protocol.handle_message.assert_called_once()
+
+    def test_unknown_peer_auto_registered_on_c2s(self):
+        p = _make_protocol()
+        assert "newsat" not in p._peers
+
+        msg = self._make_msg("hivemind/testhub/c2s/newsat", b"payload")
+        p._on_message(p._mqtt, None, msg)
+
+        assert "newsat" in p._peers
+
+    def test_bad_topic_shape_ignored(self):
+        p = _make_protocol()
+        msg = self._make_msg("bad", b"x")
+        p._on_message(p._mqtt, None, msg)
+        p.hm_protocol.handle_message.assert_not_called()
+
+
+# ---------------------------------------------------------------------------
+# Idle-timeout sweep
+# ---------------------------------------------------------------------------
+
+
+class TestIdleSweep:
+    def test_stale_peer_evicted(self):
+        p = _make_protocol()
+        p._build_client_connection("sat1", "sat1", "sat1")
+        # Back-date last_seen to force eviction.
+        p._last_seen["sat1"] = time.monotonic() - 9999
+
+        p.hm_protocol.handle_client_disconnected.reset_mock()
+        p._idle_sweep.__func__  # confirm it's a method
+
+        # Run one cycle manually by calling _disconnect_peer for stale peers.
+        now = time.monotonic()
+        idle_timeout = 300
+        stale = [sid for sid, ts in p._last_seen.items() if (now - ts) > idle_timeout]
+        for sid in stale:
+            p._disconnect_peer(sid)
+
+        assert "sat1" not in p._peers
+        p.hm_protocol.handle_client_disconnected.assert_called_once()
+
+
+# ---------------------------------------------------------------------------
+# QoS helpers
+# ---------------------------------------------------------------------------
+
+
+class TestQoS:
+    def test_default_qos_is_1(self):
+        p = _make_protocol()
+        assert p._qos(False) == 1
+
+    def test_binary_qos_is_0(self):
+        p = _make_protocol()
+        assert p._qos(True) == 0
+
+    def test_custom_qos_from_config(self):
+        p = _make_protocol({"qos": 0})
+        assert p._qos(False) == 0