swarmit 0.3.0__tar.gz → 0.4.4__tar.gz

{swarmit-0.3.0 → swarmit-0.4.4}/.gitignore

@@ -20,3 +20,6 @@ dist/
 
 # Virtual env folder
 .venv/
+
+# Tox
+.tox/

swarmit-0.4.4/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: swarmit
+Version: 0.4.4
+Summary: Run Your Own Robot Swarm Testbed.
+Project-URL: Homepage, https://github.com/DotBots/swarmit
+Project-URL: Bug Tracker, https://github.com/DotBots/swarmit/issues
+Author-email: Alexandre Abadie <alexandre.abadie@inria.fr>
+License: BSD
+License-File: AUTHORS
+License-File: LICENSE
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.7
+Requires-Dist: click==8.1.7
+Requires-Dist: cryptography==43.0.1
+Requires-Dist: marilib-pkg>=0.6.0
+Requires-Dist: pydotbot>=0.24.1
+Requires-Dist: rich==14.0.0
+Requires-Dist: structlog==24.4.0
+Requires-Dist: tqdm==4.66.5
+Description-Content-Type: text/markdown
+
+ # SwarmIT
+
+SwarmIT provides a embedded C port for nRF53 as well as Python based services to
+easily build and deploy a robotic swarm infrastructure testbed.
+ARM TrustZone is used to create a sandboxed user environment on each device
+under test, without requiring a control co-processor attached to it.
+
+https://github.com/user-attachments/assets/eff63b07-216a-41fb-9062-2e0e56f03c20
+
+## Features
+
+- Experiment management: start, stop, monitor and status check
+- Deploy a custom firmware on all or on a subset of robots of a swarm testbed
+- Resilient robot state: even when crashed by buggy user code, the robot can be reprogrammed remotely and wirelessly
+
+## Usage
+
+### Get the code
+
+Swarmit depends on the [DotBot-firmware](https://github.com/DotBots/DotBot-firmware)
+and [Mari](https://github.com/DotBots/mari) repositories. They are included
+in the codebase as [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules).
+
+Use the following command to clone the Swarmit codebase locally:
+
+```
+git clone --recurse-submodules https://github.com/DotBots/swarmit.git
+```
+
+### Embedded C code
+
+SwarmIT embedded C code can be built using
+[Segger Embedded Studio (SES)](https://www.segger.com/products/development-tools/embedded-studio/).
+Use Tools > Package manager to install the CMSIS 5 CMSIS-CORE, CMSIS-DSP and nRF packages.
+
+To provision a device, follow the following steps:
+1. open [netcore.emProject](swarmit-netcore.emProject)
+   and [bootloader.emProject](swarmit-bootloader-dotbot-v3.emProject)
+   (or [bootloader.emProject](swarmit-bootloader-dotbot-v2.emProject) depending on
+   your robot version) in SES
+2. build and load the netcore application on the nRF53 network core,
+3. build and load the bootloader application on the nRF53 application core.
+
+The device is now ready.
+
+### Gateway
+
+The communication between the computer and the swarm devices is performed via a
+gateway board connected via USB to the computer.
+
+This gateway uses the [mari](https://github.com/dotbots/mari) network stack and
+the it must run the Mari gateway firmware, either Edge or Cloud version.
+
+The documentation to setup a Mari gateway is located
+[here](https://github.com/DotBots/mari/wiki/Getting-started#running-mari-network-on-your-computer).
+
+### Python CLI script
+
+The Python CLI script provides commands for flashing, starting and stopping user
+code on the device, as well as monitoring and checking the status of devices
+in the swarm.
+
+The Python CLI script connects via a virtual COM port to the gateway connected to
+the computer.
+
+The Python CLI script is available on PyPI. Install it using:
+
+```
+pip install swarmit
+```
+
+Print usage using `swarmit --help`:
+
+```
+Usage: swarmit [OPTIONS] COMMAND [ARGS]...
+
+Options:
+  -p, --port TEXT                 Serial port to use to send the bitstream to
+                                  the gateway. Default: /dev/ttyACM0.
+  -b, --baudrate INTEGER          Serial port baudrate. Default: 1000000.
+  -H, --mqtt-host TEXT            MQTT host. Default: localhost.
+  -P, --mqtt-port INTEGER         MQTT port. Default: 1883.
+  -T, --mqtt-use_tls              Use TLS with MQTT.
+  -n, --network-id INTEGER        Marilib network ID to use. Default: 1
+  -a, --adapter [edge|cloud]
+                                  Choose the adapter to communicate with the
+                                  gateway.  [default: edge]
+  -d, --devices TEXT              Subset list of devices to interact with,
+                                  separated with ,
+  -v, --verbose                   Enable verbose mode.
+  -V, --version                   Show the version and exit.
+  -h, --help                      Show this message and exit.
+
+Commands:
+  flash    Flash a firmware to the robots.
+  message  Send a custom text message to the robots.
+  monitor  Monitor running applications.
+  reset    Reset robots locations.
+  start    Start the user application.
+  status   Print current status of the robots.
+  stop     Stop the user application.
+```

swarmit-0.4.4/README.md

@@ -0,0 +1,102 @@
+ # SwarmIT
+
+SwarmIT provides a embedded C port for nRF53 as well as Python based services to
+easily build and deploy a robotic swarm infrastructure testbed.
+ARM TrustZone is used to create a sandboxed user environment on each device
+under test, without requiring a control co-processor attached to it.
+
+https://github.com/user-attachments/assets/eff63b07-216a-41fb-9062-2e0e56f03c20
+
+## Features
+
+- Experiment management: start, stop, monitor and status check
+- Deploy a custom firmware on all or on a subset of robots of a swarm testbed
+- Resilient robot state: even when crashed by buggy user code, the robot can be reprogrammed remotely and wirelessly
+
+## Usage
+
+### Get the code
+
+Swarmit depends on the [DotBot-firmware](https://github.com/DotBots/DotBot-firmware)
+and [Mari](https://github.com/DotBots/mari) repositories. They are included
+in the codebase as [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules).
+
+Use the following command to clone the Swarmit codebase locally:
+
+```
+git clone --recurse-submodules https://github.com/DotBots/swarmit.git
+```
+
+### Embedded C code
+
+SwarmIT embedded C code can be built using
+[Segger Embedded Studio (SES)](https://www.segger.com/products/development-tools/embedded-studio/).
+Use Tools > Package manager to install the CMSIS 5 CMSIS-CORE, CMSIS-DSP and nRF packages.
+
+To provision a device, follow the following steps:
+1. open [netcore.emProject](swarmit-netcore.emProject)
+   and [bootloader.emProject](swarmit-bootloader-dotbot-v3.emProject)
+   (or [bootloader.emProject](swarmit-bootloader-dotbot-v2.emProject) depending on
+   your robot version) in SES
+2. build and load the netcore application on the nRF53 network core,
+3. build and load the bootloader application on the nRF53 application core.
+
+The device is now ready.
+
+### Gateway
+
+The communication between the computer and the swarm devices is performed via a
+gateway board connected via USB to the computer.
+
+This gateway uses the [mari](https://github.com/dotbots/mari) network stack and
+the it must run the Mari gateway firmware, either Edge or Cloud version.
+
+The documentation to setup a Mari gateway is located
+[here](https://github.com/DotBots/mari/wiki/Getting-started#running-mari-network-on-your-computer).
+
+### Python CLI script
+
+The Python CLI script provides commands for flashing, starting and stopping user
+code on the device, as well as monitoring and checking the status of devices
+in the swarm.
+
+The Python CLI script connects via a virtual COM port to the gateway connected to
+the computer.
+
+The Python CLI script is available on PyPI. Install it using:
+
+```
+pip install swarmit
+```
+
+Print usage using `swarmit --help`:
+
+```
+Usage: swarmit [OPTIONS] COMMAND [ARGS]...
+
+Options:
+  -p, --port TEXT                 Serial port to use to send the bitstream to
+                                  the gateway. Default: /dev/ttyACM0.
+  -b, --baudrate INTEGER          Serial port baudrate. Default: 1000000.
+  -H, --mqtt-host TEXT            MQTT host. Default: localhost.
+  -P, --mqtt-port INTEGER         MQTT port. Default: 1883.
+  -T, --mqtt-use_tls              Use TLS with MQTT.
+  -n, --network-id INTEGER        Marilib network ID to use. Default: 1
+  -a, --adapter [edge|cloud]
+                                  Choose the adapter to communicate with the
+                                  gateway.  [default: edge]
+  -d, --devices TEXT              Subset list of devices to interact with,
+                                  separated with ,
+  -v, --verbose                   Enable verbose mode.
+  -V, --version                   Show the version and exit.
+  -h, --help                      Show this message and exit.
+
+Commands:
+  flash    Flash a firmware to the robots.
+  message  Send a custom text message to the robots.
+  monitor  Monitor running applications.
+  reset    Reset robots locations.
+  start    Start the user application.
+  status   Print current status of the robots.
+  stop     Stop the user application.
+```

{swarmit-0.3.0 → swarmit-0.4.4}/pyproject.toml

@@ -16,6 +16,9 @@ exclude = [
 [tool.hatch.version]
 path = "testbed/swarmit/__init__.py"
 
+[tool.hatch.metadata]
+allow-direct-references = true
+
 [project]
 name = "swarmit"
 dynamic = ["version"]
@@ -25,12 +28,11 @@ authors = [
 dependencies = [
     "click          == 8.1.7",
     "cryptography   == 43.0.1",
-    "pydotbot       == 0.22.0",
-    "pyserial       == 3.5",
-    "rich           == 13.8.1",
+    "pydotbot       >= 0.24.1",
+    "rich           == 14.0.0",
     "structlog      == 24.4.0",
     "tqdm           == 4.66.5",
-    "paho-mqtt      >= 2.1.0",
+    "marilib-pkg    >= 0.6.0",
 ]
 description = "Run Your Own Robot Swarm Testbed."
 readme = "README.md"
@@ -54,7 +56,8 @@ swarmit = "testbed.cli.main:main"
 [tool.ruff]
 lint.select = ["E", "F"]
 line-length = 88
-lint.ignore = ["E501"]
+lint.ignore = ["E501", "E722"]
+exclude = ["dotbot-firmware"]
 
 [tool.isort]
 multi_line_output = 3  # Use Vertical Hanging Indent

swarmit-0.4.4/testbed/cli/main.py

@@ -0,0 +1,348 @@
+#!/usr/bin/env python
+
+import logging
+import time
+
+import click
+import serial
+import structlog
+from dotbot.serial_interface import SerialInterfaceException, get_default_port
+from rich import print
+from rich.console import Console
+from rich.pretty import pprint
+
+from testbed.swarmit import __version__
+from testbed.swarmit.controller import (
+    CHUNK_SIZE,
+    OTA_ACK_TIMEOUT_DEFAULT,
+    OTA_MAX_RETRIES_DEFAULT,
+    Controller,
+    ControllerSettings,
+    ResetLocation,
+    print_transfer_status,
+)
+
+SERIAL_PORT_DEFAULT = get_default_port()
+BAUDRATE_DEFAULT = 1000000
+MQTT_HOST_DEFAULT = "localhost"
+MQTT_PORT_DEFAULT = 1883
+# Default network ID for SwarmIT tests is 0x12**
+# See https://crystalfree.atlassian.net/wiki/spaces/Mari/pages/3324903426/Registry+of+Mari+Network+IDs
+SWARMIT_NETWORK_ID_DEFAULT = "1200"
+
+
+@click.group(context_settings=dict(help_option_names=["-h", "--help"]))
+@click.option(
+    "-p",
+    "--port",
+    type=str,
+    default=SERIAL_PORT_DEFAULT,
+    help=f"Serial port to use to send the bitstream to the gateway. Default: {SERIAL_PORT_DEFAULT}.",
+)
+@click.option(
+    "-b",
+    "--baudrate",
+    type=int,
+    default=BAUDRATE_DEFAULT,
+    help=f"Serial port baudrate. Default: {BAUDRATE_DEFAULT}.",
+)
+@click.option(
+    "-H",
+    "--mqtt-host",
+    type=str,
+    default=MQTT_HOST_DEFAULT,
+    help=f"MQTT host. Default: {MQTT_HOST_DEFAULT}.",
+)
+@click.option(
+    "-P",
+    "--mqtt-port",
+    type=int,
+    default=MQTT_PORT_DEFAULT,
+    help=f"MQTT port. Default: {MQTT_PORT_DEFAULT}.",
+)
+@click.option(
+    "-T",
+    "--mqtt-use_tls",
+    is_flag=True,
+    help="Use TLS with MQTT.",
+)
+@click.option(
+    "-n",
+    "--network-id",
+    type=str,
+    default=SWARMIT_NETWORK_ID_DEFAULT,
+    help=f"Marilib network ID to use. Default: 0x{SWARMIT_NETWORK_ID_DEFAULT}",
+)
+@click.option(
+    "-a",
+    "--adapter",
+    type=click.Choice(["edge", "cloud"], case_sensitive=True),
+    default="edge",
+    show_default=True,
+    help="Choose the adapter to communicate with the gateway.",
+)
+@click.option(
+    "-d",
+    "--devices",
+    type=str,
+    default="",
+    help="Subset list of device addresses to interact with, separated with ,",
+)
+@click.option(
+    "-v",
+    "--verbose",
+    is_flag=True,
+    help="Enable verbose mode.",
+)
+@click.version_option(__version__, "-V", "--version", prog_name="swarmit")
+@click.pass_context
+def main(
+    ctx,
+    port,
+    baudrate,
+    mqtt_host,
+    mqtt_port,
+    mqtt_use_tls,
+    network_id,
+    adapter,
+    devices,
+    verbose,
+):
+    if ctx.invoked_subcommand != "monitor":
+        # Disable logging if not monitoring
+        structlog.configure(
+            wrapper_class=structlog.make_filtering_bound_logger(
+                logging.CRITICAL
+            ),
+        )
+    ctx.ensure_object(dict)
+    ctx.obj["settings"] = ControllerSettings(
+        serial_port=port,
+        serial_baudrate=baudrate,
+        mqtt_host=mqtt_host,
+        mqtt_port=mqtt_port,
+        mqtt_use_tls=mqtt_use_tls,
+        network_id=int(network_id, 16),
+        adapter=adapter,
+        devices=[d for d in devices.split(",") if d],
+        verbose=verbose,
+    )
+
+
+@main.command()
+@click.pass_context
+def start(ctx):
+    """Start the user application."""
+    try:
+        controller = Controller(ctx.obj["settings"])
+    except (
+        SerialInterfaceException,
+        serial.serialutil.SerialException,
+    ) as exc:
+        console = Console()
+        console.print(f"[bold red]Error:[/] {exc}")
+        return
+    if controller.ready_devices:
+        controller.start()
+    else:
+        print("No device to start")
+    controller.terminate()
+
+
+@main.command()
+@click.pass_context
+def stop(ctx):
+    """Stop the user application."""
+    try:
+        controller = Controller(ctx.obj["settings"])
+    except (
+        SerialInterfaceException,
+        serial.serialutil.SerialException,
+    ) as exc:
+        console = Console()
+        console.print(f"[bold red]Error:[/] {exc}")
+        return
+    if controller.running_devices or controller.resetting_devices:
+        controller.stop()
+    else:
+        print("[bold]No device to stop[/]")
+    controller.terminate()
+
+
+@main.command()
+@click.argument(
+    "locations",
+    type=str,
+)
+@click.pass_context
+def reset(ctx, locations):
+    """Reset robots locations.
+
+    Locations are provided as '<device_addr>:<x>,<y>-<device_addr>:<x>,<y>|...'
+    """
+    try:
+        controller = Controller(ctx.obj["settings"])
+    except (
+        SerialInterfaceException,
+        serial.serialutil.SerialException,
+    ) as exc:
+        console = Console()
+        console.print(f"[bold red]Error:[/] {exc}")
+        return
+
+    devices = controller.settings.devices
+    if not devices:
+        print("No devices selected.")
+        return
+    locations = {
+        int(location.split(":")[0], 16): ResetLocation(
+            pos_x=int(float(location.split(":")[1].split(",")[0]) * 1e6),
+            pos_y=int(float(location.split(":")[1].split(",")[1]) * 1e6),
+        )
+        for location in locations.split("-")
+    }
+    if sorted(devices) and sorted(locations.keys()) != sorted(devices):
+        print("Selected devices and reset locations do not match.")
+        return
+    if not controller.ready_devices:
+        print("No device to reset.")
+        return
+    controller.reset(locations)
+    controller.terminate()
+
+
+@main.command()
+@click.option(
+    "-y",
+    "--yes",
+    is_flag=True,
+    help="Flash the firmware without prompt.",
+)
+@click.option(
+    "-s",
+    "--start",
+    is_flag=True,
+    help="Start the firmware once flashed.",
+)
+@click.option(
+    "-t",
+    "--ota-timeout",
+    type=float,
+    default=OTA_ACK_TIMEOUT_DEFAULT,
+    show_default=True,
+    help="Timeout in seconds for each OTA ACK message.",
+)
+@click.option(
+    "-r",
+    "--ota-max-retries",
+    type=int,
+    default=OTA_MAX_RETRIES_DEFAULT,
+    show_default=True,
+    help="Number of retries for each OTA message (start or chunk) transfer.",
+)
+@click.argument("firmware", type=click.File(mode="rb"), required=False)
+@click.pass_context
+def flash(ctx, yes, start, ota_timeout, ota_max_retries, firmware):
+    """Flash a firmware to the robots."""
+    console = Console()
+    if firmware is None:
+        console.print("[bold red]Error:[/] Missing firmware file. Exiting.")
+        ctx.exit()
+    ctx.obj["settings"].ota_timeout = ota_timeout
+    ctx.obj["settings"].ota_max_retries = ota_max_retries
+    fw = bytearray(firmware.read())
+    controller = Controller(ctx.obj["settings"])
+    if not controller.ready_devices:
+        console.print("[bold red]Error:[/] No ready device found. Exiting.")
+        controller.terminate()
+        return
+    print(
+        f"Devices to flash ([bold white]{len(controller.ready_devices)}):[/]"
+    )
+    pprint(controller.ready_devices, expand_all=True)
+    if yes is False:
+        click.confirm("Do you want to continue?", default=True, abort=True)
+
+    start_data = controller.start_ota(fw)
+    if controller.settings.verbose:
+        print("\n[b]Start OTA response:[/]")
+        pprint(start_data, indent_guides=False, expand_all=True)
+    if start_data["missed"]:
+        console = Console()
+        console.print(
+            f"[bold red]Error:[/] {len(start_data["missed"])} acknowledgments "
+            f"are missing ({', '.join(sorted(set(start_data["missed"])))}). "
+            "Aborting."
+        )
+        controller.stop()
+        controller.terminate()
+        raise click.Abort()
+    print()
+    print(f"Image size: [bold cyan]{len(fw)}B[/]")
+    print(
+        f"Image hash: [bold cyan]{start_data["ota"].fw_hash.hex().upper()}[/]"
+    )
+    print(
+        f"Radio chunks ([bold]{CHUNK_SIZE}B[/bold]): {start_data["ota"].chunks}"
+    )
+    start_time = time.time()
+    data = controller.transfer(fw, start_data["acked"])
+    print(f"Elapsed: [bold cyan]{time.time() - start_time:.3f}s[/bold cyan]")
+    print_transfer_status(data, start_data["ota"])
+    if controller.settings.verbose:
+        print("\n[b]Transfer data:[/]")
+        pprint(data, indent_guides=False, expand_all=True)
+    if all([device.success for device in data.values()]) is False:
+        controller.terminate()
+        console = Console()
+        console.print("[bold red]Error:[/] Transfer failed.")
+        raise click.Abort()
+
+    if start is True:
+        time.sleep(1)
+        controller.start()
+    controller.terminate()
+
+
+@main.command()
+@click.pass_context
+def monitor(ctx):
+    """Monitor running applications."""
+    try:
+        controller = Controller(ctx.obj["settings"])
+    except (
+        SerialInterfaceException,
+        serial.serialutil.SerialException,
+    ) as exc:
+        console = Console()
+        console.print(f"[bold red]Error:[/] {exc}")
+        return {}
+    try:
+        controller.monitor()
+    except KeyboardInterrupt:
+        print("Stopping monitor.")
+    finally:
+        controller.terminate()
+
+
+@main.command()
+@click.pass_context
+def status(ctx):
+    """Print current status of the robots."""
+    controller = Controller(ctx.obj["settings"])
+    controller.status()
+    controller.terminate()
+
+
+@main.command()
+@click.argument("message", type=str, required=True)
+@click.pass_context
+def message(ctx, message):
+    """Send a custom text message to the robots."""
+    controller = Controller(ctx.obj["settings"])
+    controller.send_message(message)
+    controller.terminate()
+
+
+if __name__ == "__main__":
+    main(obj={})

swarmit-0.4.4/testbed/swarmit/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.4.4"