mpflash 0.7.6__tar.gz → 0.8.0__tar.gz

{mpflash-0.7.6 → mpflash-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mpflash
-Version: 0.7.6
+Version: 0.8.0
 Summary: Flash and download tool for MicroPython firmwares
 Home-page: https://github.com/Josverl/micropython-stubber/blob/main/src/mpflash/README.md
 License: MIT
@@ -19,9 +19,11 @@ Classifier: Topic :: Software Development :: Build Tools
 Requires-Dist: beautifulsoup4 (>=4.12.3,<5.0.0)
 Requires-Dist: bincopy (>=20.0.0,<21.0.0)
 Requires-Dist: blkinfo (>=0.2.0,<0.3.0)
+Requires-Dist: cachetools (>=5.3.0,<6.0.0)
 Requires-Dist: esptool (>=4.7.0,<5.0.0)
 Requires-Dist: inquirer (>=3.2.4,<4.0.0)
 Requires-Dist: jsonlines (>=4.0.0,<5.0.0)
+Requires-Dist: jsons (>=1.6.3,<2.0.0)
 Requires-Dist: libusb (>=1.0.27,<2.0.0) ; sys_platform == "win32"
 Requires-Dist: loguru (>=0.7.2,<0.8.0)
 Requires-Dist: mpremote (>=1.22.0,<2.0.0)
@@ -31,20 +33,32 @@ Requires-Dist: psutil (>=5.9.8,<6.0.0)
 Requires-Dist: pygithub (>=2.1.1,<3.0.0)
 Requires-Dist: pyusb (>=1.2.1,<2.0.0)
 Requires-Dist: requests (>=2.31.0,<3.0.0)
-Requires-Dist: rich-click (>=1.7.3,<2.0.0)
+Requires-Dist: rich-click (>=1.8.1,<2.0.0)
 Requires-Dist: tenacity (==8.2.3)
 Project-URL: Repository, https://github.com/Josverl/micropython-stubber
 Description-Content-Type: text/markdown
 
 # MPFLASH
+  [![pypi version](https://badgen.net/pypi/v/mpflash)](https://pypi.org/project/mpflash/)
+  [![python versions](https://badgen.net/pypi/python/mpflash)](https://badgen.net/pypi/python/mpflash)
+[![Downloads](https://static.pepy.tech/badge/mpflash)](https://pepy.tech/project/mpflash)
+
 
 `mpflash` is a command-line tool for working with MicroPython firmware. It provides features to help you flash and update Micropython on one or more .
 
-This tool was initially created to be used in a CI/CD pipeline to automate the process of downloading and flashing MicroPython firmware to multiple boards, but it has been extend with a TUI to me be used for manual downloadig, flashing and development.
+This tool was initially created to be used in a CI/CD pipeline to automate the process of downloading and flashing MicroPython firmware to multiple boards, but it has been extend with a TUI to be used for manual downloadig, flashing and development.
 
-`mpflash` has been tested on Windows x64, Linux X64, but not (yet) macOS.
-Tested ports: `rp2`, `samd`, `esp32`, `esp32s3`, `esp32c3`, `esp8266` and `stm32`
+`mpflash` has been tested on:  
+ - OS: Windows x64, Linux X64, but not (yet) macOS.
+ - Micropython (hardware) ports: 
+    - `rp2`, using `.uf2`, using filecopy (macos not tested yet)
+    - `samd`, using ` .uf2`, using filecopy (macos not tested yet)
+    - `esp32`, using `.bin`, using esptool,
+    - `esp8266`, using `.bin`, using esptool
+    - `stm32`, using ` .dfu`, using pydfu
 
+Not yet implemented: `nrf`, `cc3200`, `mimxrt`
+ 
 ## Features
  1. List the connected boards including their firmware details, in a tabular or json format
  2. Download MicroPython firmware for versions, and matching a specified board or matches your current attached board.
@@ -72,8 +86,7 @@ On Windows this will not be an issue, but on Linux you can use  udev rules to gi
 ## Detailed usage
 You can list the connected boards using the following command:
 ```bash
-$ mpflash list
-D:\MyPython\micropython-stubber> mpflash list
+$> mpflash list
                                                Connected boards
 ┏━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━┓
 ┃ Serial  ┃Family       ┃Port  ┃Board                                      ┃CPU     ┃Version          ┃build ┃

{mpflash-0.7.6 → mpflash-0.8.0}/README.md

@@ -1,12 +1,24 @@
 # MPFLASH
+  [![pypi version](https://badgen.net/pypi/v/mpflash)](https://pypi.org/project/mpflash/)
+  [![python versions](https://badgen.net/pypi/python/mpflash)](https://badgen.net/pypi/python/mpflash)
+[![Downloads](https://static.pepy.tech/badge/mpflash)](https://pepy.tech/project/mpflash)
+
 
 `mpflash` is a command-line tool for working with MicroPython firmware. It provides features to help you flash and update Micropython on one or more .
 
-This tool was initially created to be used in a CI/CD pipeline to automate the process of downloading and flashing MicroPython firmware to multiple boards, but it has been extend with a TUI to me be used for manual downloadig, flashing and development.
+This tool was initially created to be used in a CI/CD pipeline to automate the process of downloading and flashing MicroPython firmware to multiple boards, but it has been extend with a TUI to be used for manual downloadig, flashing and development.
 
-`mpflash` has been tested on Windows x64, Linux X64, but not (yet) macOS.
-Tested ports: `rp2`, `samd`, `esp32`, `esp32s3`, `esp32c3`, `esp8266` and `stm32`
+`mpflash` has been tested on:  
+ - OS: Windows x64, Linux X64, but not (yet) macOS.
+ - Micropython (hardware) ports: 
+    - `rp2`, using `.uf2`, using filecopy (macos not tested yet)
+    - `samd`, using ` .uf2`, using filecopy (macos not tested yet)
+    - `esp32`, using `.bin`, using esptool,
+    - `esp8266`, using `.bin`, using esptool
+    - `stm32`, using ` .dfu`, using pydfu
 
+Not yet implemented: `nrf`, `cc3200`, `mimxrt`
+ 
 ## Features
  1. List the connected boards including their firmware details, in a tabular or json format
  2. Download MicroPython firmware for versions, and matching a specified board or matches your current attached board.
@@ -34,8 +46,7 @@ On Windows this will not be an issue, but on Linux you can use  udev rules to gi
 ## Detailed usage
 You can list the connected boards using the following command:
 ```bash
-$ mpflash list
-D:\MyPython\micropython-stubber> mpflash list
+$> mpflash list
                                                Connected boards
 ┏━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━┓
 ┃ Serial  ┃Family       ┃Port  ┃Board                                      ┃CPU     ┃Version          ┃build ┃

mpflash-0.8.0/mpflash/add_firmware.py

@@ -0,0 +1,98 @@
+import shutil
+from pathlib import Path
+from typing import Union
+
+import jsonlines
+import requests
+from loguru import logger as log
+
+# re-use logic from mpremote
+from mpremote.mip import _rewrite_url as rewrite_url  # type: ignore
+
+from mpflash.common import FWInfo
+from mpflash.config import config
+from mpflash.vendor.versions import get_preview_mp_version, get_stable_mp_version
+
+
+def add_firmware(
+    source: Union[Path, str],
+    new_fw: FWInfo,
+    *,
+    force: bool = False,
+    custom: bool = False,
+    description: str = "",
+) -> bool:
+    """Add a firmware to the firmware folder.
+
+    stored in the port folder, with the same filename as the source.
+
+    """
+    # Check minimal info needed
+    if not new_fw.port or not new_fw.board:
+        log.error("Port and board are required")
+        return False
+    if not isinstance(source, Path) and not source.startswith("http"):
+        log.error(f"Invalid source {source}")
+        return False
+
+    # use sensible defaults
+    source_2 = Path(source)
+    new_fw.ext = new_fw.ext or source_2.suffix
+    new_fw.variant = new_fw.variant or new_fw.board
+    new_fw.custom = new_fw.custom or custom
+    new_fw.description = new_fw.description or description
+    if not new_fw.version:
+        # TODO: Get version from filename
+        # or use the last preview version
+        new_fw.version = get_preview_mp_version() if new_fw.preview else get_stable_mp_version()
+
+    config.firmware_folder.mkdir(exist_ok=True)
+
+    fw_filename = config.firmware_folder / new_fw.port / source_2.name
+
+    new_fw.filename = str(fw_filename.relative_to(config.firmware_folder))
+    new_fw.firmware = source.as_uri() if isinstance(source, Path) else source
+
+    if not copy_firmware(source, fw_filename, force):
+        log.error(f"Failed to copy {source} to {fw_filename}")
+        return False
+    # add to inventory
+    with jsonlines.open(config.firmware_folder / "firmware.jsonl", "a") as writer:
+        log.info(f"Adding {new_fw.port} {new_fw.board}")
+        log.info(f"    to {fw_filename}")
+
+        writer.write(new_fw.to_dict())
+    return True
+
+
+def copy_firmware(source: Union[Path, str], fw_filename: Path, force: bool = False):
+    """Add a firmware to the firmware folder.
+    stored in the port folder, with the same filename as the source.
+    """
+    if fw_filename.exists() and not force:
+        log.error(f" {fw_filename} already exists. Use --force to overwrite")
+        return False
+    fw_filename.parent.mkdir(exist_ok=True)
+    if isinstance(source, Path):
+        if not source.exists():
+            log.error(f"File {source} does not exist")
+            return False
+        # file copy
+        log.debug(f"Copy {source} to {fw_filename}")
+        shutil.copy(source, fw_filename)
+        return True
+    # handle github urls
+    url = rewrite_url(source)
+    if str(source).startswith("http://") or str(source).startswith("https://"):
+        log.debug(f"Download {url} to {fw_filename}")
+        response = requests.get(url)
+
+        if response.status_code == 200:
+            with open(fw_filename, "wb") as file:
+                file.write(response.content)
+                log.info("File downloaded and saved successfully.")
+                return True
+        else:
+            print("Failed to download the file.")
+            return False
+    return False

{mpflash-0.7.6 → mpflash-0.8.0}/mpflash/ask_input.py

@@ -5,58 +5,33 @@ Note: The prompts can use "{version}" and "{action}" to insert the version and a
 The values are provided from the answers dictionary.
 """
 
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Dict, List, Sequence, Tuple, Union
+from typing import List, Sequence, Tuple, Union
 
 from loguru import logger as log
 
-from mpflash.config import config
-from mpflash.mpboard_id import get_known_boards_for_port, get_known_ports, known_stored_boards
-from mpflash.mpremoteboard import MPRemoteBoard
-from mpflash.vendor.versions import micropython_versions
-
-
-@dataclass
-class Params:
-    ports: List[str] = field(default_factory=list)
-    boards: List[str] = field(default_factory=list)
-    versions: List[str] = field(default_factory=list)
-    fw_folder: Path = Path()
-
-
-@dataclass
-class DownloadParams(Params):
-    clean: bool = False
-    force: bool = False
-
-
-@dataclass
-class FlashParams(Params):
-    # TODO: Should Serial port be a list?
-    serial: str = ""
-    erase: bool = True
-    bootloader: bool = True
-    cpu: str = ""
-
-
-ParamType = Union[DownloadParams, FlashParams]
+from .common import DownloadParams, FlashParams, ParamType
+from .config import config
+from .mpboard_id import get_known_boards_for_port, get_known_ports, known_stored_boards
+from .mpremoteboard import MPRemoteBoard
+from .vendor.versions import micropython_versions
 
 
 def ask_missing_params(
     params: ParamType,
-    # action: str = "download",
 ) -> ParamType:
     """
     Asks the user for parameters that have not been supplied on the commandline and returns the updated params.
 
     Args:
         params (ParamType): The parameters to be updated.
-        action (str, optional): The action to be performed. Defaults to "download".
 
     Returns:
         ParamType: The updated parameters.
     """
+    import inquirer
+
+    log.trace(f"ask_missing_params: {params}")
+
     # if action flash,  single input
     # if action download, multiple input
     multi_select = isinstance(params, DownloadParams)
@@ -64,36 +39,36 @@ def ask_missing_params(
     if not config.interactive:
         # no interactivity allowed
         return params
-    # import only when needed to reduce load time
-    import inquirer
 
     questions = []
-    answers = {"action": "download" if isinstance(params, DownloadParams) else "flash"}
+    answers: dict[str, Union[str, List]] = {"action": action}
     if not multi_select:
         if not params.serial or "?" in params.serial:
-            ask_serialport(questions)
+            questions.append(ask_serialport(multi_select=False, bluetooth=False))
         else:
             answers["serial"] = params.serial
 
-    if not params.versions or "?" in params.versions:
-        ask_versions(questions, multi_select=multi_select, action=action)
+    if params.versions == [] or "?" in params.versions:
+        questions.append(ask_mp_version(multi_select=multi_select, action=action))
     else:
         # versions is used to show only the boards for the selected versions
         answers["versions"] = params.versions  # type: ignore
 
     if not params.boards or "?" in params.boards:
-        ask_port_board(questions, multi_select=multi_select, action=action)
+        questions.extend(ask_port_board(multi_select=multi_select, action=action))
     if questions:
-        answers = inquirer.prompt(questions, answers=answers)
+        answers = inquirer.prompt(questions, answers=answers)  # type: ignore
     if not answers:
         # input cancelled by user
         return []  # type: ignore
-    # print(repr(answers))
+    log.trace(f"answers: {answers}")
     if isinstance(params, FlashParams) and "serial" in answers:
-        params.serial = answers["serial"].split()[0]  # split to remove the description
+        if isinstance(answers["serial"], str):
+            answers["serial"] = [answers["serial"]]
+        params.serial = [s.split()[0] for s in answers["serial"]]  # split to remove the description
     if "port" in answers:
         params.ports = [p for p in params.ports if p != "?"]  # remove the "?" if present
-        params.ports.append(answers["port"])
+        params.ports.extend(answers["port"])
     if "boards" in answers:
         params.boards = [b for b in params.boards if b != "?"]  # remove the "?" if present
         params.boards.extend(answers["boards"] if isinstance(answers["boards"], list) else [answers["boards"]])
@@ -108,7 +83,7 @@ def ask_missing_params(
     params.ports = list(set(params.ports))
     params.boards = list(set(params.boards))
     params.versions = list(set(params.versions))
-    log.debug(repr(params))
+    log.trace(f"ask_missing_params returns: {params}")
 
     return params
 
@@ -123,19 +98,18 @@ def filter_matching_boards(answers: dict) -> Sequence[Tuple[str, str]]:
     Returns:
         Sequence[Tuple[str, str]]: The filtered boards.
     """
+    versions = None
     # if version is not asked ; then need to get the version from the inputs
     if "versions" in answers:
-        _versions = list(answers["versions"])
-        if "stable" in _versions:
-            _versions.remove("stable")
-            _versions.append(micropython_versions()[-2])  # latest stable
-        if "preview" in _versions:
-            _versions.remove("preview")
-            _versions.extend((micropython_versions()[-1], micropython_versions()[-2]))  # latest preview and stable
-
-        some_boards = known_stored_boards(answers["port"], _versions)  #    or known_mp_boards(answers["port"])
-    else:
-        some_boards = known_stored_boards(answers["port"])
+        versions = list(answers["versions"])
+        if "stable" in versions:
+            versions.remove("stable")
+            versions.append(micropython_versions()[-2])  # latest stable
+        elif "preview" in versions:
+            versions.remove("preview")
+            versions.extend((micropython_versions()[-1], micropython_versions()[-2]))  # latest preview and stable
+
+    some_boards = known_stored_boards(answers["port"], versions)  #    or known_mp_boards(answers["port"])
 
     if some_boards:
         # Create a dictionary where the keys are the second elements of the tuples
@@ -144,11 +118,11 @@ def filter_matching_boards(answers: dict) -> Sequence[Tuple[str, str]]:
         # Get the values of the dictionary, which are the unique items from the original list
         some_boards = list(unique_dict.values())
     else:
-        some_boards = [(f"No {answers['port']} boards found for version(s) {_versions}", "")]
+        some_boards = [(f"No {answers['port']} boards found for version(s) {versions}", "")]
     return some_boards
 
 
-def ask_port_board(questions: list, *, multi_select: bool, action: str):
+def ask_port_board(*, multi_select: bool, action: str):
     """
     Asks the user for the port and board selection.
 
@@ -165,29 +139,25 @@ def ask_port_board(questions: list, *, multi_select: bool, action: str):
     # if action flash,  single input
     # if action download, multiple input
     inquirer_ux = inquirer.Checkbox if multi_select else inquirer.List
-    questions.extend(
-        (
-            inquirer.List(
-                "port",
-                message="Which port do you want to {action} " + "to {serial} ?" if action == "flash" else "?",
-                choices=get_known_ports(),
-                autocomplete=True,
-            ),
-            inquirer_ux(
-                "boards",
-                message=(
-                    "Which {port} board firmware do you want to {action} " + "to {serial} ?"
-                    if action == "flash"
-                    else "?"
-                ),
-                choices=filter_matching_boards,
-                validate=lambda _, x: True if x else "Please select at least one board",  # type: ignore
+    return [
+        inquirer.List(
+            "port",
+            message="Which port do you want to {action} " + "to {serial} ?" if action == "flash" else "?",
+            choices=get_known_ports(),
+            # autocomplete=True,
+        ),
+        inquirer_ux(
+            "boards",
+            message=(
+                "Which {port} board firmware do you want to {action} " + "to {serial} ?" if action == "flash" else "?"
             ),
-        )
-    )
+            choices=filter_matching_boards,
+            validate=lambda _, x: True if x else "Please select at least one board",  # type: ignore
+        ),
+    ]
 
 
-def ask_versions(questions: list, *, multi_select: bool, action: str):
+def ask_mp_version(multi_select: bool, action: str):
     """
     Asks the user for the version selection.
 
@@ -196,46 +166,43 @@ def ask_versions(questions: list, *, multi_select: bool, action: str):
         action (str): The action to be performed.
 
     Returns:
-        None
+
     """
     # import only when needed to reduce load time
     import inquirer
     import inquirer.errors
 
     input_ux = inquirer.Checkbox if multi_select else inquirer.List
+
     mp_versions: List[str] = micropython_versions()
-    mp_versions = [v for v in mp_versions if "preview" not in v]
+    mp_versions.reverse()  # newest first
 
     # remove the versions for which there are no known boards in the board_info.json
     # todo: this may be a little slow
-    mp_versions = [v for v in mp_versions if get_known_boards_for_port("stm32", [v])]
-
-    mp_versions.append("preview")
-    mp_versions.reverse()  # newest first
+    mp_versions = [v for v in mp_versions if "preview" in v or get_known_boards_for_port("stm32", [v])]
 
     def at_least_one_validation(answers, current) -> bool:
         if not current:
             raise inquirer.errors.ValidationError("", reason="Please select at least one version")
-        if isinstance(current, list):
-            if not any(current):
-                raise inquirer.errors.ValidationError("", reason="Please select at least one version")
+        if isinstance(current, list) and not any(current):
+            raise inquirer.errors.ValidationError("", reason="Please select at least one version")
         return True
 
-    questions.append(
-        input_ux(
-            # inquirer.List(
-            "versions",
-            message="Which version(s) do you want to {action} " + ("to {serial} ?" if action == "flash" else "?"),
-            # Hints would be nice , but needs a hint for each and every option
-            # hints=["Use space to select multiple options"],
-            choices=mp_versions,
-            autocomplete=True,
-            validate=at_least_one_validation,  # type: ignore
-        )
+    message = "Which version(s) do you want to {action} " + ("to {serial} ?" if action == "flash" else "?")
+    q = input_ux(
+        # inquirer.List(
+        "versions",
+        message=message,
+        # Hints would be nice , but needs a hint for each and every option
+        # hints=["Use space to select multiple options"],
+        choices=mp_versions,
+        autocomplete=True,
+        validate=at_least_one_validation,  # type: ignore
     )
+    return q
 
 
-def ask_serialport(questions: list, *, multi_select: bool = False, bluetooth: bool = False):
+def ask_serialport(*, multi_select: bool = False, bluetooth: bool = False):
     """
     Asks the user for the serial port selection.
 
@@ -250,14 +217,10 @@ def ask_serialport(questions: list, *, multi_select: bool = False, bluetooth: bo
     import inquirer
 
     comports = MPRemoteBoard.connected_boards(bluetooth=bluetooth, description=True)
-    questions.append(
-        inquirer.List(
-            "serial",
-            message="Which serial port do you want to {action} ?",
-            choices=comports,
-            other=True,
-            validate=lambda _, x: True if x else "Please select or enter a serial port",  # type: ignore
-        )
+    return inquirer.List(
+        "serial",
+        message="Which serial port do you want to {action} ?",
+        choices=comports,
+        other=True,
+        validate=lambda _, x: True if x else "Please select or enter a serial port",  # type: ignore
     )
-
-    return questions

{mpflash-0.7.6 → mpflash-0.8.0}/mpflash/cli_download.py

@@ -1,18 +1,18 @@
 """CLI to Download MicroPython firmware for specific ports, boards and versions."""
 
 from pathlib import Path
-from typing import List, Tuple
 
 import rich_click as click
 from loguru import logger as log
 
+from mpflash.connected import connected_ports_boards
 from mpflash.errors import MPFlashError
 from mpflash.mpboard_id import find_known_board
 from mpflash.vendor.versions import clean_version
 
-from .ask_input import DownloadParams, ask_missing_params
+from .ask_input import ask_missing_params
 from .cli_group import cli
-from .cli_list import list_mcus
+from .common import DownloadParams
 from .config import config
 from .download import download
 
@@ -50,6 +50,28 @@ from .download import download
     help="The board(s) to download the firmware for.",
     metavar="BOARD_ID or ?",
 )
+@click.option(
+    "--serial",
+    "--serial-port",
+    "-s",
+    "serial",
+    default=["*"],
+    show_default=True,
+    multiple=True,
+    help="Which serial port(s) to flash",
+    metavar="SERIALPORT",
+)
+@click.option(
+    "--ignore",
+    "-i",
+    is_eager=True,
+    help="Serial port(s) to ignore. Defaults to MPFLASH_IGNORE.",
+    multiple=True,
+    default=[],
+    envvar="MPFLASH_IGNORE",
+    show_default=True,
+    metavar="SERIALPORT",
+)
 @click.option(
     "--clean/--no-clean",
     default=True,
@@ -67,6 +89,10 @@ def cli_download(**kwargs) -> int:
     params = DownloadParams(**kwargs)
     params.versions = list(params.versions)
     params.boards = list(params.boards)
+    params.serial = list(params.serial)
+    params.ignore = list(params.ignore)
+
+    # all_boards: List[MPRemoteBoard] = []
     if params.boards:
         if not params.ports:
             # no ports specified - resolve ports from specified boards by resolving board IDs
@@ -74,12 +100,12 @@ def cli_download(**kwargs) -> int:
                 if board != "?":
                     try:
                         board_ = find_known_board(board)
-                        params.ports.append(board_["port"])
+                        params.ports.append(board_.port)
                     except MPFlashError as e:
                         log.error(f"{e}")
     else:
         # no boards specified - detect connected ports and boards
-        params.ports, params.boards = connected_ports_boards()
+        params.ports, params.boards, _ = connected_ports_boards(include=params.serial, ignore=params.ignore)
 
     params = ask_missing_params(params)
     if not params:  # Cancelled by user
@@ -100,19 +126,3 @@ def cli_download(**kwargs) -> int:
     except MPFlashError as e:
         log.error(f"{e}")
         return 1
-
-
-def connected_ports_boards() -> Tuple[List[str], List[str]]:
-    """
-    Returns a tuple containing lists of unique ports and boards from the connected MCUs.
-    Boards that are physically connected, but give no tangible response are ignored.
-
-    Returns:
-        A tuple containing two lists:
-            - A list of unique ports where MCUs are connected.
-            - A list of unique board names of the connected MCUs.
-    """
-    mpr_boards = [b for b in list_mcus() if b.connected]
-    ports = list({b.port for b in mpr_boards})
-    boards = list({b.board for b in mpr_boards})
-    return ports, boards