modpoll 1.0.0__tar.gz → 1.2.0__tar.gz

{modpoll-1.0.0 → modpoll-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: modpoll
-Version: 1.0.0
+Version: 1.2.0
 Summary: A New Command-line Tool for Modbus and MQTT
 Home-page: https://github.com/gavinying/modpoll
 Author: Ying Shaodong
@@ -12,12 +12,12 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Provides-Extra: serial
-Requires-Dist: importlib-metadata (>=8.0.0,<9.0.0)
-Requires-Dist: paho-mqtt (>=1.6.1,<2.0.0)
-Requires-Dist: prettytable (>=3.6.0,<4.0.0)
-Requires-Dist: pymodbus (>=3.5.4,<3.6.0)
+Requires-Dist: importlib-metadata (>=8.3.0,<9.0.0)
+Requires-Dist: paho-mqtt (>=2.1.0,<3.0.0)
+Requires-Dist: prettytable (>=3.9.0,<4.0.0)
+Requires-Dist: pymodbus (>=3.6.9,<3.7.0)
 Requires-Dist: pyserial (>=3.5,<4.0) ; extra == "serial"
-Requires-Dist: requests (>=2.28.1,<3.0.0)
+Requires-Dist: requests (>=2.32.0,<3.0.0)
 Project-URL: Documentation, https://gavinying.github.io/modpoll
 Project-URL: Repository, https://github.com/gavinying/modpoll
 Description-Content-Type: text/markdown
@@ -29,27 +29,27 @@ Description-Content-Type: text/markdown
 [![License](https://img.shields.io/github/license/gavinying/modpoll)](https://img.shields.io/github/license/gavinying/modpoll)
 [![Downloads](https://static.pepy.tech/badge/modpoll/week)](https://pepy.tech/project/modpoll)
 
+Show your love for this project by starring our repo 🌟, so we are encouraged to continue improving.
+
 > Learn more about *modpoll* usage at [documentation](https://gavinying.github.io/modpoll) site.
 
 ## Motivation
 
-The initial idea of creating this tool is to help myself debugging new devices during site survey. A site survey usually has limited time and space, working on-site also piles up some pressures. At that time, a portable swiss-knife toolkit is our best friend.
+The idea for creating this tool originated from my need to efficiently check new devices during site surveys. These surveys are often time-constrained and space-limited, with on-site work adding to the pressure. In such situations, a portable Swiss Army knife toolkit becomes a valuable companion.
 
-This program can be easily deployed to Raspberry Pi or similar embedded devices, polling data from modbus devices, users can choose to log data locally or publish to a MQTT broker for further debugging.
+This tool can be easily deployed on a Raspberry Pi or similar embedded devices, polling data from a Modbus network or connected devices. Users can choose to log the data locally or publish it to an MQTT broker for further troubleshooting.
 
-The MQTT broker can be setup on the same Raspberry Pi or on the cloud. Once data successfully published, users can subscribe to a specific MQTT topic to view the data via a smartphone at your fingertip.
+The MQTT broker can be set up either on the same Raspberry Pi or in the cloud. Once data is successfully published, users can subscribe to the relevant MQTT topics and conveniently view the data on their smartphones.
 
 <p align="center">
   <img src="docs/assets/modpoll-usage.png">
 </p>
 
-Moreover, you can also run this program continuously on a server as a Modbus-MQTT gateway, i.e. polling from local Modbus devices and forwarding data to a centralized cloud service.
+Moreover, you can also run this tool continuously on a server as a Modbus-MQTT gateway, i.e. polling from local Modbus devices and forwarding data to a centralized cloud service.
 
 In fact, *modpoll* helps to bridge between the traditional field-bus world and the new IoT world.
 
-> This program is designed to be a standalone tool, it works out-of-the-box on Linux/macOS/Windows.
-
-> If you are looing for a modbus python library, please consider the following great open source projects, [pymodbus](https://github.com/riptideio/pymodbus) or [minimalmodbus](https://github.com/pyhys/minimalmodbus)
+> 💡 **Tips:** This tool is designed to be a standalone executable application, which works out-of-the-box on Linux/macOS/Windows. If you are looking for a Modbus python library, please consider the following great open-source projects, [pymodbus](https://github.com/riptideio/pymodbus) or [minimalmodbus](https://github.com/pyhys/minimalmodbus)
 
 
 ## Feature
@@ -63,7 +63,7 @@ In fact, *modpoll* helps to bridge between the traditional field-bus world and t
 
 ## Installation
 
-This program tested on Python 3.8+, the package is available in the [Python Package Index](https://pypi.org/), users can easily install it using `pip` or `pipx`.
+This tool tested on Python 3.8+, the package is available in the [Python Package Index](https://pypi.org/), users can easily install it using `pip` or `pipx`.
 
 ### Using PIP
 
@@ -108,7 +108,7 @@ pipx upgrade modpoll
 Let's start exploring *modpoll* with *modsim* device, run the following command to get a first glimpse,
 
 ```bash
-modpoll \
+modpoll --once \
   --tcp modsim.topmaker.net \
   --config https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv
 ```
@@ -211,7 +211,7 @@ modpoll \
 
 ### Publish data to MQTT broker
 
-This is a useful function of this new *modpoll* tool, which provides a simple way to publish collected modbus data to MQTT broker, so users can view data from a smart phone via a MQTT client.
+This is a useful function of this new *modpoll* tool, which provides a simple way to publish collected Modbus data to MQTT broker, so users can view data from a smart phone via a MQTT client.
 
 The following example uses a public MQTT broker `mqtt.eclipseprojects.io` for test purpose. You can also set up your own MQTT broker locally using [mosquitto](https://mosquitto.org/download/).
 
@@ -224,7 +224,9 @@ modpoll \
 
 With successful data polling and publishing, you can subscribe the topic `modpoll/modsim` on the same MQTT broker `mqtt.eclipseprojects.io` to view the collected data.
 
-> The MQTT topic uses `<mqtt_topic_prefix>/<deviceid>` pattern, <mqtt_topic_prefix> is provided by `--mqtt-topic-prefix` argument, the default value is `modpoll/`  and <deviceid> is provided by the Modbus configure file.
+The MQTT topic uses `<mqtt_topic_prefix>/<device_name>` pattern, <mqtt_topic_prefix> is provided by `--mqtt-topic-prefix` argument, the default value is `modpoll`  and <device_name> is provided by the Modbus configure file.
+
+> ⚠️ **Note:** The `--mqtt-topic-prefix` argument is deprecated and will be removed in the future release. Use `--mqtt-publish-topic-pattern` and `--mqtt-subscribe-topic-pattern` instead. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility. See [document](https://gavinying.github.io/modpoll/usage.html#Named%20Arguments) for details.
 
 
 <p align="center">
@@ -234,7 +236,7 @@ With successful data polling and publishing, you can subscribe the topic `modpol
 
 ### Write registers via MQTT publish
 
-The *modpoll* tool will subscribe to the topic `<mqtt_topic_prefix>/<deviceid>/set` once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
+The *modpoll* tool will subscribe to the topic `modpoll/<device_name>/set` by default once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
 
 - To write a single holding register (address at `40001`)
 
@@ -259,13 +261,13 @@ The *modpoll* tool will subscribe to the topic `<mqtt_topic_prefix>/<deviceid>/s
 
 ## Run with docker
 
-A docker image has been provided for user to directly run the program without local installation,
+A docker image has been provided for user to directly run the tool without local installation,
 
   ```bash
   docker run --rm helloysd/modpoll
   ```
 
-It shows the version of the program by default.
+It shows the version of the tool by default.
 
 Similar to the above *modsim* test, we can poll data with `docker run`, in order to avoid printing out received data, the argument `--daemon` or `-d` is recommended to use with docker.
 

{modpoll-1.0.0 → modpoll-1.2.0}/README.md

@@ -5,27 +5,27 @@
 [![License](https://img.shields.io/github/license/gavinying/modpoll)](https://img.shields.io/github/license/gavinying/modpoll)
 [![Downloads](https://static.pepy.tech/badge/modpoll/week)](https://pepy.tech/project/modpoll)
 
+Show your love for this project by starring our repo 🌟, so we are encouraged to continue improving.
+
 > Learn more about *modpoll* usage at [documentation](https://gavinying.github.io/modpoll) site.
 
 ## Motivation
 
-The initial idea of creating this tool is to help myself debugging new devices during site survey. A site survey usually has limited time and space, working on-site also piles up some pressures. At that time, a portable swiss-knife toolkit is our best friend.
+The idea for creating this tool originated from my need to efficiently check new devices during site surveys. These surveys are often time-constrained and space-limited, with on-site work adding to the pressure. In such situations, a portable Swiss Army knife toolkit becomes a valuable companion.
 
-This program can be easily deployed to Raspberry Pi or similar embedded devices, polling data from modbus devices, users can choose to log data locally or publish to a MQTT broker for further debugging.
+This tool can be easily deployed on a Raspberry Pi or similar embedded devices, polling data from a Modbus network or connected devices. Users can choose to log the data locally or publish it to an MQTT broker for further troubleshooting.
 
-The MQTT broker can be setup on the same Raspberry Pi or on the cloud. Once data successfully published, users can subscribe to a specific MQTT topic to view the data via a smartphone at your fingertip.
+The MQTT broker can be set up either on the same Raspberry Pi or in the cloud. Once data is successfully published, users can subscribe to the relevant MQTT topics and conveniently view the data on their smartphones.
 
 <p align="center">
   <img src="docs/assets/modpoll-usage.png">
 </p>
 
-Moreover, you can also run this program continuously on a server as a Modbus-MQTT gateway, i.e. polling from local Modbus devices and forwarding data to a centralized cloud service.
+Moreover, you can also run this tool continuously on a server as a Modbus-MQTT gateway, i.e. polling from local Modbus devices and forwarding data to a centralized cloud service.
 
 In fact, *modpoll* helps to bridge between the traditional field-bus world and the new IoT world.
 
-> This program is designed to be a standalone tool, it works out-of-the-box on Linux/macOS/Windows.
-
-> If you are looing for a modbus python library, please consider the following great open source projects, [pymodbus](https://github.com/riptideio/pymodbus) or [minimalmodbus](https://github.com/pyhys/minimalmodbus)
+> 💡 **Tips:** This tool is designed to be a standalone executable application, which works out-of-the-box on Linux/macOS/Windows. If you are looking for a Modbus python library, please consider the following great open-source projects, [pymodbus](https://github.com/riptideio/pymodbus) or [minimalmodbus](https://github.com/pyhys/minimalmodbus)
 
 
 ## Feature
@@ -39,7 +39,7 @@ In fact, *modpoll* helps to bridge between the traditional field-bus world and t
 
 ## Installation
 
-This program tested on Python 3.8+, the package is available in the [Python Package Index](https://pypi.org/), users can easily install it using `pip` or `pipx`.
+This tool tested on Python 3.8+, the package is available in the [Python Package Index](https://pypi.org/), users can easily install it using `pip` or `pipx`.
 
 ### Using PIP
 
@@ -84,7 +84,7 @@ pipx upgrade modpoll
 Let's start exploring *modpoll* with *modsim* device, run the following command to get a first glimpse,
 
 ```bash
-modpoll \
+modpoll --once \
   --tcp modsim.topmaker.net \
   --config https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv
 ```
@@ -187,7 +187,7 @@ modpoll \
 
 ### Publish data to MQTT broker
 
-This is a useful function of this new *modpoll* tool, which provides a simple way to publish collected modbus data to MQTT broker, so users can view data from a smart phone via a MQTT client.
+This is a useful function of this new *modpoll* tool, which provides a simple way to publish collected Modbus data to MQTT broker, so users can view data from a smart phone via a MQTT client.
 
 The following example uses a public MQTT broker `mqtt.eclipseprojects.io` for test purpose. You can also set up your own MQTT broker locally using [mosquitto](https://mosquitto.org/download/).
 
@@ -200,7 +200,9 @@ modpoll \
 
 With successful data polling and publishing, you can subscribe the topic `modpoll/modsim` on the same MQTT broker `mqtt.eclipseprojects.io` to view the collected data.
 
-> The MQTT topic uses `<mqtt_topic_prefix>/<deviceid>` pattern, <mqtt_topic_prefix> is provided by `--mqtt-topic-prefix` argument, the default value is `modpoll/`  and <deviceid> is provided by the Modbus configure file.
+The MQTT topic uses `<mqtt_topic_prefix>/<device_name>` pattern, <mqtt_topic_prefix> is provided by `--mqtt-topic-prefix` argument, the default value is `modpoll`  and <device_name> is provided by the Modbus configure file.
+
+> ⚠️ **Note:** The `--mqtt-topic-prefix` argument is deprecated and will be removed in the future release. Use `--mqtt-publish-topic-pattern` and `--mqtt-subscribe-topic-pattern` instead. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility. See [document](https://gavinying.github.io/modpoll/usage.html#Named%20Arguments) for details.
 
 
 <p align="center">
@@ -210,7 +212,7 @@ With successful data polling and publishing, you can subscribe the topic `modpol
 
 ### Write registers via MQTT publish
 
-The *modpoll* tool will subscribe to the topic `<mqtt_topic_prefix>/<deviceid>/set` once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
+The *modpoll* tool will subscribe to the topic `modpoll/<device_name>/set` by default once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
 
 - To write a single holding register (address at `40001`)
 
@@ -235,13 +237,13 @@ The *modpoll* tool will subscribe to the topic `<mqtt_topic_prefix>/<deviceid>/s
 
 ## Run with docker
 
-A docker image has been provided for user to directly run the program without local installation,
+A docker image has been provided for user to directly run the tool without local installation,
 
   ```bash
   docker run --rm helloysd/modpoll
   ```
 
-It shows the version of the program by default.
+It shows the version of the tool by default.
 
 Similar to the above *modsim* test, we can poll data with `docker run`, in order to avoid printing out received data, the argument `--daemon` or `-d` is recommended to use with docker.
 

{modpoll-1.0.0 → modpoll-1.2.0}/modpoll/arg_parser.py

@@ -78,6 +78,12 @@ def get_parser():
         default=None,
         help="The file name to export references/registers",
     )
+    parser.add_argument(
+        "--mqtt-version",
+        choices=["3.1.1", "5.0"],
+        default="3.1.1",
+        help="MQTT version. Defaults to MQTT v3.1.1",
+    )
     parser.add_argument(
         "--mqtt-host",
         default=None,
@@ -96,8 +102,23 @@ def get_parser():
     )
     parser.add_argument(
         "--mqtt-topic-prefix",
-        default="modpoll/",
-        help='Topic prefix for MQTT subscribing/publishing. Defaults to "modpoll/"',
+        default=None,
+        help="(DEPRECATED) Topic prefix for MQTT subscribing/publishing. Defaults to None",
+    )
+    parser.add_argument(
+        "--mqtt-publish-topic-pattern",
+        default="modpoll/<device_name>",
+        help='Topic pattern for MQTT publish. Use <device_name> as placeholder for the device names in Modbus config. Defaults to "modpoll/<device_name>"',
+    )
+    parser.add_argument(
+        "--mqtt-subscribe-topic-pattern",
+        default="modpoll/<device_name>/set",
+        help='Topic pattern for MQTT subscribe. Use <device_name> as placeholder for the device names in Modbus config. Defaults to "modpoll/<device_name>/set"',
+    )
+    parser.add_argument(
+        "--mqtt-diagnostics-topic-pattern",
+        default=None,
+        help="Topic pattern for MQTT diagnostics. Use <device_name> as placeholder for the device names in Modbus config. Defaults to None",
     )
     parser.add_argument(
         "--mqtt-qos",

modpoll-1.2.0/modpoll/main.py

@@ -0,0 +1,191 @@
+import datetime
+import json
+import logging
+import re
+import signal
+import sys
+import threading
+from datetime import timezone
+
+from modpoll.arg_parser import get_parser
+from modpoll.modbus_task import (
+    modbus_setup,
+    modbus_device_list,
+    modbus_poll,
+    modbus_publish,
+    modbus_publish_diagnostics,
+    modbus_write_coil,
+    modbus_write_register,
+    modbus_export,
+    modbus_close,
+)
+from modpoll.mqtt_task import (
+    mqttc_setup,
+    mqttc_connect,
+    mqttc_receive,
+    mqttc_close,
+)
+
+from . import __version__
+
+LOG_SIMPLE = "%(asctime)s | %(levelname).1s | %(name)s | %(message)s"
+logger = None
+event_exit = threading.Event()
+
+
+def setup_logging(level, format):
+    logging.basicConfig(level=level, format=format)
+
+
+def _signal_handler(signal, frame):
+    logger.info(f"Exiting {sys.argv[0]}")
+    event_exit.set()
+
+
+def get_utc_time():
+    dt = datetime.datetime.now(timezone.utc)
+    utc_time = dt.replace(tzinfo=timezone.utc)
+    return utc_time.timestamp()
+
+
+def app(name="modpoll"):
+    print(
+        f"\nModpoll v{__version__} - A New Command-line Tool for Modbus and MQTT\n",
+        flush=True,
+    )
+
+    signal.signal(signal.SIGINT, _signal_handler)
+
+    # parse args
+    args = get_parser().parse_args()
+
+    # get logger
+    setup_logging(args.loglevel, LOG_SIMPLE)
+    global logger
+    logger = logging.getLogger(__name__)
+
+    # setup modbus
+    if not modbus_setup(args, event_exit):
+        logger.error("Failed to setup modbus client(master)")
+        modbus_close()
+        mqttc_close()
+        exit(1)
+
+    # Overwrite topic pattern if topic prefix is specified for backward compatibility
+    if args.mqtt_topic_prefix:
+        logger.warning(
+            "(DEPRECATED): The `--mqtt-topic-prefix` argument is deprecated and will be removed in the future release. Use `--mqtt-publish-topic-pattern` and `--mqtt-subscribe-topic-pattern` instead. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility."
+        )
+        if args.mqtt_topic_prefix.endswith("/"):
+            args.mqtt_topic_prefix = args.mqtt_topic_prefix[:-1]
+        args.mqtt_publish_topic_pattern = f"{args.mqtt_topic_prefix}/<device_name>"
+        args.mqtt_subscribe_topic_pattern = (
+            f"{args.mqtt_topic_prefix}/<device_name>/set"
+        )
+
+    # create mqtt subscribe topics
+    subscribe_topics = []
+    if args.mqtt_subscribe_topic_pattern:
+        if "<device_name>" in args.mqtt_subscribe_topic_pattern:
+            for device in modbus_device_list():
+                topic = args.mqtt_subscribe_topic_pattern.replace(
+                    "<device_name>", device.name
+                )
+                subscribe_topics.append(topic)
+        else:
+            subscribe_topics.append(args.mqtt_subscribe_topic_pattern)
+
+    # setup mqtt
+    if args.mqtt_host:
+        logger.info(f"Setup MQTT connection to {args.mqtt_host}")
+        if not mqttc_setup(args, subscribe_topics):
+            logger.error("Failed to setup MQTT client")
+            modbus_close()
+            mqttc_close()
+            exit(1)
+        if not mqttc_connect():
+            logger.error("Failed to connect to MQTT host")
+            modbus_close()
+            mqttc_close()
+            exit(1)
+    else:
+        logger.info("No MQTT host specified, skip MQTT setup.")
+
+    # main loop
+    last_check = 0
+    last_diag = 0
+    while not event_exit.is_set():
+        now = get_utc_time()
+        # routine check
+        if now > last_check + args.rate:
+            if last_check == 0:
+                elapsed = args.rate
+            else:
+                elapsed = round(now - last_check, 6)
+            last_check = now
+            logger.info(
+                f" ====== modpoll polling at rate:{args.rate}s, actual:{elapsed}s ======"
+            )
+            modbus_poll()
+            if event_exit.is_set():
+                break
+            if args.mqtt_host:
+                if args.timestamp:
+                    modbus_publish(timestamp=now)
+                else:
+                    modbus_publish()
+            if args.export:
+                if args.timestamp:
+                    modbus_export(args.export, timestamp=now)
+                else:
+                    modbus_export(args.export)
+        if args.diagnostics_rate > 0 and now > last_diag + args.diagnostics_rate:
+            last_diag = now
+            modbus_publish_diagnostics()
+        if event_exit.is_set():
+            break
+        # Check if receive mqtt request
+        topic, payload = mqttc_receive()
+        if topic and payload:
+            topic_regex = args.mqtt_subscribe_topic_pattern.replace(
+                "<device_name>", "([^/\n]*)"
+            )
+            device_name = re.search(topic_regex, topic).group(1)
+            if device_name:
+                logger.info(f"Received request to write data for device {device_name}")
+                try:
+                    reg = json.loads(payload)
+                    object_type = reg["object_type"]
+                    address = reg["address"]
+                    value = reg["value"]
+                    if "coil" == object_type:
+                        if modbus_write_coil(device_name, address, value):
+                            logger.info(
+                                f"Successfully write coil(s): device={device_name}, address={address}, value={value}"
+                            )
+                        else:
+                            logger.warning(
+                                f"Failed to write coil(s): device={device_name}, address={address}, value={value}"
+                            )
+                    elif "holding_register" == object_type:
+                        if modbus_write_register(device_name, address, value):
+                            logger.info(
+                                f"Successfully write register(s): device={device_name}, address={address}, value={value}"
+                            )
+                        else:
+                            logger.warning(
+                                f"Failed to write register(s): device={device_name}, address={address}, value={value}"
+                            )
+                except KeyError:
+                    logger.error(f"No required key found: {payload}")
+                except json.decoder.JSONDecodeError:
+                    logger.error(f"Failed to parse json message: {payload}")
+        if args.once:
+            event_exit.set()
+            break
+    modbus_close()
+    mqttc_close()
+
+
+if __name__ == "__main__":
+    app()