modpoll 1.2.0__tar.gz → 1.3.3__tar.gz

{modpoll-1.2.0 → modpoll-1.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: modpoll
-Version: 1.2.0
+Version: 1.3.3
 Summary: A New Command-line Tool for Modbus and MQTT
 Home-page: https://github.com/gavinying/modpoll
 Author: Ying Shaodong
@@ -11,8 +11,8 @@ Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Provides-Extra: serial
-Requires-Dist: importlib-metadata (>=8.3.0,<9.0.0)
 Requires-Dist: paho-mqtt (>=2.1.0,<3.0.0)
 Requires-Dist: prettytable (>=3.9.0,<4.0.0)
 Requires-Dist: pymodbus (>=3.6.9,<3.7.0)
@@ -29,7 +29,7 @@ Description-Content-Type: text/markdown
 [![License](https://img.shields.io/github/license/gavinying/modpoll)](https://img.shields.io/github/license/gavinying/modpoll)
 [![Downloads](https://static.pepy.tech/badge/modpoll/week)](https://pepy.tech/project/modpoll)
 
-Show your love for this project by starring our repo 🌟, so we are encouraged to continue improving.
+> 📢 **Announcement:** Since the release of v1.3.0, our official dockerhub namespace has been changed to `topmaker`, you can pull the latest images from [topmaker/modpoll](https://hub.docker.com/repository/docker/topmaker/modpoll) at dockerhub.
 
 > Learn more about *modpoll* usage at [documentation](https://gavinying.github.io/modpoll) site.
 
@@ -124,7 +124,7 @@ modpoll --once \
 The reason we can magically poll data from the online device *modsim* is because we have already provided the [Modbus configure file](https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv) for *modsim* device as following,
 
 ```CSV
-device,modsim001,1,,
+device,modsim01,1,,
 poll,coil,0,16,BE_BE
 ref,coil01-08,0,bool8,rw
 ref,coil09-16,1,bool8,rw
@@ -183,7 +183,7 @@ The configuration can be either a local file or a remote public URL resource.
 If you are blocked by company firewall for online device or prefer a local test, you can launch your own device simulator by running *modsim* locally,
 
 ```bash
-docker run --rm -p 5020:5020 helloysd/modsim
+docker run --rm -p 5020:5020 topmaker/modsim
 ```
 
 It will create a virtual Modbus TCP device running at `localhost:5020`, and you can open a new terminal, poll the virtual device using *modpoll* tool,
@@ -198,7 +198,7 @@ modpoll \
 > Use `sudo` before the docker command if you want to use the standard port `502`.
 
 ```bash
-sudo docker run --rm -p 502:5020 helloysd/modsim
+sudo docker run --rm -p 502:5020 topmaker/modsim
 ```
 
 In a new terminal,
@@ -222,11 +222,17 @@ modpoll \
   --config https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv
 ```
 
-With successful data polling and publishing, you can subscribe the topic `modpoll/modsim` on the same MQTT broker `mqtt.eclipseprojects.io` to view the collected data.
+With successful data polling and publishing, you can subscribe the default data topic `modpoll/modsim01/data` on the same MQTT broker `mqtt.eclipseprojects.io` to view the collected data.
 
-The MQTT topic uses `<mqtt_topic_prefix>/<device_name>` pattern, <mqtt_topic_prefix> is provided by `--mqtt-topic-prefix` argument, the default value is `modpoll`  and <device_name> is provided by the Modbus configure file.
+The MQTT topics can be customized by providing the following arguments,
 
-> ⚠️ **Note:** The `--mqtt-topic-prefix` argument is deprecated and will be removed in the future release. Use `--mqtt-publish-topic-pattern` and `--mqtt-subscribe-topic-pattern` instead. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility. See [document](https://gavinying.github.io/modpoll/usage.html#Named%20Arguments) for details.
+  - `mqtt-publish-topic-pattern`
+  - `mqtt-subscribe-topic-pattern`
+  - `mqtt-diagnostics-topic-pattern`
+
+See [document](https://gavinying.github.io/modpoll/usage.html#Named%20Arguments) for details.
+
+> ⚠️ **Note:** The old `--mqtt-topic-prefix` argument is now deprecated and will be removed in the future release. Suggest to use `--mqtt-xxx-topic-pattern` in new project. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility until it is removed.
 
 
 <p align="center">
@@ -236,7 +242,7 @@ The MQTT topic uses `<mqtt_topic_prefix>/<device_name>` pattern, <mqtt_topic_pre
 
 ### Write registers via MQTT publish
 
-The *modpoll* tool will subscribe to the topic `modpoll/<device_name>/set` by default once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
+The *modpoll* tool will subscribe to the topic `modpoll/{{device_name}}/set` by default once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
 
 - To write a single holding register (address at `40001`)
 
@@ -264,7 +270,7 @@ The *modpoll* tool will subscribe to the topic `modpoll/<device_name>/set` by de
 A docker image has been provided for user to directly run the tool without local installation,
 
   ```bash
-  docker run --rm helloysd/modpoll
+  docker run --rm topmaker/modpoll
   ```
 
 It shows the version of the tool by default.
@@ -272,7 +278,7 @@ It shows the version of the tool by default.
 Similar to the above *modsim* test, we can poll data with `docker run`, in order to avoid printing out received data, the argument `--daemon` or `-d` is recommended to use with docker.
 
   ```bash
-  docker run --rm helloysd/modpoll \
+  docker run --rm topmaker/modpoll \
     modpoll -d \
       --tcp modsim.topmaker.net \
       --config https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv
@@ -282,14 +288,14 @@ If you want to load a local configure file, you need to mount a local folder ont
 for example, if the child folder `examples` contains the config file `modsim.csv`, we can use it via the following command,
 
   ```bash
-  docker run --rm -v $(pwd)/examples:/app/examples helloysd/modpoll \
+  docker run --rm -v $(pwd)/examples:/app/examples topmaker/modpoll \
     modpoll -d \
       --tcp modsim.topmaker.net \
       --config /app/examples/modsim.csv
   ```
 
 
-## Basic Usage
+## Example Use Cases
 
 - Connect to Modbus TCP device
 
@@ -313,7 +319,6 @@ for example, if the child folder `examples` contains the config file `modsim.csv
   ```bash
   modpoll \
     --tcp modsim.topmaker.net \
-    --tcp-port 5020 \
     --mqtt-host mqtt.eclipseprojects.io \
     --config examples/modsim.csv
   ```
@@ -323,14 +328,22 @@ for example, if the child folder `examples` contains the config file `modsim.csv
   ```bash
   modpoll \
     --tcp modsim.topmaker.net \
-    --tcp-port 5020 \
     --export data.csv \
     --config examples/modsim.csv
   ```
 
+- Connect to Modbus TCP devices using multiple config files
+
+  ```bash
+  modpoll \
+    --tcp modsim.topmaker.net \
+    --config examples/modsim.csv examples/modsim2.csv
+  ```
+
 
-> *Refer to the [documentation](https://gavinying.github.io/modpoll) site for more details about the configuration and examples.*
+> Refer to the [documentation](https://gavinying.github.io/modpoll) site for more details about the configuration and examples.
 
+> *If you find this tool useful, please support us by starring 🌟 our repository to encourage further improvements.*
 
 ## Credits
 

{modpoll-1.2.0 → modpoll-1.3.3}/README.md

@@ -5,7 +5,7 @@
 [![License](https://img.shields.io/github/license/gavinying/modpoll)](https://img.shields.io/github/license/gavinying/modpoll)
 [![Downloads](https://static.pepy.tech/badge/modpoll/week)](https://pepy.tech/project/modpoll)
 
-Show your love for this project by starring our repo 🌟, so we are encouraged to continue improving.
+> 📢 **Announcement:** Since the release of v1.3.0, our official dockerhub namespace has been changed to `topmaker`, you can pull the latest images from [topmaker/modpoll](https://hub.docker.com/repository/docker/topmaker/modpoll) at dockerhub.
 
 > Learn more about *modpoll* usage at [documentation](https://gavinying.github.io/modpoll) site.
 
@@ -100,7 +100,7 @@ modpoll --once \
 The reason we can magically poll data from the online device *modsim* is because we have already provided the [Modbus configure file](https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv) for *modsim* device as following,
 
 ```CSV
-device,modsim001,1,,
+device,modsim01,1,,
 poll,coil,0,16,BE_BE
 ref,coil01-08,0,bool8,rw
 ref,coil09-16,1,bool8,rw
@@ -159,7 +159,7 @@ The configuration can be either a local file or a remote public URL resource.
 If you are blocked by company firewall for online device or prefer a local test, you can launch your own device simulator by running *modsim* locally,
 
 ```bash
-docker run --rm -p 5020:5020 helloysd/modsim
+docker run --rm -p 5020:5020 topmaker/modsim
 ```
 
 It will create a virtual Modbus TCP device running at `localhost:5020`, and you can open a new terminal, poll the virtual device using *modpoll* tool,
@@ -174,7 +174,7 @@ modpoll \
 > Use `sudo` before the docker command if you want to use the standard port `502`.
 
 ```bash
-sudo docker run --rm -p 502:5020 helloysd/modsim
+sudo docker run --rm -p 502:5020 topmaker/modsim
 ```
 
 In a new terminal,
@@ -198,11 +198,17 @@ modpoll \
   --config https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv
 ```
 
-With successful data polling and publishing, you can subscribe the topic `modpoll/modsim` on the same MQTT broker `mqtt.eclipseprojects.io` to view the collected data.
+With successful data polling and publishing, you can subscribe the default data topic `modpoll/modsim01/data` on the same MQTT broker `mqtt.eclipseprojects.io` to view the collected data.
 
-The MQTT topic uses `<mqtt_topic_prefix>/<device_name>` pattern, <mqtt_topic_prefix> is provided by `--mqtt-topic-prefix` argument, the default value is `modpoll`  and <device_name> is provided by the Modbus configure file.
+The MQTT topics can be customized by providing the following arguments,
 
-> ⚠️ **Note:** The `--mqtt-topic-prefix` argument is deprecated and will be removed in the future release. Use `--mqtt-publish-topic-pattern` and `--mqtt-subscribe-topic-pattern` instead. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility. See [document](https://gavinying.github.io/modpoll/usage.html#Named%20Arguments) for details.
+  - `mqtt-publish-topic-pattern`
+  - `mqtt-subscribe-topic-pattern`
+  - `mqtt-diagnostics-topic-pattern`
+
+See [document](https://gavinying.github.io/modpoll/usage.html#Named%20Arguments) for details.
+
+> ⚠️ **Note:** The old `--mqtt-topic-prefix` argument is now deprecated and will be removed in the future release. Suggest to use `--mqtt-xxx-topic-pattern` in new project. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility until it is removed.
 
 
 <p align="center">
@@ -212,7 +218,7 @@ The MQTT topic uses `<mqtt_topic_prefix>/<device_name>` pattern, <mqtt_topic_pre
 
 ### Write registers via MQTT publish
 
-The *modpoll* tool will subscribe to the topic `modpoll/<device_name>/set` by default once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
+The *modpoll* tool will subscribe to the topic `modpoll/{{device_name}}/set` by default once it successfully connected to MQTT broker, user can write device register(s) via MQTT publish,
 
 - To write a single holding register (address at `40001`)
 
@@ -240,7 +246,7 @@ The *modpoll* tool will subscribe to the topic `modpoll/<device_name>/set` by de
 A docker image has been provided for user to directly run the tool without local installation,
 
   ```bash
-  docker run --rm helloysd/modpoll
+  docker run --rm topmaker/modpoll
   ```
 
 It shows the version of the tool by default.
@@ -248,7 +254,7 @@ It shows the version of the tool by default.
 Similar to the above *modsim* test, we can poll data with `docker run`, in order to avoid printing out received data, the argument `--daemon` or `-d` is recommended to use with docker.
 
   ```bash
-  docker run --rm helloysd/modpoll \
+  docker run --rm topmaker/modpoll \
     modpoll -d \
       --tcp modsim.topmaker.net \
       --config https://raw.githubusercontent.com/gavinying/modpoll/master/examples/modsim.csv
@@ -258,14 +264,14 @@ If you want to load a local configure file, you need to mount a local folder ont
 for example, if the child folder `examples` contains the config file `modsim.csv`, we can use it via the following command,
 
   ```bash
-  docker run --rm -v $(pwd)/examples:/app/examples helloysd/modpoll \
+  docker run --rm -v $(pwd)/examples:/app/examples topmaker/modpoll \
     modpoll -d \
       --tcp modsim.topmaker.net \
       --config /app/examples/modsim.csv
   ```
 
 
-## Basic Usage
+## Example Use Cases
 
 - Connect to Modbus TCP device
 
@@ -289,7 +295,6 @@ for example, if the child folder `examples` contains the config file `modsim.csv
   ```bash
   modpoll \
     --tcp modsim.topmaker.net \
-    --tcp-port 5020 \
     --mqtt-host mqtt.eclipseprojects.io \
     --config examples/modsim.csv
   ```
@@ -299,14 +304,22 @@ for example, if the child folder `examples` contains the config file `modsim.csv
   ```bash
   modpoll \
     --tcp modsim.topmaker.net \
-    --tcp-port 5020 \
     --export data.csv \
     --config examples/modsim.csv
   ```
 
+- Connect to Modbus TCP devices using multiple config files
+
+  ```bash
+  modpoll \
+    --tcp modsim.topmaker.net \
+    --config examples/modsim.csv examples/modsim2.csv
+  ```
+
 
-> *Refer to the [documentation](https://gavinying.github.io/modpoll) site for more details about the configuration and examples.*
+> Refer to the [documentation](https://gavinying.github.io/modpoll) site for more details about the configuration and examples.
 
+> *If you find this tool useful, please support us by starring 🌟 our repository to encourage further improvements.*
 
 ## Credits
 

modpoll-1.3.3/modpoll/__init__.py

@@ -0,0 +1,3 @@
+import importlib.metadata as importlib_metadata
+
+__version__ = importlib_metadata.version(__name__)

{modpoll-1.2.0 → modpoll-1.3.3}/modpoll/arg_parser.py

@@ -5,19 +5,20 @@ from . import __version__
 
 def get_parser():
     parser = argparse.ArgumentParser(
-        description=f"modpoll v{__version__} - A New Command-line Tool for Modbus and MQTT"
+        description=f"Modpoll v{__version__} - A New Command-line Tool for Modbus and MQTT"
     )
     parser.add_argument(
         "-v",
         "--version",
         action="version",
-        version=f"modpoll v{__version__}",
+        version=f"v{__version__}",
     )
     parser.add_argument(
         "-f",
         "--config",
         required=True,
         help="A local path or URL of Modbus configuration file. Required!",
+        nargs="+",
     )
     parser.add_argument(
         "-d",
@@ -107,18 +108,18 @@ def get_parser():
     )
     parser.add_argument(
         "--mqtt-publish-topic-pattern",
-        default="modpoll/<device_name>",
-        help='Topic pattern for MQTT publish. Use <device_name> as placeholder for the device names in Modbus config. Defaults to "modpoll/<device_name>"',
+        default="modpoll/{{device_name}}/data",
+        help='Topic pattern for MQTT publish. Use {{device_name}} as placeholder for the device names in Modbus config. Defaults to "modpoll/{{device_name}}/data"',
     )
     parser.add_argument(
         "--mqtt-subscribe-topic-pattern",
-        default="modpoll/<device_name>/set",
-        help='Topic pattern for MQTT subscribe. Use <device_name> as placeholder for the device names in Modbus config. Defaults to "modpoll/<device_name>/set"',
+        default="modpoll/+/set",
+        help='Topic pattern for MQTT subscribe. Defaults to "modpoll/+/set"',
     )
     parser.add_argument(
         "--mqtt-diagnostics-topic-pattern",
-        default=None,
-        help="Topic pattern for MQTT diagnostics. Use <device_name> as placeholder for the device names in Modbus config. Defaults to None",
+        default="modpoll/{{device_name}}/diagnostics",
+        help="Topic pattern for MQTT diagnostics. Use {{device_name}} as placeholder for the device names in Modbus config. Defaults to modpoll/{{device_name}}/diagnostics",
     )
     parser.add_argument(
         "--mqtt-qos",

modpoll-1.3.3/modpoll/main.py

@@ -0,0 +1,199 @@
+import json
+import logging
+import re
+import signal
+import sys
+
+from .arg_parser import get_parser
+from .mqtt_task import MqttHandler
+from .modbus_task import setup_modbus_handlers
+
+from . import __version__
+from .utils import set_threading_event, delay_thread, on_threading_event, get_utc_time
+
+
+LOG_SIMPLE = "%(asctime)s | %(levelname).1s | %(name)s | %(message)s"
+logger = None
+
+
+def _signal_handler(signal, frame):
+    logger.info(f"Exiting {sys.argv[0]}")
+    set_threading_event()
+
+
+def setup_logging(level, format):
+    logging.basicConfig(level=level, format=format)
+
+
+def app(name="modpoll"):
+    mqtt_handler = None
+    modbus_handlers = []
+
+    print(
+        f"\nModpoll v{__version__} - A New Command-line Tool for Modbus and MQTT\n",
+        flush=True,
+    )
+
+    signal.signal(signal.SIGINT, _signal_handler)
+
+    # parse args
+    args = get_parser().parse_args()
+
+    # get logger
+    setup_logging(args.loglevel, LOG_SIMPLE)
+    global logger
+    logger = logging.getLogger(__name__)
+
+    # Overwrite topic pattern if topic prefix is specified for backward compatibility
+    if args.mqtt_topic_prefix:
+        logger.warning(
+            "(DEPRECATED): The `--mqtt-topic-prefix` argument is deprecated and will be removed in the future release. Use `--mqtt-publish-topic-pattern` and `--mqtt-subscribe-topic-pattern` instead. If both are used, `--mqtt-topic-prefix` argument will take precedence in order to keep backward compatibility."
+        )
+        if args.mqtt_topic_prefix.endswith("/"):
+            args.mqtt_topic_prefix = args.mqtt_topic_prefix[:-1]
+        args.mqtt_publish_topic_pattern = f"{args.mqtt_topic_prefix}/{{device_name}}"
+        args.mqtt_subscribe_topic_pattern = f"{args.mqtt_topic_prefix}/#/set"
+
+    # setup mqtt
+    if not args.mqtt_host:
+        logger.info("No MQTT host specified, skip MQTT setup.")
+    else:
+        logger.info(f"Setup MQTT connection to {args.mqtt_host}:{args.mqtt_port}")
+        try:
+            mqtt_handler = MqttHandler(
+                "MqttHandler",
+                args.mqtt_host,
+                args.mqtt_port,
+                args.mqtt_user,
+                args.mqtt_pass,
+                args.mqtt_clientid,
+                args.mqtt_qos,
+                subscribe_topics=[args.mqtt_subscribe_topic_pattern],
+                log_level=args.loglevel,
+            )
+            if mqtt_handler.setup() and mqtt_handler.connect():
+                logger.info("Connected to MQTT broker.")
+            else:
+                logger.error("Failed to connect with MQTT broker, exiting...")
+                mqtt_handler.close()
+                exit(1)
+        except Exception as e:
+            logger.error(f"Error setting up MQTT input: {e}, exiting...")
+            mqtt_handler.close()
+            exit(1)
+
+    # setup modbus tasks
+    modbus_handlers = setup_modbus_handlers(args, mqtt_handler)
+    if modbus_handlers:
+        logger.info(f"Loaded {len(modbus_handlers)} Modbus config(s).")
+        delay_thread(args.delay)
+    else:
+        logger.error("No Modbus config(s) defined. Exiting...")
+        if mqtt_handler:
+            mqtt_handler.close()
+        exit(1)
+
+    # main loop
+    last_check = 0
+    last_diag = 0
+    while not on_threading_event():
+        now = get_utc_time()
+        # routine check
+        if now > last_check + args.rate:
+            if last_check == 0:
+                elapsed = args.rate
+            else:
+                elapsed = round(now - last_check, 6)
+            last_check = now
+            logger.info(
+                f" === Modpoll is polling at rate:{args.rate}s, actual:{elapsed}s ==="
+            )
+            for modbus_handler in modbus_handlers:
+                modbus_handler.poll()
+                if on_threading_event():
+                    break
+                if args.mqtt_host:
+                    if args.timestamp:
+                        modbus_handler.publish_data(timestamp=now)
+                    else:
+                        modbus_handler.publish_data()
+                if args.export:
+                    if args.timestamp:
+                        modbus_handler.export(args.export, timestamp=now)
+                    else:
+                        modbus_handler.export(args.export)
+        if args.diagnostics_rate > 0 and now > last_diag + args.diagnostics_rate:
+            last_diag = now
+            for modbus_handler in modbus_handlers:
+                modbus_handler.publish_diagnostics()
+        if on_threading_event():
+            break
+        # Check if receive mqtt request
+        if mqtt_handler:
+            topic, payload = mqtt_handler.receive()
+            if topic and payload:
+                # extract device_name
+                topic_regex = args.mqtt_subscribe_topic_pattern.replace(
+                    "+", "([^/\n]*)"
+                )
+                match = re.search(topic_regex, topic)
+                if match:
+                    device_name = match.group(1)
+                    logger.info(
+                        f"Received request to write data for device {device_name}"
+                    )
+                    try:
+                        reg = json.loads(payload)
+                        object_type = reg["object_type"]
+                        address = reg["address"]
+                        value = reg["value"]
+
+                        device_found = False
+                        for modbus_handler in modbus_handlers:
+                            if device_name in [
+                                dev.name for dev in modbus_handler.get_device_list()
+                            ]:
+                                device_found = True
+                                write_success = False
+                                if object_type == "coil":
+                                    write_success = modbus_handler.write_coil(
+                                        device_name, address, value
+                                    )
+                                elif object_type == "holding_register":
+                                    write_success = modbus_handler.write_register(
+                                        device_name, address, value
+                                    )
+
+                                if write_success:
+                                    logger.info(
+                                        f"Successfully wrote {object_type}: device={device_name}, address={address}, value={value}"
+                                    )
+                                else:
+                                    logger.warning(
+                                        f"Failed to write {object_type}: device={device_name}, address={address}, value={value}"
+                                    )
+                                break
+
+                        if not device_found:
+                            logger.error(f"No device found with name: {device_name}")
+
+                    except KeyError as e:
+                        logger.error(f"Missing required key in payload: {e}")
+                    except json.JSONDecodeError:
+                        logger.error(f"Failed to parse JSON message: {payload}")
+                else:
+                    logger.error(f"Failed to extract device name from topic: {topic}")
+        if args.once:
+            set_threading_event()
+            break
+
+        delay_thread(0.01)
+
+    for modbus_handler in modbus_handlers:
+        modbus_handler.close()
+    if mqtt_handler:
+        mqtt_handler.close()
+
+
+if __name__ == "__main__":
+    app()