@bitpoolos/edge-bacnet 1.6.2 → 1.6.4

package/CHANGELOG.md

@@ -1,39 +1,80 @@
 # Changelog
 
+## [1.6.4] - 02-09-2025
+
+Minor feature:
+
+- Small update to file read / write logic for database files. Some deployments require a different storage location outside of the default. If a nodejs environment variable of BACNET_STORAGE_PATH is defined and set for that deployment, it will prefix that location to the file path, otherwise functionality will remain the same.
+
+Bug fix:
+
+- Some mstp devices were not showing in device list UI tree.
+
+## [1.6.3] - 01-06-2025
+
+Minor feature:
+
+- Device List - new right click option to MSTP NET folders - Update All Devices. Specifically updates the mstp device listed in that selected network
+- Inspector - Added statistic percentages as MQTT output in the Inspector node
+- Inspector - Added online / offline stats and Total points to read as status on node
+- Test Functions - Outputs results to both node-red console and node-red debug window now
+
+Minor update / refactor:
+
+- Datamodel - Importing and Exporting datamodel displays process status and related stats (file size etc) while importing or exporting, informing the user of current state
+- Device List - UI tree no longer stored in datamodel, as it is generated dynamically on a schedule. This significantly reduces datamodel file size, read / write time, system start up processes.
+- Datamodel - writes to file system for persistance and backup was being executed more often than needed. Process schedule is now on a larger interval, and write algorithm refactored and optimized.
+- Device List - right click -> Set Point Name feature refactored, due to many scenarios where it wasnt executing as expected.
+- BACnet read output - error and status field setting optimized
+- Inspector - updated ObjectType column values to show object type enum string instead of integer
+
+Bug fixes:
+
+- Inspector - incorrect percentages, statistics and values in the statistics bar. Fixed and tested to represent site status more accurately
+- Inpsector - not flagging offline points in error statistic and table filter. Now correctly identifies "offline" points as an Error type
+- BACnet read output - was not updating error and status correctly for full object payloads.
+- BACnet Server - was not outputting sucessfull write update MQTT msg after node-red deploy.
+
+Updating:
+
+- There shouldnt be a need for users to remove nodes or do any specific action when updating to this version, however backing up a datamodel export is advised.
+
 ## [1.6.2] - 07-05-2025
 
-Minor feature: 
-  - Added "Enable device discovery" check box to gateway settings, discovery tab. 
-    - This check box controls whether on not the auto point discovery and property discovery is enabled. 
-    - This can be used to turn off unecessary network traffic once you have discovered all of the desired devices and points.
-    - IMPORTANT - if you are updating a existing deployment, the new property will be unticked, however new installs / dragging from the pallete will by default have the option checked. So if you want to keep this enabled on an existing deployment, please check the setting. 
-    - Note - This does not turn off the whoIs task schedule.
-    - A user can use Right Click -> Update points on desired deviced in the read node tree if you would like to manually discover devices. 
+Minor feature:
+
+- Added "Enable device discovery" check box to gateway settings, discovery tab.
+  - This check box controls whether on not the auto point discovery and property discovery is enabled.
+  - This can be used to turn off unecessary network traffic once you have discovered all of the desired devices and points.
+  - IMPORTANT - if you are updating a existing deployment, the new property will be unticked, however new installs / dragging from the pallete will by default have the option checked. So if you want to keep this enabled on an existing deployment, please check the setting.
+  - Note - This does not turn off the whoIs task schedule.
+  - A user can use Right Click -> Update points on desired deviced in the read node tree if you would like to manually discover devices.
 
-Minor update: 
-  - Adjusted initial whoIs broadcast delay from 5seconds to 15seconds after node-red is started with a deployed gateway. This is to ensure large cached files are completely read and loaded.
+Minor update:
 
-Bug fixes: 
-  - Inspector:
-    - Table resized to avoid scroll bars
-    - Percentage rounding to nearest 2 decimal places rather than whole integer for more detailed data. 
-    - Loading animation added
-    - NAN years ago - removed as invalid time differential 
-    - Last seen for device points adjusted to be more accurate
+- Adjusted initial whoIs broadcast delay from 5seconds to 15seconds after node-red is started with a deployed gateway. This is to ensure large cached files are completely read and loaded.
 
-  
+Bug fixes:
+
+- Inspector:
+  - Table resized to avoid scroll bars
+  - Percentage rounding to nearest 2 decimal places rather than whole integer for more detailed data.
+  - Loading animation added
+  - NAN years ago - removed as invalid time differential
+  - Last seen for device points adjusted to be more accurate
 
 ## [1.6.1] - 14-04-2025
 
-Bug fixes: 
-  - Inspector stats were calculating incorrectly in certain scenarios
-  - Inspector downloaded HTML files had incorrect stat percentages
-  - Inspector Last_Polled_Time stat was always "UNKNOWN", now shows correct date time.
+Bug fixes:
+
+- Inspector stats were calculating incorrectly in certain scenarios
+- Inspector downloaded HTML files had incorrect stat percentages
+- Inspector Last_Polled_Time stat was always "UNKNOWN", now shows correct date time.
 
-Minor updates: 
-  - Inspector BACnet main stats now output on msg.type = getBacnetStats inject. Can be set on a scheduled inject. 
-  - Updated Examples with Inspector
+Minor updates:
 
+- Inspector BACnet main stats now output on msg.type = getBacnetStats inject. Can be set on a scheduled inject.
+- Updated Examples with Inspector
 
 ## [1.6.0] - 09-04-2025
 
@@ -49,93 +90,99 @@ New features:
   - API routes available:
 
     - /inspector
+
       - view UI in web browser
 
     - /inspector-downloadhtml
+
       - downloads the html of the inspector in its current state
 
     - /inspector-downloadhtml?filter=tableKey&value=value1,value2
-      - downloads the html of the inspector, but with a applied filter to the table. 
-      - tableKey in the above example can be any of the table columns, only 1 tableKey may be filtered on: 
+
+      - downloads the html of the inspector, but with a applied filter to the table.
+      - tableKey in the above example can be any of the table columns, only 1 tableKey may be filtered on:
         deviceID, objectType, objectInstance, presentValue, dataModelStatus, pointName, discoveredBACnetPointName, displayName, deviceName, ipAddress, area, key, topic, lastSeen, error
       - value=value1,value2 etc can be any value that the tableKey can contain. This parameter can accept many comma separated values
-      - an example filter request may look like: 
+      - an example filter request may look like:
         /inspector-downloadhtml?filter=dataModelStatus&value=error,missing
 
     - /getModelStats
+
       - returns JSON data with analysis of the BACnet model
       - contains point status, metrics, and detailed information
 
     - /pointstoread
+
       - downloads CSV file with all points in the read list
       - format: [siteName]_PointsToRead_[timestamp].csv
 
     - /getpointerrors
       - downloads CSV file with all points that have errors
       - format: [siteName]_PointErrors_[timestamp].csv
-      
     - /getmodelstatscsv
       - downloads CSV file with all model stats data in CSV format
       - format: [siteName]_ModelStats_[timestamp].csv
-      
     - /publishedpointslist
+
       - downloads CSV file with all published points and their current values
       - format: [siteName]_PublishedPointsList_[timestamp].csv
       - outputs mqtt topic and payloads with statistics about the current state of the bacnet network
       - output topics:
-          EDGE_DEVICE_{IP_ID}/STATUS/LAST_POINT_PUSHED_TIME
-          payload: Timestamp of when the last point was pushed (ISO string)
-          EDGE_DEVICE_{IP_ID}/STATUS/LAST_STAT_CALC_TIME
-          payload: Current timestamp (ISO string)
-          EDGE_DEVICE_{IP_ID}/STATUS/UPTIME
-          payload: System uptime formatted as string (e.g., "Uptime: 3 days, 5 hours, 12 minutes, 45 seconds")
-          EDGE_DEVICE_{IP_ID}/STATUS/ONLINE_POINTS
-          payload: Number of online points
-          EDGE_DEVICE_{IP_ID}/STATUS/OFFLINE_POINTS
-          payload: Number of offline points
-          EDGE_DEVICE_{IP_ID}/STATUS/TOTAL_POLLED_POINTS
-          payload: Total number of polled points
-          EDGE_DEVICE_{IP_ID}/STATUS/AVERAGE_TIME_SINCE_COV_IN_SECONDS
-          payload: Average time since last change of value in seconds
-          EDGE_DEVICE_{IP_ID}/STATUS/TOTAL_POINTS_TO_READ
-          payload: Total number of points to read
-          EDGE_DEVICE_{IP_ID}/STATUS/DISCOVERED_POINT_COUNT
-          payload: Number of discovered points
-          EDGE_DEVICE_{IP_ID}/STATUS/DISCOVERED_DEVICE_COUNT
-          payload: Number of discovered devices
-
-          where {IP_ID} is the IP address of the device with periods removed (e.g., 192.168.1.100 becomes 192168110).
-          each of these topics includes the site name as a tag in the message metadata with format geoAddr={siteName}.
-
-  - Additional input options: 
+        EDGE*DEVICE*{IP*ID}/STATUS/LAST_POINT_PUSHED_TIME
+        payload: Timestamp of when the last point was pushed (ISO string)
+        EDGE_DEVICE*{IP*ID}/STATUS/LAST_STAT_CALC_TIME
+        payload: Current timestamp (ISO string)
+        EDGE_DEVICE*{IP*ID}/STATUS/UPTIME
+        payload: System uptime formatted as string (e.g., "Uptime: 3 days, 5 hours, 12 minutes, 45 seconds")
+        EDGE_DEVICE*{IP*ID}/STATUS/ONLINE_POINTS
+        payload: Number of online points
+        EDGE_DEVICE*{IP*ID}/STATUS/OFFLINE_POINTS
+        payload: Number of offline points
+        EDGE_DEVICE*{IP*ID}/STATUS/TOTAL_POLLED_POINTS
+        payload: Total number of polled points
+        EDGE_DEVICE*{IP*ID}/STATUS/AVERAGE_TIME_SINCE_COV_IN_SECONDS
+        payload: Average time since last change of value in seconds
+        EDGE_DEVICE*{IP*ID}/STATUS/TOTAL_POINTS_TO_READ
+        payload: Total number of points to read
+        EDGE_DEVICE*{IP*ID}/STATUS/DISCOVERED_POINT_COUNT
+        payload: Number of discovered points
+        EDGE_DEVICE*{IP_ID}/STATUS/DISCOVERED_DEVICE_COUNT
+        payload: Number of discovered devices
+
+        where {IP_ID} is the IP address of the device with periods removed (e.g., 192.168.1.100 becomes 192168110).
+        each of these topics includes the site name as a tag in the message metadata with format geoAddr={siteName}.
+
+  - Additional input options:
+
     - reset - resets the complete data model used for all of the inspector analytics
+
       - msg input format: msg.reset = true
 
     - sendMqttStats - outputs additional mqtt statistics
-      - msg input format:  msg.type = sendMqttStats
-      - output topics: 
-        EDGE_DEVICE_{siteName}/BACNETSTATS/ok
+
+      - msg input format: msg.type = sendMqttStats
+      - output topics:
+        EDGE*DEVICE*{siteName}/BACNETSTATS/ok
         payload: Number of points with OK status
-        EDGE_DEVICE_{siteName}/BACNETSTATS/error
+        EDGE*DEVICE*{siteName}/BACNETSTATS/error
         payload: Number of points with error status
-        EDGE_DEVICE_{siteName}/BACNETSTATS/missing
+        EDGE*DEVICE*{siteName}/BACNETSTATS/missing
         payload: Number of missing points
-        EDGE_DEVICE_{siteName}/BACNETSTATS/warnings
+        EDGE*DEVICE*{siteName}/BACNETSTATS/warnings
         payload: Number of points with warnings
-        EDGE_DEVICE_{siteName}/BACNETSTATS/moved
+        EDGE*DEVICE*{siteName}/BACNETSTATS/moved
         payload: Number of points that have moved (e.g changed object instance)
-        EDGE_DEVICE_{siteName}/BACNETSTATS/deviceIdChange
+        EDGE*DEVICE*{siteName}/BACNETSTATS/deviceIdChange
         payload: Number of points with changed device IDs
-        EDGE_DEVICE_{siteName}/BACNETSTATS/deviceIdConflict
+        EDGE*DEVICE*{siteName}/BACNETSTATS/deviceIdConflict
         payload: Number of points with conflicting device IDs
-        EDGE_DEVICE_{siteName}/BACNETSTATS/unmapped
+        EDGE*DEVICE*{siteName}/BACNETSTATS/unmapped
         payload: Number of unmapped points
-        EDGE_DEVICE_{siteName}/BACNETSTATS/offlinePercentage
+        EDGE*DEVICE*{siteName}/BACNETSTATS/offlinePercentage
         payload: Percentage of points that are offline
 
         where {siteName} is the site name configured in the inspector node.
 
-
 - Right click -> Update Point on a individual point in the device tree. (Read node UI)
 
 - Added programmatic reinitialize/clear points on BACnet server via injecting { msg.reinitializeBacnetServer: true } into the gateway node. Optionally can include a msg.responseTopic string to get a confrimation published to that topic output from the gateway node on sucessfull reinitialize.
@@ -158,6 +205,7 @@ New features:
 Refactor:
 
 - Reading and Writing to the cache file of the datamodel.
+
   - write operations are now locked to 1 operation at a time
   - a rolling secondary backup file is now created, which can be used in case of corruption of the primary file
 

package/bacnet_client.js

@@ -11,7 +11,6 @@ const {
   parseBacnetError,
   getBacnetErrorString,
   Read_Config_Async,
-  Read_Config_Sync,
   isNumber,
   decodeBitArray,
 } = require("./common");
@@ -28,6 +27,7 @@ class BacnetClient extends EventEmitter {
     that.config = config;
     that.deviceList = [];
     that.networkTree = {};
+    that.renderList = [];
     that.lastWhoIs = null;
     that.client = null;
     that.lastNetworkPoll = null;
@@ -41,7 +41,6 @@ class BacnetClient extends EventEmitter {
     that.portRangeMatrix = config.portRangeMatrix;
 
     try {
-
       that.roundDecimal = config.roundDecimal;
       that.apduSize = config.apduSize;
       that.maxSegments = config.maxSegments;
@@ -116,13 +115,11 @@ class BacnetClient extends EventEmitter {
             }
           }, "4000");
         }, "15000");
-
       } catch (e) {
         that.logOut("Issue initializing client: ", e);
       }
 
       try {
-
         //who is callback
         that.client.on("iAm", (device) => {
           if (device.address !== that.config.localIpAdrress) {
@@ -201,7 +198,8 @@ class BacnetClient extends EventEmitter {
       const cachedData = await Read_Config_Async();
       const parsedData = JSON.parse(cachedData);
       if (parsedData && typeof parsedData == "object") {
-        if (parsedData.renderList) that.renderList = parsedData.renderList;
+        // renderList is no longer cached - will be rebuilt by tree builder
+        // if (parsedData.renderList) that.renderList = parsedData.renderList;
         if (parsedData.deviceList) {
           parsedData.deviceList.forEach(function (device) {
             let newBacnetDevice = new BacnetDevice(true, device);
@@ -209,12 +207,13 @@ class BacnetClient extends EventEmitter {
           });
         }
         if (parsedData.pointList) that.networkTree = parsedData.pointList;
-        if (parsedData.renderListCount) that.renderListCount = parsedData.renderListCount;
+        // renderListCount is no longer cached - will be recalculated by tree builder
+        // if (parsedData.renderListCount) that.renderListCount = parsedData.renderListCount;
       }
     }
   }
 
-  testFunction(address, port, type, instance, property) {
+  testFunction(address, port, type, instance, property, nodeWarnCallback) {
     let that = this;
     console.log("test function ");
 
@@ -229,6 +228,11 @@ class BacnetClient extends EventEmitter {
       console.log("1 - readPropertyMultiple:  ");
 
       console.log(value);
+
+      if (nodeWarnCallback) {
+        nodeWarnCallback(value);
+      }
+
       if (value) {
         // If the result has value, resolve the promise
         console.log(value.values[0]);
@@ -720,35 +724,30 @@ class BacnetClient extends EventEmitter {
     const that = this;
     const roundDecimal = readConfig.precision;
     const devicesToRead = Object.keys(readConfig.pointsToRead);
-    const bacnetResults = {};
-    let pendingRequests = 0;
+    let completedDevices = 0;
 
     try {
-      // Process all devices in sequence
-      for (let deviceIndex = 0; deviceIndex < devicesToRead.length; deviceIndex++) {
-        const key = devicesToRead[deviceIndex];
+      // Create array of device processing promises
+      const devicePromises = devicesToRead.map(async (key, deviceIndex) => {
         const device = that.findDeviceByKey(key);
-        if (!device) continue;
+        if (!device) return null;
 
         const deviceName = that.computeDeviceName(device);
         const deviceKey = that.createDeviceKey(device);
         const deviceObject = that.networkTree[deviceKey];
         const maxObjectCount = that.estimateMaxObjectSize(device.getMaxApdu());
 
-        if (!bacnetResults[deviceName]) {
-          bacnetResults[deviceName] = {};
-        }
+        const bacnetResults = {};
+        bacnetResults[deviceName] = {};
 
         // Process points for the current device
         const pointsToRead = readConfig.pointsToRead[key];
         const pointNames = Object.keys(pointsToRead);
         let totalPoints = pointNames.length - 1;
         let requestArray = [];
-        let processedPoints = 0; // Counter for processed points
 
         // Process each point for the device in batches
         for (let i = 0; i < pointNames.length; i++) {
-
           const pointName = pointNames[i];
           if (pointName === "deviceName") {
             continue;
@@ -771,17 +770,7 @@ class BacnetClient extends EventEmitter {
           }
 
           // Process the batch when the request array is full or the last point is reached
-          if (requestArray.length === maxObjectCount) {
-            if (device.getProtocolServiceSupport("ReadPropertyMultiple") == true) {
-              await that.processBatch(device, requestArray, deviceName, bacnetResults, that, roundDecimal);
-            } else {
-              await that.processIndividualPoints(device, requestArray, deviceName, bacnetResults, that, roundDecimal);
-            }
-
-            requestArray = [];
-            // Increment the processed points counter
-            processedPoints += maxObjectCount;
-          } else if (i === pointNames.length - 1) {
+          if (requestArray.length === maxObjectCount || i === pointNames.length - 1) {
             if (device.getProtocolServiceSupport("ReadPropertyMultiple") == true) {
               await that.processBatch(device, requestArray, deviceName, bacnetResults, that, roundDecimal);
             } else {
@@ -789,28 +778,34 @@ class BacnetClient extends EventEmitter {
             }
 
             requestArray = [];
-            // Increment the processed points counter
-            processedPoints += i;
           }
+        }
 
-          // Check if all points for the device have been processed
-          if (processedPoints >= totalPoints) {
-            pendingRequests++;
-
-            // Emit the `values` event for the current device
-            that.emit(
-              "values",
-              bacnetResults,
-              outputType,
-              objectPropertyType,
-              readNodeName,
-              pendingRequests,
-              devicesToRead.length
-            );
-            delete bacnetResults[deviceName];
-          }
+        // Return results for this device
+        return {
+          deviceName,
+          results: bacnetResults,
+          deviceIndex: deviceIndex + 1,
+          totalDevices: devicesToRead.length,
+        };
+      });
+
+      // Process all devices in parallel and emit results as they complete
+      const results = await Promise.allSettled(devicePromises);
+
+      results.forEach((result, index) => {
+        if (result.status === "fulfilled" && result.value) {
+          completedDevices++;
+          const { deviceName, results: bacnetResults, deviceIndex, totalDevices } = result.value;
+
+          // Emit the `values` event for this device immediately
+          that.emit("values", bacnetResults, outputType, objectPropertyType, readNodeName, completedDevices, totalDevices);
+        } else {
+          // Handle failed device (offline/error)
+          completedDevices++;
+          that.logOut(`Device ${devicesToRead[index]} failed:`, result.reason);
         }
-      }
+      });
     } catch (error) {
       that.logOut("doRead error: ", error);
     }
@@ -847,6 +842,7 @@ class BacnetClient extends EventEmitter {
           if (isNumber(val)) {
             pointRef.presentValue = roundDecimalPlaces(val, roundDecimal);
             pointRef.error = "none";
+            pointRef.status = "online";
             if (pointRef.meta.objectId.type == 19 || pointRef.meta.objectId.type == 13 || pointRef.meta.objectId.type == 14) {
               if (pointRef.stateTextArray && typeof pointRef.stateTextArray[0].value !== "object") {
                 if (val != 0) {
@@ -860,14 +856,18 @@ class BacnetClient extends EventEmitter {
             if (typeof val !== "object") {
               pointRef.presentValue = val;
               pointRef.error = "none";
+              pointRef.status = "online";
             } else if (val.errorClass && val.errorClass) {
               pointRef.error = getBacnetErrorString(val.errorClass, val.errorClass);
+              pointRef.status = "offline";
+            } else {
+              pointRef.error = "none";
+              pointRef.status = "online";
             }
           }
 
           pointRef.meta["device"] = deviceMetaInfo;
           pointRef.timestamp = Date.now();
-          pointRef.status = "online";
 
           // Store the point data in results
           bacnetResults[deviceName][pointNameRef] = pointRef;
@@ -891,7 +891,6 @@ class BacnetClient extends EventEmitter {
     for (const request of requestArray) {
       const { objectId, pointRef, pointName } = request;
       try {
-
         const result = await that.updatePoint(device, pointRef);
 
         if (result.objectId.type == objectId.type && result.objectId.instance == objectId.instance) {
@@ -899,6 +898,8 @@ class BacnetClient extends EventEmitter {
 
           if (isNumber(val)) {
             pointRef.presentValue = roundDecimalPlaces(val, roundDecimal);
+            pointRef.error = "none";
+            pointRef.status = "online";
 
             if (pointRef.meta.objectId.type == 19 || pointRef.meta.objectId.type == 13 || pointRef.meta.objectId.type == 14) {
               if (pointRef.stateTextArray && typeof pointRef.stateTextArray[0].value !== "object") {
@@ -910,13 +911,21 @@ class BacnetClient extends EventEmitter {
               }
             }
           } else {
-            pointRef.presentValue = val;
+            if (typeof val !== "object") {
+              pointRef.presentValue = val;
+              pointRef.error = "none";
+              pointRef.status = "online";
+            } else if (val.errorClass && val.errorClass) {
+              pointRef.error = getBacnetErrorString(val.errorClass, val.errorClass);
+              pointRef.status = "offline";
+            } else {
+              pointRef.error = "none";
+              pointRef.status = "online";
+            }
           }
 
           pointRef.meta["device"] = deviceMetaInfo;
           pointRef.timestamp = Date.now();
-          pointRef.status = "online";
-          pointRef.error = "none";
 
           // Store the point data in results
           bacnetResults[deviceName][pointName] = pointRef;
@@ -981,7 +990,7 @@ class BacnetClient extends EventEmitter {
 
       return true;
     } catch (e) {
-      throw e
+      throw e;
     }
   }
 
@@ -1007,7 +1016,7 @@ class BacnetClient extends EventEmitter {
     let settings = {
       maxSegments: maxSegments,
       maxApdu: maxApdu,
-    }
+    };
 
     return new Promise((resolve, reject) => {
       that.client.readProperty(
@@ -1681,7 +1690,7 @@ class BacnetClient extends EventEmitter {
         if (json.body.renderListCount) {
           that.renderListCount = json.body.renderListCount;
         }
-        resolve();
+        resolve(true);
       } catch (e) {
         reject(e);
       }