iobroker.esphome 0.3.2 → 0.4.0

package/CHANGELOG_OLD.md

@@ -1,4 +1,8 @@
 # Older changes
+## 0.2.3 (2021-06-29)
+* (Jey-Cee) Bugfix : Light component state not changed [#74](https://github.com/DrozmotiX/ioBroker.esphome/issues/74)
+* (DutchmanNL) Update compatibility to version 1.19.4 of ESPHome Dashboard
+
 ## 0.2.2 (2021-06-24)
 * (DutchmanNL) [!!! Breaking !!!] Make YAML file persistent, backup your configuration before updating ! solves [#57](https://github.com/DrozmotiX/ioBroker.esphome/issues/57)
 * (DutchmanNL) Update ESPHome Dashboard to 1.18.0, requires  Python >=3.7 (and ensure <4.0!)

package/README.md

@@ -112,29 +112,30 @@ If you like my work, please consider a personal donation
     ### __WORK IN PROGRESS__
     * (DutchmanNL) 
 -->
+### 0.4.0 (2023-11-03)
+* (DutchmanNL) Added cleanup capability for unused channels & states after initialisation of device, resolves #39
+* (DutchmanNL) Added button to info channel which allows to delete all offline devices from adapter tree. resolves #39
+* (DutchmanNL) [Breaking] Backup strategy changed, requires [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) and activate option for ESPHome, fixes #129
+
 ### 0.3.2 (2023-11-01)
 * (DutchmanNL) Improved error handling if devices are not reachable/disconnected
-* (DutchmanNL) Bugfix: Allow control of brightness and color for light component, fixes #173
+* (DutchmanNL) Bugfix: Allow control of brightness and color for light component, resolves #173
 
 ### 0.3.1 (2023-10-31)
-* (DutchmanNL) Bugfix: Show online state of ESP Device correctly, Fixes #106
+* (DutchmanNL) Bugfix: Show online state of ESP Device correctly, resolves #106
 
 ### 0.3.0 (2023-10-31) - Bugfixes & Improvements
 * (Dutchman & SimonFischer04) Several Bugfixes
 * (SimonFischer04) Support type "select device"
 * (DutchmanNL) ESPHome dashboard default disabled
 * (SimonFischer04) Migrate to @2colors/esphome-native-api
-* (DutchmanNL) Automatically create needed directories, Fixes #168
+* (DutchmanNL) Automatically create needed directories, resolves #168
 * (SimonFischer04) Migrate usage of python to new structure, should solve all ESPHome Dashboard related installation issues
 
 ### 0.2.4 (2021-08-24)
 * (DutchmanNL) Version of ESPHome Dashboard updated to 2021.8
 * (DutchmanNL) Add option if config of ESPHome device should be shown as states (default = FALSE, safes 8 states for each sensor)
 
-### 0.2.3 (2021-06-29)
-* (Jey-Cee) Bugfix : Light component state not changed [#74](https://github.com/DrozmotiX/ioBroker.esphome/issues/74)
-* (DutchmanNL) Update compatibility to version 1.19.4 of ESPHome Dashboard
-
 ## License
 MIT License
 

package/io-package.json

@@ -1,8 +1,21 @@
 {
   "common": {
     "name": "esphome",
-    "version": "0.3.2",
+    "version": "0.4.0",
     "news": {
+      "0.4.0": {
+        "en": "Added cleanup capability for unused channels & states after initialisation of device, resolves #39\nAdded button to info channel which allows to delete all offline devices from adapter tree. resolves #39\n[Breaking] Backup strategy changed, requires [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) and activate option for ESPHome, fixes #129",
+        "de": "Zusätzliche Reinigungsfähigkeit für ungenutzte Kanäle & Zustände nach der Initialisierung des Geräts, löst #39\nZusätzlicher Button zum Info-Kanal, der alle Offline-Geräte aus dem Adapterbaum löschen kann. löst #39\n[Breaking] Backup-Strategie geändert, erfordert [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) und aktiviert Option für ESPHome, repariert #129",
+        "ru": "Добавлена возможность очистки неиспользуемых каналов и государств после инициализации устройства, решает #39\nДобавлена кнопка к информационному каналу, что позволяет удалить все оффлайн устройства из дерева адаптера. решает #39\n[Breaking] Стратегия резервного копирования изменена, требует [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) и активировать вариант для ESPHome, исправлений #129",
+        "pt": "Adicionado capacidade de limpeza para canais não utilizados & estados após a inicialização do dispositivo, resolve #39\nAdicionado botão para info canal que permite excluir todos os dispositivos off-line da árvore do adaptador. resolve #39\n[Breaking] Estratégia de backup alterada, requer [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) e opção ativa para ESPHome, correções #129",
+        "nl": "Vervoegde schoonmaak capaciteit voor onbruikbare kanalen apparaat na de initialisatie van 39\nKnop toegevoegd aan info kanaal dat alle offline apparaten van adapter boom laat verwijderen. 39\nOndersteuningsstrategie veranderde, vereist [Backitup v2.9.1, htps:com/simateioBroker, en tactiveer de optie van 12 uur",
+        "fr": "Ajout d'une capacité de nettoyage pour les canaux non utilisés & états après l'initialisation du dispositif, résout #39\nAjout du bouton sur le canal d'info qui permet de supprimer tous les périphériques hors ligne de l'arbre d'adaptateur. résolution #39\n[Breaking] La stratégie de sauvegarde a changé, nécessite [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) et l'option active pour ESPHome, corrige #129",
+        "it": "Aggiunta funzionalità di pulizia per canali e stati non utilizzati dopo l'inizializzazione del dispositivo, risolve #39\nPulsante aggiunto al canale info che consente di eliminare tutti i dispositivi offline dall'albero dell'adattatore. risolve #39\n[Breaking] La strategia di backup è cambiata, richiede [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) e attiva l'opzione per ESPHome, corregge #129",
+        "es": "Capacidad de limpieza agregada para canales no utilizados \" estados después de la inicialización del dispositivo, resuelve #39\nSe agregó botón a canal de información que permite eliminar todos los dispositivos fuera de línea del árbol del adaptador. resuelve #39\n[Breaking] La estrategia de respaldo cambió, requiere [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) y activar la opción para ESPHome, fija #129",
+        "pl": "Po zainicjowaniu możliwości usuwania nieużywanych kanałów i stanów po inicjalizacji urządzenia, rozwiązał #39\nNaciśnięty przycisk do kanału informacyjnego, który umożliwia usunięcie wszystkich urządzeń offline z drzewa adapterowego. Rozwiązanie #39\n(Po zmianie strategii Backupa, wymaga od BOATUTUp v2.9.1(https:/github.com/simatec/ioBroker.backitup) i aktywacji opcji dla ESPHome, naprawa #129",
+        "uk": "Додано можливість очищення каналів і станів після ініціалізації пристрою, вирішує #39\nДодано кнопку на інформаційний канал, який дозволяє видалити всі offline пристрої з дерева адаптера. розчини #39\n[Breaking] Стратегія резервного копіювання змінено, вимагає [BackitUp v2.9.1](https://github.com/simatec/ioBroker.backitup) і активувати варіант для ESPHome, фіксує #129",
+        "zh-cn": "在装置初始化后增加无用渠道和各州的清洁能力,第39号决定\n添加但道托顿以电器,使所有线装置能够从适应树中删除。 行动39\n[Breaking] Backup战略改变,要求[BackitUp v2.9.1](http://github.com/simatec/ioBroker.backitup)和“ESPHome”的主动选择,第129号行动。"
+      },
       "0.3.2": {
         "en": "Improved error handling if devices are not reachable/disconnected\nBugfix: Allow control of brightness and color for light component, fixes #173",
         "de": "Verbesserte Fehlerbehandlung, wenn Geräte nicht erreichbar/vernetzt sind\nBugfix: Erlauben Sie die Kontrolle von Helligkeit und Farbe für die Lichtkomponente, fixiert #173",
@@ -77,18 +90,6 @@
         "es": "[!!! ¡¡¡Rompiendo !!!] ¡Haga que el archivo YAML sea persistente, haga una copia de seguridad de su configuración antes de actualizar! resuelve [# 57] (https://github.com/DrozmotiX/ioBroker.esphome/issues/57)\nActualice ESPHome Dashboard a 1.18.0, requiere Python> = 3.7 (¡y asegúrese de <4.0!)\nCorrección de error: vuelva a conectarse a dispositivos sin detección automática / MDNS-Broadcast en la red, resuelve [# 66] (https://github.com/DrozmotiX/ioBroker.esphome/issues/66)",
         "pl": "[!!! Breaking !!!] Utrwal plik YAML, wykonaj kopię zapasową konfiguracji przed aktualizacją! rozwiązuje [#57](https://github.com/DrozmotiX/ioBroker.esphome/issues/57)\nZaktualizuj pulpit nawigacyjny ESPHome do wersji 1.18.0, wymaga Pythona >=3.7 (i upewnij się, że <4.0!)\nBugfix: Połącz się ponownie z urządzeniami bez automatycznego wykrywania / MDNS-Broadcast w sieci, rozwiązuje [#66](https://github.com/DrozmotiX/ioBroker.esphome/issues/66)",
         "zh-cn": "[！！！打破 !!!] 使 YAML 文件持久化，更新前备份您的配置！解决 [#57](https://github.com/DrozmotiX/ioBroker.esphome/issues/57)\n将 ESPHome Dashboard 更新到 1.18.0，需要 Python >=3.7（并确保 <4.0！）\n修正：在网络中没有自动发现/MDNS-Broadcast 的情况下重新连接到设备，解决[#66](https://github.com/DrozmotiX/ioBroker.esphome/issues/66)"
-      },
-      "0.2.1-1": {
-        "en": "add cover component\nadd transitionLength for lights",
-        "de": "Abdeckungskomponente hinzufügen\nÜbergangslänge für Lichter hinzufügen",
-        "ru": "добавить компонент обложки\nдобавить transitionLength для огней",
-        "pt": "adicionar componente de capa\nadicionar comprimento de transição para luzes",
-        "nl": "voeg een omslagcomponent toe\nvoeg transitionLength toe voor lichten",
-        "fr": "ajouter un composant de couverture\najouter une longueur de transition pour les lumières",
-        "it": "aggiungi il componente di copertina\naggiungi TransitionLength per le luci",
-        "es": "agregar componente de portada\nagregar transiciónLength para luces",
-        "pl": "dodaj komponent okładki\ndodaj długość przejścia dla świateł",
-        "zh-cn": "添加封面组件\n为灯光添加transitionLength"
       }
     },
     "title": "Control ESPHome managed devices",
@@ -145,7 +146,6 @@
         "dsn": "https://52227cfa250b49e99b1f28408273402c@sentry.iobroker.net/122"
       }
     },
-    "dataFolder": "esphome.%INSTANCE%",
     "localLink": "http://%ESPHomeDashboardIP%:%ESPHomeDashboardPort%",
     "adminTab": {
       "link": "http://%ESPHomeDashboardIP%:%ESPHomeDashboardPort%",

package/main.js

@@ -14,8 +14,10 @@ const stateAttr = require(__dirname + '/lib/stateAttr.js'); // Load attribute li
 const disableSentry = false; // Ensure to set to true during development!
 const warnMessages = {}; // Store warn messages to avoid multiple sending to sentry
 const fs = require('fs');
+const {clearTimeout} = require('timers');
 const client = {};
-let reconnectTimer, reconnectInterval, apiPass, autodiscovery, dashboardProcess, createConfigStates;
+const resetTimers = {}; // Memory allocation for all running timers
+let reconnectInterval, apiPass, autodiscovery, dashboardProcess, createConfigStates;
 
 class Esphome extends utils.Adapter {
 
@@ -70,6 +72,21 @@ class Esphome extends utils.Adapter {
 				this.log.info(`Native Integration of ESPHome Dashboard disabled `);
 			}
 
+			// Create & Subscribe to button handling offline Device cleanup
+			this.extendObjectAsync('esphome.0.info.deviceCleanup',
+				{
+					'type': 'state',
+					'common': {
+						'role': 'button',
+						'name': 'Device or service connected',
+						'type': 'boolean',
+						'read': false,
+						'write': true,
+						'def': false
+					}
+				});
+			this.subscribeStates('esphome.0.info.deviceCleanup');
+
 		} catch (e) {
 			this.log.error(`[Adapter start] Fatal error occurred ${e}`);
 		}
@@ -154,7 +171,7 @@ class Esphome extends utils.Adapter {
 
 			// Get basic data of known devices and start reading data
 			for (const i in knownDevices) {
-				this.connectDevices(knownDevices[i].native.ip, this.decrypt(knownDevices[i].native.passWord));
+				this.connectDevices(knownDevices[i].native.ip);
 			}
 		} catch (e) {
 			this.sendSentry(`[tryKnownDevices] ${e}`);
@@ -173,7 +190,7 @@ class Esphome extends utils.Adapter {
 					this.log.debug(`Discovery message ${JSON.stringify(message)}`);
 					if (this.deviceInfo[message.address] == null) {
 						this.log.info(`[AutoDiscovery] New ESPHome device found at IP ${message.address}, trying to initialize`);
-						this.connectDevices(`${message.address}`, `${apiPass}`);
+						this.connectDevices(`${message.address}`);
 					}
 				} catch (e) {
 					this.log.error(`[deviceDiscovery handler] ${e}`);
@@ -188,9 +205,8 @@ class Esphome extends utils.Adapter {
 	/**
 	 * Handle Socket connections
 	 * @param {string} host IP adress of device
-	 * @param {string | number | boolean} pass Password used for APU
 	 */
-	connectDevices(host, pass) {
+	connectDevices(host) {
 		try {
 			// const host = espDevices[device].ip;
 			this.log.info(`Try to connect to ${host}`);
@@ -237,6 +253,14 @@ class Esphome extends utils.Adapter {
 					if (this.deviceInfo[host].deviceName != null) {
 						this.setState(`${this.deviceInfo[host].deviceName}.info._online`, {val: false, ack: true});
 
+						// Check all created states in memory if their are related to this device
+						for (const state in this.createdStatesDetails) {
+							// Remove states from cache
+							if (state.split('.')[0] === this.deviceInfo[host].deviceName) {
+								delete this.createdStatesDetails[state];
+							}
+						}
+
 						// Cache relevant data before clearing memory space of device
 						const cacheDeviceInformation = {
 							deviceName: this.deviceInfo[host].deviceName,
@@ -266,6 +290,12 @@ class Esphome extends utils.Adapter {
 			client[host].on('initialized', () => {
 				this.log.info(`ESPHome  client ${this.deviceInfo[host].deviceInfoName} on ip ${host} initialized`);
 				this.deviceInfo[host].initialized = true;
+
+				// Start timer to cleanup unneeded objects
+				if (resetTimers[host]) resetTimers[host] = clearTimeout(resetTimers[host]);
+				resetTimers[host] = setTimeout(async () => {
+					await this.objectCleanup(host);
+				}, (10000));
 			});
 
 			// Log message listener
@@ -286,7 +316,12 @@ class Esphome extends utils.Adapter {
 					// Store device information into memory
 					const deviceName = this.replaceAll(deviceInfo.macAddress, `:`, ``);
 					this.deviceInfo[host] = {
+						adapterObjects : {
+							channels : []
+						},
 						ip: host,
+						connectError : false,
+						connected : true,
 						mac: deviceInfo.macAddress,
 						deviceInfo: deviceInfo,
 						deviceName: deviceName,
@@ -368,6 +403,11 @@ class Esphome extends utils.Adapter {
 						native: {},
 					});
 
+					// Cache created channel in device memory
+					if (!this.deviceInfo[host].adapterObjects.channels.includes(`${this.namespace}.${this.deviceInfo[host].deviceName}.${entity.type}`)) {
+						this.deviceInfo[host].adapterObjects.channels.push(`${this.namespace}.${this.deviceInfo[host].deviceName}.${entity.type}`);
+					}
+
 					// Create state specific channel by id
 					await this.extendObjectAsync(`${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}`, {
 						type: 'channel',
@@ -377,6 +417,11 @@ class Esphome extends utils.Adapter {
 						native: {},
 					});
 
+					// Cache created channel in device memory
+					if (!this.deviceInfo[host].adapterObjects.channels.includes(`${this.namespace}.${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}`)) {
+						this.deviceInfo[host].adapterObjects.channels.push(`${this.namespace}.${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}`);
+					}
+
 					//Check if config channel should be created
 					if (!createConfigStates) {
 						// Delete folder structure if already present
@@ -397,6 +442,12 @@ class Esphome extends utils.Adapter {
 							},
 							native: {},
 						});
+
+						// Cache created channel in device memory
+						if (!this.deviceInfo[host].adapterObjects.channels.includes(`${this.namespace}.${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}.config`)) {
+							this.deviceInfo[host].adapterObjects.channels.push(`${this.namespace}.${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}.config`);
+						}
+
 						// Handle Entity JSON structure and write related config channel data
 						await this.TraverseJson(entity.config, `${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}.config`);
 					}
@@ -492,7 +543,7 @@ class Esphome extends utils.Adapter {
 
 
 				} catch (e) {
-					this.log.error(`Connection issue for ${entity.name} ${e}`);
+					this.log.error(`Connection issue for ${entity.name} ${e} | ${e.stack}`);
 				}
 
 			});
@@ -672,6 +723,8 @@ class Esphome extends utils.Adapter {
 						// Check if state contains value
 						if (transitionLength) {
 							this.deviceInfo[host][entity.id].states.transitionLength = transitionLength.val;
+							// Run create state routine to ensure state is cached in memory
+							await this.stateSetCreate(`${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}.transitionLength`, `${stateName} of ${entity.config.name}`, transitionLength.val, `s`, writable);
 						} else { // Else just create it
 							await this.stateSetCreate(`${this.deviceInfo[host].deviceName}.${entity.type}.${entity.id}.transitionLength`, `${stateName} of ${entity.config.name}`, 0, `s`, writable);
 							this.deviceInfo[host][entity.id].states.transitionLength = 0;
@@ -948,6 +1001,8 @@ class Esphome extends utils.Adapter {
 	onUnload(callback) {
 		try {
 			this.log.debug(JSON.stringify(this.deviceInfo));
+
+			// Set all online states to false
 			for (const device in this.deviceInfo) {
 
 				// Ensure all known online states are set to false
@@ -962,9 +1017,12 @@ class Esphome extends utils.Adapter {
 					this.log.debug(`[onUnload] ${JSON.stringify(e)}`);
 				}
 			}
-			if (reconnectTimer) {
-				reconnectTimer = clearTimeout();
+
+			// Ensure all possible running timers are cleared
+			for (const timer in resetTimers) {
+				if (resetTimers[timer]) resetTimers[timer] = clearTimeout(resetTimers[timer]);
 			}
+
 			try {
 				if (dashboardProcess) {
 					dashboardProcess.kill('SIGTERM', {
@@ -1029,7 +1087,7 @@ class Esphome extends utils.Adapter {
 						this.log.info(`Valid IP address received`);
 						this.messageResponse[obj.message['device-ip']] = obj;
 						// const pass = this.decrypt(obj.message['device-pass']);
-						await this.connectDevices(obj.message['device-ip'], apiPass);
+						await this.connectDevices(obj.message['device-ip']);
 					}
 					break;
 			}
@@ -1079,6 +1137,17 @@ class Esphome extends utils.Adapter {
 		try {
 			if (state && state.ack === false) {
 				const device = id.split('.');
+
+				try {
+					// Verify if trigger is related to device-cleanup
+					if (id.split('.')[3] === 'deviceCleanup'){
+						await this.offlineDeviceCleanup();
+						return;
+					}
+				} catch (e) {
+					// Skip action
+				}
+
 				const deviceIP = this.deviceStateRelation[device[2]].ip;
 
 				// Handle Switch State
@@ -1233,6 +1302,77 @@ class Esphome extends utils.Adapter {
 			this.log.error(`[resetOnlineState] ${e}`);
 		}
 	}
+
+	async objectCleanup(ip){
+		try {
+			this.log.debug(`[objectCleanup] Starting channel and state cleanup for ${this.deviceInfo[ip].deviceName} | ${ip} | ${this.deviceInfo[ip].ip}`);
+
+			// Cancel cleanup operation in case device is not connected anymore
+			if (this.deviceInfo[ip].connectionError || !this.deviceInfo[ip].connected) return;
+
+			// Set parameters for object view to only include objects within adapter namespace
+			const params = {
+				startkey : `${this.namespace}.${this.deviceInfo[ip].deviceName}.`,
+				endkey : `${this.namespace}.\u9999`,
+			};
+
+			// Get all current channels
+			const _channels = await this.getObjectViewAsync('system', 'channel', params);
+			// List all found channels & compare with memory, delete unneeded channels
+			for (const currDevice in _channels.rows) {
+				// @ts-ignore
+				if (!this.deviceInfo[ip].adapterObjects.channels.includes(_channels.rows[currDevice].id)
+					&& _channels.rows[currDevice].id.split('.')[2] === this.deviceInfo[ip].deviceName){
+					this.log.debug(`[objectCleanup] Unknown Channel found, delete ${_channels.rows[currDevice].id}`);
+					await this.delObjectAsync(_channels.rows[currDevice].id, {recursive: true});
+				}
+			}
+
+			// Get all current states in adapter tree
+			const _states = await this.getObjectViewAsync('system', 'state', params);
+			// List all found states & compare with memory, delete unneeded states
+			for (const currDevice in _states.rows) {
+				if (!this.createdStatesDetails[_states.rows[currDevice].id.replace(`esphome.0.`, ``)]
+					&& _states.rows[currDevice].id.split('.')[2] === this.deviceInfo[ip].deviceName){
+					this.log.debug(`[objectCleanup] Unknown State found, delete ${_states.rows[currDevice].id}`);
+					// await this.delObjectAsync(_states.rows[currDevice].id);
+				}
+			}
+
+		} catch (e) {
+			this.log.error(`[objectCleanup] Fatal error ${e} | ${e.stack}`);
+		}
+	}
+
+	async offlineDeviceCleanup () {
+		this.log.info(`Offline Device cleanup started`);
+
+		try {
+
+			// Get an overview of all current devices known by adapter
+			const knownDevices = await this.getDevicesAsync();
+			console.log(`KnownDevices: ${knownDevices}`);
+
+			// Loop to all devices, check if online state = TRUE otherwise delete device
+			for (const device in knownDevices){
+
+				// Get online value
+				const online = await this.getStateAsync(`${knownDevices[device]._id}.info._online`);
+				if (!online || !online.val){
+					this.log.info(`Offline device ${knownDevices[device]._id.split('.')[2]} expected on ip ${knownDevices[device].native.ip} removed`);
+					await this.delObjectAsync(knownDevices[device]._id, {recursive: true});
+				}
+
+			}
+
+			if (!knownDevices) return; // exit function if no known device are detected
+			if (knownDevices.length > 0) this.log.info(`Try to contact ${knownDevices.length} known devices`);
+		} catch (e) {
+			this.log.error(`[offlineDeviceCleanup] Fatal error occurred, cannot cleanup offline devices ${e} | ${e.stack}`);
+
+		}
+	}
+
 }
 
 if (require.main !== module) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "iobroker.esphome",
-  "version": "0.3.2",
+  "version": "0.4.0",
   "description": "Control your ESP8266/ESP32 with simple yet powerful configuration files created and managed by ESPHome",
   "author": {
     "name": "DutchmanNL",