node-red-contrib-dmx-for-ha 0.1.0

package/docs/master_todo.md

@@ -0,0 +1,428 @@
+# DMX Node-RED Project — Master TODO & Roadmap
+**Last updated:** Session 3  
+**Discord:** @deswaggy
+
+---
+
+## Project Versions
+
+| Node | Current Version | Status |
+|---|---|---|
+| DMX NODE | 0.5.9 | World tested — field testing pending |
+| DMX GROUP NODE | 0.3.8 | World tested — field testing pending |
+| 230V RELAY NODE | 4.0.2 | World tested — field testing pending |
+| BUTTON NODE | 5.0.3 | World tested — field testing pending |
+| PIR / BINARY SENSOR NODE | 1.0.3 | World tested — field testing pending |
+
+---
+
+## Versioning Convention
+
+```
+MAJOR.MINOR.PATCH
+  │     │     └── Bug fix / small addition (+0.0.1)
+  │     └──────── New feature / significant change, no breaking (+0.1.0, reset patch)
+  └────────────── Breaking change / full rewrite (+1.0.0, reset minor + patch)
+```
+
+---
+
+## DMX NODE (v0.5.9)
+
+### World Testing
+- [ ] Worst-case perf test — 200–300 rgbw subflows simultaneous
+- [ ] Snapshot into _dev & beta nodes
+- [ ] HA automations wired to new nodes
+- [ ] Begin migrating rules from NR to HA — NR remains as virtual lighting desk only, no rules post-migration
+- [ ] Tune GAMMA constant per fixture type based on real-world fade observation
+- [ ] Tune `<DMX_transition_ticks_ps>` per fixture based on deployment size
+- [ ] Validate `<DMX_ct_min_mireds>` & `<DMX_ct_max_mireds>` against fixture spec sheets
+- [ ] Validate `<DMX_Brightness_Initial_on_value>` bump level per room/fixture
+- [ ] Stress test disk-save timer under rapid command bursts (colour wheel drag)
+- [ ] Confirm all effects self-terminate and `restoreAfterEffect()` correctly on all color modes
+- [ ] Verify recovery state payload on reboot for each supported color mode
+
+### Post Testing — Release & Documentation
+- [x] Audit discovery payload field names — `default_entity_id` is not a valid HA MQTT
+      discovery field. The correct field is `object_id`. DMX and Group Nodes likely need
+      this corrected before packaging — check if entity_ids are being generated correctly
+      in HA or defaulting to name-based slugs. Button and PIR nodes use `object_id` correctly.
+- [x] Package as a proper Node-RED node (package.json, node .js + .html)
+- [ ] Git repository setup with branch structure (_dev / beta / main)
+- [x] README — project overview, what the node does, where it fits in HA → NR → DMX chain
+- [ ] README — architecture diagram: HA dashboard → MQTT → NR subflow → DMX controller
+- [x] README — installation & setup (NR dependencies, settings.js context store config, MQTT broker)
+- [ ] README — how to deploy a new fixture (step by step using the env checklist)
+- [ ] README — env variable reference (fold in dmx_node_env_reference.md)
+- [ ] README — transition & effects guide (rate limit vs ticks, tuning for deployment size)
+- [x] README — troubleshooting guide (common faults, what to check in NR context store panel)
+- [ ] README — gamma & DMX min output note for lighting designers:
+      Explain that GAMMA=2.2 is perceptually correct but collapses sub-threshold
+      brightness levels to DMX zero. <DMX_min_output>=1 (default) prevents the
+      fixture appearing off when the HA slider is visibly above 0%. Lighting
+      designers wanting pure gamma response can set <DMX_min_output>=0 and tune
+      the GAMMA constant (2.2 = sRGB standard, 2.5–2.8 suits most LED fixtures).
+      Note: 5% on the HA slider is genuinely sub-threshold at GAMMA=2.2 — this
+      is mathematically correct, not a bug.
+- [ ] README — version history
+- [ ] README — wiring diagram (how subflow connects to SYSTEM node, MQTT in/out, etc.)
+- [ ] Submit to Node-RED flows library (flows.nodered.org)
+
+---
+
+## DMX GROUP NODE (v0.3.8)
+
+### World Testing
+- [ ] Wire into test environment — single-level (Group Node → DMX Node × N)
+- [ ] Test recursive group — Group Node → Group Node → DMX Node
+- [ ] Test Bedroom 1 real-world hierarchy (mixed color modes across children)
+- [ ] Verify `dmx_trace` path and depth are correct at each hop
+- [ ] Trigger loop guard — circular wiring, confirm `node.warn` + red status + hard stop
+- [ ] Confirm group effects forward simultaneously and children stay in sync
+- [ ] Stress test — rapid commands from HA dashboard across 3+ group levels
+- [ ] Confirm recovery state on reboot for all color modes
+- [ ] Verify disk-save timer resets correctly on rapid commands
+- [ ] Confirm `<DMX_Effects_Group_Controlled>=true` blocks direct HA effects correctly
+- [ ] Confirm `<DMX_Effects_Group_Controlled>=false` allows direct HA effects correctly
+
+### Post Testing — Release & Documentation
+- [x] Package as a proper Node-RED node alongside DMX Node
+- [x] Shared utility decision: INLINE for v1 — extract to shared module in v2 post-stability
+- [ ] Git repository alongside DMX Node (_dev / beta / main)
+- [ ] README — Group Node overview and role in HA → NR → DMX chain
+- [ ] README — topology diagram (Parent Group → Sub Group → DMX Node)
+- [ ] README — `dmx_trace` explained (source, path, depth, loop detection)
+- [ ] README — env variable reference for Group Node
+- [ ] README — wiring guide (5 outputs, AUX input, recursive topology)
+- [ ] README — effects guide (synced group effects vs individual fixture effects)
+- [ ] README — state aggregation explained (last sent wins, no upward propagation)
+- [ ] README — troubleshooting (loop guard, unknown message warn, G-prefix context keys)
+- [ ] Add Group Node env vars to `dmx_node_env_reference.md`
+
+### Group Node — Future Design
+- [ ] AUX input contract finalised — confirm `dmx_trace` payload structure is stable
+- [ ] State aggregation — confirm "last sent wins" holds in real-world automation scenarios
+- [ ] Consider HA automation helper for edge case (e.g. wall button turns on single child — should parent reflect this?)
+
+---
+
+## DMX GROUP NODE — Future: Group of Groups
+- [ ] Bedroom 1 style hierarchy tested end to end (5 sub-groups, mixed color modes)
+- [ ] Ceiling style hierarchy tested (3 sub-groups, same color mode)
+- [ ] Confirm `dmx_trace.depth` increments correctly through 3+ levels
+- [ ] Confirm loop guard fires correctly at configured depth limit
+
+---
+
+## BUTTON NODE (v5.0.3)
+> Pure MQTT — no websocket dependency. Dual HA entity discovery:
+> binary_sensor (automation trigger) + button (UI dashboard mirror).
+> HA off_delay handles auto-clear. 4 outputs. Subflow input kept for SYSTEM node.
+
+### World Testing
+- [ ] Wire into test environment
+- [ ] Test device add — binary_sensor and UI button both appear in HA device registry
+- [ ] Test physical press — binary_sensor ON then auto-clears via off_delay
+- [ ] Test UI button press — same result as physical press
+- [ ] Test device remove — both entities removed from HA
+- [ ] Test sibling payloads silently ignored (no false triggers)
+- [ ] Test with old firmware (flat "buttons" topic) via translation layer
+- [ ] Test with new firmware (structured topic) once firmware updated
+- [ ] Validate serial_number clean and unique across all test buttons
+- [ ] Migrate existing HA automations from button.press to binary_sensor ON triggers
+
+### Post Testing — Release & Documentation
+- [x] Package as proper Node-RED node
+- [ ] README — overview, dual entity design, why no websocket dependency
+- [ ] README — payload format "{panelId}-{GPIOpin}" explained
+- [ ] README — button position conventions (A/B/C/D for 1/2/3/4 gang plates)
+- [ ] README — translation layer guide for old firmware rollout
+- [ ] README — env variable reference
+- [ ] README — troubleshooting
+- [ ] Add Button Node env vars to master env reference document
+
+### Env Vars — Changes from v4.x
+- Renamed: `<HA_BUTTON_icon>` → `<HA_icon>`
+- Renamed: `<DEVICE_area_case>` → `<DEVICE_situation>`
+- Added:   `<BUTTON_hold_time_seconds>` (float, default 0.5)
+- Kept:    `<BUTTON_Type>` (default "Wall" — retained for future flexibility)
+- Kept:    `<CONTROLLER_Model>` (used in hw_version device info)
+- Deleted: `<HA_Payload>`, `<HA_device_class>`, `<HA_entity_category>`, `<HA_schema>`
+
+---
+
+## BINARY SENSOR NODE / PIR SENSOR NODE (v1.0.3)
+> Pure MQTT binary_sensor with device_class: motion.
+> HA off_delay handles auto-clear. Configurable warm-up delay on boot.
+> Time-based NR debounce reduces MQTT traffic. 4 outputs. Subflow input kept.
+
+### World Testing
+- [ ] Wire into test environment
+- [ ] Test device add — PIR appears "offline" then "online" after warmup delay
+- [ ] Test PIR trigger — binary_sensor ON then auto-clears via off_delay
+- [ ] Test debounce — rapid triggers during hold period don't re-publish to HA
+- [ ] Test device:avty "online"/"offline" from SYSTEM node and admin panel
+- [ ] Test device remove — entity removed, availability goes offline
+- [ ] Test reboot scenario — PIR stays offline during warm-up, no mass light injection
+- [ ] Test with old firmware ("MW3D/Master/PIR_Sensors") via translation layer
+- [ ] Test with new firmware ("MW3D/Master/13/pir") once firmware updated
+- [ ] Validate serial_number clean and unique across test sensors
+- [ ] Confirm sibling PIR payloads silently ignored
+
+### Post Testing — Release & Documentation
+- [ ] Package as proper Node-RED node
+- [ ] README — overview, availability topic rationale (the 3am story)
+- [ ] README — payload format explained
+- [ ] README — warm-up sequence and why it exists
+- [ ] README — env variable reference
+- [ ] README — translation layer guide for old firmware rollout
+- [ ] README — troubleshooting (PIR stuck offline, false triggers)
+- [ ] Add PIR Node env vars to master env reference document
+- [ ] Consider heartbeat mechanism to replace daily sun-event "add" inject
+
+### Env Vars — Changes from v0.1.x
+- Renamed: `<HA_PIR_icon>` → `<HA_icon>`
+- Renamed: `<DEVICE_area_case>` → `<DEVICE_situation>`
+- Added:   `<PIR_hold_time_seconds>` (num, default 15)
+- Added:   `<PIR_warmup_seconds>` (num, default 120)
+- Kept:    `<PIR_Cable_Color>` (now used in hw_version device info — "Purple Cat5 cable")
+- Deleted: `<PIR_Friendly_Name>`, `<PIR_Friendly_Sensor>`, `<HA_Payload>`, `<CONTROLLER_Model>`, `<HA_device_class>`, `<HA_schema>`
+
+---
+
+## 230V RELAY NODE (v4.0.0)
+> **Important:** The 230V Relay Node has NO relationship to DMX whatsoever.
+> It is a completely separate control path, separate protocol, separate hardware.
+> Do not reference DMX in this node's code, comments, or documentation.
+> See "DMX Power Devices" section below for the DMX side of 230V control.
+
+### World Testing
+- [ ] Wire into test environment
+- [ ] Test device add — discovery payload, MQTT subscribe, recovery state
+- [ ] Test ON/OFF from HA — confirm relay payload is integer 1/0 not string
+- [ ] Test flash_short and flash_long — confirm relay fires then restores
+- [ ] Test strobe — confirm 500ms minimum interval respected
+- [ ] Test device remove — unsubscribe, memory cleared, status red
+- [ ] Test AUX input from DMX Group Node
+- [ ] Test recovery state on reboot — physical relay re-asserted to match HA
+- [ ] Validate serial_number is clean and unique across test fixtures
+- [ ] Test with mechanical relay — confirm strobe safe at 500ms
+- [ ] Test with SSR — confirm strobe acceptable at 500ms
+
+### Post Testing — Release & Documentation
+- [ ] Package as proper Node-RED node alongside DMX Node and Group Node
+- [ ] Git repository alongside other nodes
+- [ ] README — overview and what this node is NOT (not DMX)
+- [ ] README — relay MQTT topic format and integer payload convention
+- [ ] README — env variable reference
+- [ ] README — wiring guide (4 HA outputs + 1 relay controller output)
+- [ ] README — effects guide + mechanical relay vs SSR note for strobe
+- [ ] README — troubleshooting
+- [ ] Add 230V Relay Node env vars to master env reference document
+
+---
+
+## DMX POWER DEVICES (Future Discussion)
+> **Clarification:** DMX Power Devices are controlled via the standard **DMX Node**,
+> not the 230V Relay Node. They are DMX-decodable hardware that happen to switch
+> or dim 230V circuits. The DMX Node sends DMX payloads over MQTT as normal —
+> the hardware on the other end handles the 230V switching internally.
+> The 230V Relay Node is an entirely separate, non-DMX control path.
+
+- [ ] DMX 230V Dimmer — use DMX Node with `brightness` color mode
+      Examples: theatre profile lights, fresnels, tungsten fixtures on a DMX dimmer rack
+- [ ] DMX 230V Relay — use DMX Node with `onoff` color mode
+      Examples: fans, socket outlets, remote power switching via DMX relay card
+- [ ] Prefix convention (shared across ALL node types — end user sees device intent, not implementation):
+      `L`  = Light  (DMX Node, 230V Relay Node in lighting role)
+      `P`  = Power  (DMX Node in power role, 230V Relay Node in power role)
+      `LG` = Light Group (DMX Group Node)
+      `PG` = Power Group (DMX Group Node in power role)
+      The end user sees L/P/LG/PG — they don't need to know whether it's
+      DMX, relay, or any other underlying technology. Keeps HA dashboard,
+      cable labels, Master Fixture Control Document, and Electrical Plans
+      clean and consistent. L and P fixtures are on separate plan pages
+      to keep IDs short and unambiguous.
+- [ ] Confirm DMX Node handles power devices correctly as-is (color mode selection sufficient)
+- [ ] Document use case in README under "Power Devices" — clearly separate from 230V Relay Node
+- [ ] Add note: a DMX Power Device group uses DMX Group Node with `onoff` or `brightness`,
+      prefixed `PG` — same pattern as lighting groups
+
+---
+
+## Shared Utility Module (Future — Node Package Build)
+- [ ] Extract shared helpers from DMX Node and Group Node:
+  - `recall(ramKey, diskKey, fallback)`
+  - `rememberRam(values)`
+  - `rememberDisk(values)`
+  - `startDiskSaveTimer(onComplete)`
+  - `sendStatus(fill, shape, text)`
+  - `hsvToRgb(h, s, v)`
+- [ ] Decide module format (CommonJS vs ES module) based on NR node package requirements
+- [ ] Write unit tests for shared helpers
+
+---
+
+## Git Repository (Future)
+- [ ] Repository name TBD
+- [ ] Branch structure: `main` / `beta` / `_dev`
+- [ ] Nodes as separate packages or monorepo — TBD
+- [ ] Contribution guidelines
+- [ ] Issue templates (bug report, feature request)
+- [ ] License selection
+- [ ] Open source contribution / Node-RED flows library submission
+
+---
+
+## DMX NODE — Pending Field Testing Decisions
+
+### Gamma + Minimum Floor Behaviour
+- [ ] FIELD TEST: At low combined brightness (e.g. brightness 17% + color 1%),
+      scaleToDmx() rounds to 0 before the DMX min floor guard fires.
+      Current guard checks `linear > 0` but `linear` has already been rounded to 0.
+      Fix ready: change guard to `colorValue > 0 && brightness > 0` instead.
+      Needs visual confirmation in the field — does the user WANT DMX=1 or DMX=0
+      when both sliders are above zero but the combined value is < 0.5?
+      Lighting designer answer vs UX answer may differ.
+      Code fix is one line in scaleToDmx() in dmx_node.js.
+
+---
+
+## CONTROLLER FIRMWARE (Future — Post NR/HA Completion)
+> Do not mix firmware and UI software development — complete the NR/HA node
+> layer first, then revisit controller firmware as a separate body of work.
+
+### Button Controller Firmware
+- [ ] Update topic structure: flat "buttons" → "MW3D/{zone}/{panelId}/buttons"
+      (one line change in config.h — topicString variable)
+      Deploy to one test controller first, translation layer covers the rest
+- [ ] Investigate adding long press detection:
+      Measure time between GPIO press and release in firmware
+      If > threshold (e.g. 800ms) publish "{panelId}-{GPIOpin}-L" payload
+      NR Button Node filters "-L" suffix → separate binary_sensor entity in HA
+- [ ] Investigate double-tap detection:
+      Count press events within a short window (~400ms) in firmware
+      If 2 presses detected publish "{panelId}-{GPIOpin}-D" payload
+- [ ] Agree on timing thresholds for long press and double tap
+      (needs real-world tuning per installation — wall button feel varies)
+- [ ] Update Button Node to support "-L" and "-D" payload suffixes if firmware updated
+- [ ] Review I2C stability (shared concern with relay controller)
+- [ ] Document controller firmware versions
+
+### PIR Controller Firmware
+- [ ] Update topic structure: "MW3D/Master/PIR_Sensors" → "MW3D/{zone}/{panelId}/pir"
+      (one line change in config.h — motionTopic variable, note: needs String not const char*)
+      Deploy to one test controller first, translation layer covers the rest
+- [ ] Review startup delay behaviour — firmware already ignores first trigger on boot,
+      confirm this still works correctly with new topic structure
+- [ ] Confirm 48 GPIO pins (not 64 as previously assumed — Mega has 54 digital pins)
+- [ ] Document OLED display usage and any planned improvements
+
+### 230V Relay Controller Firmware
+- [ ] Review I2C stability improvements
+      (rapid relay switching caused lockup — 1pF cap on reset pin was the fix,
+      firmware hardening still worth exploring to prevent recurrence)
+- [ ] Review DMX controller firmware version compatibility with new node payloads
+- [ ] Document controller firmware versions alongside NR node versions
+- [ ] Consider firmware OTA update process documentation
+
+---
+
+## Session 4 — World Testing Complete
+
+### Bugs fixed this session
+- [x] msg.device string vs object — all nodes handle both formats
+- [x] Zone env var mismatch fixed across DMX Node and Group Node
+- [x] device.identifiers now includes postfix — L-992-A and L-992-B are separate HA devices
+- [x] Serial number and device name missing postfix — fixed all nodes
+- [x] Button/PIR MQTT In datatype json → auto-detect (plain string payloads)
+- [x] Button/PIR postfix options now include dash (-A, -B) matching DMX convention
+- [x] GROUP_name now used as HA entity friendly name
+- [x] DMX Group Node forwards device:add/remove via AUX to child DMX Nodes
+- [x] transitionsEnabled missing from CFG — fixed
+- [x] defaultOn values undefined — added fallbacks (full brightness)
+- [x] buildDmxPayload null guard — skips undefined channels
+- [x] Status node — replaced tostatus debug with proper status node in all subflows
+- [x] Subflow layout restored from canonical uploaded files
+- [x] NR 4.1.6 — definition + instance cannot import in same JSON (documented)
+
+### Confirmed working
+- [x] DMX Group Node → AUX → DMX Node fan-out (colour palette syncs all children)
+- [x] DMX Node direct (no group parent)
+- [x] Button physical press simulation via inject
+- [x] Button UI press via HA dashboard
+- [x] Button hold time (off_delay)
+- [x] PIR trigger simulation via inject
+- [x] Relay ON/OFF and state recovery
+
+### Pending field testing
+- [ ] Gamma floor at low combined brightness (fix ready, needs visual confirmation)
+- [ ] Button with real hardware controller
+- [ ] PIR with real hardware controller
+- [ ] Full multi-fixture deployment stress test
+- [ ] Firmware topic structure updates (one line each in config.h)
+
+---
+
+## Node Packaging — node-red-contrib-dmx-for-ha
+
+### Completed
+- [x] Package name: `node-red-contrib-dmx-for-ha`
+- [x] Palette category: `DMX for HA`
+- [x] Display names: DMX, DMX Group, Relay, Button, PIR
+- [x] Internal types: ha-mqtt-dmx, ha-mqtt-dmx-group, ha-mqtt-relay, ha-mqtt-button, ha-mqtt-pir
+- [x] Config node: `ha-mqtt-config` — one per zone, holds broker + site-wide settings
+- [x] Cascade output label: `Link` (DMX Group only)
+- [x] Zero outputs on DMX, Relay, Button, PIR
+- [x] MQTT handled internally — no external MQTT In/Out nodes needed
+- [x] Shared utility: inline for v1
+- [x] All passthrough references removed from code
+- [x] Areas and sub-areas lists finalised (38 areas, 21 sub-areas)
+- [x] ha-mqtt-config.html + .js written
+- [x] ha-mqtt-dmx.html + .js written
+- [x] ha-mqtt-dmx-group.html + .js written
+- [x] ha-mqtt-relay.html + .js written
+- [x] ha-mqtt-button.html + .js written
+- [x] ha-mqtt-pir.html + .js written
+- [x] package.json written
+- [x] Package folder structure created: node-red-contrib-dmx-for-ha/nodes/
+- [x] Logic review — 2 real bugs fixed:
+      - groupSync was blocking ALL group commands (should only block effect sync)
+      - DMX Group was forwarding device:add via Link (children get it from SYSTEM node)
+
+### Pending — Before First Install Test
+- [ ] settings.js — document disk context store requirement:
+      contextStorage: { default: {module:"memory"}, disk: {module:"localfilesystem"} }
+- [ ] README.md — package overview, setup guide, SYSTEM node wiring diagram
+- [ ] Test local install: npm install /path/to/node-red-contrib-dmx-for-ha
+- [ ] Verify nodes appear in NR palette under "DMX for HA"
+- [ ] Verify config node picker works on each node
+- [ ] Verify ha-mqtt-dmx discovery reaches HA
+- [ ] Verify ha-mqtt-dmx-group Link output fans out correctly
+- [ ] Verify ha-mqtt-relay publishes integer to relay controller topic
+- [ ] Verify ha-mqtt-button dual entity (binary_sensor + button) in HA
+- [ ] Verify ha-mqtt-pir warm-up sequence and availability topic
+
+### Pending — Before Publishing
+- [ ] Git repository setup (node-red-contrib-dmx-for-ha)
+- [ ] Branch structure: main / beta / _dev
+- [ ] README.md — complete
+- [ ] CHANGELOG.md
+- [ ] LICENSE file (Apache-2.0)
+- [ ] npm publish (or flows.nodered.org submission)
+- [ ] Version bump to 1.0.0 for first public release
+
+---
+
+## General / Cross-Node
+- [ ] Decide on consistent output label naming convention before node packaging
+- [ ] Rename "AUX output" on DMX Group Node and Relay Node to a clearer end-user label
+      (discussion deferred — agreed "AUX" is somewhat obscure but no replacement agreed yet)
+- [ ] Review all `node.warn` messages for consistency and helpfulness
+- [ ] Audit all env variable names across nodes for naming consistency
+- [ ] Master env variable reference document — expand `dmx_node_env_reference.md` to cover all nodes
+- [ ] Add Button Node, PIR Node, and Relay Node env vars to `dmx_node_env_reference.md`
+- [ ] "Hit by a bus" documentation — HOW-TO 101 / onboarding guide for new developers
+- [ ] SYSTEM node review — startup injection timing, daily sun-event add, heartbeat consideration
+      Current timing: T+15s first add, T+25s first ON, T+75s second add, T+85s second ON, T+120s PIR online
+      Consider replacing daily sun-event inject with HA-side heartbeat + NR timeout node