hm-simulator 1.3.2 → 1.5.0

package/CHANGELOG.md

@@ -3,6 +3,38 @@
 All notable changes of hm-simulator. The release workflow takes each version's section from here for the
 GitHub release notes.
 
+## 1.5.0 — 2026-10-30
+
+- `getParamsetId(address, paramset)`, which answered the unknown-method fault: hmipserver's id
+  built from the device type, channel index and key (`hmipw-drbl4_2_link`), and on rfd and
+  BidCos-Wired the id the device file gives the paramset (`cc_tc_ch_wsr_link`), from the new
+  `data/paramset-ids.json` (261 BidCos device types, generated by `tools/paramset-ids.js` from
+  OpenCCU-Base's device files), with rfd's `-3 Unknown paramset` and its reading of any other key
+  as a peer address - all measured on 3.89.11. The option `paramsetIds` answers first.
+- The fixture `data/fixtures/hmip-wired.json`: an HmIPW-DRBL4 blind actuator, HmIPW-DRI32 and
+  HmIPW-DRI16 input modules and an HmIPW-WRC6 wall button with their real descriptions (MASTER,
+  VALUES, LINK, SERVICE), and five links to the actuator whose LINK paramsets hold what the CCU's
+  WebUI writes for the shutter profile chosen (one of them as the WebUI of firmware 3.87 wrote it,
+  with the long press's `LONG_PROFILE_ACTION_TYPE` 5 the current WebUI no longer writes).
+- The option `paramsets` (and the fixture property of that name for `--devices`): the stored
+  values MASTER, VALUES, SERVICE and link paramsets start with, on top of the defaults.
+- The interface option `stickyWriteEvents` (default `true`, as before): with `false`, a client's
+  `setValue` or `putParamset(VALUES)` on a sticky datapoint (`FLAGS & 16`, the acknowledgement of
+  `STICKY_UNREACH`) changes the value and sends no event. A registered client on rfd 3.89.11 got
+  none when another client acknowledged the message, so a client that waits for the event shows
+  the device as "was unreachable" until its next read; a consumer's test can now ask for that
+  (found through occulited's service messages).
+
+## 1.4.0 — 2026-10-03
+
+- `setInstallModeWithWhitelist(on, time, [{ADDRESS, KEY_MODE, KEY}])` on hmipserver, which answered
+  the unknown-method fault: it opens the install mode for the listed SGTINs (Homematic Manager's
+  pairing with the key from the sticker), `on: false` closes it. The scenario call `scriptInclusion`
+  puts an HmIP device in reach of the next install mode: it joins, or - with a whitelist `LOCAL` key
+  that is not its own - is declined, nothing joins and the device stays in reach; `getInclusions`
+  answers every request with its result (`paired`, `declined`, `ignored`), `getInstallWhitelist` the
+  open install mode's list. The simulator's model, not a measurement.
+
 ## 1.3.2
 
 - `getParamsetDescription` of an address the interface does not know answers the `unknownInstance`
@@ -10,12 +42,11 @@ GitHub release notes.
   as both interface processes do on 3.89.11; it answered `''`. A paramset the channel does not list
   answers the new table entry `unknownParamsetDescription` (`-3 Unknown Paramset` / `-3 Unknown
 paramset`, measured); a paramset the channel has but the description set lacks still answers `''`
-  (B-5, found through node-red-contrib-ccu's handling of the fault on a real CCU).
+  (found through node-red-contrib-ccu's handling of the fault on a real CCU).
 - `getLinkPeers` of a device address faults on `rfd` with the `notSupported` entry (`-1 Failure`
   under rfd's table, as rfd 3.89.11 answers; `-1 Generic error` under the default one), where it
   answered the peers of all channels; on hmipserver and the other interfaces it answers the channels'
-  lists one after the other, a peer linked with two of the channels twice, as hmipserver 3.89.11 does
-  (B-4).
+  lists one after the other, a peer linked with two of the channels twice, as hmipserver 3.89.11 does.
 
 ## 1.3.1
 
@@ -23,14 +54,14 @@ paramset`, measured); a paramset the channel has but the description set lacks s
   `rfd` - a relay's own button on the relay, a dimmer's channel on its virtual channels - as rfd
   3.89.11 does although such a link works; filtered by an address the link carries its stored flags,
   0, as rfd answers there too. A test that shows a client how it handles rfd's flag can now make the
-  link with `addLink` instead of seeding it (B-3, found through Homematic Manager's links list).
+  link with `addLink` instead of seeding it (found through Homematic Manager's links list).
 - `replaceDevice` answers `true`, as the specification's `bool` says, where it answered `''` (a
   client that takes the answer as the success saw a replacement that did happen as failed).
 - rfd's `listReplaceableDevices` names only the devices of the same type that are unreachable
   (`setReachable(iface, address, false)`); a device that still answers is not replaced.
 - The declarations gain `clients` (the registered logic layers per interface, read-only, `Client`),
   `dispatch` with its `DispatchContext` (the servers' entry, which a test may wrap to record calls),
-  and `replaceDevice`'s `true` (B-2, found through Homematic Manager's tests on 1.3.0).
+  and `replaceDevice`'s `true` (found through Homematic Manager's tests on 1.3.0).
 
 ## 1.3.0
 
@@ -113,7 +144,7 @@ paramset`, measured); a paramset the channel has but the description set lacks s
   the members in `TEAM_CHANNELS`; `setTeam(channel, team)` moves it and deletes a team nobody is
   left in, `setTeam(channel, '')` puts it back into a team of its own, created like the one it
   left (`newDevices`, `deleteDevices`, `updateDevice` to the logic layers). The shape is rfd's
-  as Homematic Manager task 58 read it from a CCU3; when rfd deletes and creates the team devices
+  as Homematic Manager read it from a CCU3; when rfd deletes and creates the team devices
   is the simulator's model, not a measurement.
 
 ## 1.0.0

package/README.md

@@ -122,17 +122,17 @@ that starts with the bytes `Bin` is BIN-RPC, anything else XML-RPC over HTTP.
 
 **Incoming RPC methods.**
 
-|                  |                                                                                                                                                                                                                                                   |
-| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| session          | `init` (register and de-register a logic layer), `ping`, `system.listMethods`, `system.methodHelp`, `system.multicall`                                                                                                                            |
-| devices          | `listDevices`, `getDeviceDescription`, `deleteDevice`, `replaceDevice`, `listReplaceableDevices`, `setInstallMode`, `getInstallMode`, `addDevice` (BidCos only), `getKeyMismatchDevice` (rfd, hmipserver)                                         |
-| paramsets        | `getParamsetDescription`, `getParamset`, `putParamset`, `getValue`, `setValue`, `determineParameter`, `reportValueUsage`                                                                                                                          |
-| links            | `getLinks`, `getLinkPeers`, `getLinkInfo`, `setLinkInfo`, `addLink`, `removeLink`, `activateLinkParamset`                                                                                                                                         |
-| interface        | `rssiInfo`, `listBidcosInterfaces`, `setBidcosInterface`, `getServiceMessages`, `setTempKey` (BidCos only), `changeKey` (rfd, hmipserver), `logLevel`, `getVersion`, `setInterfaceClock` (rfd, hmipserver), `getLGWStatus` (only when configured) |
-| metadata         | `getMetadata`, `setMetadata` (rfd, hs485d, hmipserver), `getAllMetadata` (BidCos only)                                                                                                                                                            |
-| service messages | `suppressServiceMessages`, `getSuppressedServiceMessages` (hmipserver only)                                                                                                                                                                       |
-| teams            | `listTeams`, `setTeam` (BidCos only): the smoke detector teams as rfd keeps them, pseudo devices `*<serial>`                                                                                                                                      |
-| maintenance      | `clearConfigCache`, `restoreConfigToDevice`, `updateFirmware`, `installFirmware`, `refreshDeployedDeviceFirmwareList` (rfd, hmipserver)                                                                                                           |
+|                  |                                                                                                                                                                                                                                                       |
+| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| session          | `init` (register and de-register a logic layer), `ping`, `system.listMethods`, `system.methodHelp`, `system.multicall`                                                                                                                                |
+| devices          | `listDevices`, `getDeviceDescription`, `deleteDevice`, `replaceDevice`, `listReplaceableDevices`, `setInstallMode`, `setInstallModeWithWhitelist` (hmipserver), `getInstallMode`, `addDevice` (BidCos only), `getKeyMismatchDevice` (rfd, hmipserver) |
+| paramsets        | `getParamsetDescription`, `getParamsetId`, `getParamset`, `putParamset`, `getValue`, `setValue`, `determineParameter`, `reportValueUsage`                                                                                                             |
+| links            | `getLinks`, `getLinkPeers`, `getLinkInfo`, `setLinkInfo`, `addLink`, `removeLink`, `activateLinkParamset`                                                                                                                                             |
+| interface        | `rssiInfo`, `listBidcosInterfaces`, `setBidcosInterface`, `getServiceMessages`, `setTempKey` (BidCos only), `changeKey` (rfd, hmipserver), `logLevel`, `getVersion`, `setInterfaceClock` (rfd, hmipserver), `getLGWStatus` (only when configured)     |
+| metadata         | `getMetadata`, `setMetadata` (rfd, hs485d, hmipserver), `getAllMetadata` (BidCos only)                                                                                                                                                                |
+| service messages | `suppressServiceMessages`, `getSuppressedServiceMessages` (hmipserver only)                                                                                                                                                                           |
+| teams            | `listTeams`, `setTeam` (BidCos only): the smoke detector teams as rfd keeps them, pseudo devices `*<serial>`                                                                                                                                          |
+| maintenance      | `clearConfigCache`, `restoreConfigToDevice`, `updateFirmware`, `installFirmware`, `refreshDeployedDeviceFirmwareList` (rfd, hmipserver)                                                                                                               |
 
 Every write is checked against the paramset description (type, range, `VALUE_LIST`,
 `OPERATIONS`) and answered the way the interface process answers it - see
@@ -196,6 +196,8 @@ UTF-8. BIN-RPC strings go out as ISO-8859-1, as rfd sends them.
 | `newDevices`               | `{}`                                                 | devices that appear when the install mode is switched on, `{<iface>: {devices, delay}}`       |
 | `defaultRssi`              | `-65`                                                | what `rssiInfo` answers where a device has no RSSI datapoint                                  |
 | `metadata`                 | `{}`                                                 | what `getMetadata` answers before any `setMetadata`, `{<iface>: {<address>: {<key>: value}}}` |
+| `paramsets`                | `{}`                                                 | stored values to start with, `{<iface>: {<address>: {<MASTER\|VALUES\|SERVICE\|peer>: {}}}}`  |
+| `paramsetIds`              | `{}`                                                 | what `getParamsetId` answers first, `{<iface>: {<address>: {<paramset>: id}}}`                |
 | `config.listenAddress`     | all interfaces                                       | `'127.0.0.1'` in tests                                                                        |
 | `config.binrpcListenPort`  | –                                                    | rfd                                                                                           |
 | `config.xmlrpcListenPort`  | –                                                    | hmipserver                                                                                    |
@@ -226,6 +228,7 @@ UTF-8. BIN-RPC strings go out as ISO-8859-1, as rfd sends them.
 | `pingDelay`                    | `0`                                                                            | milliseconds between `ping` and its `PONG`                                                                                                                                                                    |
 | `startDelay`                   | `0`                                                                            | milliseconds after `whenReady()` until the port accepts connections: a process that starts late (`sim.ports` knows the port from the start)                                                                   |
 | `unreachWrites`                | `'accept'`                                                                     | what `setValue`/`putParamset` to an unreachable device answer: `'accept'`, or `'fault'` (`notReachable`). The simulator's model, not a measurement                                                            |
+| `stickyWriteEvents`            | `true`                                                                         | `false`: a client's `setValue`/`putParamset` on a sticky datapoint (`FLAGS & 16`, the acknowledgement of `STICKY_UNREACH`) sends no event, see [Device and radio health](#device-and-radio-health)            |
 | `firmwareUpdateDelay`          | `0`                                                                            | milliseconds per step of a firmware update                                                                                                                                                                    |
 | `getServiceMessagesFault`      | `false`                                                                        | answer `getServiceMessages` with the unknown-method fault (`Invalid XML-RPC message`), as VirtualDevices, CUxD and hmipserver were seen to on 3.89.x                                                          |
 | `version`                      | `'2.6.0'` for `rfd` and `wired`, `'3.89.11.20260919'` for `hmip`               | what `getVersion` answers; the rest has no `getVersion`                                                                                                                                                       |
@@ -246,6 +249,14 @@ sim.getDevice('rfd', 'LAB0000002:1'); // the description, false when unknown
 // a device that holds another system's key: the next install mode hears it (getKeyMismatchDevice
 // names it) until setTempKey('their-key') lets it pair; addDevice('LAB0000009') does the same
 sim.scriptKeyMismatch('rfd', 'LAB0000009', {key: 'their-key', devices: [device, channel0, channel1]});
+// an HmIP device in factory state asks the next install mode to join; with a whitelist whose LOCAL
+// key is not its key it is declined, and stays in reach for the next one
+sim.scriptInclusion('hmip', '3014F711A000000000000088', {
+  key: '00112233445566778899AABBCCDDEEFF',
+  devices: [hmipDevice],
+});
+sim.getInclusions('hmip'); // [{sgtin, result: 'paired' | 'declined' | 'ignored', ts}]
+sim.getInstallWhitelist('hmip'); // the open install mode's [{ADDRESS, KEY_MODE, KEY}], [] without one
 ```
 
 **Values and events**
@@ -311,6 +322,13 @@ sim.regaSim.scripts; // every script the ReGa mock received
   with `setServiceMessage`. `STICKY_UNREACH` stays after the device is back until a client writes
   it `false`, as rfd does. A write to an unreachable device is accepted unless `unreachWrites` is
   `'fault'`.
+- That acknowledgement is answered with the channel's events, like every other write. On rfd
+  3.89.11 a registered client got **no** event when another client wrote `STICKY_UNREACH` `false`
+  (observed once, from the receiving client's log; a second measurement is pending): the
+  interface option `stickyWriteEvents: false` models that - a client's write to a datapoint with
+  the sticky flag (`FLAGS & 16`) changes the value and sends nothing, so only a read finds it
+  acknowledged. Use it to test that a client does not wait for an event that never comes. The
+  default stays `true` until the measurement is repeated.
 - The radio module is the HmIP device whose address is the last 14 characters of the interface's
   address in `listBidcosInterfaces` (its SGTIN), or a device of a radio module's type
   (`RPI-RF-MOD`, `HmIP-RFUSB`); without one only `listBidcosInterfaces` changes.
@@ -412,6 +430,7 @@ HMIP-PS, HMIP-SWDO, HMIP-WTH, HmIP-SMI and a radio module). Without `paramsetDes
 | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | `data/fixtures/lab-2026-09.json` | real devices of four test systems with the descriptions of their exact firmware, links and radio modules - the recommended one                                                    |
 | `data/fixtures/devices.json`     | generated from node-red-contrib-ccu's `paramsets.json`: real descriptions, synthesised channel indexes - HmIP-PDT, HmIPW-DRS8, HmIPW-DRI16, HmIPW-DRAP, HM-LC-Sw1-Pl, HM-CC-RT-DN |
+| `data/fixtures/hmip-wired.json`  | an HmIP-Wired distribution board: HmIPW-DRBL4, HmIPW-DRI32, HmIPW-DRI16, HmIPW-WRC6, links to the DRBL4 with the LINK paramsets the WebUI writes                                  |
 | `data/fixtures/lab-devices.json` | a real `listDevices` of two CCUs on firmware 3.89.8 (2026-09-05), no descriptions of its own - pass the ones of `devices.json` or the bundled set alongside                       |
 
 **`data/fixtures/lab-2026-09.json`**: four test systems (openccu-lite, the interface processes of
@@ -435,6 +454,30 @@ Real hardware's oddities stay: the CCU's own `HmIP-RCV-50` sends a trailing **em
 `CHILDREN`, for instance. The file is also a device file for the command line (`--devices`; add
 `--wired-port 0` or a port for BidCos-Wired).
 
+**`data/fixtures/hmip-wired.json`**: a wired blind actuator HmIPW-DRBL4 (firmware 1.6.0, four
+channels run as shutters), the input modules HmIPW-DRI32 (1.2.2) and HmIPW-DRI16 (1.4.6, as in
+`lab-2026-09.json`) with their inputs in key mode, and the wall button HmIPW-WRC6 (1.0.8). The
+descriptions of the DRBL4, DRI32 and WRC6 are hmipserver's answers as
+[pydevccu](https://github.com/danielperna84/pydevccu) (MIT) recorded them. Five links go to the
+DRBL4, and their `paramsets` hold what the CCU's WebUI writes for the profile chosen - the presets
+of its easymode file (`SHUTTER_VIRTUAL_RECEIVER/MULTI_MODE_INPUT_TRANSMITTER_1.tcl` and
+`…/KEY_TRANSCEIVER.tcl`, the first value where it lists several) on the description's defaults. The
+WebUI is eQ-3's current source, except for one link made the way the WebUI of firmware 3.87 made it:
+its shutter profiles still wrote `LONG_PROFILE_ACTION_TYPE` 5 for the long press, where the current
+one writes 1 (changed in June 2026), so a CCU keeps links of both kinds.
+
+| Sender     | Receiver    | Profile                                     |
+| ---------- | ----------- | ------------------------------------------- |
+| DRI32 `:1` | DRBL4 `:2`  | 3, "open / closed & up / down"              |
+| DRI32 `:2` | DRBL4 `:3`  | 3, "open / closed & up / down" (WebUI 3.87) |
+| DRI16 `:1` | DRBL4 `:6`  | 1, "open / up"                              |
+| WRC6 `:1`  | DRBL4 `:10` | 3, "open / closed & up / down"              |
+| WRC6 `:2`  | DRBL4 `:14` | 4, "move to position"                       |
+
+Pass the four properties (`devices`, `paramsetDescriptions`, `links`, `paramsets`) to the
+constructor, or the file to `--devices`. `tools/fixture-hmip-wired.js` builds it from a pydevccu
+package and an OpenCCU-Base checkout.
+
 **How descriptions are found.** A description is looked up by
 `<interface>/<type>/<firmware>/<version>/<channel type>/<paramset>`, and a device reports the
 firmware it runs - which a description set often does not have. Such a device uses the description
@@ -484,7 +527,7 @@ a run of the generator, or, better, by dumping a CCU that has one.
 ## Interface behaviour, measured
 
 Where the simulator imitates an interface process, this says whether the shape was measured on a
-CCU or is the simulator's model. The measurements were made for Homematic Manager (tasks 6 and 58)
+CCU or is the simulator's model. The measurements were made for Homematic Manager
 on CCUs with firmware 3.89.8 and 3.89.x.
 
 ### Direct links
@@ -535,6 +578,29 @@ What a `putParamset MASTER` does, per interface (`interfaces.<iface>.configPendi
 The two models are the explanations that competed before the measurement (Homematic Manager issue
 #98); they stay for applications that want to test against them.
 
+### Paramset ids
+
+`getParamsetId(address, paramset)` is the CCU's own name of a paramset; its WebUI picks the form of a
+channel or a link side by it (`easymodes/<id>.tcl`). Measured 2026-10-30 on firmware 3.89.11:
+
+- **hmipserver builds it** from the device type, the channel index and the key, lower case and
+  joined by `_`: `hmipw-drbl4_2_link`, `hmipw-dri16_1_master`, `hmipw-dri16_master` for a device
+  address. It does that for any key - `VALUES`, `SERVICE`, a peer address
+  (`hmipw-dri16_1_00000000000102:1`), a name the channel does not have, `null` without one - and
+  for a channel without a LINK paramset as well. The simulator does the same on `hmip`, `virtual`
+  and `cuxd` (VirtualDevices and CUxD not measured).
+- **rfd and hs485d answer the `id` of the `<paramset>` in the device file** (`firmware/rftypes`,
+  `firmware/hs485types`): `cc_tc_dev_master` for a device address (MASTER only), `maint_ch_master`,
+  `cc_tc_ch_wsr_link`, `hmw_input_ch_values` for a channel. Another key on a channel is taken as a
+  peer address and answers the LINK id; a paramset the address does not have - LINK or VALUES of a
+  device, LINK of a maintenance channel, any other key there - is `-3 Unknown paramset`. The
+  simulator answers from `data/paramset-ids.json` (261 device types, by type and the device file's
+  version, which is the description's `VERSION`; built by `tools/paramset-ids.js` from
+  OpenCCU-Base), the `unknownParamsetDescription` entry of the fault table for a paramset the
+  address does not list, and `''` for a type the files do not describe.
+- An unknown address is `unknownInstance` on all of them. `paramsetIds` in the constructor options
+  answers before the rules above.
+
 ### Fault codes
 
 The default table is **hmipserver's**, because that is the one an application has to survive
@@ -649,6 +715,17 @@ knows only its own addresses; `setInterfaceClock` is recorded, not measured (it
 devices get). `addDevice(serial)` pairs only the device of `scriptKeyMismatch`, with the right
 temporary key; any other serial is `unknownInstance`, the mismatch a `notSupported` fault.
 
+`setInstallModeWithWhitelist(on, time, [{ADDRESS, KEY_MODE, KEY}])` (hmipserver only; the others
+answer the unknown-method fault) opens the install mode for the listed SGTINs, as Homematic Manager
+sends it for a device paired with the key from its sticker (`KEY_MODE: 'LOCAL'`, `KEY` the 32 hex
+digits of the QR code), and `on: false` closes it; a list that is not an array of structs with an
+`ADDRESS` is `invalidArguments`. Which devices it admits is the simulator's model, not a
+measurement: a device of `scriptInclusion` joins when it is on the list and the entry's `LOCAL` key
+is its own (or the entry names another key mode, the key server's), is **declined** when the `LOCAL`
+key is another - nothing joins, no RPC client is told, `getInclusions` records it, the device stays
+in reach - and is not admitted (`ignored`) when it is not on the list. Opened with `setInstallMode`,
+the device joins.
+
 ## ReGa mock
 
 `rega: {port: 8181, ...}` starts an HTTP server that answers `POST /<name>.exe` as the CCU's
@@ -710,8 +787,8 @@ Usage: hm-simulator [options]
                             as one line to stdout with "-" (the log then goes to stderr)
   --config <file>           constructor options as a .json or .js file; the flags override it
   --devices <file>          a device file: {devices: {rfd, hmip, ...}, paramsetDescriptions, links,
-                            bidcosInterfaces}, such as data/fixtures/lab-2026-09.json; default:
-                            the bundled rfd and hmip lists
+                            bidcosInterfaces, paramsets}, such as data/fixtures/lab-2026-09.json;
+                            default: the bundled rfd and hmip lists
   --tls                     serve the XML-RPC servers over TLS with a generated certificate
   --tls-cert-out <file>     write that certificate (PEM) to the file
   --auth <user:password>    HTTP basic auth for the XML-RPC servers and ReGa
@@ -760,7 +837,8 @@ there. The flags win over the file:
 arguments. The answer is `{"result": ...}`; a fault is `400` with `faultCode` and `faultString`, an
 unknown call `404`. `GET /ports` answers the ports. The calls: `addDevice`, `removeDevice`,
 `fireEvent`, `fireEvents`, `setValue` (the device reporting a value, as a behaviour script does),
-`setServiceMessage`, `scriptNewDevices`, `scriptKeyMismatch`, `dropConnection`, `stopInterface`, `startInterface`,
+`setServiceMessage`, `scriptNewDevices`, `scriptKeyMismatch`, `scriptInclusion`, `getInclusions`,
+`getInstallWhitelist`, `dropConnection`, `stopInterface`, `startInterface`,
 `restartInterface`, `injectFault`, `clearFaults`, `setReachable`, `setLowBattery`, `setDutyCycle`,
 `setCarrierSense`, `offerFirmware`, `schedule`, `getDevice`, `getWriteLog`, `getConfigPending`,
 `getPoisonedChannels`, `getMissingParamsetDescriptions`, `getCallbackLog`, `getTempKey`,