nwp500-python 9.6.0__tar.gz → 9.7.0__tar.gz

{nwp500_python-9.6.0 → nwp500_python-9.7.0}/CHANGELOG.rst

@@ -2,6 +2,28 @@
 Changelog
 =========
 
+Version 9.7.0 (2026-10-21)
+==========================
+
+Added
+-----
+- **Calibrate from Home Assistant's history** (#179), for a heater with
+  no recording of its statuses.
+  ``sim_home_assistant.read_heater_history`` reads the heater's own
+  entities a day at a time: the probes, what is heating, the in-use
+  flag, the stop threshold, the operation mode, the error code, the
+  power, the energy to setpoint and the heater's own air, each number
+  from the unit it was recorded in. An entity that changes on every
+  response (``heard``) says when the heater was observed.
+  ``sim_calibrate.calibrate_changes`` fits on what it read, and gives
+  the same ``Calibrated`` and ``report()`` as ``calibrate``.
+- ``sim_adapters.ChangeRows`` and ``sim_adapters.change_history``: the
+  calibration frame from each value's changes and the instants the
+  heater was heard from, for any store of that history.
+- ``History.states``: every row's state as written, numbers or not.
+  ``HomeAssistantClient.read_history`` fills it (``parse_states``).
+- ``sim_home_assistant.to_w`` and ``to_wh``.
+
 Version 9.6.0 (2026-10-21)
 ==========================
 

{nwp500_python-9.6.0/src/nwp500_python.egg-info → nwp500_python-9.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nwp500-python
-Version: 9.6.0
+Version: 9.7.0
 Summary: A library for controlling Navien NWP500 Water Heaters via NaviLink
 Home-page: https://github.com/eman/nwp500-python
 Author: Emmanuel Levijarvi

{nwp500_python-9.6.0 → nwp500_python-9.7.0}/docs/how-to/calibrate-simulator.rst

@@ -232,6 +232,86 @@ the draw response and the elements also run there: see "Improve
 accuracy" and "Fit the elements" in :doc:`simulate-heater`.
 
 
+Calibrate from Home Assistant's history
+=======================================
+
+A heater whose integration writes its values to Home Assistant can be
+calibrated from that history, with no recording of its statuses. Home
+Assistant keeps each entity's changes, and a value that holds writes
+none, so the history also needs an entity that changes every time the
+heater answers, such as the integration's last response time. Its
+changes are when the heater was heard from; a silence between them
+longer than 4 times their usual spacing (and at least 5 minutes) is
+unobserved, as a gap between statuses is.
+
+.. code-block:: python
+
+   from datetime import UTC, datetime, timedelta
+
+   from nwp500 import sim_calibrate
+   from nwp500 import sim_home_assistant as ha
+
+   entities = ha.HeaterEntities(
+       heard="sensor.water_heater_last_mqtt_response",
+       upper="sensor.water_heater_tank_upper_temperature",
+       lower="sensor.water_heater_tank_lower_temperature",
+       compressor="binary_sensor.water_heater_compressor_running",
+       upper_element="binary_sensor.water_heater_upper_element",
+       lower_element="binary_sensor.water_heater_lower_element",
+       heat_source="sensor.water_heater_current_heat_source",
+       dhw_in_use="binary_sensor.water_heater_dhw_in_use",
+       target="sensor.water_heater_hp_lower_off_temperature",
+       operation_mode="sensor.water_heater_current_operation_mode",
+       error_code="sensor.water_heater_error_code",
+       power="sensor.water_heater_current_power",
+       energy_to_setpoint="sensor.water_heater_energy_to_setpoint",
+       ambient="sensor.water_heater_ambient_temperature",
+   )
+   end = datetime.now(UTC)
+   start = end - timedelta(days=60)
+   async with ha.HomeAssistantClient(url, token) as client:
+       history = await ha.read_heater_history(client, entities, start, end)
+       record = await ha.record_inputs(client, ha.Entities(air=..., humidity=...),
+                                       start, end)
+   done = sim_calibrate.calibrate_changes(
+       history.rows, history.observed, record=record, notes=history.notes
+   )
+
+Use your own entity ids. Only ``heard`` and the two probes are needed;
+leave out what your installation does not record. ``read_heater_history``:
+
+- reads a day at a time, so the heartbeat's rows are never all held as
+  Home Assistant gives them;
+- takes each number from the unit it was recorded in (°F, °C or K; W or
+  kW; Wh or kWh), ``on`` and ``off`` from the binary sensors, and the
+  heat source, operation mode and error code by their enum names
+  (``HeatSource``, ``CurrentOperationMode``, ``ErrorCode``);
+- takes ``unavailable`` or ``unknown`` on any of the entities as an
+  outage of the heater, unobserved (an enum's own ``UNKNOWN`` is a
+  value);
+- takes the heat source as the compressor's state where no compressor
+  entity is given;
+- notes a value it could not read, and an entity whose history starts
+  after ``start``.
+
+Give ``anti_legionella`` where it is recorded. Without it an
+anti-legionella cycle is not kept out of the recoveries, and the notes
+say so.
+
+Home Assistant keeps 10 days of history by default, so ``start`` reaches
+back only that far unless the recorder's ``purge_keep_days`` is raised
+to cover the record you want to fit. The same is true of the air.
+
+Rows from another store of the same history (an export, say) go in as
+``sim_adapters.ChangeRows``, each value's changes as ``(instant,
+value)`` pairs, with the instants heard from:
+``sim_calibrate.calibrate_changes(rows, observed, ...)``.
+
+On a desktop computer, 60 days with a heartbeat every 10 seconds
+(about 550,000 rows) took about 13 seconds to read, not counting Home
+Assistant's own queries, and 11 to fit, in about 180 MB.
+
+
 Limitations
 ===========
 
@@ -252,4 +332,6 @@ See also
 
 * :doc:`simulate-heater`: run the simulator with your parameters
 * :doc:`/reference/python_api/sim`: ``sim_calibrate``,
-  ``sim.calibration``, ``sim_adapters.history`` and ``status_history``
+  ``sim.calibration``, ``sim_adapters.history``,
+  ``sim_adapters.change_history``, ``status_history`` and
+  ``sim_home_assistant.read_heater_history``

{nwp500_python-9.6.0 → nwp500_python-9.7.0}/docs/reference/python_api/sim.rst

@@ -298,7 +298,7 @@ Recorded history
    :members:
 
 .. automodule:: nwp500.sim_calibrate
-   :members: calibrate, Calibrated, report, Term, max_gap,
+   :members: calibrate, calibrate_changes, Calibrated, report, Term, max_gap,
       CalibrationInputError, DEFAULT_HUMIDITY_PCT, GAP_SAMPLES, MIN_GAP,
       DEFAULT_BASE_NOTE, NO_AIR_NOTE
 
@@ -309,6 +309,8 @@ Home Assistant sources
 .. automodule:: nwp500.sim_home_assistant
    :members: HomeAssistantEnvironment, HomeAssistantClient, record_inputs,
       RecordInputs, Sources, Entities, sources_from, State, parse_state,
-      parse_history, parse_outages, History, history_for, parse_forecast,
+      parse_history, parse_outages, parse_states, History, history_for,
+      parse_forecast, read_heater_history, HeaterEntities, HeaterHistory,
+      HISTORY_WINDOW, to_w, to_wh,
       pump_runs, draw_spans, busy, covers_start, reports_after, percent,
       fresh, to_f, to_gpm, to_gal, to_rpm, HomeAssistantError

{nwp500_python-9.6.0 → nwp500_python-9.7.0}/src/nwp500/sim_adapters.py

@@ -760,15 +760,187 @@ def add_inputs(
     for name, spans in (("drawing", drawing), ("circulating", circulating)):
         if spans is not None:
             frame[name] = _interval_column(index, spans)
+    unobserved = np.zeros(len(frame), dtype=bool)
     if max_gap is not None and len(frame) > 1:
-        unobserved = np.zeros(len(frame), dtype=bool)
         unobserved[:-1] = np.diff(seconds) > max_gap.total_seconds()
+    if "heater_unobserved" in frame.columns:
+        # ``change_history``'s outages, into the draws added here too.
+        unobserved |= frame["heater_unobserved"].to_numpy(dtype=bool)
+    if unobserved.any():
         for name in ("dhw_in_use", "drawing", "excluded"):
             if name in frame.columns:
                 frame[name] = frame[name].to_numpy(dtype=bool) | unobserved
     return frame
 
 
+# ── a heater's history from its values' changes ───────────────────
+
+#: A value's changes: each instant it took a new value, and the value.
+Changes = Sequence[tuple[datetime, Any]]
+
+
+@dataclass(frozen=True)
+class ChangeRows:
+    """A heater's history as the changes of each value a status carries
+    (Home Assistant's entity history, say): each series the instants a
+    value changed and what it changed to, in time order. A value holds
+    until its next change. Temperatures are °F, power W, energy Wh.
+
+    Only the probes are needed. Without a value, its column keeps
+    ``history``'s default: the flags off, a number unknown.
+    """
+
+    upper_f: Changes
+    lower_f: Changes
+    compressor_on: Changes = ()
+    upper_element_on: Changes = ()
+    lower_element_on: Changes = ()
+    #: ``HeatSource`` values.
+    heat_source: Changes = ()
+    dhw_in_use: Changes = ()
+    #: The lower probe's heat-pump stop threshold.
+    target_f: Changes = ()
+    #: ``CurrentOperationMode`` values.
+    operation_mode: Changes = ()
+    anti_legionella: Changes = ()
+    #: ``ErrorCode`` values.
+    error_code: Changes = ()
+    power_w: Changes = ()
+    energy_to_setpoint_wh: Changes = ()
+    #: The heater's own air reading, used only while its compressor runs.
+    heater_air_f: Changes = ()
+    #: The spans a value was unknown (Home Assistant's ``unavailable``
+    #: or ``unknown``): unobserved, like a gap between statuses.
+    outages: Sequence[tuple[datetime, datetime]] = ()
+
+
+def _ns(instants: Iterable[datetime]) -> np.ndarray:
+    """``instants`` as UTC nanoseconds."""
+    import pandas as pd
+
+    return np.asarray(
+        pd.to_datetime(list(instants), utc=True).as_unit("ns").asi8,
+        dtype=np.int64,
+    )
+
+
+def _held_at(
+    changes: Changes, at: np.ndarray, missing: Any, dtype: Any
+) -> np.ndarray:
+    """``changes`` held from each change to the next, read at ``at``
+    (ns); ``missing`` before the first."""
+    out = np.full(len(at), missing, dtype=dtype)
+    if not changes:
+        return out
+    rows = sorted(changes, key=lambda r: r[0])
+    times = _ns(t for t, _ in rows)
+    values = np.array([v for _, v in rows], dtype=dtype)
+    i = np.searchsorted(times, at, side="right") - 1
+    seen = i >= 0
+    out[seen] = values[i[seen]]
+    return out
+
+
+def change_history(
+    rows: ChangeRows, observed: Iterable[datetime]
+) -> pd.DataFrame:
+    """``history``'s frame from a heater's values' changes (``rows``).
+
+    A value that holds writes no change, so the changes cannot tell an
+    observed span from one nobody saw: ``observed`` is the instants the
+    heater was heard from (an entity that changes on every status
+    response, say). The frame has a row at each of them and at each
+    change between the first and the last, each value as it stood
+    then. Rows over an outage are marked water in use and excluded, as
+    ``add_inputs`` marks a gap, and ``heater_unobserved``. Without a
+    compressor flag, the heat source says when the compressor runs.
+    """
+    import pandas as pd
+
+    heard = _ns(observed)
+    if not len(heard):
+        at = heard
+    else:
+        first, last = heard.min(), heard.max()
+        changed = [
+            _ns(t for t, _ in series)
+            for series in (
+                rows.upper_f,
+                rows.lower_f,
+                rows.compressor_on,
+                rows.upper_element_on,
+                rows.lower_element_on,
+                rows.heat_source,
+                rows.dhw_in_use,
+                rows.target_f,
+                rows.operation_mode,
+                rows.anti_legionella,
+                rows.error_code,
+                rows.power_w,
+                rows.energy_to_setpoint_wh,
+                rows.heater_air_f,
+            )
+            if series
+        ]
+        at = np.unique(np.concatenate([heard, *changed]))
+        at = at[(at >= first) & (at <= last)]
+    nan = float("nan")
+    source = _held_at(rows.heat_source, at, -1, np.int64)
+    compressor = (
+        _held_at(rows.compressor_on, at, False, bool)
+        if rows.compressor_on
+        # Without its own flag, the heat source says when it runs.
+        else np.isin(
+            source,
+            [int(HeatSource.HEATPUMP), int(HeatSource.HEATPUMP_HEATELEMENT)],
+        )
+    )
+    mode = _held_at(rows.operation_mode, at, -1, np.int64)
+    error = _held_at(rows.error_code, at, int(ErrorCode.NO_ERROR), np.int64)
+
+    def optional(series: Changes) -> np.ndarray | None:
+        return _held_at(series, at, nan, float) if series else None
+
+    columns: dict[str, np.ndarray | None] = {
+        "upper_f": _held_at(rows.upper_f, at, nan, float),
+        "lower_f": _held_at(rows.lower_f, at, nan, float),
+        "compressor_on": compressor,
+        "element_on": _held_at(rows.upper_element_on, at, False, bool)
+        | _held_at(rows.lower_element_on, at, False, bool)
+        | np.isin(
+            source,
+            [int(HeatSource.HEATELEMENT), int(HeatSource.HEATPUMP_HEATELEMENT)],
+        ),
+        "dhw_in_use": _held_at(rows.dhw_in_use, at, False, bool),
+        "target_f": optional(rows.target_f),
+        "excluded": np.isin(
+            mode,
+            [
+                int(CurrentOperationMode.HYBRID_EFFICIENCY_MODE),
+                int(CurrentOperationMode.HYBRID_BOOST_MODE),
+            ],
+        )
+        | _held_at(rows.anti_legionella, at, False, bool)
+        | (error != int(ErrorCode.NO_ERROR)),
+        "power_w": optional(rows.power_w),
+        "energy_to_setpoint_wh": optional(rows.energy_to_setpoint_wh),
+        "heater_air_f": np.where(
+            compressor, _held_at(rows.heater_air_f, at, nan, float), nan
+        ),
+    }
+    # ``history``'s columns, in its order, less the values not given.
+    data = {k: v for k, v in columns.items() if v is not None}
+    index = pd.DatetimeIndex(pd.to_datetime(at, unit="ns", utc=True))
+    frame = pd.DataFrame(data, index=index)
+    if rows.outages and len(frame):
+        down = np.asarray(_interval_column(index, rows.outages))
+        for name in ("dhw_in_use", "excluded"):
+            frame[name] = frame[name].to_numpy(dtype=bool) | down
+        # Kept, so ``add_inputs`` marks a meter's draws over them too.
+        frame["heater_unobserved"] = down
+    return frame
+
+
 # ── a past scenario, from the statuses the heater reported ─────────
 
 #: A tank reading older than this at the instant is no state to start

{nwp500_python-9.6.0 → nwp500_python-9.7.0}/src/nwp500/sim_calibrate.py

@@ -34,8 +34,9 @@ statuses::
 This module needs the ``sim`` extra.
 """
 
+import itertools
 from collections.abc import Iterable, Mapping, Sequence
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
 from datetime import datetime, timedelta
 from typing import TYPE_CHECKING, Any, cast
 
@@ -66,6 +67,7 @@ __all__ = (
     "Calibrated",
     "Term",
     "calibrate",
+    "calibrate_changes",
     "max_gap",
     "report",
 )
@@ -176,6 +178,20 @@ def _base(params: HeaterParams | Any) -> HeaterParams:
         ) from None
 
 
+def _begin(
+    air: Sequence[Reading] | None,
+    ambient: float | None,
+    params: HeaterParams | Any,
+) -> tuple[HeaterParams, list[str]]:
+    """The base to refit and the notes so far; the inputs checked before
+    a history is read."""
+    if air is not None and ambient is not None:
+        raise CalibrationInputError("give air or ambient, not both")
+    if params is None:
+        return sim.default_params(), [DEFAULT_BASE_NOTE]
+    return _base(params), []
+
+
 def calibrate(
     readings: Iterable[tuple[datetime, DeviceStatus]],
     *,
@@ -204,17 +220,69 @@ def calibrate(
     for both ``air`` and ``ambient``, a ``params`` that is not a
     parameters document, or no readings.
     """
-    if air is not None and ambient is not None:
-        raise CalibrationInputError("give air or ambient, not both")
-    notes: list[str] = []
-    if params is None:
-        base = sim.default_params()
-        notes.append(DEFAULT_BASE_NOTE)
-    else:
-        base = _base(params)
+    base, notes = _begin(air, ambient, params)
     frame = sa.history(readings)
     if frame.empty:
         raise CalibrationInputError("the history holds no statuses")
+    return _fit(frame, base, notes, air, ambient, humidity, record)
+
+
+def calibrate_changes(
+    rows: sa.ChangeRows,
+    observed: Iterable[datetime],
+    *,
+    air: Sequence[Reading] | None = None,
+    ambient: float | None = None,
+    humidity: Sequence[Reading] | None = None,
+    params: HeaterParams | Any = None,
+    record: RecordInputs | None = None,
+    notes: Iterable[str] = (),
+) -> Calibrated:
+    """``calibrate`` on a heater's values' changes (``rows``) instead of
+    its statuses, with ``observed`` the instants it was heard from
+    (``sim_adapters.change_history``; ``sim_home_assistant.
+    read_heater_history`` reads both from Home Assistant). ``notes``
+    open the result's (a reader's). The rows are counted in
+    ``n_statuses``. Raises ``CalibrationInputError`` as ``calibrate``
+    does, or for no ``observed`` instants.
+    """
+    base, more = _begin(air, ambient, params)
+    heard = sorted(observed)
+    # The gaps are the silences between the instants heard from: a
+    # change inside one says nothing of the time around it.
+    gap = max_gap(heard)
+    silences = [(a, b) for a, b in itertools.pairwise(heard) if b - a > gap]
+    frame = sa.change_history(
+        replace(rows, outages=(*rows.outages, *silences)), heard
+    )
+    if frame.empty:
+        raise CalibrationInputError("the history holds no observed instants")
+    return _fit(
+        frame,
+        base,
+        [*notes, *more],
+        air,
+        ambient,
+        humidity,
+        record,
+        gaps=(gap, len(silences)),
+    )
+
+
+def _fit(
+    frame: pd.DataFrame,
+    base: HeaterParams,
+    notes: list[str],
+    air: Sequence[Reading] | None,
+    ambient: float | None,
+    humidity: Sequence[Reading] | None,
+    record: RecordInputs | None,
+    gaps: tuple[timedelta, int] | None = None,
+) -> Calibrated:
+    """The fits on ``frame`` (``history``'s or ``change_history``'s),
+    with the air, the humidity, the record's spans and the gaps added:
+    ``gaps``' longest spacing taken as observed and how many spans are
+    longer, else the frame's own."""
     index = cast("pd.DatetimeIndex", frame.index)
     # Timestamps, which are datetimes.
     first, last = cast(datetime, index[0]), cast(datetime, index[-1])
@@ -254,8 +322,11 @@ def calibrate(
         # The fits' own default, given explicitly: an absent humidity
         # reaches the evaporator's dew point as NaN, which reads as 100 %.
         humidity = [(first, DEFAULT_HUMIDITY_PCT), (last, DEFAULT_HUMIDITY_PCT)]
-    gap = max_gap(index)
-    n_gaps = int((_steps(index) > gap.total_seconds() * 1e9).sum())
+    if gaps is None:
+        gap = max_gap(index)
+        n_gaps = int((_steps(index) > gap.total_seconds() * 1e9).sum())
+    else:
+        gap, n_gaps = gaps
     sa.add_inputs(
         frame,
         air=series,