nwp500-python 9.5.1__tar.gz → 9.6.0__tar.gz

{nwp500_python-9.5.1 → nwp500_python-9.6.0}/CHANGELOG.rst

@@ -2,6 +2,43 @@
 Changelog
 =========
 
+Version 9.6.0 (2026-10-21)
+==========================
+
+Added
+-----
+- **Calibration outside the CLI** (``nwp500.sim_calibrate``, #177).
+  ``calibrate``, ``Calibrated``, ``report``, ``Term`` and ``max_gap``
+  moved here from ``nwp500.cli.calibration`` so an application can
+  calibrate without ``click`` or ``rich``. ``calibrate`` takes
+  ``HeaterParams`` or a parameters document as ``params`` and raises
+  ``CalibrationInputError`` (a ``ValueError``). Its notes no longer name
+  the command's options; ``nwp-cli simulate calibrate`` adds them.
+- ``status_history.iter_read`` yields a recording's statuses one at a
+  time.
+- ``sim_adapters.add_inputs`` adds the air, its humidity, a meter's
+  draws, the circulation's runs and the unobserved gaps to a frame
+  ``sim_adapters.history`` built.
+
+Changed
+-------
+- **A long history calibrates in bounded memory** (#177).
+  ``sim_adapters.history`` reads its statuses one at a time and keeps
+  only the frame's values, and ``sim_calibrate.calibrate`` takes the
+  record's span and gaps from the frame, so a generator over day files
+  is read once. 60 days of statuses (about 540,000) now take about
+  150 MB instead of about 6 GB of parsed statuses. The drawing and
+  circulating columns are computed per span instead of per row and
+  span.
+
+Documentation
+-------------
+- The calibration how-to says the recorder writes every status the
+  heater sends and what that costs on disk, shows calibrating from
+  day files with ``sim_calibrate``, adds calibrating in an
+  application, and says what Home Assistant's 10-day retention means
+  for a longer record.
+
 Version 9.5.1 (2026-10-21)
 ==========================
 

{nwp500_python-9.5.1/src/nwp500_python.egg-info → nwp500_python-9.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nwp500-python
-Version: 9.5.1
+Version: 9.6.0
 Summary: A library for controlling Navien NWP500 Water Heaters via NaviLink
 Home-page: https://github.com/eman/nwp500-python
 Author: Emmanuel Levijarvi

{nwp500_python-9.5.1 → nwp500_python-9.6.0}/docs/how-to/calibrate-simulator.rst

@@ -14,9 +14,13 @@ fit its standby loss, its compressor and its power to that record.
 Record the history
 ==================
 
-``nwp-cli simulate record`` asks the heater for its status every 30
-seconds and appends each one to a file until you stop it (Ctrl+C). It
-needs the ``cli`` extra and no other:
+``nwp-cli simulate record`` appends every status the heater sends to a
+file until you stop it (Ctrl+C): the answers to the status requests it
+makes every 30 seconds (``--every``), and the statuses the heater sends
+between them. One heater sent about 370 an hour, about half of them
+identical to the one before. A line is about 2.3 kB, so a day of
+recording is about 20 MB, or 0.2 MB gzipped. The command needs the
+``cli`` extra and no other:
 
 .. code-block:: bash
 
@@ -35,11 +39,15 @@ can have gaps. The heater may have heated or delivered water during a
 gap, so ``calibrate`` takes any spacing between statuses of more than
 4 times the record's usual spacing (and at least 5 minutes) as
 unobserved. No quiet window and no recovery spans such a gap, and the
-report counts them. Write the file any other way you like (an MQTT callback, an
-export from a home-automation recorder) as long as each line has this
-shape: ``nwp500.status_history.line(instant, raw_status)`` writes one,
-and ``status_history.read(lines)`` reads them back as
-``(instant, DeviceStatus)`` pairs.
+report counts them. The identical statuses are what show the heater was
+observed between changes, so keep them.
+
+Write the file any other way you like (an MQTT callback, an export
+from a home-automation recorder) as long as each line has this shape:
+``nwp500.status_history.line(instant, raw_status)`` writes one.
+``status_history.read(lines)`` reads them back as
+``(instant, DeviceStatus)`` pairs, and ``status_history.iter_read(lines)``
+yields them one at a time, for a long recording.
 
 Record through quiet nights and full recoveries. The standby fit needs
 windows of 3 hours or more with no heating and no hot water in use. The
@@ -75,10 +83,14 @@ Idle, it reads the cabinet, about 6 °F warm. So give one of these:
   air beside the heater and its humidity over the record. It also reads
   the meter's draws and the circulation's runs, which make the quiet
   test exact: the heater's own "hot water in use" flag also fires on
-  circulation. Home Assistant keeps 10 days of history by default. The
-  air is used only over the span its history covers, and not while its
-  sensor was ``unavailable``; the humidity takes the 50 % default
-  outside that span, and the report says how much that is. Where the
+  circulation. The air is used only over the span its history covers,
+  and not while its sensor was ``unavailable``; the humidity takes the
+  50 % default outside that span, and the report says how much that
+  is. Home Assistant keeps 10 days of history by default, so of a
+  60-day record only the last 10 days have the air, and the standby
+  loss is fitted from those alone. For the whole record, raise the
+  recorder's ``purge_keep_days`` to cover it before you start
+  recording, or give the air another way (``--air``). Where the
   meter's or the circulation's history doesn't cover the whole record,
   the heater's own flag is used instead. ``--air``, ``--ambient`` and
   ``--humidity`` override the sources.
@@ -158,37 +170,66 @@ meter's draws, which the command does not take (see below).
 From Python
 ===========
 
-The command runs ``sim.calibration.calibrate``:
+``nwp500.sim_calibrate.calibrate`` does what the command does, and
+``report`` gives the command's rows as ``Term`` values. It takes the
+statuses one at a time, so a recording kept as a file per day can be
+read in order without holding every status:
 
 .. code-block:: python
 
-   from datetime import timedelta
+   from pathlib import Path
+
+   from nwp500 import sim, sim_calibrate, status_history
 
-   from nwp500 import sim, sim_adapters as sa, status_history
-   from nwp500.sim import calibration
+   def recorded(days):
+       for day in days:
+           with day.open(encoding="utf-8") as f:
+               yield from status_history.iter_read(f)
 
-   with open("history.jsonl") as f:
-       readings = status_history.read(f)
-   history = sa.history(
-       readings,
+   done = sim_calibrate.calibrate(
+       recorded(sorted(Path("history").glob("*.jsonl"))),
        air=air_readings,            # [(instant, temperature), ...]
        humidity=humidity_readings,  # [(instant, %RH), ...]
-       max_gap=timedelta(minutes=5),
+       params=sim.default_params(),  # or your own HeaterParams
    )
-   fitted = calibration.calibrate(history, sim.default_params(),
-                                  recovery_air="air_f")
-   params = fitted.params
-
-Give the humidity; if you have none, pass a constant 50 %, as the
-command does. ``max_gap`` is what
-marks a recording's gaps as unobserved; without it every span between
-two statuses is taken as observed.
-
-From Python you can also give a meter's draws and the circulation's
-runs, which make the quiet test exact and price the heat each cycle's
-draws took. The fits for the readiness terms, the heat book, the draw
-response and the elements also run there: see "Improve accuracy"
-and "Fit the elements" in :doc:`simulate-heater`.
+   params = done.calibration.params
+   for term in sim_calibrate.report(done):
+       print(term.name, term.value, "fitted" if term.fitted else term.why)
+
+Give ``ambient`` instead of ``air`` for a constant, or neither: the
+standby loss then keeps the base's value. Without ``humidity`` the fits
+get 50 %. An input it cannot use raises ``CalibrationInputError``.
+
+On a desktop computer, 60 days of statuses (about 540,000) took about
+40 seconds to read and 10 to fit, in about 150 MB.
+
+Calibrate in an application
+---------------------------
+
+An application that records the heater itself, such as a Home
+Assistant integration, can calibrate the same way:
+
+1. Write each status it receives with ``status_history.line``, a file
+   per UTC day, and keep as many days as the fit should cover.
+2. Read the air beside the heater and its humidity over the same span
+   with ``sim_home_assistant.record_inputs(client, entities, start,
+   end)``, and pass the result as ``record=``. It also brings the
+   meter's draws and the circulation's runs. Any client with the same
+   ``read_history`` as ``HomeAssistantClient`` works. The recorder's
+   retention limits the air, as above.
+3. Call ``sim_calibrate.calibrate`` in a worker thread. It takes the
+   gaps from the statuses' own spacing: a span longer than 4 times
+   their usual spacing, and at least 5 minutes, is unobserved.
+4. Store ``params_to_document(done.calibration.params)``, and show the
+   ``report(done)`` rows and ``done.notes``.
+
+``sim.calibration.calibrate`` underneath takes the frame
+``sim_adapters.history`` builds; ``sim_adapters.add_inputs`` adds the
+air, the humidity, a meter's draws and the circulation's runs to it.
+From there you can also give the draws that price the heat each
+cycle's draws took. The fits for the readiness terms, the heat book,
+the draw response and the elements also run there: see "Improve
+accuracy" and "Fit the elements" in :doc:`simulate-heater`.
 
 
 Limitations
@@ -202,13 +243,13 @@ Limitations
   heater's own "hot water in use" flag, which also fires on
   circulation. A loop's heat that does not trip the flag is invisible
   to the heater and lands in the standby loss. From Python, pass the
-  circulation's runs (``sa.history(..., circulating=...)``) to keep
-  them out of the quiet windows.
+  circulation's runs (``record=`` or ``sa.history(...,
+  circulating=...)``) to keep them out of the quiet windows.
 
 
 See also
 ========
 
 * :doc:`simulate-heater`: run the simulator with your parameters
-* :doc:`/reference/python_api/sim`: ``sim.calibration``,
-  ``sim_adapters.history`` and ``status_history``
+* :doc:`/reference/python_api/sim`: ``sim_calibrate``,
+  ``sim.calibration``, ``sim_adapters.history`` and ``status_history``

{nwp500_python-9.5.1 → nwp500_python-9.6.0}/docs/reference/python_api/sim.rst

@@ -297,6 +297,11 @@ Recorded history
 .. automodule:: nwp500.status_history
    :members:
 
+.. automodule:: nwp500.sim_calibrate
+   :members: calibrate, Calibrated, report, Term, max_gap,
+      CalibrationInputError, DEFAULT_HUMIDITY_PCT, GAP_SAMPLES, MIN_GAP,
+      DEFAULT_BASE_NOTE, NO_AIR_NOTE
+
 
 Home Assistant sources
 ======================

{nwp500_python-9.5.1 → nwp500_python-9.6.0}/setup.cfg

@@ -112,6 +112,16 @@ forbidden_modules =
 	awsiot
 	logging
 
+[importlinter:contract:calibrate-no-cli]
+name = An application calibrates without the CLI's dependencies
+type = forbidden
+source_modules = 
+	nwp500.sim_calibrate
+forbidden_modules = 
+	nwp500.cli
+	click
+	rich
+
 [importlinter:contract:ble-optional]
 name = Only nwp500.ble imports bleak, and only nwp-cli's ble backend imports nwp500.ble
 type = forbidden
@@ -138,6 +148,7 @@ source_modules =
 	nwp500.reservations
 	nwp500.sim
 	nwp500.sim_adapters
+	nwp500.sim_calibrate
 	nwp500.sim_home_assistant
 	nwp500.sim_live
 	nwp500.status_history

nwp500_python-9.6.0/src/nwp500/cli/calibration.py

@@ -0,0 +1,190 @@
+"""``nwp-cli simulate calibrate``: fit the heater's parameters to its
+recorded history.
+
+The history is the statuses the heater reported
+(``nwp500.status_history``, as ``nwp-cli simulate record`` writes them).
+The fits and the report's terms are ``nwp500.sim_calibrate``'s; this
+module reads the command's files and prints the report.
+
+This module needs the ``sim`` extra.
+"""
+
+import csv
+import json
+import logging
+import math
+import sys
+from collections.abc import Sequence
+from dataclasses import replace
+from datetime import datetime
+from pathlib import Path
+from typing import Any, TextIO
+
+from rich.console import Console
+from rich.table import Table
+
+from nwp500 import sim_calibrate as sc
+from nwp500 import status_history
+from nwp500.sim.params import params_to_document
+from nwp500.sim_calibrate import Calibrated, Term, report
+
+from .simulate import InputError, read_params
+
+__author__ = "Emmanuel Levijarvi"
+__copyright__ = "Emmanuel Levijarvi"
+__license__ = "MIT"
+
+_logger = logging.getLogger(__name__)
+
+#: A reading: an instant and a value.
+Reading = tuple[datetime, float]
+
+
+def read_history(path: str | Path) -> list[status_history.TimedStatus]:
+    """The statuses recorded at ``path``."""
+    try:
+        with Path(path).open(encoding="utf-8") as f:
+            readings = status_history.read(f)
+    except (OSError, ValueError) as e:
+        raise InputError(f"cannot read the history {path}: {e}") from None
+    if not readings:
+        raise InputError(f"the history {path} holds no statuses")
+    return readings
+
+
+def read_readings(path: str | Path, what: str) -> list[Reading]:
+    """A CSV file's readings: an ISO 8601 instant with its offset, then
+    a value, on each row. A first row that is not one is a header."""
+    try:
+        with Path(path).open(encoding="utf-8", newline="") as f:
+            rows = [r for r in csv.reader(f) if r and any(c.strip() for c in r)]
+    except OSError as e:
+        raise InputError(f"cannot read the {what} {path}: {e}") from None
+    out: list[Reading] = []
+    for number, row in enumerate(rows, start=1):
+        try:
+            at = datetime.fromisoformat(row[0].strip())
+            value = float(row[1])
+        except IndexError, ValueError:
+            if number == 1:
+                continue
+            raise InputError(
+                f"row {number} of the {what} {path} is not an instant "
+                "and a value"
+            ) from None
+        if at.tzinfo is None:
+            raise InputError(
+                f"row {number} of the {what} {path}: give the instant's "
+                "UTC offset"
+            )
+        if math.isfinite(value):
+            out.append((at, value))
+    if not out:
+        raise InputError(f"the {what} {path} holds no readings")
+    return out
+
+
+async def read_record(
+    sources: Any, readings: Sequence[status_history.TimedStatus]
+) -> Any:
+    """What ``sources`` recorded over ``readings``' span
+    (``sim_home_assistant.record_inputs``)."""
+    from nwp500 import sim_home_assistant as ha
+
+    first = min(t for t, _ in readings)
+    last = max(t for t, _ in readings)
+    async with ha.HomeAssistantClient(sources.url, sources.token) as client:
+        return await ha.record_inputs(client, sources.entities, first, last)
+
+
+#: The library's notes, naming the command's options.
+_OPTION_NOTES = {
+    sc.DEFAULT_BASE_NOTE: "refitting the 65 gal defaults (give --params "
+    "for another tank size)",
+    sc.NO_AIR_NOTE: "no air given (--air, --ambient or --sources): the "
+    "standby loss is not fitted",
+}
+
+
+def calibrate(
+    readings: Sequence[status_history.TimedStatus],
+    *,
+    air: Sequence[Reading] | None = None,
+    ambient: float | None = None,
+    humidity: Sequence[Reading] | None = None,
+    params: Any = None,
+    record: Any = None,
+) -> Calibrated:
+    """``sim_calibrate.calibrate`` with ``params`` a parameters
+    document, its errors as ``InputError`` and its notes naming the
+    command's options."""
+    base = None if params is None else read_params(params, "--params")
+    try:
+        done = sc.calibrate(
+            readings,
+            air=air,
+            ambient=ambient,
+            humidity=humidity,
+            params=base,
+            record=record,
+        )
+    except sc.CalibrationInputError as e:
+        raise InputError(str(e)) from None
+    return replace(
+        done, notes=tuple(_OPTION_NOTES.get(n, n) for n in done.notes)
+    )
+
+
+def _kept(t: Term) -> str:
+    if t.fitted:
+        return f"fitted: {t.note}" if t.note else "fitted"
+    return f"base: {t.why}" if t.why else "base"
+
+
+def print_report(done: Calibrated, console: Console) -> None:
+    """The report as tables on ``console``."""
+    c = done.calibration
+    console.print(
+        f"{done.n_statuses} statuses; quiet test: {c.standby.quiet_by}; "
+        f"{len(c.recoveries)} recoveries found"
+    )
+    if done.n_gaps:
+        minutes = done.max_gap.total_seconds() / 60
+        console.print(
+            f"{done.n_gaps} gap{'' if done.n_gaps == 1 else 's'} of over "
+            f"{minutes:g} min between statuses: no fit spans them"
+        )
+    console.print(
+        {
+            "readings": "Air: the readings given",
+            "constant": "Air: the constant given",
+            "source": "Air: the sources' record beside the heater",
+            None: "Air: none given; recoveries use the heater's own reading",
+        }[done.air]
+    )
+    table = Table(title="Calibration")
+    for column in ("Term", "Value", "From", "Error", "Kept"):
+        table.add_column(column)
+    for t in report(done):
+        table.add_row(
+            t.name,
+            t.value,
+            t.samples,
+            t.error,
+            _kept(t),
+        )
+    console.print(table)
+    unfitted = ", ".join(c.params.unfitted) or "none"
+    console.print(f"Unfitted: {unfitted}")
+
+
+def run(
+    done: Calibrated, output: str | Path | None, out: TextIO | None = None
+) -> None:
+    """Print the report and write the parameters to ``output``."""
+    for note in done.notes:
+        _logger.warning(note)
+    print_report(done, Console(file=out or sys.stdout))
+    if output is not None:
+        text = json.dumps(params_to_document(done.calibration.params), indent=2)
+        Path(output).write_text(text + "\n", encoding="utf-8")

{nwp500_python-9.5.1 → nwp500_python-9.6.0}/src/nwp500/sim_adapters.py

@@ -44,10 +44,11 @@ Example::
 
 import itertools
 import math
+from array import array
 from collections.abc import Callable, Iterable, Mapping, Sequence
 from dataclasses import dataclass, replace
 from datetime import UTC, datetime, time, timedelta
-from typing import TYPE_CHECKING, Any
+from typing import TYPE_CHECKING, Any, cast
 from zoneinfo import ZoneInfo
 
 import numpy as np
@@ -581,21 +582,80 @@ def _hybrid(status: DeviceStatus) -> bool:
 
 def _interval_column(
     index: pd.DatetimeIndex, spans: Iterable[tuple[datetime, datetime]]
-) -> list[bool]:
+) -> np.ndarray:
     """Each row's interval (its instant to the next row's) that overlaps
-    a span: a span wholly between two samples marks the first."""
+    a span: a span wholly between two samples marks the first. The
+    index is in time order."""
+    import pandas as pd
+
+    t = index.as_unit("ns").asi8
+    n = len(t)
+    out = np.zeros(n, dtype=bool)
     spans = list(spans)
-    times = list(index)
-    out: list[bool] = []
-    for i, t in enumerate(times):
-        if i + 1 < len(times):
-            nxt = times[i + 1]
-            out.append(any(a < nxt and b > t for a, b in spans))
-        else:
-            out.append(any(a <= t < b for a, b in spans))
+    if not spans or not n:
+        return out
+    # utc=True takes instants with differing offsets (across a change
+    # of clocks).
+    starts = pd.to_datetime([a for a, _ in spans], utc=True)
+    ends = pd.to_datetime([b for _, b in spans], utc=True)
+    a = starts.as_unit("ns").asi8
+    b = ends.as_unit("ns").asi8
+    # Row i < n - 1 is marked when a < t[i + 1] and b > t[i]: from the
+    # last row at or before a to the last row before b.
+    lo = np.maximum(np.searchsorted(t, a, side="right") - 1, 0)
+    hi = np.minimum(np.searchsorted(t, b, side="left"), n - 1)
+    keep = hi > lo
+    marks = np.zeros(n + 1, dtype=np.int64)
+    np.add.at(marks, lo[keep], 1)
+    np.add.at(marks, hi[keep], -1)
+    out[:] = np.cumsum(marks[:n]) > 0
+    # The last row has no next: a span holding its instant marks it.
+    out[-1] = bool(((a <= t[-1]) & (t[-1] < b)).any())
     return out
 
 
+#: The epoch, for an instant as whole microseconds.
+_EPOCH = datetime(1970, 1, 1, tzinfo=UTC)
+_US = timedelta(microseconds=1)
+
+#: The history's columns from a status, in order, and their type code.
+_COLUMNS = (
+    ("upper_f", "d"),
+    ("lower_f", "d"),
+    ("compressor_on", "b"),
+    ("element_on", "b"),
+    ("dhw_in_use", "b"),
+    ("target_f", "d"),
+    ("excluded", "b"),
+    ("power_w", "d"),
+    ("energy_to_setpoint_wh", "d"),
+    ("heater_air_f", "d"),
+)
+
+
+def _row(s: DeviceStatus) -> tuple[float | bool, ...]:
+    """A status's values, in ``_COLUMNS``' order."""
+    return (
+        _from_device(DeciCelsius(s.tank_upper_temperature_raw)),
+        _from_device(DeciCelsius(s.tank_lower_temperature_raw)),
+        bool(s.comp_use),
+        bool(s.heat_upper_use)
+        or bool(s.heat_lower_use)
+        or s.current_heat_use
+        in (HeatSource.HEATELEMENT, HeatSource.HEATPUMP_HEATELEMENT),
+        bool(s.dhw_use),
+        _from_device(HalfCelsius(s.hp_lower_off_temp_setting_raw)),
+        _hybrid(s)
+        or bool(s.anti_legionella_operation_busy)
+        or s.error_code != ErrorCode.NO_ERROR,
+        float(s.current_inst_power),
+        float(s.energy_to_setpoint),
+        _from_device(DeciCelsius(s.ambient_temperature_raw))
+        if s.comp_use
+        else float("nan"),
+    )
+
+
 def history(
     readings: Iterable[tuple[datetime, DeviceStatus]],
     *,
@@ -611,10 +671,59 @@ def history(
     Everything a status carries is read from it: the probes, what is
     heating, ``dhw_use``, the lower probe's stop threshold, what rules a
     sample out of a heat-pump recovery, the power it draws, the energy
-    still to heat, and
-    the heater's own air
-    reading while its compressor runs. What it does not carry is the
-    caller's, in the preferred unit:
+    still to heat, and the heater's own air reading while its compressor
+    runs. ``readings`` is taken one at a time and only those values are
+    kept, so a generator over a long recording
+    (``status_history.iter_read``) needs memory for the history, not
+    for its statuses.
+
+    What a status does not carry is the caller's, as
+    ``add_inputs`` takes it.
+    """
+    import pandas as pd
+
+    times: array[int] = array("q")
+    values: list[array[Any]] = [array(code) for _, code in _COLUMNS]
+    for at, status in readings:
+        times.append((at - _EPOCH) // _US)
+        for column, value in zip(values, _row(status), strict=True):
+            column.append(value)
+    stamps = (
+        np.frombuffer(times, dtype=np.int64) if times else np.empty(0, np.int64)
+    )
+    order = np.argsort(stamps, kind="stable")
+    ordered = bool(np.all(order == np.arange(len(order))))
+    data: dict[str, np.ndarray] = {}
+    for (name, code), column in zip(_COLUMNS, values, strict=True):
+        a = np.array(column, dtype=float if code == "d" else bool)
+        data[name] = a if ordered else a[order]
+    index = pd.DatetimeIndex(
+        pd.to_datetime(
+            stamps if ordered else stamps[order], unit="us", utc=True
+        )
+    ).as_unit("ns")
+    frame = pd.DataFrame(data, index=index)
+    return add_inputs(
+        frame,
+        air=air,
+        humidity=humidity,
+        drawing=drawing,
+        circulating=circulating,
+        max_gap=max_gap,
+    )
+
+
+def add_inputs(
+    frame: pd.DataFrame,
+    *,
+    air: Iterable[tuple[datetime, float]] | None = None,
+    humidity: Iterable[tuple[datetime, float]] | None = None,
+    drawing: Iterable[tuple[datetime, datetime]] | None = None,
+    circulating: Iterable[tuple[datetime, datetime]] | None = None,
+    max_gap: timedelta | None = None,
+) -> pd.DataFrame:
+    """``frame`` (``history``'s) with what its statuses do not carry,
+    added in place, in the preferred unit:
 
     - ``air``: the air where the heater stands, interpolated in time onto
       the readings (a sensor's, or ``location_air``'s projection); the
@@ -634,53 +743,8 @@ def history(
     window and no recovery spans it. Without it, every span between two
     readings is taken as observed.
     """
-    import pandas as pd
-
-    rows = sorted(readings, key=lambda r: r[0])
-    index = pd.DatetimeIndex([at for at, _ in rows]).tz_convert("UTC")
-    frame = pd.DataFrame(
-        {
-            "upper_f": [
-                _from_device(DeciCelsius(s.tank_upper_temperature_raw))
-                for _, s in rows
-            ],
-            "lower_f": [
-                _from_device(DeciCelsius(s.tank_lower_temperature_raw))
-                for _, s in rows
-            ],
-            "compressor_on": [bool(s.comp_use) for _, s in rows],
-            "element_on": [
-                bool(s.heat_upper_use)
-                or bool(s.heat_lower_use)
-                or s.current_heat_use
-                in (HeatSource.HEATELEMENT, HeatSource.HEATPUMP_HEATELEMENT)
-                for _, s in rows
-            ],
-            "dhw_in_use": [bool(s.dhw_use) for _, s in rows],
-            "target_f": [
-                _from_device(HalfCelsius(s.hp_lower_off_temp_setting_raw))
-                for _, s in rows
-            ],
-            "excluded": [
-                _hybrid(s)
-                or bool(s.anti_legionella_operation_busy)
-                or s.error_code != ErrorCode.NO_ERROR
-                for _, s in rows
-            ],
-            "power_w": [float(s.current_inst_power) for _, s in rows],
-            "energy_to_setpoint_wh": [
-                float(s.energy_to_setpoint) for _, s in rows
-            ],
-            "heater_air_f": [
-                _from_device(DeciCelsius(s.ambient_temperature_raw))
-                if s.comp_use
-                else float("nan")
-                for _, s in rows
-            ],
-        },
-        index=index,
-    )
-    seconds = [t.timestamp() for t in index]
+    index = cast("pd.DatetimeIndex", frame.index)
+    seconds = index.as_unit("ns").asi8 / 1e9
     for name, series in (("air_f", air), ("humidity_pct", humidity)):
         if series is None:
             continue