nwp500-python 9.5.0__tar.gz → 9.6.0__tar.gz

{nwp500_python-9.5.0 → nwp500_python-9.6.0}/CHANGELOG.rst

@@ -2,6 +2,57 @@
 Changelog
 =========
 
+Version 9.6.0 (2026-10-21)
+==========================
+
+Added
+-----
+- **Calibration outside the CLI** (``nwp500.sim_calibrate``, #177).
+  ``calibrate``, ``Calibrated``, ``report``, ``Term`` and ``max_gap``
+  moved here from ``nwp500.cli.calibration`` so an application can
+  calibrate without ``click`` or ``rich``. ``calibrate`` takes
+  ``HeaterParams`` or a parameters document as ``params`` and raises
+  ``CalibrationInputError`` (a ``ValueError``). Its notes no longer name
+  the command's options; ``nwp-cli simulate calibrate`` adds them.
+- ``status_history.iter_read`` yields a recording's statuses one at a
+  time.
+- ``sim_adapters.add_inputs`` adds the air, its humidity, a meter's
+  draws, the circulation's runs and the unobserved gaps to a frame
+  ``sim_adapters.history`` built.
+
+Changed
+-------
+- **A long history calibrates in bounded memory** (#177).
+  ``sim_adapters.history`` reads its statuses one at a time and keeps
+  only the frame's values, and ``sim_calibrate.calibrate`` takes the
+  record's span and gaps from the frame, so a generator over day files
+  is read once. 60 days of statuses (about 540,000) now take about
+  150 MB instead of about 6 GB of parsed statuses. The drawing and
+  circulating columns are computed per span instead of per row and
+  span.
+
+Documentation
+-------------
+- The calibration how-to says the recorder writes every status the
+  heater sends and what that costs on disk, shows calibrating from
+  day files with ``sim_calibrate``, adds calibrating in an
+  application, and says what Home Assistant's 10-day retention means
+  for a longer record.
+
+Version 9.5.1 (2026-10-21)
+==========================
+
+Fixed
+-----
+- **A steady Home Assistant sensor no longer goes stale in the
+  simulator** (#174). ``sim_home_assistant.parse_state`` dated a state by
+  ``last_updated``, which moves only when the value or its attributes
+  change. An air sensor holding the same value for over an hour failed
+  ``fresh()``, and the simulation fell back to the default air
+  temperature. A state is now dated by ``last_reported`` (Home Assistant
+  2024.3 and later), then ``last_updated``, then ``last_changed``. The
+  history parsers still date each change row by its change.
+
 Version 9.5.0 (2026-10-20)
 ==========================
 

{nwp500_python-9.5.0/src/nwp500_python.egg-info → nwp500_python-9.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nwp500-python
-Version: 9.5.0
+Version: 9.6.0
 Summary: A library for controlling Navien NWP500 Water Heaters via NaviLink
 Home-page: https://github.com/eman/nwp500-python
 Author: Emmanuel Levijarvi
@@ -72,9 +72,7 @@ Features
 * **Monitoring:** Real-time tracking of temperature, power usage, tank charge, and component status.
 * **Control:** Remote control of target temperatures, operation modes, and vacation settings.
 * **Scheduling:** Reservations, time-of-use (TOU) schedules and anti-legionella cycles.
-* **Typed models:** Pydantic models with validation and unit handling.
-* **Async:** Built on asyncio.
-* **Simulator (optional):** A physics model of the heater, ``nwp500.sim``, that answers "when will the tank be hot?", "how much hot water is there?" and "what would this change do?", and calibrates from a heater's own history. ``nwp-cli simulate`` runs it on the heater as it is now or on a scenario file. Install it with ``pip install nwp500-python[sim]``.
+* **Simulator (optional):** ``nwp500.sim``, a physics model of the heater. It estimates recovery time and available hot water, projects the effect of a change to the schedule or settings, and calibrates against a heater's own history. ``nwp-cli simulate`` runs it against the live heater or a scenario file. Install with ``pip install nwp500-python[sim]``.
 
 Getting Started
 ===============

{nwp500_python-9.5.0 → nwp500_python-9.6.0}/README.rst

@@ -18,9 +18,7 @@ Features
 * **Monitoring:** Real-time tracking of temperature, power usage, tank charge, and component status.
 * **Control:** Remote control of target temperatures, operation modes, and vacation settings.
 * **Scheduling:** Reservations, time-of-use (TOU) schedules and anti-legionella cycles.
-* **Typed models:** Pydantic models with validation and unit handling.
-* **Async:** Built on asyncio.
-* **Simulator (optional):** A physics model of the heater, ``nwp500.sim``, that answers "when will the tank be hot?", "how much hot water is there?" and "what would this change do?", and calibrates from a heater's own history. ``nwp-cli simulate`` runs it on the heater as it is now or on a scenario file. Install it with ``pip install nwp500-python[sim]``.
+* **Simulator (optional):** ``nwp500.sim``, a physics model of the heater. It estimates recovery time and available hot water, projects the effect of a change to the schedule or settings, and calibrates against a heater's own history. ``nwp-cli simulate`` runs it against the live heater or a scenario file. Install with ``pip install nwp500-python[sim]``.
 
 Getting Started
 ===============

{nwp500_python-9.5.0 → nwp500_python-9.6.0}/docs/how-to/calibrate-simulator.rst

@@ -14,9 +14,13 @@ fit its standby loss, its compressor and its power to that record.
 Record the history
 ==================
 
-``nwp-cli simulate record`` asks the heater for its status every 30
-seconds and appends each one to a file until you stop it (Ctrl+C). It
-needs the ``cli`` extra and no other:
+``nwp-cli simulate record`` appends every status the heater sends to a
+file until you stop it (Ctrl+C): the answers to the status requests it
+makes every 30 seconds (``--every``), and the statuses the heater sends
+between them. One heater sent about 370 an hour, about half of them
+identical to the one before. A line is about 2.3 kB, so a day of
+recording is about 20 MB, or 0.2 MB gzipped. The command needs the
+``cli`` extra and no other:
 
 .. code-block:: bash
 
@@ -35,11 +39,15 @@ can have gaps. The heater may have heated or delivered water during a
 gap, so ``calibrate`` takes any spacing between statuses of more than
 4 times the record's usual spacing (and at least 5 minutes) as
 unobserved. No quiet window and no recovery spans such a gap, and the
-report counts them. Write the file any other way you like (an MQTT callback, an
-export from a home-automation recorder) as long as each line has this
-shape: ``nwp500.status_history.line(instant, raw_status)`` writes one,
-and ``status_history.read(lines)`` reads them back as
-``(instant, DeviceStatus)`` pairs.
+report counts them. The identical statuses are what show the heater was
+observed between changes, so keep them.
+
+Write the file any other way you like (an MQTT callback, an export
+from a home-automation recorder) as long as each line has this shape:
+``nwp500.status_history.line(instant, raw_status)`` writes one.
+``status_history.read(lines)`` reads them back as
+``(instant, DeviceStatus)`` pairs, and ``status_history.iter_read(lines)``
+yields them one at a time, for a long recording.
 
 Record through quiet nights and full recoveries. The standby fit needs
 windows of 3 hours or more with no heating and no hot water in use. The
@@ -75,10 +83,14 @@ Idle, it reads the cabinet, about 6 °F warm. So give one of these:
   air beside the heater and its humidity over the record. It also reads
   the meter's draws and the circulation's runs, which make the quiet
   test exact: the heater's own "hot water in use" flag also fires on
-  circulation. Home Assistant keeps 10 days of history by default. The
-  air is used only over the span its history covers, and not while its
-  sensor was ``unavailable``; the humidity takes the 50 % default
-  outside that span, and the report says how much that is. Where the
+  circulation. The air is used only over the span its history covers,
+  and not while its sensor was ``unavailable``; the humidity takes the
+  50 % default outside that span, and the report says how much that
+  is. Home Assistant keeps 10 days of history by default, so of a
+  60-day record only the last 10 days have the air, and the standby
+  loss is fitted from those alone. For the whole record, raise the
+  recorder's ``purge_keep_days`` to cover it before you start
+  recording, or give the air another way (``--air``). Where the
   meter's or the circulation's history doesn't cover the whole record,
   the heater's own flag is used instead. ``--air``, ``--ambient`` and
   ``--humidity`` override the sources.
@@ -158,37 +170,66 @@ meter's draws, which the command does not take (see below).
 From Python
 ===========
 
-The command runs ``sim.calibration.calibrate``:
+``nwp500.sim_calibrate.calibrate`` does what the command does, and
+``report`` gives the command's rows as ``Term`` values. It takes the
+statuses one at a time, so a recording kept as a file per day can be
+read in order without holding every status:
 
 .. code-block:: python
 
-   from datetime import timedelta
+   from pathlib import Path
+
+   from nwp500 import sim, sim_calibrate, status_history
 
-   from nwp500 import sim, sim_adapters as sa, status_history
-   from nwp500.sim import calibration
+   def recorded(days):
+       for day in days:
+           with day.open(encoding="utf-8") as f:
+               yield from status_history.iter_read(f)
 
-   with open("history.jsonl") as f:
-       readings = status_history.read(f)
-   history = sa.history(
-       readings,
+   done = sim_calibrate.calibrate(
+       recorded(sorted(Path("history").glob("*.jsonl"))),
        air=air_readings,            # [(instant, temperature), ...]
        humidity=humidity_readings,  # [(instant, %RH), ...]
-       max_gap=timedelta(minutes=5),
+       params=sim.default_params(),  # or your own HeaterParams
    )
-   fitted = calibration.calibrate(history, sim.default_params(),
-                                  recovery_air="air_f")
-   params = fitted.params
-
-Give the humidity; if you have none, pass a constant 50 %, as the
-command does. ``max_gap`` is what
-marks a recording's gaps as unobserved; without it every span between
-two statuses is taken as observed.
-
-From Python you can also give a meter's draws and the circulation's
-runs, which make the quiet test exact and price the heat each cycle's
-draws took. The fits for the readiness terms, the heat book, the draw
-response and the elements also run there: see "Improve accuracy"
-and "Fit the elements" in :doc:`simulate-heater`.
+   params = done.calibration.params
+   for term in sim_calibrate.report(done):
+       print(term.name, term.value, "fitted" if term.fitted else term.why)
+
+Give ``ambient`` instead of ``air`` for a constant, or neither: the
+standby loss then keeps the base's value. Without ``humidity`` the fits
+get 50 %. An input it cannot use raises ``CalibrationInputError``.
+
+On a desktop computer, 60 days of statuses (about 540,000) took about
+40 seconds to read and 10 to fit, in about 150 MB.
+
+Calibrate in an application
+---------------------------
+
+An application that records the heater itself, such as a Home
+Assistant integration, can calibrate the same way:
+
+1. Write each status it receives with ``status_history.line``, a file
+   per UTC day, and keep as many days as the fit should cover.
+2. Read the air beside the heater and its humidity over the same span
+   with ``sim_home_assistant.record_inputs(client, entities, start,
+   end)``, and pass the result as ``record=``. It also brings the
+   meter's draws and the circulation's runs. Any client with the same
+   ``read_history`` as ``HomeAssistantClient`` works. The recorder's
+   retention limits the air, as above.
+3. Call ``sim_calibrate.calibrate`` in a worker thread. It takes the
+   gaps from the statuses' own spacing: a span longer than 4 times
+   their usual spacing, and at least 5 minutes, is unobserved.
+4. Store ``params_to_document(done.calibration.params)``, and show the
+   ``report(done)`` rows and ``done.notes``.
+
+``sim.calibration.calibrate`` underneath takes the frame
+``sim_adapters.history`` builds; ``sim_adapters.add_inputs`` adds the
+air, the humidity, a meter's draws and the circulation's runs to it.
+From there you can also give the draws that price the heat each
+cycle's draws took. The fits for the readiness terms, the heat book,
+the draw response and the elements also run there: see "Improve
+accuracy" and "Fit the elements" in :doc:`simulate-heater`.
 
 
 Limitations
@@ -202,13 +243,13 @@ Limitations
   heater's own "hot water in use" flag, which also fires on
   circulation. A loop's heat that does not trip the flag is invisible
   to the heater and lands in the standby loss. From Python, pass the
-  circulation's runs (``sa.history(..., circulating=...)``) to keep
-  them out of the quiet windows.
+  circulation's runs (``record=`` or ``sa.history(...,
+  circulating=...)``) to keep them out of the quiet windows.
 
 
 See also
 ========
 
 * :doc:`simulate-heater`: run the simulator with your parameters
-* :doc:`/reference/python_api/sim`: ``sim.calibration``,
-  ``sim_adapters.history`` and ``status_history``
+* :doc:`/reference/python_api/sim`: ``sim_calibrate``,
+  ``sim.calibration``, ``sim_adapters.history`` and ``status_history``

{nwp500_python-9.5.0 → nwp500_python-9.6.0}/docs/reference/python_api/sim.rst

@@ -297,6 +297,11 @@ Recorded history
 .. automodule:: nwp500.status_history
    :members:
 
+.. automodule:: nwp500.sim_calibrate
+   :members: calibrate, Calibrated, report, Term, max_gap,
+      CalibrationInputError, DEFAULT_HUMIDITY_PCT, GAP_SAMPLES, MIN_GAP,
+      DEFAULT_BASE_NOTE, NO_AIR_NOTE
+
 
 Home Assistant sources
 ======================

{nwp500_python-9.5.0 → nwp500_python-9.6.0}/setup.cfg

@@ -112,6 +112,16 @@ forbidden_modules =
 	awsiot
 	logging
 
+[importlinter:contract:calibrate-no-cli]
+name = An application calibrates without the CLI's dependencies
+type = forbidden
+source_modules = 
+	nwp500.sim_calibrate
+forbidden_modules = 
+	nwp500.cli
+	click
+	rich
+
 [importlinter:contract:ble-optional]
 name = Only nwp500.ble imports bleak, and only nwp-cli's ble backend imports nwp500.ble
 type = forbidden
@@ -138,6 +148,7 @@ source_modules =
 	nwp500.reservations
 	nwp500.sim
 	nwp500.sim_adapters
+	nwp500.sim_calibrate
 	nwp500.sim_home_assistant
 	nwp500.sim_live
 	nwp500.status_history

nwp500_python-9.6.0/src/nwp500/cli/calibration.py

@@ -0,0 +1,190 @@
+"""``nwp-cli simulate calibrate``: fit the heater's parameters to its
+recorded history.
+
+The history is the statuses the heater reported
+(``nwp500.status_history``, as ``nwp-cli simulate record`` writes them).
+The fits and the report's terms are ``nwp500.sim_calibrate``'s; this
+module reads the command's files and prints the report.
+
+This module needs the ``sim`` extra.
+"""
+
+import csv
+import json
+import logging
+import math
+import sys
+from collections.abc import Sequence
+from dataclasses import replace
+from datetime import datetime
+from pathlib import Path
+from typing import Any, TextIO
+
+from rich.console import Console
+from rich.table import Table
+
+from nwp500 import sim_calibrate as sc
+from nwp500 import status_history
+from nwp500.sim.params import params_to_document
+from nwp500.sim_calibrate import Calibrated, Term, report
+
+from .simulate import InputError, read_params
+
+__author__ = "Emmanuel Levijarvi"
+__copyright__ = "Emmanuel Levijarvi"
+__license__ = "MIT"
+
+_logger = logging.getLogger(__name__)
+
+#: A reading: an instant and a value.
+Reading = tuple[datetime, float]
+
+
+def read_history(path: str | Path) -> list[status_history.TimedStatus]:
+    """The statuses recorded at ``path``."""
+    try:
+        with Path(path).open(encoding="utf-8") as f:
+            readings = status_history.read(f)
+    except (OSError, ValueError) as e:
+        raise InputError(f"cannot read the history {path}: {e}") from None
+    if not readings:
+        raise InputError(f"the history {path} holds no statuses")
+    return readings
+
+
+def read_readings(path: str | Path, what: str) -> list[Reading]:
+    """A CSV file's readings: an ISO 8601 instant with its offset, then
+    a value, on each row. A first row that is not one is a header."""
+    try:
+        with Path(path).open(encoding="utf-8", newline="") as f:
+            rows = [r for r in csv.reader(f) if r and any(c.strip() for c in r)]
+    except OSError as e:
+        raise InputError(f"cannot read the {what} {path}: {e}") from None
+    out: list[Reading] = []
+    for number, row in enumerate(rows, start=1):
+        try:
+            at = datetime.fromisoformat(row[0].strip())
+            value = float(row[1])
+        except IndexError, ValueError:
+            if number == 1:
+                continue
+            raise InputError(
+                f"row {number} of the {what} {path} is not an instant "
+                "and a value"
+            ) from None
+        if at.tzinfo is None:
+            raise InputError(
+                f"row {number} of the {what} {path}: give the instant's "
+                "UTC offset"
+            )
+        if math.isfinite(value):
+            out.append((at, value))
+    if not out:
+        raise InputError(f"the {what} {path} holds no readings")
+    return out
+
+
+async def read_record(
+    sources: Any, readings: Sequence[status_history.TimedStatus]
+) -> Any:
+    """What ``sources`` recorded over ``readings``' span
+    (``sim_home_assistant.record_inputs``)."""
+    from nwp500 import sim_home_assistant as ha
+
+    first = min(t for t, _ in readings)
+    last = max(t for t, _ in readings)
+    async with ha.HomeAssistantClient(sources.url, sources.token) as client:
+        return await ha.record_inputs(client, sources.entities, first, last)
+
+
+#: The library's notes, naming the command's options.
+_OPTION_NOTES = {
+    sc.DEFAULT_BASE_NOTE: "refitting the 65 gal defaults (give --params "
+    "for another tank size)",
+    sc.NO_AIR_NOTE: "no air given (--air, --ambient or --sources): the "
+    "standby loss is not fitted",
+}
+
+
+def calibrate(
+    readings: Sequence[status_history.TimedStatus],
+    *,
+    air: Sequence[Reading] | None = None,
+    ambient: float | None = None,
+    humidity: Sequence[Reading] | None = None,
+    params: Any = None,
+    record: Any = None,
+) -> Calibrated:
+    """``sim_calibrate.calibrate`` with ``params`` a parameters
+    document, its errors as ``InputError`` and its notes naming the
+    command's options."""
+    base = None if params is None else read_params(params, "--params")
+    try:
+        done = sc.calibrate(
+            readings,
+            air=air,
+            ambient=ambient,
+            humidity=humidity,
+            params=base,
+            record=record,
+        )
+    except sc.CalibrationInputError as e:
+        raise InputError(str(e)) from None
+    return replace(
+        done, notes=tuple(_OPTION_NOTES.get(n, n) for n in done.notes)
+    )
+
+
+def _kept(t: Term) -> str:
+    if t.fitted:
+        return f"fitted: {t.note}" if t.note else "fitted"
+    return f"base: {t.why}" if t.why else "base"
+
+
+def print_report(done: Calibrated, console: Console) -> None:
+    """The report as tables on ``console``."""
+    c = done.calibration
+    console.print(
+        f"{done.n_statuses} statuses; quiet test: {c.standby.quiet_by}; "
+        f"{len(c.recoveries)} recoveries found"
+    )
+    if done.n_gaps:
+        minutes = done.max_gap.total_seconds() / 60
+        console.print(
+            f"{done.n_gaps} gap{'' if done.n_gaps == 1 else 's'} of over "
+            f"{minutes:g} min between statuses: no fit spans them"
+        )
+    console.print(
+        {
+            "readings": "Air: the readings given",
+            "constant": "Air: the constant given",
+            "source": "Air: the sources' record beside the heater",
+            None: "Air: none given; recoveries use the heater's own reading",
+        }[done.air]
+    )
+    table = Table(title="Calibration")
+    for column in ("Term", "Value", "From", "Error", "Kept"):
+        table.add_column(column)
+    for t in report(done):
+        table.add_row(
+            t.name,
+            t.value,
+            t.samples,
+            t.error,
+            _kept(t),
+        )
+    console.print(table)
+    unfitted = ", ".join(c.params.unfitted) or "none"
+    console.print(f"Unfitted: {unfitted}")
+
+
+def run(
+    done: Calibrated, output: str | Path | None, out: TextIO | None = None
+) -> None:
+    """Print the report and write the parameters to ``output``."""
+    for note in done.notes:
+        _logger.warning(note)
+    print_report(done, Console(file=out or sys.stdout))
+    if output is not None:
+        text = json.dumps(params_to_document(done.calibration.params), indent=2)
+        Path(output).write_text(text + "\n", encoding="utf-8")